70285 June 28, 2010 Regional Study on Targeting Systems and Practices Draft Policy Note Human Development Unit South Asia World Bank 1 Contents Team and Acknowledgements ......................................................................................................................... 4 Executive Summary ........................................................................................................................................... 5 A. Introduction.................................................................................................................................................10 A.1. Background and context ....................................................................................................................10 A.2. Why target? ..........................................................................................................................................11 A.3. How is targeting done? .......................................................................................................................12 A.4. This study .............................................................................................................................................15 B. Overview of Targeted Government Programs and Targeting Practices in South Asia....................17 B.1 Brief overview of targeted program types ........................................................................................18 B.2 Overview of regional targeting systems ............................................................................................20 Use of specific targeting approaches ....................................................................................................20 Architecture of targeting systems ..........................................................................................................21 B.3 Targeting effectiveness ........................................................................................................................25 Determinants of targeting effectiveness ...............................................................................................28 C. Overview of World Bank-Led Regional Work on Targeting ...............................................................42 C.1 Methodological approach....................................................................................................................43 C.2. Rationale for targeting in World Bank projects: To target or not to target ................................46 C.3 The quality of World Bank-led regional targeting work .................................................................48 Quality at entry.........................................................................................................................................49 Quality of implementation and monitoring .........................................................................................53 Quality at exit ...........................................................................................................................................56 C.4. The impact of World-Bank led regional targeting work................................................................57 Role as a vehicle for innovation and knowledge sharing ...................................................................58 Strengthening administrative and technical capacity within government .......................................58 Strengthening the architecture of existing government systems ......................................................61 D. Concluding Remarks ..................................................................................................................................63 References .........................................................................................................................................................65 Annex 1: Targeted programs and targeting systems in South Asia ..........................................................67 The Targeting of Social Safety Programs in South Asia: A selected annotated bibliography ..........67 2 Table A1: Indicators of program-specific targeting effectiveness........................................................83 Annex 2: List of World Bank projects surveyed .........................................................................................87 Annex 3: Standardized questionnaire for TTLs and project teams ..........................................................89 List of Tables Table 1: Poverty Indicators for South Asia ..................................................................................................10 Table 2: Targeting methods by type of program and country ..................................................................21 Table 3: Inclusion and exclusion errors ........................................................................................................26 Table 4: Overview of surveyed projects by country and type of project/advisory task........................44 Table 5: Targeting approaches by sector and type of program .................................................................49 List of Boxes Box 1: Main types of targeting methods.......................................................................................................13 Box 2: National targeting systems .................................................................................................................14 Box 3: Identifying the poor using an integrated targeting system in India..............................................23 Box 4: Targeting cash transfers in the aftermath of disaster: The Pakistan earthquake ........................25 Box 5: Targeting outcomes and effectiveness in South Asia.....................................................................27 Box 6: Incentives and outreach in RSBY .....................................................................................................35 Box 7: Impact of program implementation on targeting effectiveness: Social pensions and the PDS in India ...............................................................................................................................................................40 Box 8: Participatory approaches to targeting: Concepts and operational challenges.............................47 Box 9: Piloting before choosing a targeting method ..................................................................................50 Box 10: Proxy means testing in South Asia: Emerging interest and recent experience .......................52 Box 11: Poverty mapping ...............................................................................................................................59 Box 12: The impact of investments in database management on government targeting systems: the RSBY case .........................................................................................................................................................60 3 TEAM AND ACKNOWLEDGEMENTS This note has been prepared by Carolina Sanchez-Paramo, Puja Vasudeva Dutta and Yashodhan Ghorpade, under the guidance of Mansoora Rashid. Ben Powis (SASSD, consultant) prepared the background note on participatory approaches to targeting. Yashodhan Ghorpade prepared the background note on proxy means testing in South Asia. Tanusree Talukdar provided team support during preparation. The authors would like to thank Meena Munshi, Biswajit Sen, Barbara Verardo, Asmeen Khan, Jeffrey S. Hammer, Qazi Azmat Isa, Samik Das, Parmesh Shah, Meera Shekhar, Caryn Bredenkamp, Ihsan Ajwad, Kees Kostermans, Agnes Couffinhall, Benjamin Loevinsohn, Venkatesh Sundararaman, Amit Dar, Deepa Sankar, Rajendra Joshi, Tahseen Sayed, Sofia Shakil, Reema Nayar, Dushyanth Raju, Helen Craig, Harsha Aturupane, Andrea Vermehren, Iffath Sharif, Xiaohui Hou, Aylin Isik-Dikmelik, Udhav Bhandari, Jishnu Das, Robert Palacios, Tara Vishwanath and Rinku Murgai for valuable information and contributions. The authors would also like to thank peer reviewers, Margaret Grosh and Philip O‘Keefe, for insightful and helpful comments. 4 EXECUTIVE SUMMARY Targeting is a tool policy makers sometimes employ to make a program more efficient by increasing its direct impact on the poor and vulnerable. Designing, administering and monitoring an effective targeting system is a complex and often costly task, but one that can potentially produce important benefits in the medium-term. Redistributive packages to protect the vulnerable exist in all South Asian countries, but they often suffer from low coverage rates and poor targeting which significantly undermine their impact on the poor. Increased targeting efficiency can contribute to improving the quality and poverty impact of expenditures within the existing budget envelope—an attractive prospect for countries with tight fiscal situations. More inclusive growth will require, among other factors, adequately targeted interventions to support the poor and vulnerable. Even in programs delivering universal benefits or services, some element of targeting may be useful in ensuring the inclusion of the particularly disadvantaged groups. In addition, well administered and effectively targeted programs assume an even greater significance during times of crisis when the quickest response is often to scale up existing programs. Recent years have been witness to an increased emphasis on the part of the World Bank on the need to adequately target program and project benefits to the intended population. This can be attributed to both a growing demand from regional governments for this type of support and a renewed institutional interest on ensuring maximum (poverty) impact of World Bank-supported initiatives. As a result attention to targeting-related issues has been increasing within analytical and operational portfolios. This study This policy note aims to critically take stock of regional experiences in the area of targeting, both in the context of government systems and the World Bank‘s operational work, in South Asia. The main objectives are the following:  Review targeting systems and practices in the context of government programs  Critically review the role for and impact of targeting in the World Bank‘s operational work  Extract lessons that can be used to deepen the relevance and impact of the World Bank‘s operational work in South Asia We expect the evidence presented in this note will serve as a resource for those interested in and/or planning some work on targeting related work in the region. In this sense, by presenting information on both country systems and performance of World Bank-led work, the note targets both practitioners and managers (particularly parts C and D). Analytical framework and methodology The first part of the analysis focuses on the architecture of targeting systems in South Asia and on the determinants of targeting effectiveness, including (a) the choice and design of the targeting tool; (b) implementation and monitoring of the targeting tool; and (c) the design, implementation and monitoring of the targeted program. This analysis draws entirely upon existing literature. In fact, in many cases, the most comprehensive sources of information are recent World Bank country-specific reports, such as poverty assessment, social protection or social sector reports. This prior analytical 5 work in many of the countries in the region provides a solid diagnostic basis for compiling information on targeting performance of government programs (particularly of safety net programs) and social services. The review of the World Bank‘s operational work is focused primarily on targeting work undertaken as part of projects that target benefits or services to the poor and associated technical assistance. The analysis is based on primary qualitative research carried out specifically for this policy note in 2008. This involved collection of project-specific information in a systematic manner through a standardized questionnaire for project teams, usually followed by meetings, and reliance on project documentation and technical papers to supplement this information. We also include in this analysis a small sample of standalone technical assistance activities that have focused primarily on developing tools for national targeting systems. In some cases, the tools are developed with a view of piloting through a particular government program, but the intention often is to rely on the demonstration effect for wider adoption. A total of 25 projects were initially identified by regional management, out of which 20 form our final sample. These were selected based on discussions with project teams on the relative importance of targeting-related work within the overall project. Response rates varied across groups, with the final pool of surveyed projects including mainly rural livelihood projects, social protection projects, and projects aiming to increase access to basic social services. The distribution of projects across countries is also somewhat uneven, with half in India and no relevant projects identified in Afghanistan, Bhutan or Maldives. Finally, as targeting is a relatively new element within World Bank operational work in the region, a conscious decision was made to focus on more recent projects. As a result, at the time of the survey, most of the projects considered for the analysis were either under preparation or implementation, with only a few having recently closed. The implications of these project characteristics are that our information is particularly rich with respect to the design of the targeting tool, with some information on implementation and monitoring, but very little on evaluation. The analytical framework for this analysis mirrors that of government systems to some extent in terms of its focus on issues of design; implementation and monitoring; and evaluation and impact. We develop the following typology for examining the nature, relevance and impact of the World Bank‘s operational work in the South Asia region:  Quality at entry: At the design stage, the World Bank could increase exposure to and in- country knowledge of international best and innovative practices, provide strategic technical advice to strengthen the architecture of existing systems, and help governments choose the right targeting approach given the country context or the nature of the intervention to be targeted.  Quality of implementation: At the implementation stage, the World Bank can leverage knowledge, expertise and resources to build institutional and technical capacity for adequate administration and monitoring. This is not only true in those cases where operational work is framed within a broader government program, but applies to self-standing operational work as well where demonstration effects can be quite powerful.  Quality at exit: Finally, the World Bank can play an important role in what regards evaluation of existing systems by providing information on targeting effectiveness and supporting governments in extracting and acting upon lessons learned. 6 Finally it must be taken into account that the project information discussed in the note was collected between July and December, 2008 and therefore reflects the situation at that point. It is possible for some of the project features to have changed since as lessons arising from project supervision and evaluation are incorporated into the project‘s design. Consequently, the discussion below should be taken as a whole and not as reflecting the merits or defects of specific projects or advisory tasks. Main messages None of the South Asian countries have an integrated targeting system, though India, and now Pakistan display some elements of this. In this respect, South Asian countries differ markedly from many countries in Eastern Europe, Central Asia and Latin America, where national targeting systems are common. An integrated national targeting architecture typically includes the development of a backbone targeting method to be used for the majority of programs (but with room for program-specific targeting tools depending on the program design, e.g., self-targeting for public works) with clearly defined processes for data collection, eligibility determination and beneficiary database management, clear institutional roles and responsibilities and systems for monitoring, verification, grievance redressal and fraud control. As a result, targeting is usually tackled at the program level, both in terms of design and implementation and monitoring arrangements, often leading to high levels of fragmentation, ineffective use of resources and excessive administrative costs and ultimately poor targeting outcomes both at the country and program-levels. This is often compounded by confusion among potential beneficiaries of various programs with apparently similar objectives but different beneficiary pools, and the subsequent negative impact on program perception. In this context, there is significant potential for the World Bank to add value at entry, implementation and exit, thereby supporting countries in strengthening and improving targeting systems and outcomes. Our analysis suggests that much of the World Bank‘s engagement and impact as been at the design stage (i.e., quality at entry) and less so in implementation and monitoring (i.e., quality at implementation) and evaluation (i.e., quality at exit). The World Bank’s operational work on targeting issues has undoubtedly served as a vehicle for innovation and knowledge sharing in the region. First, the introduction of introduction of targeting components in World Bank projects has helped reinforce the notion that targeting can be an effective tool to increase transparency and accountability and maximize impact in the face of limited resources. Second, building on international best practices, a variety of new targeting approaches have been introduced in the region, including the use of community-based targeting in India, proxy-means tests in Bangladesh, Nepal, Pakistan and Sri Lanka and poverty mapping in Bangladesh, India and Sri Lanka. In both instances, there existed no prior (minimum scale) in- country experience with either targeting approach, so that an opportunity was created to foster debate and build capacity. However, these efforts have not always translated into improved in-country technical and implementation capacity or improved government targeting systems. A variety of different approaches have been used by different project teams, with little or no administrative or technical capacity being created within government on the most part. Closely resembling the fragmentation in government programs, most of the World Bank operational work on targeting has taken place at the program or project level. Some projects supporting existing targeted government programs tend to 7 adopt and at times improve upon the government targeting systems in use. However, much operational work on targeting relates to existing untargeted programs or new interventions where World Bank projects have often promoted de facto parallel systems for the purpose of targeting. In these cases, these targeting choices can be seen as feeding into, rather than mitigating, system fragmentation. As a consequence, the impact of this work on the policy debate on targeting, the performance of government systems and, ultimately, on targeting effectiveness has been limited, with a few exceptions. The World Bank‘s operational work has, on the whole, played a limited role in fostering discussion on the targeting effectiveness of existing government systems, either by evaluating existing systems and projects with a view to strengthening the existing system or exploring the adoption of alternative approaches based on learning from projects. In contrast, much of the standalone analytical and advisory work has focused on developing targeting tools that can be applied nationally. However, this dialogue is at an early stage and the ultimate impact on government systems remains to be seen. Going forward, there remains significant potential to deepen the relevance and impact of targeting in operational work by the World Bank in South Asia. Several governments are grappling with problems of improving targeting outcomes and there have been recent debates on introducing new or scaling up existing programs in response to the food, fuel and financial crises. In several countries, there is a renewed focus on targeting and a lively debate around the introduction of new methods and systems (e.g., in Pakistan) or the reform of existing systems (e.g., in India). In fact, there is a more fundamental debate underway in India on targeted versus universal provision of programs and services that has its roots in the growth of a rights-based approach to development in the country. While the World Bank‘s operational work has introduced several innovations to the region, this could be further strengthened by greater rigor during the design, implementation and monitoring and evaluation of targeting tools. While the World Bank’s operational work has introduced several innovations to the region, this emphasis on design needs to be complemented by an equal emphasis on implementation arrangements and monitoring. This calls for better documentation of processes (e.g., the rationale for the choice of the targeting tool and any subsequent changes) and improved monitoring of the targeting tool. It is also critically important to build technical and administrative capacity locally and within government systems at the design, implementation and monitoring and evaluation stage. This could be through initial exposure to international good practices and subsequent technical collaboration on the design, implementation and monitoring and evaluation of targeting tools. The extent of the impact on building capacity would depend on the mode of engagement and collaboration during operational work. Even without direct collaboration, particular approaches can serve as a powerful demonstration effect. International evidence suggests that a large fraction of the observed differences in targeting effectiveness across systems and programs can be attributed to factors related to implementation and monitoring. This implies that investments aimed at correcting resource, capacity and logistic limitations in government systems could go a long way in improving targeting outcomes in the region. There is also potential for greater learning and harmonization across projects, with a view to developing more integrated targeting systems that would minimize fragmentation and maximize targeting effectiveness. All too often the focus of experimentation, systematic improvement and documentation is the project design, with inadequate attention given to the 8 targeting tool. There is a need for renewed emphasis on targeting in World Bank projects. This could take the form of greater rigor during design of the targeting tool (e.g., through more experimentation), improved documentation and systematic evaluation of the targeting tool. An effort by World Bank teams to share lessons learned and to achieve some level of harmonization in targeting approaches could have high pay-offs in terms of increased savings and overall effectiveness. The recent experience of Pakistan demonstrates that this is indeed possible. In this case, different project teams have managed to combine the work that had been done by each team independently (using entirely different targeting methods) in the form of a single national targeting instrument to be used across projects and government programs. There is a case for supporting governments in developing more integrated targeting systems that would minimize fragmentation and maximize targeting effectiveness. This does not necessarily imply the imposition of a single targeting method for all programs and projects. A single targeting method could be used across the majority programs, with room for program-specific targeting tools depending on program design and social context. It would be important to ascertain, through rigorous experimentation and evaluation, the most appropriate targeting method as the backbone of the integrated targeting system given the specific country context. If a convergence of methods is not possible in a given country context, a viable strategy would be to have the other elements of an integrated system in place – i.e., clearly defined processes for data collection (whether through census operations or community resources) and systems for household registry, beneficiary database management, monitoring, verification, grievance redressal and fraud control. These practices would not only strengthen the quality of the technical advice on targeting provided by the World Bank in the context of operational work, but it would also create a more solid platform for building on lessons learned in the context of specific projects to foster the broader policy dialogue on the architecture of targeting systems in the region. 9 A. INTRODUCTION A.1. Background and context South Asian countries have experienced significant economic growth and, consequently, a substantial reduction in poverty over the past few years. Since 1996, average annual GDP growth in Bangladesh, Bhutan, India, Maldives, Nepal and Sri Lanka has exceeded 5 percent; Pakistan has been growing at this rate since 2000; and Afghanistan has exhibited double-digit growth albeit starting from a very low base. Economic growth has contributed to poverty reduction. During the 1990s, poverty rates in India, Bangladesh and Nepal declined by 7, 9 and 11 percentage points respectively; while poverty in Pakistan and Sri Lanka fell by about 5 and 3 percentage points respectively during the first half of this decade. And the 2000s have continued to be witness to further reductions in poverty (see Table 1). Table 1: Poverty Indicators for South Asia Country Poverty HCR using HCR using HCR using GDP growth rate (2007)a National Poverty Lines $1.08/ day in $1.25/ day in (year) 2005 PPP 2005 PPP Afghanistan 45 (2007) - - - Bangladesh 40 (2005) 39 (2005) 56 (2000) 6.4 Bhutan 36 (2004) 20 (2003) 26 (2003) - India (rural) 28 (2004) 31 (2004) 44 (2004) 9.0 India (urban) 26 (2004) 28 (2004) 36 (2004) Maldives 16 (2007) - - - Nepal 31 (2004) 47 (2003) 55 (2003) 2.6 Pakistan 33 (1999) 13 (2004) 23 (2004) 6.0 Sri Lanka 23 (2002) 8 (2002) 14 (2002) 6.8 Notes: a\ Growth rate calculated as percentage change in GDP; GDP measured in constant 2000 US Dollars. Afghanistan, Bhutan and Maldives were not estimated due to data limitations. Source: Poverty incidence using national poverty lines - World Development Indicators, RGoB (2004) for Bhutan, GoM (2007) for Maldives. Poverty incidence using $1.08 and $1.25 per day poverty lines - Staff calculations using PovCal; GDP growth rate using Global Economic Prospects 2009. Accompanying South Asia‘s economic growth and poverty reduction, however, there has been an increase in inequality. This increase reflects rising differences across both regions and households within regions. The gap among regions, sectors, and people can be bridged with coherent economic and labor policies as well as policies that improve service delivery in health, education, and infrastructure, and protect the vulnerable, including in lagging regions. Increasing food and energy prices across the world and the job losses in the wake of the recent food, fuel and financial crisess have significantly eroded purchasing power among vulnerable and poor households, threatening to reverse a large portion of the poverty gains made during the last few years and putting additional pressure on governments in the region and elsewhere to provide adequate income and other support to affected households. Redistributive packages to protect the vulnerable exist in all South Asian countries, but they often suffer from low coverage rates, under-coverage of the poor and leakage of program benefits to the non-poor which significantly undermine their impact on the poor. In this context policy makers may 10 find it desirable to concentrate program resources on the poor and vulnerable, particularly as part of any reform package aimed at promoting more inclusive growth. Ultimately economic reforms may be more palatable when accompanied by safety net systems that truly protect losers and those in need. Well administered and effectively targeted programs assume an even greater significance during times of crisis when the quickest response often is to scale up existing programs. A.2. Why target? We understand by targeting the design and implementation of mechanisms aimed at identifying a particular group of communities, households or individuals to whom a program expects to cater. In this sense targeting is a tool policy makers sometimes employ to make a program more efficient by increasing its direct impact on the poor and vulnerable. Concentrating resources on these groups can increase the benefits that they can achieve within the existing budget envelope, an attractive prospect for countries with tight fiscal situations, or can achieve a given impact at the lowest cost. In this context the theoretical gains from targeting appear to be large. For instance, a recent review of international experience finds that the median targeted program provides approximately 25% more resources to the poor relative to untargeted random allocations (Coady, Grosh and Hoddinott 2004). In practice, these gains are never fully realized because there are several costs associated with targeting. These costs include administrative costs borne by the program, transaction and social costs borne by program beneficiaries, incentive costs that may affect overall benefits to society, and political costs that may affect support for the program. Also, targeting does not always work – 25% of the programs studied were regressive. For these reasons, the decision to target should be preceded by an informed debate on the desirability and feasibility of targeting, i.e., an assessment of whether the benefits outweigh the costs. The administrative, social and political costs associated with targeting need to be factored into the decision to target or not to target. For instance, the argument of gains from targeting in terms of concentrating limited resources to the poor is valid if the overall budget is fixed. On the other hand, if the budget as well as the eligibility rules were joint policy choices, an overly narrow targeting of transfers may well reduce the political support for the program and consequently, the size of the transfer to the poor (see for example Gelbach and Pritchett 1997). In addition, a poor track record in concentrating resources on the poor and vulnerable could potentially undermine the credibility and broad-based support for a targeted transfer. More fundamentally, fine-tuned targeting design may in itself be inherently challenging given the pattern of poverty and vulnerability in a particular country context. For instance, a large share of the population in India clusters around the poverty line, and thus the observable differences between ―poor‖ and ―non-poor‖ households are often not easy to measure and may be small in absolute terms. In addition, the literature on poverty dynamics and vulnerability in India suggests that variability of incomes and coping with recurring shocks are a common feature. In other words, regardless of the current poverty status of a household, households may fall into poverty after exposure to large, severe and/or frequent shocks, especially when clustered close to the poverty line (World Bank 2010a, b). This in turn raises a fundamental question of whether targeting is feasible, and indeed desirable, in such a context. This is the basis of a heated debate currently underway in India and other countries in the region on targeted versus universal provision of programs and services. This debate has its 11 roots in the growth of a rights-based approach to development in the country – the notion of targeted provision contradicts this framework if access to food, education, employment and health are seen as universal rights. Though no conclusive policy direction has emerged so far, there is a distinct move towards the formulation of social programs in a rights-based framework – notably the National Rural Employment Guarantee Act (a public works program that is legally guarantees the right to work) and the ongoing debate on the National Food Security Act (to guarantee to the right to food). Depending on the social, political and administrative imperatives, targeting may or may be the appropriate policy choice. Available evidence suggests that programs can focus resources on the poor to a moderate or high degree without incurring unacceptably high errors of exclusion and administrative, private or incentive costs, although not all do so. Even in the context of universal provision, an element of targeting can emerge in the form of affirmative action for specific groups that have traditionally been excluded or are particularly disadvantaged. The discussion in this policy note is limited to the targeting effectiveness of government programs and World Bank operational work, and not to the broader question of targeting versus universalisation. A.3. How is targeting done?1 A number of different targeting methods are available for directing resources to a particular group (see Box 1 for a brief description of the most common targeting methods; see Section B3 for a fuller discussion). A more detailed discussion, together with references on specific methods, can be found in Grosh, del Ninno, Tesliuc and Ouerghi (2008). 1 This section is an extract from Chapter 4 in Grosh, del Ninno, Tesliuc and Ouerghi (2008). 12 Box 1: Main types of targeting methods a. Means tests A means test seeks to collect (nearly) complete information on households‘ income and/or wealth. Sometimes this information is fully/partially verified against independent sources, other times verification is not possible and program intake workers simply record what the applicant says. Verified means tests are considered to be the gold standard of targeting but are usually difficult to implement in developing countries. b. Proxy means tests Proxy means tests generate a score for each applicant household based on fairly easy-to-observe household characteristics believed to be correlated with the household welfare status. The information provided by program applicants is usually partially verified either by program officials. Eligibility is determined by comparing the household score against a predetermined cutoff. c. Community-based targeting Community-based targeting uses a group of community members or leaders whose principal functions in the community are not related to the program to decide who in the community should benefit. d. Geographic targeting With geographic targeting, location determines eligibility for benefits: people who live in the designated areas are eligible and those who live elsewhere are not. Few programs target only on the basis of geography, but many programs combined this criterion with other targeting methods. e. Demographic/categorical targeting The usual and simple forms of demographic targeting are based on age (e.g. children and elderly), or on some other individual characteristic commonly perceived to be associated with a higher likelihood of being poor, vulnerable or socially excluded (e.g. orphans, widows, disabled, certain cast groups in the case of India). f. Self-targeting Self-targeted programs are technically open to everyone, but are designed in such a way that the take-up is expected to be much higher among the poor than the non-poor or the level of benefits is to be higher among the poor. A common example is the use of low wages in public work programs to induce participation primarily among the poor. Source: Grosh et al. (2008) Chapter 4. Some demand an assessment of eligibility for each applicant (individual or household). Others grant eligibility to broad categories of people, for instance, all those residing in certain areas or all those of certain age. Finally a third type of instrument is designed to discourage the non-needy from entering the program, but do not actually prohibit them from doing so. A few targeting methods and types of programs go hand-in-hand, but most normally several methods can be used to target a particular type of program. In addition it is common for a single program to use a number of methods simultaneously, which tends to yield better targeting outcomes than using any single approach. There is no targeting method that clearly dominates all other in terms of targeting effectiveness. A review of 122 targeted social programs by Coady, Grosh and Hoddinott (2004) suggests that 80 percent of the variability in targeting performance was due to differences within targeting methods and only 20 percent was associated with differences across methods. Finally, international evidence on targeted programs and targeting methods reveal marked differences between regions. There seems to be a weak mapping between the nature of interventions and targeting methods used across regions. Thus, cash transfer programs dominate in Eastern Europe and Central Asia (ECA) and Latin America (LAC), universal food subsidies in Middle East and North Africa (MENA) and a mix of cash and food transfers in South Asia (SAR). Mirroring this 13 mapping to some extent, the use of means or proxy means tests is common in ECA and LAC, while self-targeting for food subsidies is common in MENA (Coady, Grosh and Hoddinott 2004). In SAR, in contrast, a wide range of methods is used for targeting resources, ranging from geographic, self- selection (for workfare programs), categorical and community-based; but proxy means testing is relatively uncommon in the region. Box 2: National targeting systems There are several elements of a national targeting system. Some of the key considerations in defining each step are outlined below: (a) Clearly defined eligibility rules: This is essentially a policy decision on who should be made eligible for assistance and would depend on the underlying patterns of and attitudes towards poverty and vulnerability. Considerations of budgetary implications as well as political and administrative feasibility would also be paramount. The need for policy coordination across programs would also play a role. (b) Clearly defined institutional roles and responsibilities: In terms of the targeting tool, reliance on standard national criteria or thresholds in large federal countries with wide disparities across states/provinces or between rural and urban areas could lead to inaccurate targeting. At the implementation stage, clarifying roles and responsibilities with respect to database management across administrative levels is important for maintenance, verification and dynamic updating of household information. (c) Appropriate mix of targeting tools: International evidence suggests that no single targeting method would adequately meet all requirements. The question then is identifying an appropriate and cost-effective mix of methods that minimizes duplication. Typically, this takes the form of a single method for the majority of programs but with other targeting methods as required by the program design and/or social context. The key principles include (i) maximizing coverage of the poor; (ii) minimizing leakage to the non-poor; (iii) cost efficiency; and (iv) transparency. (d) Systems for household registration: Depending on the existing administrative capacity, systems for data collection (on-demand or census or community meetings) and household registration need to be developed that balance the need for accurate, verifiable data with low administrative and transaction costs. These are often combined with outreach and communication efforts to minimize exclusion. (e) Beneficiary database management: By comparing household characteristics in the household registry against eligibility rules, program-specific beneficiary lists can be generated. In the case of community-based targeting tools, these lists are generated at the local level in community meeting. (f) Systems for monitoring and fraud control, grievance redressal and recertification: A number of mechanisms have been developed for verification and monitoring of information, including automated checks across linked administrative databases, random interviews, citizen oversight, and audits. Recertification and updating are important for fraud control and to reflect changes in household status. Source: Castenada and Lindert (2005), Grosh et al (2008). Another key source of differences across regions is the existence of an integrated targeting system: most countries in Eastern Europe and Central Asia (ECA) and Latin America (LAC) have one, almost all South Asian countries do not (though India and now Pakistan have some elements of one). This reflects to some extent differences in income levels, institutions and administrative capacity. An integrated national targeting architecture typically comprises includes the development of a backbone targeting method to be used for the majority of programs (but with room for program-specific targeting tools depending on the program design, e.g., self-targeting for public works) with clearly defined processes for data collection and household registry, eligibility determination and beneficiary database management, clear institutional roles and responsibilities and systems for monitoring, verification, grievance redressal and fraud control (Castenada and Lindert 14 2005). However, variations on this theme are possible – at least with respect to the backbone targeting method. Thus, this does not necessarily imply the imposition of a single targeting method for all programs and projects. For instance, even in India where a core administrative targeting method exists for targeting fiscal allocations across space and for many government programs, other targeting methods co-exist where program design and/or social context makes it relevant. Thus, public works programs continue to rely on self-targeting, while other programs combine one or more method – e.g., social pensions to the destitute elderly combine categorical with BPL targeting. Nor is it necessary that proxy means testing be the backbone targeting method. Depending on administrative capacity, poverty dynamics, local institutions and social context, other targeting methods, including community-based targeting may well be more appropriate. It would be important to ascertain, through rigorous experimentation and evaluation, the most appropriate targeting method for the backbone of the integrated targeting system given the specific country context. If a convergence of methods is not possible in a given country context, a viable strategy would be to have, at a bare minimum, the other elements of an integrated system in place as noted above. The argument here is for developing an integrated national targeting architecture that minimizes fragmentation and maximizes effectiveness. The focus on targeting within programs or projects alone overwhelms any discussion of improving targeting effectiveness overall. Some of the considerations in designing an integrated targeting system are outlined briefly in Box 2 above. A.4. Objectives and scope of the study The main objectives of this note are to systematically and critically take stock of regional experiences in the area of targeting, with a special focus on government systems and on work on targeting issues under World Bank activities in South Asia. In particular, this note aims to do the following:  take stock of targeting systems and practices in government programs in the region;  critically review targeting systems and practices in the World Bank‘s operational work and the impact on government systems; and  on the basis of this diagnosis, extract lessons that can be used to inform the way forward. The rest of this note is structured as follows. Section B presents a description of targeted programs in the region and discusses government targeting systems and their performance. This analysis of government systems has drawn upon and been limited to existing secondary sources (i.e. no original analytical work has been undertaken) and has focused on: (i) objective and nature of targeted programs; (ii) structure of regional targeting systems; and (iii) targeting effectiveness. We expect this note and the accompanying materials to serve as a resource for those interested in or planning some work on targeting related work in the region. We hope the evidence presented here will have important implications for and can help improve the design and implementation of targeting systems in the region. Section C takes stock of regional work by the World Bank in the area of targeting with a special focus on operations. Information on World Bank-led work has been collected following a multi- pronged approach that includes an extensive review of relevant project and other documentation, development of a standardized questionnaire to be completed by Task Team Leaders (TTLs) involved in operations including targeting work, and follow-up discussions with TTLs. The questionnaire, fielded in 2008, includes questions on: (i) choice and design of targeting tool; (ii) development and implementation arrangements; (iii) evaluation and impact of targeting-related work. We have then developed a framework to analyze and systematize the information gathered 15 through these exercises and to extract relevant lessons for World Bank work in this area. This section also presents some insights from several ongoing innovative pilot activities in order to explore the challenges of targeting specifically in the context of South Asia. These activities include targeting pilots in Bangladesh, India and Pakistan whose focus is to comparatively evaluate various targeting approaches (ranging from community wealth ranking to proxy-means testing to a combination of methods). Finally section D concludes with some observations and recommendations as to how to expand and deepen the relevance and impact of targeting operational work by the World Bank in South Asia. Our analysis suggests that government targeting systems in the region are fragmented and targeting outcomes are poor, with a few exceptions. In this respect, South Asian countries differ markedly from many countries in Eastern Europe, Central Asia and Latin America, where national targeting systems are common. In South Asia, this absence of an integrated targeting architecture combined with low administrative and monitoring capacity within the public sector implies a potentially large role of the World Bank to support countries in improving and strengthening existing systems at both the design and implementation stages. A review of targeting-related work under 20 projects reveals that the World Bank has played an important role in terms of innovation and knowledge sharing in the region. However, targeting operational work in the region has sometimes lacked the necessary rigor and has often failed to translate into tangible improvements in overall country systems or in-country technical and implementation capacity. As a consequence, the impact of this work on the policy debate on targeting, the performance of government systems and, ultimately, on targeting effectiveness has been limited, with a few exceptions. Going forward, there remains significant potential to deepen the relevance and impact of targeting operational work by the World Bank in South Asia. An effort by World Bank teams to share lessons learned and to achieve some level of harmonization in targeting approaches could have high pay- offs in terms of increased savings and overall effectiveness. Technical support to correct resource, capacity and logistic limitations in government systems could also go a long way in improving targeting outcomes in the region. These practices would not only strengthen the quality of the technical advice on targeting provided by the World Bank in the context of operational work, but it would also create a more solid platform for building on lessons learned in the context of specific projects to foster the broader policy dialogue on the architecture of targeting systems in the region. 16 B. OVERVIEW OF TARGETED GOVERNMENT PROGRAMS AND TARGETING PRACTICES IN SOUTH ASIA The decision to target and to develop mechanisms to do so is rarely taken in a void but rather appears to be intimately related to the government‘s desire or need to broadly channel resources and target specific programs to particular population groups. In this sense targeting systems, their design, administration and effectiveness cannot be examined independently from the programs to be targeted. All governments in the region support pro-poor or targeted interventions within their budgets, but there are significant differences across countries in terms of the nature of programs and benefits being targeted and, especially, in terms of the way targeting systems are designed and implemented. For instance, distribution of selected subsidized food items is targeted to the poor in India, but universal in Pakistan; social pensions for the elderly and other vulnerable groups are targeted to those most in need in India but granted universally in Sri Lanka and so on. Similarly administrative arrangements for targeting systems vary widely across countries. In India most pro-poor federal level programs (i.e. centrally sponsored schemes) and a large majority of pro-poor state level programs are targeted using a common proxy-means indicator (BPL). In contrast, the choice of targeting strategies appears to be program specific in other countries in the region. Having said this, most pro-poor or targeted programs tend to be safety net or social security programs—i.e., programs that distribute cash or in-kind benefits (or, more generally, provide support) to needy and vulnerable households. This contrasts with other social interventions where universal coverage and access are often a priority2. Given this, for the purpose of this section we will mainly look at targeting systems through the lens of safety net programs and the subset of targeted social interventions, where relevant. Our expectation is that this approach will help to keep the discussion focused while still allowing us to make broader observations on existing targeting systems that are pertinent beyond the realm of safety nets. The discussion draws exclusively from secondary sources, ranging from World Bank reports, to academic papers, to third-party program evaluations.3 An annotated bibliography of all sources consulted for this purpose is provided in Annex 1. In some cases existing information was patchy or insufficient to build a full assessment of the effectiveness of targeting systems within a specific program or country, while in others we found a wealth of analysis and data. These limitations partly reflect varying degrees of interest in and level of ―sophistication‖ of the debate around issues of targeting and, more broadly, differences in the desire or need to effectively reach the poor across countries and have undoubtedly shaped the scope of the discussion presented below. The rest of this section is structured as follows. We first present a brief overview of targeted programs in the region, paying particular attention to program features that may have a special bearing on the issue of targeting. This overview is intended to provide the background context for 2 There are several programs in the education and health sectors where targeting plays a role. This is the case, for instance, of school stipend programs or free-access to public health care for the poor. As discussed in the text we include these types of programs in the analysis to the extent that available data permits. 3 A comprehensive, comparative review of regional targeting systems, including primary data analysis to evaluate targeting effectiveness could well constitute a follow up activity to this note. 17 the more detailed discussion on targeting systems and practices that follows. In this discussion we focus both on design and implementation issues and try to assess targeting effectiveness, understood as the relative success in identifying and reaching the intended beneficiary population at a reasonable cost. The discussion below also highlights how South Asia differs from other regions, notably East Europe and Central Asia and Latin America with respect to targeting systems. This reflects, to some extent, differences in the nature of targeted programs, but more importantly, in income levels, institutions and administrative capacity. As mentioned in the previous section, most countries in Eastern Europe, Central Asia and Latin America are characterized by national targeting systems that rely on means or proxy means testing of household circumstances (Coady, Grosh and Hoddinott 2004, Castañeda and Lindert 2005). In sharp contrast, South Asian countries use a wide range of methods for targeting pro-poor interventions, ranging from geographic, self-selection (for workfare programs), categorical and community-based; proxy means testing is relatively new in the region. B.1 Brief overview of targeted program types This section describes the types of targeted program common in the region. The focus is primarily on safety net programs, with some references to a subset of other targeted social interventions (including education, health, nutrition and insurance). There are two main reasons why we believe focusing on safety net programs as an introductory platform to the issue of targeting is a good strategy. First, although not all safety net programs are targeted and not all targeted interventions are safety net programs, it is fair to say that these programs account for most of the (explicitly) pro-poor public spending. Second, safety net programs exhibit a large variety of features that can be successfully exploited to illustrate the factors underlying the choice, design, implementation and monitoring of targeting systems, as well as determining their overall effectiveness. These features include (i) the need to reach a specific beneficiary population, (ii) significant variation in the nature and frequency of benefits, and (iii) large diversity in implementation and monitoring arrangements. The first one often motivates the need to target and determines to some extent the choice of targeting tool, while (ii) and (iii), although not apparently related to the design and implementation of the targeting system, can bear significant impact on implementation and ultimately targeting effectiveness, as discussed below. In order to widen the scope of the discussion as much as possible, we have adopted a broad definition of safety nets to include all programs that provide income and other support to poor and vulnerable communities, households or individuals and assist them in managing income and other risks that could potentially render them poor(er) if unattended to.4 Within this definition we have then focused on the following types of programs: a. Cash transfer programs These are programs that pay unconditional or conditional monetary benefits to designated beneficiaries with the objective of permanently or temporarily (i) providing additional income, and (ii) promoting behavioral changes (by associating payments with access to or use of social and other services). Examples of such programs include subsistence allowances (e.g. Samurdhi in Sri Lanka 4Note that this broad definition includes types of programs and interventions that are not traditionally considered safety nets (e.g., ex-ante risk mitigation through insurance is usually classified under social security). This broad definition has been adopted purely for the purposes of exposition of targeting methods in government programs and World Bank operations. 18 and Food Support program in Pakistan), social pensions (e.g. old age allowances in most countries in the region), school stipends (e.g. stipends for schooling-age girls in Pakistan and Bangladesh) and post disaster relief transfer programs (e.g. post-tsunami relief efforts). Together with in-kind transfers (see below), these programs take the lion‘s share of government resources devoted to safety nets in the region. b. In-kind transfer programs These are programs that distribute food and other goods to designated beneficiaries with the objective of permanently or temporarily (i) providing non-monetary support5 and (ii) promoting behavioral changes6. Examples of such programs include subsidized food distribution (e.g. Public Distribution System in India), school feeding programs (e.g. Mid-Day Meals program in India and Food for Education program in Nepal), distribution of non-food items (e.g. disability assistance program in Maldives) and post disaster relief programs (e.g. Vulnerable Group Feeding program in Bangladesh). In-kind transfers remain a fairly popular option in South Asia relative to other regions. c. Workfare programs These are programs that distribute cash and/or food in exchange for labor to designated beneficiaries with the objective of (i) providing income or other support, (ii) generating employment and (iii) developing small-scale infrastructure at the local level. Examples of such programs include cash for work programs (e.g. National Rural Employment Guarantee Scheme in India) and food- for-work programs (e.g. Food-for-Work program in Nepal). All countries in the region have or have recently experimented with workfare programs. d. Social care services These are programs that aim to serve vulnerable and other groups with the objective of (i) facilitating access to and promoting use of social services and (ii) providing specialized care. Examples of such programs include promotion of access to medical facilities and services (e.g. Lady Health Workers program in Pakistan), and provision of specialized services for vulnerable groups (e.g. Integrated Child Development Services in India and disability assistance program in Maldives). e. Micro-credit and insurance These are programs that seek to mobilize community-level resources with the objective of (i) mitigating the impact of shocks on household income, (ii) promoting access to economic opportunities and livelihoods and (iii) providing mechanisms for consumption-smoothing. Examples of such programs include the SGSY (Sampoorna Grameen Swarozgari Yojana) rural credit scheme in India and the Grameen Bank in Bangladesh. These programs, often administered by community-level organizations and linked to the formal banking system, have become a prominent feature in the rural safety net landscape across South Asia. Some NGOs, trade unions and other community-based organizations operate micro-insurance programs. A recent large-scale government program is the RSBY (Rashtriya Swastha Bima Yojna) offering subsidized health insurance to poor households in India. A detailed list of the specific programs covered and their main features is provided in Table A.1 in Annex 1. Although this list excludes programs that are universal by design, such as blanket 5 Particularly in the face of chronic poverty and food insecurity or in the absence of thick enough local markets—e.g. in the aftermath of a disaster. 6 By associating the distribution of in-kind goods to access to social and other services (e.g., school feeding programs). 19 interventions in the health and education sectors, or programs with a pro-poor impact but broader targeting approaches, such as slum upgrading or infrastructure programs, we strongly believe that the observations and conclusions on targeting systems presented in the next section are entirely applicable to these types of programs and sectors were they to be transformed into targeted interventions. B.2 Overview of regional targeting systems The need and desire to target budgetary resources to poor and vulnerable groups, households and individuals varies across countries in the region, as does the political feasibility of targeted versus universal transfers. Consequently, the level of development and sophistication of existing targeting systems also exhibits significant variation, as does the degree of targeting effectiveness achieved under different programs. This section presents an overview of regional targeting systems. We do so by providing a general systematizing framework and using country-specific examples for illustrative purposes. The discussion draws from existing work and is intended for a general audience, while the interested reader can find more detailed information on specific countries and programs in Annex 1. Use of specific targeting approaches All the main targeting approaches discussed in Box 1 are in use in the region, often in combination (Table 2)7. This results partly from the program mix within and across countries but also reflects policy choices since, as we argued earlier, various approaches can be used to target a specific type of program. Workfare programs across the region (and elsewhere) tend to rely on self-targeting mechanisms (e.g. National Rural Employment Guarantee scheme in India), combined in some cases with geographic targeting (e.g. Food-for-Work program in Nepal). The main exception is the Food-for-Work program in Bangladesh, which uses categorical targeting to determine program eligibility. For most other programs, a variety of targeting methods are used. Cash and in-kind transfers are mostly targeted using some form of proxy-means testing or categorical targeting, sometimes in combination with geographic targeting as a first step for selection of program areas. Examples of such arrangements are the Public Distribution System in India (PMT targeting), and the Absolute Poverty Scheme in Maldives (categorical targeting). There are, however, exceptions, including the Vulnerable Development Group (VDG) program in Bangladesh or the Relief for Displaced Persons in Sri Lanka, which apply a combination of means testing and categorical or geographical targeting to identify beneficiaries, and the Samurdhi program in Sri Lanka, which uses community-based targeting. Social care services are also targeted using a variety of approaches, ranging from categorical (e.g. ICDS in India) to geographical targeting (e.g. Lady Health Workers program in Pakistan). Finally micro-credit and micro-insurance programs use a mix of geographic, community- based targeting and self-targeting (through self-selection into self-help or savings groups). The exceptions are SGSY (rural credit) and RSBY (health insurance) programs in India that rely on proxy means testing. 7The information presented in Table 2 corresponds to the programs reviewed for the purpose of this note and may hence not fully portray the complete mapping of programs into targeting systems for all countries (see Table A1 in Annex 1 for a more detailed list). 20 Across the region, India relies heavily on proxy-means testing for targeting of various government programs as a result of the adoption of the BPL indicator as the main national targeting tool in rural areas (see Box 3 for a detailed description of the history and methodology underpinning the BPL indicator)8. In contrast a variety of targeting approaches coexist in Bangladesh, Maldives, Nepal, Pakistan, and Sri Lanka. Finally, as discussed below, very few programs are targeted in Afghanistan and Bhutan.9 Table 2: Targeting methods by type of program and country Type of program Cash In-kind Workfare Social care Micro-credit transfers transfers services and Targeting approach insurance Means test SL BD, MA, SL BD Proxy means test IN, PK0 IN IN Community-based targeting SL, NP PK Geographic targeting NP, MA1, BD, PK, NP NP IN, NP BD PK1, SL1 Demographic or categorical BD, IN, MA, BD, IN, PK, BD NP, PK targeting PK, SL, NP SL Self-targeting IN, NP IN2, PK2 BD3, PK3 ,NP2 Notes: Country acronyms are as follows AF (Afghanistan), BD (Bangladesh), BU (Bhutan), IN (India), MA (Maldives), NP (Nepal), PK (Pakistan) and SL (Sri Lanka). 0 Currently in pilot stage. 1 Post-disaster relief. 2 Universal access with self-selection on program take-up. 3Geographical targeting in coverage with self-selection into self-help groups. Architecture of targeting systems Despite significant variation in targeting practices and outcomes across countries and programs, here exist certain common elements that characterize the architecture of targeting systems in the region:  Program-specific targeting instead of integrated targeting systems, leading to duplication;  Targeting implementation by lower administrative levels combined with inadequate capacity and monitoring systems, leading to variability in targeting effectiveness across localities;  Strong involvement of donors and non-government actors in choice, design and implementation of targeting tools and systems;  Inadequate systems to target benefits in the aftermath of natural disasters. First, with the exception of India (see Box 3) and Pakistan (currently piloting the proxy-means test based poverty score card for adoption as the national targeting system), all countries in South Asia lack an integrated targeting system so that targeting approaches are generally program specific. This translates into high levels of fragmentation and duplication, and excessive administrative costs. It also means that often programs with similar target beneficiary populations end up serving somewhat different groups even within a particular area, undermining the credibility of the targeting process and potentially the overall program. For instance, until 2009 both the Zakat and the Food Support 8 The process for the identification of poor households is much more fragmented and less well documented than that in rural areas. Various departments and programs follow different methodologies and there is often little coordination or overlap. Kerala is a notable exception (see Box 3). 9 Bhutan does not have any safety net programs and households rely on informal networks as coping mechanisms. The government does, however, provide free education and health services (World Bank 2008). 21 programs in Pakistan provided income support to the poor across the country, with the poor being identified by program-specific local committees according to broadly defined criteria. Lack of coordination in the formation and operation of local committees effectively translated into higher aggregate administrative costs, while non-standardization of eligibility criteria across programs generated significant differences between their respective beneficiary populations, which was often interpreted as proof of political capture and cronyism among local program administrators. Similar examples can be found in most other countries. Second, while broad decisions on the choice and design of targeting tools are usually made at the federal/central levels, actual implementation and monitoring are handled at the state/provincial and, most frequently, local levels. In the absence of strong administrative and monitoring systems capable of feeding information back and forth across various levels of government, this has often translated into a high degree of discretion being exercised at the local level in the implementation of the targeting tool.10 This can have potentially negative effects on the integrity and overall effectiveness of the proposed targeting approach. For example, the basic design of the rural BPL indicator in India is the responsibility of the Central Planning Commission, while collection of the information to be used to calculate household-level welfare scores is done at the local level with little standardization of processes, little or no coordination with and supervision and quality control from higher levels of administration. Similarly day-to-day administration of the rural BPL database, including updates, is handled independently by states and local governments. As a consequence the accuracy and reliability of the rural BPL lists varies widely across the country and even within states. A similar problem occurs in urban areas, where no unique BPL list exists, but rather different lists are used for different programs with no clear systems for unifying/updating over time or across administrative levels. 10We use the term discretion in this instance to refer to fragmentation and lack of standardization in the implementation of the targeting tool. This is in contrast with the use of local knowledge by local government officials and administrators to improve targeting outcomes. 22 Box 3: Identifying the poor using an integrated targeting system in India The core administrative mechanism used for identifying poor households is the Below Poverty Line (BPL) method. The design and process is clearly documented for rural areas; the process for urban areas is less clear. The discussion here focuses on the rural BPL census, conducted every five years or so by the nodal ministry. The first such census in 1992 used self-reported household incomes to identify BPL households. However, given the difficulties of measuring income, the self-reported income approach was abandoned in the 1997 BPL census. The 1997 exercise tried to first screen out the ―visibly non-poor‖ on the basis of five asset/income related questions and then further narrow down the eligible households to those whose annual household expenditure was below the poverty line. This approach too suffered from certain shortcomings, including very stringent exclusion criteria, the non-availability of official poverty lines for all states/ Union Territories and the uniformity of criteria across states. For the 2002 census, the Central Planning Commission decided to move to a rudimentary form of a proxy means test to identify BPL households. All households are given a score on 13 indicators. However, these are defined by a committee as proxy indicators of poverty but are not underpinned through an analysis of household survey data as in traditional PMTs. The score would be aggregated and a cut-off assigned that would determine which families would be classified as BPL. In principle, these cut-off could vary across villages, but in practice, states have opted for uniform intra-state cut-off thresholds. Analytical work by the World Bank and others has criticized this methodology on various grounds, including poor choice of indicators, application equal weights for all (disparate) indicators and the uniformity of indicators across states. Technical support in the context of operational work has also highlighted the poor quality of database management. The Central Planning Commission is currently engaged in the redesign of the BPL method. The proposal under discussion proposes automatic exclusion of visibly non-poor households and the automatic inclusion of the most vulnerable households, with a survey and scoring only of the remaining households. The remaining households would be ranked on the basis of social group, vulnerability and occupational status. The proposed method also overlays geographic targeting criteria by specifying district, block and village-level quotas with respect to the proportion of BPL households. In districts that have a very high proportion of BPL households (over 80%), all households (except the visibly non-poor) would be considered to be poor and automatically included in the BPL list. The threshold for identifying the poor could vary across states and for programs, implying no single BPL list (even for a state), instead, multiple beneficiary lists using the same ordinal ranking of households. This exercise is proposed to be conducted once in 10 years with a review of lists for additions and deletions every two years in the interim period. In a significant departure from implementation of previous censuses, the proposal is to employ external (outside the community) enumerators to conduct the BPL census. These changes, if accepted, should be implemented towards the end of 2009. However, there remains some degree of fragmentation as two separate BPL databases are maintained by separate departments in some states – one, the core administrative BPL list as described above; and two, BPL ration cards for the Public Distribution System for subsidized food. Several states have converged these two lists into the core BPL list; in others, the political economy considerations may make convergence challenging. No comparable central guidelines for a census of poor households exists for urban areas. Typically, the identification of poor households is carried out separately by different departments using different methodologies and there is little systematic information on this process. Kerala is a notable exception where a well-documented process is followed in both urban and rural areas. The BPL list so produced is used for all programs. Source: Jalan and Murgai (2008), Sundaram (2003), Mehrotra and Mander (2009), World Bank (2009). Third, the absence of an integrated targeting architecture, combined with low administrative and monitoring capacity within the public sector, has opened up the space for strong involvement of non-government actors, particularly donor agencies, in the design, implementation and monitoring 23 of various targeting tools. This involvement has taken many forms, ranging from technical assistance to strengthen and reform existing government systems to the creation and implementation of de facto parallel systems for the purpose of targeting donor-financed programs. The cases of Sri Lanka and Pakistan are good illustrations of the first scenario. In Sri Lanka, the World Bank has provided technical support to the government on targeting of the Samurdhi program, the largest safety net program in the country. Initially, this involved the development of a new proxy-means test. However, this approach was ultimately abandoned for political and other reasons and Samurdhi now relies on a community-based targeting system (similar to that used by the World Bank-funded Gemi Diriya project). Currently, there is a joint study between the World Bank and Samurdhi to assess how the community-based targeting system can play a more effective role in identifying the poorest population in times of disasters. In Pakistan, different World Bank teams have provided coordinated support to government in order to develop a new targeting tool (a proxy-means test based poverty score card) that is currently being piloted with the intention to be adopted as the national targeting system, depending on the findings of the pilot phase. In contrast, in Nepal, the UNICEF has adopted the Disadvantage Group Mapping (DAG) methodology to target their interventions and actively promoted the adoption of similar approaches by other donors working in the country (UNICEF forthcoming). World Bank projects too have developed different targeting methods under different projects, including a PMT and a community- based targeting system. Given that most (safety net) interventions in Nepal are funded by donors this has led to the development of a parallel, and fragmented, system for targeting of poverty alleviation programs independent of government structures. Finally, most countries in the South Asia region have been exposed to either conflict or natural disasters in the last few years. In conflict situations, targeting becomes extremely difficult due to capacity and logistic constraints, and may not even be desirable or politically feasible if large fractions of the population are considered poor or vulnerable. Such is, for instance, the case of Afghanistan, where the government is primarily focusing on expansion of universal access to basic services and assistance rather than on granting selective access to these same services to a subgroup of the population. Targeted assistance in the wake of natural disasters also present formidable challenges given the widespread destruction and urgency for action. Community-based targeting is often infeasible in such situations as communities are displaced, fractured and scattered. This limits social mobilization and community participation and makes community consensus on targeting difficult to achieve. In the aftermath of a natural disaster existing targeting systems may be rendered non-operational and new methods to identify those affected by the disaster and in need of support have to be designed and put in place at high speed. In some cases the use of geographic and categorical targeting has proven to be quite effective. For instance, in the case of the 2005 earthquake in Pakistan, no existing targeting systems were in place and new categorical targeting methods had to be developed. This case demonstrates how cash transfers can be effectively targeted through clear eligibility rules, sound data collection and management practices, and by actively engaging with government and other agencies and local communities (see Box 4). Interestingly among some countries in the region frequently and/or regularly affected by natural disasters, such as Bangladesh, the design and implementation of targeting systems and safety net programs already incorporates elements that make them relatively amenable to rapid scaling up following a disaster. In Sri Lanka, there is an on-going joint study between the World Bank and Samurdhi to assess how the community-based targeting system can play a more effective role in identifying the poorest population in times of disasters. 24 Box 4: Targeting cash transfers in the aftermath of disaster: The Pakistan earthquake A cash transfer program targeted to the poor and vulnerable families affected by the earthquake was designed and implemented in response to the earthquake in October 2005 in Pakistan. This experience demonstrates how cash transfers can be effectively targeted in these emergency situations through clear eligibility rules, sound data collection and management practices, and by actively engaging with government and other agencies and local communities. Given the lack of an existing targeting system and challenges with community-based targeting in such situations, an administrative selection of beneficiaries, based on assessments of poverty and vulnerability was chosen as the appropriate means of identifying beneficiaries of the proposed cash transfer. The approach followed involved creating a database simultaneously with the application process. First, sub- district level administrative units were selected on the basis of estimates of the damage suffered. An information campaign was undertaken to draw applicants. Basic information was collected from households by local officials and leaders. These application forms were computerized and checked centrally by the National Database and Registration Authority (NADRA). Second, eligibility criteria for the cash transfers were defined with the help of simulations from this database. At the first stage, all households that had reported housing damage and that had no government employee were deemed to be the ―universe of eligible families‖. Eligibility was then determined by households meeting any one of the following conditions: (i) families headed by currently unmarried women, (ii) families with any disabled member and (iii) families with five or more children, including orphans. These criteria were developed in consultation with the Earthquake Reconstruction and Rehabilitation Authority (ERRA) and were endorsed by local agencies. The entire exercise involved over 3,000 personnel on the ground, demonstrating the magnitude of the disaster as well as the high level of coordination needed in setting up an effective targeting mechanism in a very short period of time. Much of the success is due to the massive efforts to build local capacity for implementation, effective database management and third-party quality control checks. The experience also underlines the importance of community involvement (in data collection, accountability and transparency), flexibility in World Bank processes (to simultaneously allow damage assessment and intervention design) and the need for data systems (during non- emergency times, which can be of immense value post-disaster) for quick response and successful targeting in the wake of natural disasters. Source: Vishwanath and Narayan (2006). In sum, the lack of national targeting systems in most countries in the region has translated into a multiplicity of approaches being used simultaneously across programs, often leading to high levels of fragmentation and duplication, and excessive administrative costs. The question then arises as to the impact of these (and other) factors on targeting effectiveness of both country systems and program- specific approaches. We turn to this issue next. B.3 Targeting effectiveness The idea of targeting effectiveness refers to the relative success of a targeting tool and/or a program in correctly identifying and reaching the intended beneficiary population. Various indicators can be used to measure targeting effectiveness. Data availability and cross-country comparability have led us to focus on the following:  Inclusion errors  Exclusion errors  Benefit incidence  Leakage 25 This is not to suggest that other indicators, such as cost or transparency, are not important, but rather results from the need to work with systematic and representative information. Table 3: Targeting efficiency indicators Inclusion errors Exclusion errors Benefit incidence Leakage •Percentage of •Percentage of target •Share of program •Percentage of program beneficiaries group wrongly benefits accruing to program resources that are wrongly identified as non- various income that are altogether identified as eligible eligible or simply does gorups, ranging from lost or that accrue to for the program not participate in the poorest to richest program beneficiaries program wrongly identified as eligible Each one of these indicators has advantages and disadvantages when used independently so ideally one would like to present systematic evidence on more than one measure when attempting to compare targeting effectives across systems and programs. Unfortunately this is not always possible and has proven to be extremely difficult for the purpose of this review. Existing evidence on targeting effectiveness in South Asia is at best patchy and comes in all forms and shapes with program-specific information being rarely comparable across interventions, let alone countries. There are various reasons for this including (i) very limited information on access to programs in household surveys, which include information on consumption and other social indicators and would hence allow for the identification of poor/non-poor households and individuals among program participants; and (ii) low quality of program administrative data, which could potentially be used to examine the characteristics of program participants. Taking into account all these limitations, we have chosen to present a summary of the available information as follows. Box 5 provides a brief overview of overall targeting effectiveness by country, while Table A1 in Annex 1 summarizes program specific information on one or more of the four indicators described above. We draw primarily on analysis done by the World Bank and others in the context of the preparation of country-specific safety set reports, as well as on academic papers by regional and international researchers on particular interventions—usually the largest, most important programs in the region. 26 Box 5: Targeting outcomes and effectiveness in South Asia Targeting outcomes and effectiveness vary across countries in the region as does the amount and nature of the information available on each country. We briefly summarize here country-wide evidence presented in country Social Protection reports prepared by the World Bank and others, while program-specific information is provided in Annex 2. Afghanistan (Afghanistan National Development Strategy 2007, World Bank 2005a)  Afghanistan does not have a national targeting system.  Over the last few years government efforts have primarily focused on expansion of universal access to basic services and assistance rather than on granting selective access to these same services to a subgroup of the population.  Moving forward the National Development Strategy proposes a more active use of geographical and categorical and demographic targeting for social interventions. The former is already used under the Food- for-Work program administered by the World Food Program. The latter is currently used for social pensions and further applications would focus on population at ―risk‖ (very broadly defined at the moment) and war survivors. Bangladesh (World Bank, 2005b, 2006a)  Bangladesh uses a combination of targeting methods.  Evidence from household surveys suggests that there are significant inclusion and exclusion errors and leakage in cash and in-kind transfer programs, due to (i) poor choice, design and implementation of targeting tools, and (ii) limited capacity for program administration and monitoring.  Relatively effective use of geographic targeting for delivery of (short-term) post-disaster relief, but less successful use of this approach in the context of other programs.  Programs with better targeting outcomes, such as micro-finance and micro-insurance, do not necessarily reach the extreme poor due to low program participation among this group.  Public works program is generally well-targeted and the government has recently introduced the 100-day guarantee. Bhutan (World Bank 2008)  Bhutan does not have any safety nets or targeted social programs - instead the government provides free education and health services. India (World Bank 2010)  India relies largely on a national targeting system - the Below Poverty Line (BPL) indicator - for most centrally-sponsored schemes in rural areas and a large fraction of those financed at the state level. This is, however, is poorly designed and implemented leading to substantial inclusion errors and worrying exclusion among the poorest, albeit overall BPL card-holding appears to be progressive. Targeting effectiveness of the BPL is further compromised by poor program implementation and high leakage rates although there is significant variation in targeting outcomes across programs (see Box 7).  The use of self-targeting has yielded relatively good results under public works programs, with more progressive targeting outcomes relative to other programs.  Similarly, the use of combined categorical and BPL targeting for social pensions promotes coverage among the poorest quintile and greater benefit incidence among the bottom quintile.  BPL-based targeting under PDS and rural housing programs performs less well among targeted programs, although they are mildly progressive.  A combination of methods is used for targeted programs in urban areas. Maldives (World Bank 2006b)  Maldives uses a combination of targeting methods.  Poorly defined targeting criteria, combined with limited information on existing programs among potential beneficiaries and weak and highly discretionary program implementation structures translate into low targeting effectiveness. Approximately 50 percent of government resources assigned to social assistance accrue to the richest 30 percent of the population and only 26 percent accrue to the poorest 30 percent. 27 Box 5: Targeting outcomes and effectiveness in South Asia … (contd.) Nepal (World Bank 2005, World Bank 2009b)  Nepal uses a combination of targeting methods.  Decisions on the choice, design and implementation of various targeting approaches are often made by donors, who have favored geographical and categorical and demographic targeting. Recently, there is increasing use of community-based targeting through village assembly groups. Lack of coordination among donors and even across programs supported by the same donor has resulted in a multiplicity of concomitant approaches and a high degree of fragmentation. Pakistan (World Bank, 2007)  Pakistan uses a combination of targeting methods, but has recently developed a PMT-based poverty score card and is currently piloting this method, with a view to adopt nationally across programs.  Evidence from household surveys suggest that there are significant inclusion and, particularly, exclusion errors and leakage in cash and in-kind transfer programs, due to (i) insufficient program budgets, and (ii) poor design and implementation of targeting tools, combined with (iii) limited capacity for program administration and monitoring.  Relatively effective use of geographical and categorical/demographical targeting for delivery of (short-term) post-disaster relief, but mixed outcomes when used for targeting of other programs.  Poor experience with self-targeting for public works due to inadequate design of targeting parameters. Sri Lanka (World Bank, 2006c)  Sri Lanka uses a combination of targeting methods.  The Samurdhi program, the largest in the country with coverage close to 40 percent of the population, has traditionally suffered from important inclusion errors and high leakage, primarily associated with a poorly designed and implemented targeting system. Recent attempts at improving targeting effectiveness under the program have motivated a series of changes in targeting approaches, the most recent of which has led to the adoption of community-based targeting. Information on targeting outcomes under this new system is not yet available.  Exclusion errors are high for other social programs using categorical targeting due primarily to limited budgetary resources.  Geographic (cash assistance) and self-targeting (public works) mechanisms were successfully used to deliver (temporary) post-disaster relief after the tsunami. Source: Government documents, country-specific World Bank Social Protection and Poverty Assessment reports and Targeting Reviews as noted above. See Annex 2 for country-specific and program-specific details. Determinants of targeting effectiveness In this section we build upon the information on targeting effectiveness of specific programs to motivate and illustrate a discussion on the determinants of targeting effectiveness. Program specific design and implementation details and quality will have a significant impact on targeting outcomes, but a few basic targeting principles can be applied to most tools and programs. Reducing errors of inclusion requires a definition of eligibility that correctly discriminates between the poor and the non-poor and that can be implemented at reasonable cost. Similarly programs need extensive outreach and communication campaigns and adequate budgetary resources to keep errors of exclusion low. Finally targeting systems should be dynamic to allow for entrance of new poor beneficiaries and exit of those who are no longer eligible. 28 We structure the discussion on targeting effectiveness around the following three issues: i) choice and design of targeting tool, (ii) implementation and monitoring of targeting tool and (iii) implementation and monitoring of targeted program (see below). We explicitly draw a distinction between the implementation and monitoring of the targeting tool and that of the program to be targeted to highlight that fact that well-designed and administered targeting tools can still yield poor targeting outcomes if used to target poorly implemented programs, and vice versa. Getting it right in terms of any one of these dimensions will go a long way in ensuring good targeting outcomes, but will not guarantee adequate targeting in the absence of other factors. In other words, making the right choice and having a solid technical design and adequate implementation and monitoring systems for the targeting tool are necessary but not sufficient conditions for effective targeting. Having said this, our focus here is on (i) and (ii)—i.e. targeting systems—while we provide a short comment and further references on (iii)—i.e. programs. Choice and design of the targeting tool11 A few methods of targeting and types of programs go hand-in-hand—for instance self-targeting and workfare programs, but more generally different methods can often be used to target a particular type of program. Hence, whether the choice of targeting approach is right should be judge not only against program objectives but, most importantly, with respect to existing resources and capacity (an issue we discuss under the next heading). The following as key elements to be taken into account when choosing a specific targeting approach: (i) nature and distribution of poverty or other social dimension to be addressed by program, (ii) existing administrative capacity, and (iii) nature of program(s), including coverage and level of benefits. Once a particular method is chosen, technical design details still need to be worked out to tailor the general approach to the specific context in which it is to be applied. Significant targeting inefficiencies across programs in the region can be traced back to this stage. A few illustrations follow for the most frequently used approaches in the region. a. Means testing Verified means tests are considered to be the gold standard of targeting but are usually difficult to implement in developing countries since they work best in situations with high levels of literacy and documentation of economic transactions. They are also administratively demanding when combined with meaningful attempts at verification. Finally means testing is also the form of targeting most likely to discourage work effort, because eligibility is linked directly to current income. Effective means-testing requires high quality, reliable and widely available information on household or individual income. For this reason this method is most appropriate when declared income is verifiable, where some sort of self-selection limited applicants from non-target groups, where administrative capacity was high, or where benefit levels where high enough to justify the administration of a means test. 11 This sub-section draws on Grosh, del Ninno, Tesliuc and Ouerghi (2008) for definitions of targeting methods; a background note on Participatory Approaches to Targeting: Concepts and Operational Challenges by Ben Powis (SASSD, consultant) and inputs from Tara Vishwanathan (PREM) on the experience in South Asia with proxy means testing. 29 Bangladesh, Maldives and Sri Lanka appear to use some form of means-testing to target cash and in- kind transfer programs. Bangladesh also uses means-testing as one of the targeting criteria for work fare. In none of these countries, however, are the conditions outlined above in place (with the potential exception of workfare in Bangladesh where a potential element of self-selection could be introduced through the wage rate to be paid to program participants). b. Proxy-means targeting A good proxy means test should: (i) be based on easily observable/verifiable household and/or individual characteristics that can adequately help distinguish between the poor and the non-poor and are likely to remain stable over a certain period of time, (ii) strike a balance between adequately discriminating between eligible and non-eligible households and keeping the number of indicators on which information needs to be collected down to a manageable minimum; and (iii) aggregate this information so as to maximize the correlation between the welfare score and poverty status. The advantage of proxy means testing is that it requires les information than true means testing, and yet is objective. Moreover, because it does not actually measure income, it may discourage work effort less than a means test would. However, administering it requires a large number of literate and probably computer trained staff and moderate levels of information technology. Finally proxy means tests distinguish the chronically poor well but can be insensitive to quick changes in household welfare or disposable income, which may be frequent and large when an economy is suffering from a large turndown. As a result, this approach is most appropriate in the presence of reasonable administrative capacity, for programs meant to address chronic poverty in stable situations and where they are used to target a single program with large benefits or to target several programs so as to maximize the return for a fixed overhead. India is the only country in the region that uses proxy-means testing at a large scale at least in rural areas, but its experience is rather illustrative.12 The BPL indicator fails to comply with any of the basic design principles outlined above. In fact, it is constructed from proxy indicators of poverty defined by a committee and not through rigorous analysis of household survey data as in traditional PMTs. Work by the World Bank (see Jalan and Murgai 2008, World Bank 2009) indicates that across India the BPL score misclassifies half of the poor as non-poor, and conversely, half of those identified as poor are in fact non-poor. There is significant variation in targeting errors across states, with exclusion errors being higher in richer states, but errors are substantial for all states. The authors attribute the score‘s poor performance primarily to (i) the wrong choice of indicators and (ii) the wrong aggregation procedure (see Box 3 above). In addition, although the BPL being used to target a multiplicity of program may have provided some justification for the choice of proxy-means testing, the fact that targeting errors are aggravated once the BPL is actually implemented suggest that existing administrative and monitoring capacity is not adequate to handle this kind of tool. c. Community-based targeting The advantage of this approach is that it relies on local information on individual circumstances, which may be more accurate and less costly to collect than usual methods—although the costs of 12 As noted above, Pakistan has recently developed, with World Bank technical assistance, a PMT-based poverty score card and is currently piloting this method, with a view to adopt nationally across programs. 30 facilitation could become high. They also permit local definitions of welfare and are typically effective at identifying the extreme poor if designed and administered correctly. At the same time, local actors may have other incentives besides good targeting of the program so that such a system may continue or exacerbate any existing patterns of social exclusion. Finally, if local definitions of welfare are used, spatial comparisons of poverty and welfare become more difficult and ambiguous. Effective community-based targeting requires active and extensive participation across all groups within a particular village or target unit, as well as adequate facilitation. For these reasons this method is most appropriate where local communities are clearly defined and cohesive, for programs that plan to include just a small portion of the population. It is also best suited for programs that cannot support administrative structures of their own or programs where allocation of resources across space is not a function of targeting. Community-targeting is not very widely prevalent within government programs in the region13. The Samurdhi program in Sri Lanka and the Indira Kranti Patham (IKP; formerly Velugu) program in the state of Andhra Pradesh in India, as well as the Challenging the Frontiers of Poverty Reduction – Targeting the Ultra Poor (CFPR/TUP) program administered by BRAC in Bangladesh are the main exceptions. None of these programs is small in terms of resource allocation or coverage and they all have their own administrative and monitoring structures. Instead the main rationale for the choice of community-based targeting appears to be the desire to consider multidimensional definitions of poverty for targeting purposes and, particularly, to tap into local knowledge and information about household welfare. No information is currently available on the targeting effectiveness achieved under these programs. d. Geographical targeting The advantage of this approach is that it is administratively simple, requiring none of the machinery for individual assessment programs described above. It will have no disincentive and is unlikely to result in stigmatization, as all households in an area are eligible. However geographic targeting will perform poorly when poverty (or the indicators(s) of choice—e.g. malnutrition, school enrolment) is not spatially concentrated. It also depends on the accuracy of the poverty map. Finally political compromises may be required, as politicians from each jurisdiction will lobby to have their districts included and this could mean that a few districts in each area, rather than the poorest districts, will benefit. The most appropriate circumstances for geographic targeting are when living standards across regions vary significantly, when administration capacity is too limited or transfer amounts too low to make individual assessment methods sensible, for programs delivering public or non-excludable goods or services, and/or when additional self-targeting can be induced through the use of some public service used mainly by the poor. Good geographic targeting requires the use of high quality, up-to-date and sufficiently disaggregated information on poverty or adequate poverty correlates. For instance, both the India ICDS and Bangladesh Health SWAP projects developed an index of nutritionally backward regions in order to target project resources. The former was constructed from indicators directly related to the priorities 13This contrasts with the frequent use of community-based targeting by the World Bank and others to target programs and interventions financed with donor funds. More details on this are provided in section C of this note. 31 of the project, such as proportion of underweight children, anemia among pregnant women and gender parity in primary enrolments; the latter relied on a close proxy – poverty incidence – and one direct indicator (proportion of underweight children). Information on the relative targeting effectiveness of these projects is not available. In addition the most appropriate circumstances for geographic targeting are when living standards across regions vary significantly or poverty is geographically concentrated, for programs delivering public or non-excludable goods or services, and when additional self-targeting can be induced through the use of some public service used mainly by the poor. The Tawana program in Pakistan, a school feeding program, and the use of food insecurity maps in Bangladesh provide two interesting illustrations on the issues of data quality and poverty concentration. The primary objective of the Tawana program was to provide mid-day meals to girls in primary schools in the poorest districts in the country. For this purpose district level information on nutritional outcomes was combined with administrative data on existing schools to identify a total of 5,300 eligible schools. In reality both the nutrition data and the administrative data provided by the Department of Education were of poor quality and/or significantly outdated, which led to the selection of a large number of non-eligible, non-functional schools and a subsequent increase in program costs associated with the necessary adjustments (Shahnaz, 2008). Similarly food insecurity maps are used to target various food distribution interventions in Bangladesh. These maps are constructed at the upazila level, with programs targeting households in the poorest 50 percent of upazilas. While this makes it more likely that food assistance does get to the poor, it also means that a large number of poor households residing outside eligible upazilas are denied such assistance, including large numbers of households affected by severe food insecurity. In contrast, the use of geographic targeting for the delivery of post-disaster relief, be it in the form of cash, food or other supplies, has been very successful in the region in recent years as the discussion in Box 5 illustrated. Two key factors behind this success were high concentration of target households and individuals in a well-defined geographical area and significant technical assistance for targeting and program implementation and monitoring from donor agencies and others. e. Categorical and demographic targeting Demographic targeting is obviously administratively simple and it carries the appeal of universality, hence minimizing stigma within a specific group. This tends to make it politically popular. Demographic targeting is a low-cost targeting method and is particularly useful when a specific individual characteristic is highly correlated with poverty (or the indicator of choice) or for programs that include a complementary element of self-targeting. Good categorical and demographic targeting requires that the indicator(s) of choice be (i) unambiguously defined, (ii) highly correlated with poverty or the specific outcome the program is trying to impact, and (iii) easily verifiable. In practice, very few indicators satisfy these requirements and, while considering multiple indicators simultaneously may lead to better discrimination between the poor and non-poor, in the absence of clearly defined aggregation and decision rules, it can also translate into higher levels of discretion in beneficiary selection. 32 Information on targeting outcomes for the Vulnerable Group Development (VGD) and Primary Education Stipend (PESP) programs in Bangladesh provides a useful example. Both programs target households with specific characteristics assumed to be correlated with poverty and vulnerability and use multiple eligibility criteria (see details in Annex 1). Despite these efforts, survey data indicate that 27 percent of VGD and 47 percent of PESP beneficiaries are non-poor, suggesting that the selected eligibility criteria do not have the desired discriminatory power (Ahmed 2004). e. Self-targeting The administrative costs of self-targeting are quite low, although often the administrative costs of the associated program are high (e.g. public works). In addition errors of exclusion (associated with non-participation) and stigmatization tend to be high when self-targeting is used. This approach is most effective when used to target goods or services that are more heavily consumed by the poor than the non-poor, and for programs where monitoring costs associated with verification of eligibility are high. Good self-targeting relies heavily on decisions regarding the nature and level of the discrimination parameter. For instance, the targeting success of subsidized food distribution programs will hinge, among other factors, on the choice of the right commodity to be subsidized. Similarly adequate targeting of work fare programs will depend on the choice of the wage rate to be paid under the program relative to the prevalent market rate. For these reasons this approach is most effective when used to target goods and services that are more heavily consumed by the poor than the non- poor, and for programs were the costs associated with monitoring eligibility are high. Evidence on targeting within the National Rural Employment Guarantee (NREG) program in India can be used to illustrate some of these points. Mehrotra (2008) uses program administrative and survey data to compare wage rates paid under the NREG with market rates for similar occupations in program areas and concludes that program rates are significantly higher than prevalent market rates. This appears to be particularly true for women who tend to command lower market wage rates than men. In most public works programs, the fact that program wage rates are above those prevalent locally in program areas could potentially compromise targeting outcomes, especially if program wage rates are high enough to attract non-poor (or non-extreme poor) participants. However, NREG confers a right to work to all rural households (and not just poor households). As a result, the self-targeting element is limited to the expectation is that the nature of the work (unskilled manual labor) would attract mostly poor households. In this context, higher real program wages are seen as obvious advantages for participating households. There would be cause for concern only in the event the supply of worksites is inadequate to meet demand for work, leading to rationing of employment in a manner such that the poor are excluded. From this we conclude that, even though the discrimination parameter (in this case, the wage rate) seems to have been fixed at too high a level for effective self-targeting, the fact that program participants are asked to perform hard labor may have contributed to disproportionately attract the extreme poor and poor (i.e. hard labor to be considered as a commodity likely to be ―consumed‖ more heavily by these groups) hence saving the day. In sum, existing evidence suggests that, although exceptions can be found, technical capacity and information constraints have often translated into inadequate choices and poor design of targeting tools across the region and, ultimately, compromised targeting effectiveness. Although getting it right at the design stage is only the first step towards satisfactory targeting outcomes, it is a necessary one. In this context efforts to increase exposure to international best targeting practices, to support 33 more systematic thinking around design-related issues, and to build technical capacity could have a high payoff in terms of improved targeting performance. Implementation and monitoring of targeting tool Most of the variation in targeting efficiency across programs can be explained by differences within targeting methods, as opposed to variation across targeting methods. In Coady, Grosh and Hoddinott‘s (2004) comparisons, country context explained some, but by no means all the variability. Targeting performance improved with country income levels (used as a proxy for implementation capacity), the extent to which governments were held accountable for their actions, and the degree of inequality. Unobserved factors, however, explained many of the differences in targeting success suggesting that improvements in the implementation and monitoring of targeting methods have great potential. We focus on three specific aspects of implementation and monitoring here: (i) outreach and communication efforts, (ii) implementation framework and capacity, and (iii) monitoring and oversight. Outreach and communication One of the reasons why households and individuals fail to come forward to be identified as beneficiaries of programs for which they are potentially eligible is that they do not have sufficient information about the program or the steps that need to be followed for identification and registration. Maximum outreach can be achieved through a combination of actions such as (i) ensuring adequate dissemination of information about the eligibility criteria, the identification process and, more generally, the program, (ii) designing identification and registration procedures so as to minimize transaction costs to eligible households, and (iii) ensuring that people can register at any time by having an open application process. Success on these fronts often hinges on resource availability and good logistic planning. See also Box 6 for the interaction between a sound outreach and communications plan and the use of incentives to improve access and reduce exclusion errors under the RSBY health insurance program in India. 34 Box 6: Incentives and outreach in RSBY The Rashtriya Swastha Bima Yojna (RSBY) is a national program that offers subsidized health insurance to poor households in India. Households are identified using the BPL method, with all its attendant problems (see Box 3). The combination of a sensible incentive structure and emphasis on a sound outreach and communications strategy suggest a greater potential for the program to deliver benefits to the target group. The program design incorporates an incentive structure that induces the actors involved – beneficiaries, hospitals, insurance firms and others – to respond in a way designed to improve targeting outcomes. For instance, unlike several government insurance programs, the enrolment process requires direct contact with the beneficiary (ensuring minimal awareness that the benefit exists and conferring some degree of ownership). In addition, insurance firms are paid per household enrolled and thus, have an incentive to reach out to potential beneficiaries and maximize enrolments. The use of smart cards and biometric information provides additional confidence to implementers that the contact with beneficiaries was actually made. As with other central programs (including most safety nets), program design is typically undertaken at the center while states are responsible for implementation. As a result, there is variation in communication/outreach, depending on the state government implementing the program; the following case study describes a good example. Enrolments in the first year of RSBY were low in the state studied. Part of the problem was an inaccurate and outdated the BPL list - a household survey, drawn from the BPL list, revealed that a large number of households on the BPL list could not be located at the address listed and were in truth ―ghost‖ households. In addition, lack of information and of effective demand among the beneficiary population could also be driving low enrolments. A survey of households post RSBY enrolment revealed that, among those who said they were aware of the scheme, 82 percent had applied. In order to address this issue, a door-to-door outreach and communications campaign was piloted in the state and 7500 randomly selected households were visited by representatives of a firm who explained the RSBY program, including the enrolment process. Among the households contacted through these pilot program, enrolment rates were double compared to the households that were not contacted. Since these households were chosen at random, we could conclude that (a) lack of awareness of the scheme was a major problem in enrolment and (b) that door-to-door visits and the provision of information did increase enrolment. Partly based on these activities, the state nodal agency is planning a large campaign with 80 NGOs to provide door-to-door information about the program. Source: Notes by Jishnu Das and Robert Palacios. Evidence from the pilot of the Child Support Program (CSP, a conditional cash transfer program) in Pakistan in 2006-2008 provides a useful example here. For the sake of simplicity, it was decided that all households currently enrolled under the Food Support Program (the largest national cash transfer program at that time) would be eligible to receive payments from the CSP provided they had children ages 5 to 12. It was determined that approximately 22,000 households satisfied these criteria in the three pilot districts. However only 11,000 actually reported to the registration teams deployed to the areas. Detailed process analysis later revealed that the communication campaign launched to inform qualifying households of their eligibility and to invite them to the registration process had to a large extent failed. Specifically the executing agency chose to send invitation letters to all eligible households as their primary means of communication. The combination of high illiteracy rates and a poorly functioning post office system rendered this approach ineffective, leading to exclusion errors of approximately 50 percent. As a consequence a second registration process preceded by an extensive multimedia information campaign had to launched, at significant additional costs, which then brought program take-up rates to up to 80 percent. Similarly evidence presented in the 35 Maldives Safety Net report (World Bank, 2006b) suggests that poor coverage and high exclusion errors in transfer programs are due partly to the lack of adequate information among eligible households outside the main capital and/or residing in remote atolls. In contrast, the organization of disability certification camps in several states in India represents a good illustration of how transaction costs can significantly be reduced and hence exclusion errors minimized. Disabled individuals are entitled to receive a social pension as long as they are in possession of a disability certificate. These certificates are typically given by assessment and certification teams at district hospitals. However, there are problems with this process, due to physical and financial cost of going to district hospitals, supply-side issues such as vacant posts and lack of doctors. In response, some states have started to organize special camps where information on social pensions is provided and disability identification and certification are done on the spot. Implementation framework and administrative capacity Effective selection and registration of eligible households requires a clear definition of roles and responsibilities across all actors involved and sufficient administrative capacity. The specific requirements will depend on the targeting method selected and the implementation details, so this section cannot be comprehensive, but it does illustrate some of the issues pertaining to administrative capacity and how they may influence targeting effectiveness. Eligibility rules must be clear and well defined. Lack of clarity or definition regarding eligibility often leads to inconsistencies in decision making, while excessive detail may mean that field staff and other implementation staff are unlikely to know or understand all rules or to be able to effectively convey these to the public. Similarly, institutional roles and responsibilities must be clearly mapped and respond to existing capacity and program needs. There is no single way of doing this, but existing evidence suggests that local management presents a comparative advantage in data collection (provided adequate training), while the central level is best equipped for database management. Failures on any of these fronts can lead to significant targeting errors. For instance, the Bangladesh Primary Education Stipend Program (PESP) uses categorical targeting to identify program beneficiaries who meet any one of five different criteria. These criteria are poorly defined and are difficult to verify, which implies that they are often used as indicative rather than binding. Using survey data Ahmed (2004) shows that 11 percent of participants of the Primary Education Stipend Program (PESP) did not meet any of the eligibility criteria for program participation and that almost none of the beneficiaries meet at least three of the criteria. Similarly, eligibility to receive Zakat benefits in Pakistan is determined by local committees according to very loosely defined criteria (e.g., the ‗deserving needy‘), which allows for significant discretion in the identification criteria for beneficiary selection. As a result, while program applicants are to a large extent poor (i.e. 60 percent of applicants belong to the poorest two quintiles), only 40 percent of those actually receiving benefits can be considered poor. In contrast, eligibility criteria used during the post-earthquake relief effort in Pakistan in 2005 were clear, well defined and easy to understand by both program implementers and potential beneficiaries in affected areas. This not only contributed to achieving good targeting outcomes (i.e. About 90% of the total of 285,000 estimated eligible households have received at least one payment, and about 85% of the eligible households have received four or more payments so far) but, more generally, fostered a positive perception of the program among local officials and households residing in affected areas. 36 Having said this, a word of caution is in order since clear and transparent eligibility criteria could lead to ―learning‖ among program beneficiaries and administrators regarding what factors make a particular household or individual eligible for a program and could, subsequently, translate into attempts to game the system. In the case of community-based targeting, where eligibility criteria themselves are generally defined by the communities, the emphasis should be on clearly defined and standardized processes for facilitation and subsequent identification of the poor. A good example is provided by the detailed operations manual and training of facilitators in the Andhra Pradesh IKP program. Anecdotal evidence from other community-driven projects also suggests that the commonality of local indicators across communities and the implicit need for some standardization or comparability across communities in order to allocate program resources across space has led to the adoption of ‗guidelines‘ for the characterization of various poverty strata (Powis, 2009). A second key element for successful implementation is ensuring that capacity is appropriate, where capacity can be understood to encompass staffing and other material inputs, and data management systems. Adequate staff time and general skills are required, with specific details relating to the particular targeting method to be used. There is some evidence that non-availability of trained staff in underprivileged areas resulted in poor outreach and insufficient coverage in the Lady Health Worker‘s Program in Pakistan (Arif 2006). However, in general there is rarely any documentation on ex-ante planning on staffing or ex-post results. Similarly, a suitable share of material inputs is necessary for the implementation process to run effectively. Frequently, errors of exclusion are aggravated because of lack of application forms, transportation funds for registration teams and so on. Finally, although a good targeting system can be administered and managed on the basis of pen and paper, computerization and adequate database management are essential for cross-checking eligibility, keeping household histories and effectively handling changes in eligibility status. The BPL in India provides a useful example. The information necessary to construct the BPL score is updated every 5 years or so through a census-like operation in rural areas (see Box 3 for details). Financing for this operation is provided by the central government but logistic planning and implementation are entirely the responsibility of local authorities. Generally these authorities receive relative short notice regarding the deadline for data collection and thus have to hurriedly put systems in place to gather and process large amounts of information over too short a period of time. As a result, staff time and training are often insufficient to adequately handle the task and this negatively impacts data quality and eventually targeting efficiency. In addition, since the task is performed by teachers, health workers and other village-level government employees, this responsibility comes in addition to their regular work load.14 Jalan and Murgai (2008) calculate state-level under-coverage rates15 associated with the BPL design and implementation separately and show that in most cases under-coverage is exacerbated by poor implementation. Take, for instance, the figures for Rajasthan. The 2004-5 rural poverty rate in the state was 18.3 percent, while the share of BPL card holders was 17.9 (a close enough figure). However this should not be interpreted as evidence of good targeting. In fact, simulations using household survey data suggest that 60 percent of those considered poor 14 Based on anecdotal evidence from localized surveys and confirmed by preliminary results from a household survey on targeting methods in two states. A recent proposal to reform the BPL system includes a suggestion to employ external (outside the community) enumerators to conduct the census (Mehtrotra and Mander 2009). 15 Percentage of the poor population wrongly classified as BPL non-poor (under-coverage de jure or by design) or not possessing a BPL card (under-coverage de facto or by implementation). 37 would be classified as non-BPL poor given the indicator‘s current design (see discussion above on this issue), and that an even higher 67 percent would not have access to a BPL card. The difference between the under-coverage rate associated with the instrument‘s design and that observed on the ground (i.e. 7 percentage points or a 10 percent increase) can then be attributed to poor implementation. In contrast, adequate financing, staffing, training and supervision were all factors contributing to high targeting effectiveness within the post-disaster relief program in earthquake affected areas in Pakistan. In contrast, adequate financing, staffing, training and supervision were all factors contributing to high targeting effectiveness within the post-disaster relief program in earthquake affected areas in Pakistan. This included a comprehensive package of capacity building for data management of local staff, effective training and coordination of the over 3,000 ground staff, an intensive information campaign for spreading information on the process of application and appeal, as well as operational third-party checks for monitoring of payments. In the IKP program in Andhra Pradesh (India), the community-based targeting mechanism (referred to as Participatory Identification of the Poor or PIP) builds on thorough training of field staff and the use of a well- developed field PIP Operations Manual. The operation manual gives detailed information on how to proceed with establishing rapport with the community, mobilizing participation, conducting wealth ranking and social mapping exercises and finally ratifying the wealth ranking lists. By addressing several possible questions and difficulties that could arise in the field, the operations manual, combined with monitoring systems to check compliance in the field, assures a certain degree of consistency and accuracy of facilitation, and therefore of the effectiveness of community-based targeting. Monitoring and oversight Monitoring and oversight are crucial to all systems. For the purpose of this discussion we will apply a broad definition of these two concepts so as to include (i) fraud control, but also specific processes related to the accuracy of the targeting information such as (ii) grievances and appeals and (iii) recertification. A number of mechanisms are available to minimize fraud and leakages, including process audits and random controls, cross-checks, and social controls. These tools can be used independently or jointly for maximum impact. In any case, in every system and program mistakes happen or, even more frequently, people believe a mistake has been made. Having mechanisms to adequately handle appeals and grievances is therefore important both for correcting the mistakes and for perceived fairness. Finally over time some households may cease to be eligible for a particular program, while others may become eligible due to a change in their circumstances and/or demographic characteristics. Adequately handling these changes will require both open exit and entry systems and a well-designed and administered recertification policy. The case of the Food Support Program in Pakistan is illustrative of how the lack of an adequate database management system can compromise the effectiveness of monitoring and oversight mechanisms and ultimately overall targeting effectiveness. Before its disappearance in 2009, program eligibility and beneficiary selection were handled at the district level. District officials received applications through an open registration process and classify applicants as eligible or ineligible after basic information is verified through a home visit. Eligible applicants were then registered on a waiting list and enrolled in the program as slots become available (each district is assigned a program quota and resources are only sufficient to pay those included in the quota). Unfortunately, the lack of adequate monitoring systems at the district level, combined with very primitive database management, often implied that waiting lists were not properly maintained and updated as, as a 38 result, district officials and politicians got to exercise significant discretion in the allocation of available program slots. The same problem afflicted the grievances and appeals process. In the absence of a program database and of clearly defined procedures to handle these cases, decisions were often made in an ad hoc manner and are poorly documented. Although no hard data is available on the impact of these caveats on targeting effectiveness, anecdotal evidence suggests that the program suffered from capture by local elites with non-poor households being given priority over eligible ones at times. In contrast, the RSBY (Rashtriya Swastha Bima Yojna) health insurance program in India, launched in 2008, is a good example of how anti-fraud controls can be built into the system design making creative use of technology. Smart cards containing individual biometric information for all eligible people within target households are handed out at the time of registration. These cards then have to be presented and the information read and validated each time a program transaction is performed in order to ensure that programs benefits only accrue to eligible individuals. The card also stores information on the local government official who facilitated the registration so that information inconsistencies and errors can be traced back to the responsible official. Some states, notably Andhra Pradesh and Karnataka, are experimenting with the use of such IT-solutions in other safety nets, including social pensions and workfare. Counter verifications of beneficiary lists, to establish their veracity can also help tremendously in reducing fraud by increasing its detectability and visible liability. In the CFPR/TUP program run by BRAC in Bangladesh, while there is tremendous emphasis on local engagement in the community wealth ranking process (which forms the basis of beneficiary selection), this exercise is followed by home visits to those deemed poor by project staff in order to verify their actual income and well- being status. Households short-listed in the community wealth ranking exercise are further evaluated against a set of inclusion and exclusion criteria in order for BRAC to finally confirm their eligibility (BRAC 2004). Where this two-stage verification process may not be possible for all potential beneficiaries, spot checks and qualitative process evaluations could help indicate if things are going wrong. In sum, the discussion above shows that, as was the case at the design stage, substantial resource, capacity and logistic limitations associated with the implementation and monitoring of targeting instruments significantly hinder targeting effectiveness. International evidence suggests that a large fraction of the observed differences in targeting effectiveness across systems and programs can be attributed to factors related to implementation and monitoring. This implies that investments aimed at correcting some or all of these limitations could go a long way in improving targeting outcomes in the region. Implementation and monitoring of targeted program Targeting systems are rarely designed in a void but rather are developed and implemented to be used as part of the implementation of a particular program or series of programs. In this sense targeting effectiveness would not only be a function of the design, implementation and monitoring of the targeting tool itself, but would also be affected by the implementation and monitoring of the targeted program (see Box 7 for an illustration). A detailed analysis of implementation and monitoring details across targeted interventions in the region is beyond the scope of this note and we refer the interested reader to the annotated bibliography in Annex 1 for more information on specific programs. 39 Box 7: Impact of program implementation on targeting effectiveness: Social pensions and the PDS in India Targeting effectiveness under a specific program depends on three key factors: (i) the choice and design of targeting tool, (ii) the implementation and monitoring of the targeting system and, (iii) implementation and monitoring of the targeted program. As a result, programs targeted using the same tool can exhibit radically different targeting outcomes. To illustrate this point we examine targeting results under the Public Distribution System and the Social Pensions program in Rajasthan. Both interventions are targeted using the BPL indicator (combined with categorical targeting in the case of social pensions). However, the programs present varying degrees of targeting effectiveness which we then attribute to differences in program implementation and monitoring. The Targeted Public Distribution System (TPDS) in India is the largest social safety net program in the world under which a price subsidy is offered to identify beneficiaries for essential commodities, most prominently rice and wheat. This operates under a two-tier structure in which differential prices are set for APL and BPL families. The Antyodaya Anna Yojana (AAY), a sub-scheme that operates under the aegis of the TPDS offers a subset of the BPL beneficiaries (most vulnerable households) additional benefits, both in terms of additional rationed grain allocations and a lower price. In Rajasthan, the TPDS uses the BPL census lists to ascertain eligibility for the varying levels of benefit to be received by households. The Social Pensions program seeks to address chronic poverty among especially vulnerable groups (destitute elderly, widows and disabled people) In Rajasthan, these programs use a combination of categorical targeting and the BPL census list (the latter for identifying the destitute). The targeting of the TPDS in Rajasthan, relative to the rest of the country has been particularly poor. Umali- Deininger and Deininger (2006) estimate using NSS (60th Round) data that the ratio of PDS coverage in the bottom two to that in the top two quintiles for Rajasthan was among the lowest in India, at 1.4, compared to an all-India figure of 2.2; indicative of very limited difference in PDS usage across the income spectrum. Comparing officially reported wheat and rice off-take figures with reported purchases of PDS grains reveals that for Rajasthan the leakage of PDS grains is over 90% (ibid.) In contrast elderly and widow social pensions in Rajasthan are generally progressive (Dutta 2008). Much of these differing targeting outcomes using the same targeting tool can be explained by the nature of the program design and delivery. While the PDS system is operated by heavy bureaucratic machinery involving many stages and levels of intermediaries, including a network of fair price shops; the social pensions program is fairly simple in design, making direct payments to pensioners. Further the PDS deals in providing monthly rations of food grains etc. while the social pensions programs offer regular small and regular cash benefits (as against a large lump-sum) which are typically better targeted. The regularity of payments, small amount of each pension payment and monetization of benefits under the social pensions program make it a more effectively delivered and better targeted program as compared to the PDS. Source: World Bank (2010). A few common lessons, however, can be extracted from regional (and international) experiences and summarized in the form of good principles. Some of these principles are similar to those discussed above in reference to the implementation and monitoring of targeting systems and other are more closely related to actual service delivery. Specifically, the following practices have been found to contribute to better targeting outcomes across targeting systems and programs of various types. Clear program design with well-defined program rules and institutional roles and responsibilities is a critical prerequisite. The nature and frequency of benefits also has implications for minimizing fraud 40 and leakage – (i) cash transfers appear to be preferable than in-kind transfers; and (ii) regular (smaller) payments appear to be preferable than one-time large payments. With respect to program implementation, the following contribute to maximize access and minimize de facto exclusion errors, as well as to reduce program leakage: (i) adequate outreach and information campaigns; (ii) streamlined enrolment process to as to minimize transaction costs (i.e. minimum required documentation, open enrolment, etc); (iii) lean administrative structures (i.e. minimum number of program intermediaries); and (iv) creative use of IT for payment delivery (e.g. use of ATMs, mobile banking and Point of Service solutions). Similarly, adequate program monitoring can contribute to reduce inclusion (and exclusion) errors and fraud, through: (i) creative use of IT technologies for database management and fraud control; and (ii) explicit inclusion of targeting outcomes as part of program‘s M&E framework. In sum, we have argued in this section that targeting systems in the region are fragmented and targeting outcomes are poor, with a few exceptions. High levels of fragmentation can be attributed to the absence of national targeting systems. And weak targeting effectiveness appears to be the product of faulty design and poor implementation and monitoring, both of the targeting tool and the program for which the tool has been developed. This absence of an integrated targeting architecture combined with low administrative and monitoring capacity within the public sector has opened up the space for strong involvement of non-government actors, particularly donor agencies, in the design, implementation and monitoring of various targeting tools. 41 C. OVERVIEW OF WORLD BANK-LED REGIONAL WORK ON TARGETING Recent years have been witness to an increased emphasis on the part of the World Bank on the need to adequately target program and project benefits to the intended population. This can be attributed to both a growing demand from regional governments for this type of support and a renewed institutional interest on ensuring maximum (poverty) impact of World Bank-supported initiatives. As a result, attention to targeting-related issues has been increasing within the analytical and operational portfolios. In fact, much of the analysis of targeting in government programs in Section B draws on recent World Bank country-specific reports on poverty and on government programs (including safety net and social security programs as well as social services). This prior analytical work has provided the basis for assessing the targeting effectiveness in government programs and the potential role of the World Bank in influencing these outcomes and the broader policy debate. In this context, there is significant potential for the World Bank to add value and support countries in improving and strengthening existing systems through both analytical and operational work. In this section we primarily focus on targeting work undertaken as part of World Bank operational work, i.e., projects that target benefits or services to the poor and associated technical assistance. We also include some discussion of standalone technical assistance that has focused primarily on developing national targeting systems. We examine the extent to which this work has addressed some of the main constraints and challenges identified in section B above. A complementary overview of innovative analytical and advisory work on targeting issues will be provided through a series of companion background notes to be included as part of the overall regional study on targeting systems and practices.16 Some early insights from some of this analytical work are included in the discussion below. In order to emphasize the connections between this and the previous section, we will structure the discussion on the nature, relevance and impact of operational targeting work around the issues of (a) design, (b) implementation and monitoring, (c) and evaluation and impact. The World Bank could potentially have value added in one or more of these three areas, as follows: Build Innovate Evaluate Quality institutional and and learn Quality at and Quality strengthen from at entry impleme technical at exit existing targeting ntation capacity for systems systems targeting At the design stage, it could increase exposure to and in-country knowledge of international best and innovative practices, provide strategic technical advice to strengthen the architecture of existing systems, and help governments choose the right targeting approach given the country context or the nature of the intervention to be targeted. We can think of this as contributing to improve ―quality at entry‖. At the implementation stage, the World Bank could leverage knowledge, expertise and resources to build institutional and technical capacity for adequate administration and monitoring. 16 See footnote 2 for details on the notes commissioned. 42 This is not only true in those cases where operational work is framed within a broader government program, but applies to self-standing operational work as well where demonstration effects can be quite powerful. We can think of this as ―quality of implementation‖. Finally the World Bank could play an important role in what regards evaluation of existing systems by providing information on targeting effectiveness and supporting governments in extracting and acting upon lessons learned. We can think of this as ―quality at exit‖. The rest of this section is structured as follows. We first describe the methodological approach adopted for the identification and selection of surveyed projects and advisory work, as well as for the collection of project-specific information. We also highlight some of the limitations associated with the proposed approach. We then turn to the presentation and discussion of the results that emerge from the analysis. We initially focus on results relating to the quality and rigor of World Bank operational work with respect to targeting, following the framework described above. Next, we draw out the interactions of this operational work with and impact on government systems and the policy debate on targeting in the region. We find that the World Bank has clearly played an important role in terms of innovation and knowledge sharing. However, these efforts could be further strengthened through greater rigor and systematic learning across projects. As a result, these efforts have not always translated into improved in-country technical and implementation capacity or improved government targeting systems. As a consequence, the impact of this work on the policy debate on targeting, the performance of government systems and, ultimately, on targeting effectiveness has been limited, with a few exceptions. C.1 Methodological approach Selection and nature of surveyed projects Identification of relevant projects (i.e. including a targeting component) was undertaken in coordination with the regional management. Specifically we requested all sector managers to provide information on potentially eligible projects, as well as on current task team leaders, as a starting point for the analysis. A total of 25 projects were initially identified, out of which 21 were determined to be relevant for the purpose of this study17. Inclusion of individual projects was decided upon on the basis of the information provided by project teams regarding the relative importance of targeting-related work within the overall project. Out of 21 selected projects, we were able to gather information for 20, which then constitute the final sample for analysis (Annex 2 presents a complete list). Response rates varied across groups, with the final pool of potentially eligible projects containing mostly projects led by the Sustainable Development (especially the Agricultural and Rural Development group) and Human Development networks. As a result the projects analyzed here include mainly rural livelihood projects (e.g. the Andhra Pradesh District Poverty Initiatives project 17 This is not to imply that projects not included in the analysis do not contain an element of targeting. For instance, infrastructure and transport, urban housing, urban slum upgrading and other projects are also likely to have an element of targeting. In fact, most World Bank projects are targeted in one way or another, with the most common targeting approach being geographic targeting—i.e. project catchment is limited to areas with particular characteristics. The majority of the projects included in this report also have some element of household or individual targeting. 43 or the Gemi Diriya project in Sri Lanka), safety net projects (e.g. the Bangladesh National Social Protection project or the Support to Safety Net reform project in Pakistan), and projects aiming to increase access to basic social services (e.g. Health and Nutrition Population Sector program). Table 3 presents an overview of all surveyed projects and TA/advisory tasks. The distribution of projects across countries is also somewhat uneven, with a total of 10 projects in India, followed by 4 in Pakistan, 3 in Bangladesh, 3 in Sri Lanka, and 2 in Nepal. The one project in Afghanistan originally identified as potentially eligible has not been included in the final sample upon discussions with the TTL, while no relevant projects were identified in Bhutan or Maldives. In addition, though a few budget support operations were included in the initial list of potentially eligible projects, most of those considered for the final analysis were investment operations where the design or implementation of a targeting system for the distribution of project benefits constituted an important element of the overall project (i.e. 19 out of 22 projects considered are investment operations). Table 4: Overview of surveyed projects by country and type of project/advisory task (as of November 2008) Focus of project Technical Access to Total assistance to Country CDD/ basic social number of develop Livelihoods services Safety nets Other projects targeting tools Bangladesh 1 1 1 .. 3 1 India 6 2 .. 1 9 .. Nepal 1 1 .. .. 2 .. Pakistan 1 2 1 .. 4 1 Sri Lanka 2 .. .. .. 2 1 Total 11 6 2 1 20 3 Note: This list does not include the 4 projects identified as not relevant for this purpose and 1project no-response. See Annex 2 for a detailed list of projects and TA/advisory tasks. Finally, targeting is a relatively new element within World Bank operational work in the region. For instance, out of the 20 operations considered, 5 are repeat or second generation projects that incorporate a targeting component for the first time; the rest are new or first generation operations. Consequently a conscious decision was made to focus on more recent projects. As a result most of the projects considered for the analysis are either under preparation (7) or implementation (10), with only a few (3) having recently closed. This has implications in terms of the nature and amount of information we were able to collect, as discussed below. Selection and nature of surveyed advisory and technical assistance activities In recent years, the World Bank has also been requested by regional governments to provide technical assistance in developing targeting tools. Unlike those developed under the projects surveyed for this study, these standalone activities typically focus on developing targeting tools for government programs more broadly. In some cases, the tools are developed with a view of piloting through a particular government program, but the intention often is to rely on the demonstration 44 effect for wider adoption. The activities surveyed in this report include TA to develop PMTs for Bangladesh, Pakistan and Sri Lanka. A detailed overview of the World Bank-led development of targeting tools, including community-based methods, poverty maps and PMTs will be provided through companion background notes to this study. Collection of information on targeting-related work In order the collect project-specific information in a systematic manner, we developed a standardized questionnaire (Annex 3) to be completed by TTLs and other project team members. The questionnaire was circulated electronically and follow-up meetings were scheduled to review and supplement the information gathered in this manner. The questionnaire includes both multiple choice and open-ended questions and is divided in 4 sections: (i) basic project information, (ii) choice and design of targeting tool, (iii) development and implementation arrangements for targeting, and (iv) evaluation and impact of targeting tool. The last three sections focus on targeting-related issues and can be roughly mapped into the three areas identified above; namely, quality at entry, quality of implementation and quality at exit.18 In addition, respondents were given the opportunity to provide a brief assessment of the overall experience with targeting, with a focus on lessons learned, as well as asked to cite any relevant written references that could supplement the information provided through the questionnaire. These written inputs were followed by detailed discussions with the TTL. Limitations of proposed approach The combination of the responses obtained through the standardized questionnaires and the information gathered through follow up meetings has provided us with a good basis for analysis, especially in what regards questions related to quality at entry. However the fact that most of the projects considered are either under preparation or in the early stages of implementation implies that information on issues related to implementation and impact is significantly thinner. This is particularly true of the TA on developing PMTs, where the policy dialogue is at the nascent stage of design and piloting, rather than implementation. In addition, while Project Appraisal and other documents drafted during project preparation contain detailed information on the project design and intended implementation and monitoring arrangements, information on actual implementation is significantly sketchier and less systematic. We have heavily relied on direct inputs from TTLs and other team members to fill in the gaps, but still feel that the discussion on topics covered under sections (iii) and (iv) of the questionnaire is based on more anecdotal evidence. The presentation below takes this into account while trying to draw some conclusions and ideas for further discussion on all three areas. 18 More detailed information about the issues covered under each section included the following: (a) Basic project information: This section includes information on (i) project objective and description, (ii) implementation arrangements and duration and (iii) intended beneficiary population and/or estimated coverage. (b) Choice and design of targeting tool: This section includes information on (i) rationale for targeting, (ii) choice of targeting approach, (iii) relationship to (existing) government systems, and (iv) design arrangements. (c) Development and implementation arrangements for targeting: This section includes information on (i) piloting, (ii) scaling up, (iii) appeals and grievances, (iv) database management and overall monitoring, and (v) updates/recertification. (d) Evaluation and impact: This section includes information on (i) targeting effectiveness, and (ii) impact beyond realm of project. See Annex 3 for details. 45 Finally it must be taken into account that the project information discussed below was collected between July and December, 2008 and therefore reflects the situation at that point. It is possible for some of the project features to have changed since as lessons arising from project supervision and evaluation are incorporated into the project‘s design. Consequently, the discussion below should be taken as a whole and not as reflecting the merits or defects of specific projects or advisory tasks. C.2. Rationale for targeting in World Bank projects: To target or not to target The rationale for targeting appears to be similar across all surveyed projects; namely, the need or desire to effectively react a population of choice in order to maximize (project) impact. For this purpose, different projects use different means that allow for spatial, household or individual-level targeting, according to project objectives. Interestingly, however, not all projects that include a targeting component actually target benefits— at least not in the way suggested by the rationale above—especially at the household and individual level. Why are then resources invested in developing, and implementing, elaborate targeting mechanisms at the household or individual level? In most cases the results emanating from these exercises are being used for three complementary purposes. One reason is that many project teams view community-wealth ranking and similar participatory targeting exercises as a means to increase awareness about poverty and inclusion among community members. In addition, many projects aimed at promoting livelihoods or community-driven development include multiple components, including several that deliver community-level benefits rather than solely household-level benefits. In these cases, exercises to identify the poor may be undertaken but projects benefits may be provided to multiple and diverse actors. For instance, through ―geographic saturation‖ so that all households resident in project catchment areas are considered beneficiaries. Raising community awareness around poverty and inclusion-related issues and improving project monitoring in a participatory manner are very valuable goals. The question is whether instruments that have been primarily developed for the purpose of targeting are the most adequate for these purposes—especially taking into account that the information they generate is not being used for the purpose of targeting benefits or services. Experience with this kind of approaches and their use within World Bank projects is fairly recent and hence we do not have definite answers to these questions (see Box 7). Instead, these are some of the issues that will be tackled in more detail as part of the companion note on the experience of community-based targeting in the region. 46 Box 8: Participatory approaches to targeting: Concepts and operational challenges Participatory targeting (PT) methods have long been considered as a part of the standard participatory rural appraisal (PRA) tool kit used by researchers and field level development agencies. In the broadest sense, the technique seeks to allow communities to decide for themselves who they consider to be poor, based on their own assessment of the definition of ‗what it is to be poor‘. In this sense, participatory targeting is seen as more than simply a method of identifying the poor – it is a part of a process of developing local capacities for collective action and integrally linked to the broader concept of empowerment. Core principles include: (a) an understanding of poverty as complex and contextual; (b) a belief that ‗the community knows best‘; (c) greater ownership and accountability over the process of targeting if decentralized to the community level; and (d) participation is an empowering process in itself. Participatory targeting is currently in an experimental phase and remains limited to the project context in South Asia and, with very few exceptions, exists in parallel to more standardized methods of targeting. Currently, a number of different terminologies are in use to describe similar methods – including community wealth ranking (CWR) and participatory identification of the poor (PIP). The variances between these different ‗methods‘ tend to reflect the specific the project context, rather than any fundamental methodological differences. These methods are currently at an experimental stage and it is therefore not possible to talk in terms of a standard PT toolkit. PT methods are process and resource intensive yet can potentially yield significant and sustainable additional benefits that can assist communities overcome poverty. One of the major challenges in using these methods to target government programs at scale would depend on the extent to which government targeted interventions, including safety nets and social services, can be aligned with process intensive targeting methods. A second conceptual difficulty lies in the fact that PT methods essentially capture relative rather than absolute poverty within identified communities. This makes comparability across space especially problematic. While ―bottom-up‖ local information might correctly identify the poor at the community level, some framework would be required to marry this with ―top-down‖ allocations of fiscal resources across communities. In addition, the following operational challenges would need to be addressed with respect to scaling up PT methods (with concomitant reduction in process intensity) if they are to be used in government programs:  Defining community - Balancing the sociological necessity of cohesion with the operational compulsions of scale. The unit of the exercise should be small and socially cohesive, but this adds to the cost of the exercise and leads to an additional problem of aggregation of these locally collected lists to an operational/administrative unit level.  Mobilizing participation - The role of expectation of immediate benefits in a project setting as opposed to more diffuse expectations of future benefits as part of a targeting exercise delinked from a specific project.  Monitoring the quality of participation - Standardized and high quality facilitation to minimize dominance.  Defining poverty and categorizing the poor - Balancing the maxim of allowing communities to define characteristics of the poor with the operational need to ensure an acceptable degree of comparability across communities (even in a project-context). Typically projects have tended to adopt ‗guidelines‘ for the characterization of various poverty strata.  Institutionalizing the process – Setting up systems for verification of poverty lists and for regular updating of poverty status. Source: Powis (2009). 47 A common second rationale is that project teams need to assess the extent to which project benefits or services are reaching those at the bottom of the distribution, even in the absence of any explicit household-level targeting of the project. As a result, household-level targeting tools are necessary to develop this poverty profile of beneficiary households. This is important for projects or project benefits are, by design, less attractive to the poor. Two examples of very different projects illustrate this point. For example, the Bangladesh Health Nutrition and Population Sector Project uses the project-specific PMT to identify the profile of people using the services offered by the project, with a view to encouraging greater usage among the poorest. Similarly, the Pakistan Poverty Alleviation Fund (PPAF) uses information on poverty status to track the extent to which access to microcredit and other services offered by the project among extremely poor households is changing over time. It is well known that extremely poor households often have difficulty participating in such schemes given their very limited savings capacity. As a result, as part of the preparation of the PPAF 3, the project team is now considering the introduction of explicit participation quotas for extremely poor households in self-help or community groups as a pre-requisite for these groups to have access to project benefits and services, with the expectation that minimum quotas will help provide incentives for broader access and inclusion. Similar approaches are being considered by other CDD/livelihood project teams working in India and other countries. Third, some teams are exploring the possibility of using the information on poverty status to track changes in welfare levels among households in project areas by periodically repeating the community-wealth ranking or similar exercise. C.3 The quality of World Bank-led regional targeting work The surveyed projects can be classified into the following typology: (i) projects in support of existing targeted programs, (ii) projects in support of existing universal programs, and (iii) projects in support of new programs or interventions. Decisions regarding the choice and design of the targeting tool, as well as its implementation appear to be closely related to project type, as defined above. This section examines the quality and rigor of World Bank operational work at entry, during implementation and at exit. As mentioned earlier, there is far more systematic information on quality at entry than for subsequent stages. While we briefly indicate the potential significance of this work outside the project boundaries, a more detailed discussion of broader impact of this operational work on government systems is left for the next section. We find that the World Bank‘s operational work on targeting issues has introduced several innovations to the region. However, this contribution to knowledge sharing could be further strengthened by greater rigor during the design, implementation and monitoring of targeting tools. In addition, there is potential for greater learning across projects, and for government systems more generally, through improved documentation and systematic evaluation of targeting effectiveness. All too often the focus of experimentation, systematic improvement and documentation is the project design, with inadequate attention given to the targeting tool. It is equally important for teams to confirm their a priori expectations about the goodness and adequacy of a particular targeting approach with more systematic evidence. This can be done at the pilot stage by comparing the preferred approach with other viable alternatives (including existing government systems) or as part of the evaluation of project impact at a later stage. This practice would not only strengthen the quality of the technical advice on targeting provided by the World Bank in the context of operational work, but it would also create a more solid platform for building on lessons learned in the context of 48 specific projects to foster the broader policy dialogue on the architecture of targeting systems in the region. Quality at entry We focus here on rationale behind the choice and design of targeting tools for specific projects. As mentioned above, technically rigorous and high quality operational work at this stage can serve as a vehicle for innovation and to strengthen targeting architecture more broadly. We find that the technical work supporting decisions regarding the choice and design of selected targeting instruments by project teams could sometimes be strengthened and often better documented. Choice of targeting tool All but two of the methods described in section B (Box 1) have been applied for targeting purposes within World Bank projects (Table 5). There appears to be a weak mapping between the focus of the project (or the program it supports) and the choice of targeting approach. For instance, livelihood projects have favored the use of community-based targeting (often in combination with geographical targeting to limit the project catchment area), while safety net projects have relied on proxy-means testing. Projects that promote access to basic social services have used a wider variety of approaches, including proxy-means testing, geographical and categorical or demographic targeting—the last two often in combination. Table 5: Targeting approaches by sector and type of program Targeting approach Means- Proxy-means Community- Categorical Self- Geographical Focus of project testing testing based Demographic targeting CDD/Livelihoods1 .. 2 6 2 1 .. Access to basic .. .. .. 1 3 2 social services Safety nets .. 2 .. .. .. .. Other .. 1 .. .. .. .. All included 0 6 6 5 3 0 Note: 1\ In some projects a combination of targeting methods is used, e.g., geographic with categorical or community- based with categorical targeting. Why were these specific approaches chosen? In a few instances, project teams were requested to use existing (government) systems. These are mostly projects supporting existing targeted programs. For the most part, however, teams enjoyed significant freedom of choice. In these cases, ‗expected targeting effectiveness of the selected approach‘ was the main reason provided by project teams for their choices. Specifically, 14 out of 19 teams where the use of government systems was not mandated identified this as the primary reason for their choice (see Figure). This expectation appears to be based on the credibility of the chosen targeting method with government or communities. This 49 Box 9: Piloting before choosing a targeting method The Gemi Diriya project in Sri Lanka piloted three different targeting modalities during the first phase of implementation in order to determine which one would produce better targeting outcomes in the areas to be covered by the project. Although all three approaches built upon the notion of community-based targeting, their implementation details were distinct enough to generate significantly different results. Thus, the following three modalities were tested: (i) community-targeting, without facilitation; (ii) community-targeting with support from a local NGO; and (iii) community-targeting with external facilitation. The first and second approaches led to high levels of program capture by local elites, thus undermining targeting effectiveness and ultimately program impact. Consequently, the third approach was selected and adopted for the purpose of recertification within areas covered during the first phase of implementation, as well as targeting in new areas covered during subsequent phases. Taking this a step further, recent efforts by the World Bank and Samurdhi to incorporate contingency planning in the program include an empirical comparison of rankings from the community-based targeting system with household consumption levels. This study will help gauge the extent to which using a set of household characteristics could help improve the current targeting system. Similarly, a joint study by the Rural Development and Social Protection teams aims at systematically comparing the targeting effectiveness of proxy-means targeting (i.e. current BPL indicator and alternative PMT formulations) and community-based targeting in two states in India. The livelihoods project in one of the study states (Orissa) is currently exploring two possibilities: one, expanding the list of beneficiaries by community verification and subsequent changes to the BPL list; and two, using community-based targeting methods instead of the BPL system. expectation, however, does not appear to be founded in any systematic comparison between the selected approach and other alternatives, either prior to project implementation or during the pilot phase. Although piloting of the selected approach—itself a good practice—is a common feature across projects, it tends to focus on fine-tuning the proposed instrument rather than on informing the choice of targeting tool. Two interesting exceptions are described in Box 9. In addition, irrespective of project-specific considerations, it appears to be the case that in general the choice of a targeting instrument is to some extent endogenous to the skills of the project team. For instance, PREM and HD team are more likely to chose PMT than SD teams, while the latter are more likely to chose community-based methods. Alhtough these choices are to some extent driven by the type of intervention to be targeted, very few interventions go hand-in-hand with specific targeting instruments so that other considerations appear to play a role as well. The question then arises as to whether these choices seem appropriate in light of the discussion on various targeting approaches presented in section B.3. The answer is somewhat mixed. We argued above that community-based targeting was most appropriate where communities are clearly defined and cohesive, for programs that plan to include just a small portion of the population and for temporary or low-benefit programs that cannot support administrative structures of their own. These conditions, however, are often not met in those instances when community-based targeting has been chosen. For instance caste and other social divisions in rural India, where community- based targeting has been most frequently used, run contrary to the notion of homogenous and cohesive communities. As a consequence, some livelihood projects target entire communities, rather than households or individuals within communities. In this context, household-level targeting instruments may end up being used for purposes other than identifying beneficiaries (see previous section). 50 Similar issues arise with the use of proxy-means testing. We discussed earlier that this approach is most appropriately used when a country has reasonably high capacity, for programs meant to address chronic poverty in stable situations and when used to target more than one program so as to maximize the return for a fixed overhead. While it is true that most safety net projects aim to target the chronic poor, capacity remains a challenge and can seriously jeopardize targeting effectiveness, as argued in section B.3 above (see also Box 10). 51 Box 10: Proxy means testing in South Asia: Emerging interest and recent experience Proxy means testing is a fairly common targeting tool in several Latin American and some Eastern European countries. Colombia, Chile, Costa Rica and Mexico use PMT systems in education, health, food and nutrition interventions, and social insurance and assistance programs. In contrast, PMTs are relatively new in South Asia. India is the only country in the region with a core targeting method – the Below Poverty line (BPL) census that is somewhat akin to a PMT (the design is decided by consensus among a committee on pertinent proxies of poverty but not underpinned through rigorous analyses based on household surveys). Recently, there is increasing interest in the use of PMT for targeting safety nets and social services and the World Bank is supporting several governments in developing PMTs either in the context of specific World Bank projects or more broadly in the context of government programs. For instance, in Pakistan, the World Bank is supporting government efforts in developing a PMT-based national targeting system. This is currently being piloted, with the intention of targeting resources under the recently introduced large-scale cash transfer program, Benazir Income Support Program. In Sri Lanka, the World Bank had helped pilot a PMT for the Samurdhi program in 2004, though a community-based targeting system was later adopted. In Bangladesh, three separate PMTs have been developed by education, health and social protection projects in the context of targeting project benefits, with a possibility of adoption for relevant government programs. The following lessons emerge from this recent experience in developing and implementing PMTs in the region:  Collaboration matters to garner understanding and acceptance: There is interest in and knowledge of the technique is quite well known by now, local capacity to construct and understand PMTs is not. Collaborative development of these tools seems to be an essential ingredient.  PMTs need to be technically sound and defensible: The challenge of constructing a sound PMT is three- fold: (a) it needs to be solidly grounded to ensure that the final model captures the best possible proxies to predict poverty in rural and urban settings; (b) the indicators should be relatively easily verifiable and not easily manipulated ; (c) there should be ex-ante considerations for errors of exclusion and inclusion arising from the choice of indicators. This is a technically demanding task that also requires an in-depth understanding of country context.  Pilots are critical in improving the design of the PMT: This is an essential, but often omitted, step in finalizing the PMT and to ensure that risks of exclusion/inclusion are minimized. Pilots help to (a) recalibrate based on current data, since PMTs are constructed based on household surveys that are typically a few years earlier than the date of implementation; and (b) validate the PMT by testing against other indicators of household welfare.  Setting up systems for implementation: The use of PMT as a targeting tool requires institutional capacity for collection of household-level information and for subsequent database management. Governments or project teams may decide to collect information through a door-to-door census type exercise or through targeting centers (combined with outreach and communication strategies to minimize exclusion). Crucial elements for successful implementation include: (a) clear definitions of roles and responsibilities across administrative levels; (b) sufficient time for data collection by well-trained staff; (c) setting up good systems for data collection, management, monitoring, verification and redressal are crucial elements for successful implementation.  Evaluating the targeting effectiveness of PMTs ex-post is important for improving the targeting system. In the context of project-specific PMTs, this is critical for contributing to the policy debate on targeting.  Timing: PMTs are relatively insensitive to quick changes in welfare and require time to design, pilot and set up systems for implementation, monitoring and redressal. As a result, PMTs are best suited for targeting the chronic poor in stable situations and not as a response to crises. An exception was the earthquake livelihood support program in Pakistan which was a PMT-style approach but was defined to be a short- term benefit only. Source: Narayan and Yoshida (2005); Sharif (2009); Vishwanath et al (2009); note from Tara Vishwanath. 52 Perhaps it is the use of geographical and categorical targeting in the context of projects that promote access to basic social services that appears most in line with the criteria presented in section B. There we mentioned that both approaches were particularly useful when poverty or the outcome of choice was spatially concentrated (geographical targeting) or highly correlated with one or more household or individual characteristics (categorical or demographic targeting). Under most of these projects maps have been constructed for the purpose of selecting target areas using information on poverty and, especially, education, nutrition and health outcomes, while service provision within selected areas has been either universal or focused on clearly identifiable (vulnerable) groups (i.e. girls of primary-schooling age, scheduled and backward castes). This approach not only appears to be consistent with the stated objective of improved outcomes through wider access, but is also relatively low cost and effective in situations where capacity is limited. Having said this, it is clear that one size does not always fit all and hence general criteria such as the ones described in section B.3 have to be evaluated against the specific context in which a particular project would be implemented. Certain contexts, more than others, may require teams to take into account local social and political factors, in addition to technical ones. In fact, several project teams mentioned ‗credibility with the community‘ (11 out of 20) and ‗credibility with the government‘ (9 out of 20) as important reasons for their choice of targeting approach (see Figure X above). Credibility with the community appears to be of particular importance for CDD/livelihood projects, while credibility with the government is more frequently mentioned in the context of projects that support existing programs. Design of targeting tool Overwhelmingly, the design details of the selected targeting tool are taken care of by World Bank staff (17 out of 20 surveyed projects, in two of the three TA activities to develop PMTs), sometimes in collaboration with international and local consultants. Government agencies appear to play an explicit role in projects supporting existing targeted programs and hence relying on (or trying to improve upon) existing targeting tools. This approach may be entirely justified by the need to ensure adequate quality and technical rigor, especially in those cases where new methodologies are being introduced and hence little or no local expertise is available. It is important, however, to make certain that this does not happen at the expense of building technical capacity on the ground (more on this in section C.4). Quality of implementation and monitoring As we mentioned at the beginning of this section, the information collected from project teams on development and implementation arrangements and on evaluation and impact (to be dealt with below) is significantly patchier than that on design-related issues. This results from a combination of factors, including the fact that only 10 of the surveyed projects are actually under execution and the lack of systematic and detailed documentation on issues related to project implementation. Consequently the discussion under this and the next headings is significantly shorter and more anecdotal than the one presented above. We focus here on three specific issues:  Arrangements for collection of targeting data/information,  Arrangements for administration of targeting tool, and 53  Arrangements for monitoring of targeting tool, with a focus on appeals and grievances and recertification. Successful engagement on these aspects of implementation can directly build technical and administrative capacity for targeting in the country and/or serve as a powerful demonstration effect. We find a variety of arrangements being used across projects to tackle the issues mentioned above, which makes it hard to extract concrete lessons that can then be generalized and translated into recommendations. A few patterns emerge, however, and we comment on these below. Arrangements for collection of targeting data/information Three distinct arrangements seem to be in place when it comes to collecting the information necessary to develop and operationalize the selected targeting approach. Projects relying on household-level, quantitative data-intensive approaches, such as proxy-means targeting, generally outsource data collection to independent survey firms, which then operate under the supervision of the project team. Because data quality, consistency and comparability across space are important concerns in these cases, standardized questionnaires and other tools are developed and used for the purpose of data collection. Survey firms are sometimes assisted by local facilitators or organizations while in the field to ensure maximum outreach and minimize exclusion errors. In contrast, projects relying on community-based approaches tend to outsource the collection of information to local facilitators or NGOs directly. Because the process by which the information is obtained is as important as the information itself, adequate participation and facilitation are key factor for success, which is reflected in the amount of training being provided to those responsible for the process. Finally, projects relying on geographical and categorical or demographic targeting, often used in combination, tend to build upon existing data on poverty and other outcomes, as well as on information that can be easily collected by program administrators/government officials at the time of enrolment (e.g. a separate data collection stage that pre-dates the launching of the actual intervention is often not required). The first two approaches bypass government structures for the most part, at least in what regards actual data collection, although (local) government officials are often involved in the process as observers. They may also be asked to vet the targeting information once it has been processed. Government officials play a more active role when targeting relies on existing (government) systems (e.g. in rural India when the BPL is used) or when it is conducted simultaneously with enrolment. In these cases, eligibility is often determined on the basis of government-issued documentation, such as the BPL card or a disability certificate. Logistic arrangements for data collection are often piloted prior to scale up and changes to such arrangements and the instruments to be used for data collection are made on the basis of the lessons emerging from these pilots. For instance, instruments used for the PMT developed in the SHEP project in Nepal were modified based on findings from the pilot. Rarely, however, is targeting effectiveness of the proposed tool evaluated during the pilot phase, as discussed above. A notable exception is the introduction of the poverty score card (i.e. a PMT-based targeting tool developed with technical support from the World Bank) in Pakistan. This was developed initially as an integrated targeting system and later in the context of the recently launched Benazir Income Support Program (BISP) to deliver cash transfers to eligible households. This new targeting mechanism superseded the initial method of parliamentarians nominating beneficiary households for BISP. 54 Given the newness of the approach, a comprehensive test phase was designed to roll-out the poverty scorecard in 16 districts in May 2009. This would provide more information on administrative costs of administering this system and other logistic problems in data collection of this scale (through targeting centers rather than a door-to-door census). The pilot would also help in fine-tuning the cut-off threshold to determine eligibility. The targeting outcomes of this initial test phase will be evaluated at the end of six months and the national rollout is contingent on findings of this evaluation. Arrangements for administration of targeting tool, including beneficiary database Our focus here is mostly on beneficiary information and database management for targeting purposes, as distinct from overall program administration. Almost all projects reported the use of centralized databases and/or multiple connected databases and registers, which allow for both central and local information management and monitoring. However, no single model of level of responsibilities and the role of project teams applies to all projects. But a few elements appear to be common across several of them. In those cases when a new targeting tool is developed for the purpose of the project (i.e. the majority of the surveyed projects), the administration of the targeting management information system is more often than not the responsibility of the project team, in conjunction with the project implementation unit. Government agencies, NGOs and communities are sometimes given a role, but these are really just a few instances. In contrast, projects supporting existing targeted programs or using existing targeting tools, such as the BPL in India, tend to rely on government systems already in place. In both instances, however, there is typically little capacity built outside the confines of the project (more on this below). Arrangements for monitoring of targeting tool We pay special attention to two specific processes under this heading: reception and handling of appeals and grievances, and recertification of program/project beneficiaries. Of all arrangements reviewed so far as part of our discussion on quality of implementation, local and higher tier governments appear to play their largest role in the reception and (to a lesser extent) handling of targeting appeals and grievances, irrespective of project nature and focus. Their main responsibility is that of staffing and managing processing windows for complaints coming from NGOs, communities and other agents working closely with beneficiaries, while final decisions on inclusion and exclusion are often taken in coordination with (or under the supervision of) the project team and the project implementation unit. Unfortunately we were unable to collect detailed or meaningful information on the specific arrangements and capacity needs associated with these processes through the standardized questionnaires and follow up interviews and hence can say little about the impact that specific projects have had on existing capacity and structures. Most projects targeting households or individuals include provisions for the recertification of eligibility criteria at some point during the life of the project. However, given that most of the surveyed projects are either under preparation or in the early stages of implementation, actual recertification has taken place in very few instances. And when it has, it appears to have been poorly conducted and/or documented. Recertification instruments and implementation arrangements were not always designed in order to ensure full comparability with previous targeting efforts, and project teams were often unable to produce basic information on either the recertification process (e.g. 55 changes in participation rates and dynamics of community-based targeting exercises; potential learning about targeting criteria among potential beneficiaries) nor its main outcomes (e.g. changes in the size and composition of project beneficiary population). This is unfortunate for various reasons. First, significant (and probably unplanned) changes in project targeting can affect project implementation and undermine its overall effectiveness. Second, the task of evaluating project impact becomes harder in the absence of adequate documentation regarding changes in the project‘s target population. Quality at exit We focus here on the role of the World Bank in promoting a culture of evaluation by generating information and fostering discussion on the targeting effectiveness of existing systems. We find that efforts to evaluate targeting effectiveness within World Bank projects are at best limited, though the impact on the policy debate on targeting is greater than this would suggest partly due to high quality analytical and advisory work (more below). Evaluation of targeting effectiveness within World Bank projects As argued in section B when targeting effectiveness of government systems was reviewed, targeting effectiveness within World Bank projects (or any other targeted program) can be examined from two complementary angles: effectiveness of the selected targeting tool and effectiveness of the particular intervention, given a specific targeting tool. Assessing the effectiveness of the selected targeting tool involves answering the question ―Does the tool adequately identify the intended target population?‖, while assessing targeting effectiveness under a particular intervention involves answering the (broader) question ―Was the target population reached?‖ The answer to the fist question is primarily a function of the design of the targeting tool. For example, evaluating the targeting effectiveness of a proxy-means test would require a comparison between those who are classified as poor and non-poor when the test is applied with those who are actually poor and non- poor based on their consumption levels (or other relevant welfare metric). The smaller the inclusion and exclusion errors, the higher the targeting effectiveness of the proxy-means test. The answer to the second questions depends on the implementation and monitoring arrangements for both the targeting tool and the program itself. We already mentioned above that piloting of targeting instruments during project preparation or in the early stages of implementation is mostly used for the purpose of fine-tuning the selected tool, rather than to inform the choice of tools (by, for instance, comparing the targeting effectiveness, administrative requirements and financial costs of alternative approaches). The question then arises as to whether targeting effectiveness of the selected tool is evaluated later on the context of full- blown project implementation—say, for instance, as part of an overall project impact evaluation. The information collected from project teams suggests that little is done to this effect. 19 Not all projects are formally evaluated and very few include an assessment of the targeting effectiveness of the selected targeting instrument as one of the objectives of this exercise. The Nepal Poverty Alleviation Fund project is one of the few exceptions. A baseline household survey was collected prior to the launch of the project and a follow up survey has recently been fielded. Both surveys contain information that will allow the project team to evaluate, among other things, the design (and 19However, this paucity of information also reflects the fact that our sample of surveyed projects are relatively recent projects, with the majority under preparation or a few years into implementation. 56 implementation) of the targeting tool and, based on this analysis, to modify it as necessary in order to ensure maximum targeting effectiveness. The extent to which the target population (as identified through the selected targeting tool) has been reached seems to be more systematically monitored and documented. This is partly due to the fact that many projects include targeting-related monitoring indicators and outcomes as part of their results framework. In the absence of a proper assessment of the effectiveness of targeting tool, this information has to be interpreted carefully. Take for instance a project that supports a program targeting BPL households. The project team can document how many BPL households benefited from the project, but will not be able to and should not conclude from this information that the project has been successful in reaching the poor—at least not in the absence of an assessment of the BPL‘s effectiveness in correctly identifying the poor (which would most likely be outside the scope of the project, but could be incorporated into the design of the overall project evaluation). What the team will be able to do instead on the basis of these data is to determine whether program eligibility criteria were adequately applied and established procedures followed at the time of enrolment. And, if it emerges that they were not, the team could conduct a process audit or evaluation to better understand why this is the case and put in place the necessary corrective measures. The same reasoning applies to any project targeted using an instrument whose targeting effectiveness has not been evaluated. Unfortunately, we find that reaching the target population, as identified by the targeting tool, is more often than not equated to successfully reaching the ―true‖ poor. We also come across few instances where process audits or similar exercises were undertaken, particularly in what regards the implementation and monitoring of the targeting tool. C.4. Impact of World-Bank led regional targeting work To conclude the discussion on World Bank-led targeting work in the region, we focus here on the potential channels through which the World Bank‘s operational work could contribute to strengthening targeting systems in the region and to the policy debate on targeting. The discussion under the previous heading in this section suggests that much of the World Bank‘s engagement and impact as been at the design stage (i.e., quality at entry) and less so in implementation and monitoring (i.e., quality at implementation) and evaluation (i.e., quality at exit). We focus here on the following three specific channels of potential impact:  The role of the World Bank as a knowledge agent; and  The extent to which operational work on targeting issues has contributed to: o Strengthening the architecture of existing systems, and o Strengthening administrative and technical capacity We find that the World Bank has clearly played an important role in terms of innovation and knowledge sharing. However, these efforts have not always translated into improved in-country technical and implementation capacity or improved government targeting systems. As a consequence, the impact of this work on the policy debate on targeting, the performance of government systems and, ultimately, on targeting effectiveness has been limited, with a few exceptions. 57 Role as a vehicle for innovation and knowledge sharing Operational work on targeting issues has undoubtedly served as a vehicle for innovation and knowledge sharing in the region both on its own right and by building upon the results emanating from analytical work in this area. First, although not all programs are or should be targeted, the introduction of targeting components in World Bank projects has helped reinforce the notion that targeting can be an effective tool to increase transparency and accountability and maximize impact in the face of limited resources. This has been particularly true for projects supporting existing ‗untargeted‘ programs and new interventions. Arguing for and demonstrating the advantages of adequate targeting are important contributions for various reasons. Just to mention a couple; recent economic growth in the region has not always been accompanied by declines in poverty rates of the expected magnitude, and has often been coupled with increases in inequality. More inclusive growth will require, among other factors, adequately targeted interventions to support the poor and vulnerable. Similarly, crucial structural reforms in several sectors have been postponed for years for fear of serious political and social backlash in the absence of adequate compensation mechanisms. A more informed debate about the role that targeted interventions can play within broader reform packages can help break this deadlock. Well administered and effectively targeted programs assume an even greater significance during times of crisis, as now, when the quickest response often is to scale up existing programs. Second building on international best practices, a variety of new targeting approaches have been introduced in the region in the context of operational work—and accompanying technical assistance. This is the case, for example, with the use of community-based targeting for CDD/livelihood projects in India, or with technical support for the design of proxy-means tests for specific projects and/or for national targeting systems in Bangladesh, Nepal, Pakistan and Sri Lanka. In both instances, there existed no prior (minimum scale) in-country experience with either targeting approach, so that an opportunity was created to foster debate and build capacity. The same can be said about poverty mapping, an area in which the World Bank has played a leading role in the region (see Box 11 for more details on these innovations). Strengthening administrative and technical capacity within government Operational work in the context of projects and technical assistance can have potentially large impacts on administrative and technical capacity. This can be through initial exposure to international good practices and subsequent technical collaboration on the design, implementation and monitoring and evaluation of targeting tools. The extent of the impact on building capacity would depend on the mode of engagement and collaboration. Even without direct collaboration, particular approaches can serve as a powerful demonstration effect. In practice, a variety of different approaches have been used, with little or no capacity being created within government on the most part. 58 At the design stage, identifying and establishing strategic partnerships with counterpart agencies with potential technical capacities in order to encourage learning by doing (even if it is results in a longer preparation period), carefully documenting all steps of the design process, organizing knowledge events to discuss the chosen methodological approach and main results with government and other counterparts and, eventually disseminating this information more widely could go a long way in achieving this goal without compromising the quality and validity of the targeting tool. Unfortunately, we find little evidence of such systematic attempts to build capacity, although there are some good examples of closer collaboration. For instance, technical work to assist the Government of Sri Lanka in the development of a new proxy-means test for their main safety net program, the Samurdhi, was undertaken in close collaboration with the Welfare Benefit Board, a counterpart institution created by the government for this purpose. Although this approach was ultimately abandoned for political and other reasons, technical capacity building was an integral part of the task. Similarly, poverty maps for Bangladesh, India and Sri Lanka have been developed with Box 11: Poverty mapping Since the late 1990s researchers in the World Bank research department and in academia have been developing detailed ―poverty maps‖ in a growing number of developing countries. These maps provide estimates of poverty and inequality at the local level – such as the district, sub-district, and even village level. Such information is not commonly available because household surveys are typically too small in sample size to permit sufficiently fine disaggregation. Yet, with ongoing efforts to apply detailed spatial targeting of public interventions, or to realize the gains from decentralization and community-centered development, there is a pressing need for information on distributional outcomes at the local level. In particular, poverty maps can be an important part of a strategy to marry ―top down‖ targeting (essential for fiscal allocations of resources across space) with ―bottom-up‖ targeting of households or individuals (e.g., through community- based methods that rely on local knowledge but are not comparable across space). The poverty mapping method, originally introduced in Hentschel, et. al. (2000) and refined further in Elbers, et al (2002, 2003) – hereafter referred to as the ―ELL‖ approach - combines household survey data with unit record data from the population census in order to effectively expand availability of pertinent information. These poverty maps provide not only estimates of poverty or inequality levels at the local level. It is important to bear in mind that poverty mapping uses statistical techniques to circumvent the absence of statistically representative household data at a disaggregated level, which can introduce imputation errors. Procedures suggested in ELL can be applied to get a sense of the precision of the estimates. In South Asia, the World Bank has supported the governments of Bangladesh, India and Sri Lanka in producing poverty maps. Sri Lanka was the first country where estimates of poverty incidence were produced for sub-district administrative areas. In India, a pilot effort to produce poverty maps in three states - West Bengal, Orissa and Andhra Pradesh – is ongoing. In all three countries, the work has been carried out in a collaborative effort between government statistical agencies (the Bangladesh Bureau of Statistics, Registrar General of the Census in India, the Department of Census and Statistics in Sri Lanka), local research institutes (e.g., the India Development Foundation) and the World Bank. There appears to be strong country ownership of these poverty maps. For instance, government endorsed the results of the poverty maps for Bangladesh as robust and reliable at the upazila level. Source: World Bank (2010, 2009,d,e, 2007). technical expertise from the World Bank, but in partnership with government agencies (including the Census offices) and local research institutions. At the implementation stage, the discussion in the previous section suggests that, on the whole, little or no capacity is built within the government structure. Information collection processes largely 59 bypass government structure for the most part (especially when using project-specific targeting tools). An interesting exception is the collection of information in the test phase for the newly developed poverty score card for Pakistan, where one of the organizations involved in household Box 12: The impact of investments in database management on government targeting systems: the RSBY case RSBY program administration and monitoring relies heavily on a fairly sophisticated MIS platform that allows for (quasi) real-time information sharing between program administrators (i.e. government), service providers (i.e. public and private hospitals) and insurance companies. The program uses the BPL method to identify beneficiaries. Validation checks of the BPL administrative database reveals two critical problems in the BPL database that could seriously jeopardize program implementation. One, internal inconsistencies and gaps in the database; and two, lack of clarity on a master BPL list at the state-level given dynamic updating at local levels, with no corresponding mirror changes at the state-level. For instance, information updates (including those that have a bearing on eligibility) are handled at the local level, with no mirror changes being made to the master database kept at the state level and no clear delineation of responsibilities. Consequently, differences between the state and local-level registers are not only significant but bound to grow over time as more appeals and grievances are being processed, severely undermining the system‘s integrity, transparency and credibility. In response, the project developed a detailed data validation protocol (i.e., a series of tools and filters) to identify and minimize such problems and clarified institutional roles and responsibilities in this process. The project has also developed a detailed monitoring plan, with household survey based verification of administrative BPL data. This has had immediate impacts not just on targeting effectiveness in the project and program but also on the BPL system, which is used to allocate financial resources across states and to target most federal and some state level programs. First, the immediate impact has been an improvement in overall database management capacity of state government units in charge of the BPL administration as these tools and filters have gradually been incorporated as part of the BPL information management system. Second, this process has improved the quality of the BPL database. In some states, dissatisfaction with the quality of the BPL database (as revealed by this process) has prompted new BPL data collection. This process has also raised pertinent questions on institutional arrangements and the nature of centre-state relations with respect to targeting and the devolution of data administration (with respect to collection and management). data collection is the national statistical organization. Arrangements of the administration of beneficiary information and database management also typically build little capacity outside the projects. Project implementation units frequently operate as semi-parallel entities and, as a result, capacity built within these units is often lost upon project closure. Similarly information and MIS administrative structures are frequently taken as given by project teams working within existing government systems, while capacity building and implementation efforts are focused on other elements of the program. An interesting exception is the work on information management and database administration under the RSBY program in India (supported by the Social Security to the Unorganized Sector project – see Box 12). This case also provides an illustration of how project or program-specific investments in database management can be leveraged to have a positive impact on government targeting systems more generally (more on this in the next point). 60 Strengthening the architecture of existing government systems The World Bank‘s operational work on targeting issues can in theory contribute immensely to strengthening the architecture of existing country systems, both in terms of improved design and more efficient implementation and monitoring. By emphasizing evaluation of targeting tools, it can create a platform for building on lessons learned or to foster the broader policy dialogue on the architecture of targeting systems in the region. We ask the following questions to assess the impact of the World Bank‘s operational work on targeting systems in the region. To what extent has this work mitigated the problems of program fragmentation and poor targeting effectiveness common in government programs in the region? To what extent has this work contributed to the policy debate on targeting in the individual countries? We find that the impact beyond projects on broader government targeting systems and on the broader policy debate has been limited, with a few exceptions. As the discussion in section B made clear, most countries in the region do not have a coherent targeting system. Instead targeting is usually tackled at the program level, both in terms of design and implementation and monitoring arrangements, often leading to high levels of fragmentation, ineffective use of resources and ultimately poor targeting outcomes. Not to mention confusion among potential beneficiaries of various programs with apparently similar objectives (i.e. to provide income and other support the poor) but different beneficiary pools, and the subsequent negative impact on program perception. In close resemblance, most of the World Bank operational work on targeting has taken place at the program/project level. This is true even in India, where analytical work by the World Bank and others on the design and targeting effectiveness of the BPL has not translated into (a request from the government of India for) support to reform the system. In contrast, much of the standalone analytical and advisory work has focused on developing targeting tools that can be applied nationally. Whether this intention gets translated into genuine program- and country-wide strengthening of the targeting architecture remains to be seen; at least in one of the countries – Pakistan – the process seems to be on track. Promoting learning across projects and fostering harmonization of targeting methods The question is then whether World Bank involvement in the design and implementation of targeting systems, even if structured around specific projects and programs, has contributed to palliate some of the weaknesses and constraints described in section B and briefly mentioned above. Information collected from project teams suggests the picture is at best mixed, especially in what regards duplication and fragmentation. Some projects supporting existing targeted programs tend to adopt and at times improve upon the systems in use. A notable example is the significant impact on the targeting effectiveness of the BPL system through technical assistance accompanying the RSBY project (see Box 12 above). However, much operational work on targeting relates to existing untargeted programs or new interventions. In these cases we find numerous instances in which choices made by World Bank teams can be seen as feeding into, rather than mitigating system fragmentation. For example, in Nepal the World Bank is currently financing (at least) three different activities whose main objective is to identify and provide support to the poor. The first one uses proxy-means testing for this purpose (to be developed using project resources), the second one uses a mix of geographical and 61 categorical targeting (again designed as part of the project), and the third one provides technical assistance for the construction of a socio-economic index to be used by the government to allocate fiscal resources in a pro-poor fashion. Although there may be valuable reasons that justify the choice of different approaches in these three cases (even if the ultimate target population is the same), there appears to be little ex ante (design) or ex post (validation) coordination across all three activities. The case in Bangladesh is similar, with three different PMTs being designed as part of three different projects. Given existing resource and capacity constraints in the country and in the absence of an integrated targeting system, an effort by World Bank teams to share lessons learned and to achieve some level of harmonization in targeting approaches could have high pay-offs in terms of increased savings and overall effectiveness. This can be done as illustrated by recent experiences in Pakistan, where teams working on projects containing a targeting component have managed to combine the work that had been done by each team independently in the form of a single national targeting instrument to be used across projects and government programs. More importantly, different targeting approaches had been initially selected (i.e. community-based targeting and proxy-means testing) forcing the teams to creatively revisit the work that had already been done in order to converge upon a common instrument. Contribution to a broader policy debate on targeting This World Bank‘s operational work has, on the whole, played a limited role in fostering discussion on the targeting effectiveness of existing systems, either by evaluating existing systems and projects with a view to strengthening the existing system or exploring the adoption of alternative approaches based on learning from projects. The experience of BPL reform in India is illustrative in this case. There are numerous problems in India‘s BPL targeting system (see Box 3 above) and there is a lively debate in India on the design of the BPL. The World Bank has made important contributions to this debate through analytical work on the targeting effectiveness of the BPL system (see Jalan and Murgai 2008, World Bank 2010). Recommendations for a substantially modified system have recently been developed by government and will undergo piloting over the coming months (Mehtrotra and Mander 2009). There are also important implementation problems in this system. Like most government programs, the BPL system does not adequately address the issue of monitoring, management and dynamic updating of beneficiary databases (see Box 12 above). In this regard, it is possible that the adoption and showcasing of good information processing and management practices under World Bank projects could potentially have a positive impact on existing country systems through demonstration effects. These shortcomings of the BPL system have led to various project teams deciding to adopt alternative approaches for the purpose of targeting benefits under their specific projects. This is particularly true for those projects that support new interventions. Although this decision may be justified on the basis of increased targeting effectiveness and other reasons (see discussion in Section C above), little has been done to build upon the experience of these projects to inform the broader debate on the reform of the BPL system. This is possible as illustrated by the Andhra Pradesh experience and, possibly the Gemi Diriya project in Sri Lanka, where positive results with community-based targeting have led the state government to use this information to target some of their own programs. 62 D. CONCLUDING REMARKS Recent years have been witness to a renewed focus on the need to adequately target benefits to the intended population, both among countries in the region and within the World Bank. There remains significant potential to deepen the relevance and impact of targeting operational work by the World Bank in South Asia. The current context is particularly timely with several governments grappling with problems of improving targeting outcomes and introducing new or scaling up existing programs in response to the food, fuel and financial crises. In several countries, there is a renewed focus on targeting and a lively debate around the introduction of new methods and systems (e.g., in Pakistan) or the reform of existing systems (e.g., in India). This policy note critically reviewed regional experiences in the area of targeting, both in the context of government systems and the World Bank‘s operational work and assessed the impact of the latter on country targeting systems, on the basis of this diagnosis. Based on this analysis, the note extracts some lessons that can be used to inform the way forward. We find that targeting systems in all countries suffer from common ailments, including high levels of fragmentation and duplication. In this respect, South Asian countries differ markedly from many countries in Eastern Europe, Central Asia and Latin America, where national targeting systems are common. With the exception of India, none of the South Asian countries have an integrated targeting system or tool. In South Asia, this absence of an integrated targeting architecture implies a multiplicity of approaches being used simultaneously across programs. This often leads to excessive administrative costs, and poor targeting outcomes both at the country and program-levels. These problems can be largely attributed to faulty design and poor implementation and monitoring. Designing, administering and monitoring an effective targeting system is a complex and often costly task, but one that can produce important benefits in the medium-term. In this context, there is significant potential for the World Bank to add value and support countries in strengthening and improving targeting systems and outcomes. At the design stage, it can increase promote innovation and provide support to strengthen the architecture of existing systems, thereby contributing to improve ―quality at entry‖. At the implementation stage, the World Bank can contribute to ―quality of implementation‖ by leveraging knowledge, expertise and resources to build institutional and technical capacity for adequate administration and monitoring. Finally, by fostering a culture of evaluation and supporting governments in extracting and acting upon lessons learned, the World Bank can contribute to improved ―quality at exit‖. However, the analysis of targeting-related work under 20 projects (and some insights from standalone advisory activities) suggests that much of the World Bank‘s engagement and impact as been at the design stage and less so in implementation and monitoring and evaluation. The World Bank has indeed played an important role in terms of innovation and knowledge sharing in the region. However, the problems of fragmentation and program- or project-specific approaches are apparent in the operational work by the World Bank on targeting in the region. World Bank-led targeting operational work has sometimes lacked the necessary rigor and has often failed to translate into tangible improvements in overall country systems or in-country technical and implementation capacity. As a consequence, the impact of this work on the policy debate on targeting, the performance of government systems and, ultimately, on targeting effectiveness has been limited, with a few exceptions. 63 The World Bank‘s contribution to improved quality at entry could be further strengthened by greater rigor during the design, implementation and monitoring of targeting tools. In addition, there is potential for greater learning across projects, and for country systems more generally, through improved documentation and systematic evaluation of targeting effectiveness. All too often the focus of experimentation, systematic improvement and documentation is the project design, with inadequate attention given to the targeting tool. There is a need for renewed emphasis on targeting in World Bank projects. This could take the form of greater rigor during design of the targeting tool (e.g., through more experimentation), better documentation of processes (e.g., the choice of the targeting tool and any subsequent changes), and improved monitoring and evaluation of the targeting tool. An effort by World Bank teams to share lessons learned and to achieve some level of harmonization in targeting approaches could have high pay-offs in terms of increased savings and overall effectiveness. It is also critically important to build technical and administrative capacity locally and within government systems at the design, implementation and monitoring and evaluation stage. This could be through initial exposure to international good practices and subsequent technical collaboration on the design, implementation and monitoring and evaluation of targeting tools. The extent of the impact on building capacity would depend on the mode of engagement and collaboration during operational work. Even without direct collaboration, particular approaches can create a powerful demonstration effect. International evidence suggests that a large fraction of the observed differences in targeting effectiveness across systems and programs can be attributed to factors related to implementation and monitoring. This implies that investments aimed at correcting resource, capacity and logistic limitations in government systems could go a long way in improving targeting outcomes in the region. These practices would not only strengthen the quality of the technical advice on targeting provided by the World Bank in the context of operational work, but would also create a more solid platform for building on lessons learned in the context of specific projects to foster the broader policy dialogue on the architecture of targeting systems in the region. 64 REFERENCES Ahmed, S. (2004) Delivery Mechanism for Cash Transfer Programs to the Poor in Bangladesh. Social Protection Discussion Paper Series 0520, World Bank. Ahmed, S. S 2007. ―Social Safety Nets in Bangladesh‖ Background Paper for Bangladesh Poverty Assessment. Manuscript. World Bank, Washington DC ANDS (2007) Afghanistan National Development Strategy: 2008 - 2013. Government of the Islamic Republic of Afghanistan, Kabul. Arif, G. M. (2006) Targeting Efficiency of Poverty Reduction Programs in Pakistan. Working Paper No. 4 Asian Development Bank Pakistan Resident Mission, Islamabad. BRAC(2004) Stories of Targeting: Process Documentation of Selecting the Ultra poor for CFPR/TUP Program. CFPR Working paper Series No. 1. BRAC, Dhaka. Castañeda and Kathy Lindert (2005). Designing and Implementing Household Targeting Systems: Lessons from Latin American and The United States, Social Protection Discussion Paper No. 0526. Coady, David, Margaret Grosh and John Hoddinott (2004). Targeting of Transfers in Developing Countries: Review of Lessons and Experience. World Bank, Washington DC. Dreze, J. and R. Khera (2009). The Battle for Employment Guarantee. Frontline. January 16. Elbers, et al (2002, 2003) Grosh, M., C. del Ninno, E. Tesliuc and A. Ouerghi (2008). For Protection And Promotion: The Design And Implementation Of Effective Safety Nets. World Bank, Washington DC. Hentschel, et. al. (2000) Jalan, J. and R. Murgai (2008) ―An effective ―targeting shortcut‖? Analysis of the BPL scheme in reaching the poor.‖ Manuscript, The World Bank, New Delhi. Mehrotra, S. (2008) NREG Two Years On: Where Do We Go from Here?, Economic and Political Weekly, August 2. Mehtrotra, S. and Mander, H. (2009). How to identify the poor: A proposal, Economic and Political Weekly, Vol. XLIV (19), May 9. Narayan, Ambar and Nobuo Yoshida (2005). Proxy Means Test for Targeting Welfare Benefits in Sri Lanka, Report No. SASPR-7, World Bank. Powis, B. (2009). Participatory Approaches to Targeting: Concepts and Operational Challenges, Background paper for Regional Study on Targeting Systems and Practices. 65 Shahnaz (2008) Targeting of Social Programs in Pakistan: An Evaluation. Manuscript, World Bank, Washington DC. Sharif, I. (2009). Building a Targeting System for Bangladesh based on Proxy Means Testing. Manuscript. World Bank, Washington DC. Sundaram, K. (2003) ―On Identification of Households Below Poverty Line in BPL Census 2002: Some Comments on the Proposed Methodology‖, Economic and Political Weekly, March 1. UNICEF (forthcoming) Participatory Method for Mapping of Disadvantaged Groups (DAG) and its Application in DACAW Districts: A Summary Report. UNICEF, Kathmandu. Vishwanath, T. and A. Narayan (2006) Livelihood Support Cash Grants in the aftermath of disasters: Sri Lanka and Pakistan. Manuscript, World Bank. Washington DC. Vishwanath, T. , X. Hou and N. Yoshida (2009). Poverty Scorecard for Pakistan: A Recommended Approach for Targeting the Poor. Manuscript. World Bank, Washington DC. World Bank (2010) Social Protection for a Changing India. World Bank, Washington DC. World Bank (2009b) Targeting Systems for Safety Nets in Nepal: A Review. World Bank, Washington DC. World Bank (2009c). Social Protection in Nepal. January 20, 2009. Human Development Unit, South Asia Region: the World Bank World Bank (2009d). Perspectives on Poverty in India: Stylized Facts from Survey Data. PREM, South Asia Region: World Bank. World Bank (2008) Bhutan: Human Development Strategy Report. World Bank, Washington DC. World Bank (2007) Social Protection in Pakistan: Managing Household Risks and Vulnerability. World Bank, Washington DC. World Bank (2006a) Social Safety Nets in Bangladesh: An Assessment. Bangladesh Development Series Paper No. 9. World Bank, Dhaka. World Bank (2006b) Social Protection in the Maldives: Options for Reforming Pensions and Safety Nets. World Bank, Washington DC. World Bank (2006c) Sri Lanka: Strengthening Social Protection. Part I. World Bank, Washington. World Bank (2005a) Afghanistan: Poverty, Vulnerability and Social Protection – An Initial Assessment. World Bank, Washington DC. World Bank (2005b) Targeting Resources for the Poor in Bangladesh. Bangladesh Development Series Paper No. 5. World Bank, Washington DC. 66 ANNEX 1: TARGETED PROGRAMS AND TARGETING SYSTEMS IN SOUTH ASIA The Targeting of Social Safety Programs in South Asia: A selected annotated bibliography Notes: This annotated bibliography focuses largely on overview papers, academic studies and World Bank reports on targeting of safety net programs in South Asia. Recent resources (post 2000) that cover the larger national safety net programs, especially those referred to in detail in Section B of this policy note, are given priority. This is by no means an exhaustive compendium of the South-Asia literature on safety nets or on targeting, and aims to direct the reader to more detailed documents that elucidate some of the points referred to in the text. Many of the documents reviewed have a broader focus than targeting alone. We have therefore been selective in picking out the key messages vis-à-vis targeting in our descriptions of the various studies and reports included. The bibliography is organized by country (in the following order: Afghanistan, Bangladesh, India, Maldives, Nepal, Pakistan, Sri Lanka). We begin with reports that comment on safety net programs and targeting for the country as a whole, before moving on to the program-specific literature. The final section consists of general resources on targeting more broadly. AFGHANISTAN 1. ANDS (2007) Afghanistan National Development Strategy: 2008 - 2013. Government of the Islamic Republic of Afghanistan, Kabul. This is a broad-based report outlining the development priorities of Afghanistan over a period of time and addresses several sectors and crosscutting issues. The report provides a poverty profile and examines the macroeconomic framework of Afghanistan. It articulates priorities related to security, governance and economic and social development. Social protection, mainly comprising a few categorically targeted pensions, is discussed under the latter (albeit briefly) and the chief areas accorded priority include: strengthening the pension system, strengthening disaster preparedness and response, improving coordination between government agencies and donor bodies. Goals set out include: the following major outcomes: (i) poverty and vulnerability reduction; (ii) improved social inclusion: (iii) lower infant and maternal mortality; (iv) reduction in harmful child labor; (v) reduction in drug demand within the country; (vi) improved employment; (vii) reduction in vulnerability to natural disasters; (viii) improved aid effectiveness. Fiscally sound and well-targeted social protection interventions are identified as being of critical importance for improving poverty outcomes. 2. World Bank (2005) Afghanistan: Poverty, Vulnerability and Social Protection – An Initial Assessment. World Bank, Washington DC. This report on poverty, vulnerability and social protection seeks to provide a first comprehensive study o f the situation facing the people of Afghanistan, highlighting the relationship between poverty, risk and vulnerability. The report draws extensively on the Nationwide Risk and Vulnerability Assessment (NRVA), which was carried out in July-September 2003 under the umbrella of the Ministry of Rural Rehabilitation and Development (MRRD) with collaboration from other ministries, and with assistance from several international agencies. Beginning with a short historical overview of poverty and vulnerability, the report focuses on livelihoods, poverty and vulnerability in rural and urban Afghanistan separately. The final chapter of the report sets out the policy directions for poverty reduction, and suggests that the essential path for poverty reduction is sustained, equitable, broad-based economic growth, with the State playing the role of facilitator, regular and financier of high-return infrastructure and services, but leaving provision largely in the hands of the private sector, communities and NGOs. BANGLADESH 67 1. World Bank (2006a) Social Safety Nets in Bangladesh: An Assessment. Bangladesh Development Series Paper No. 9. World Bank, Dhaka. This report provides a comprehensive overview of the poverty and vulnerability situation in Bangladesh and provides an overview of the existing safety nets systems. While assessing the impacts of existing safety nets systems, the report highlights key challenges as well as proposes a number of solutions; pertaining to both design and management aspects. Issues of coverage and targeting are separately highlighted, as are the challenges of natural disasters and the role of microcredit. 2. World Bank (2005) Targeting Resources for the Poor in Bangladesh. Bangladesh Development Series Paper No. 5. World Bank, Washington DC. While commenting on the current state of inequitable health outcomes in Bangladesh, this report reviews a number of poverty alleviation programs in the public and NGO sectors, examines the methods and techniques used for identifying and targeting the poor, and defines the strengths and weaknesses of these programs. This study relied primarily on secondary information to document the government's and NGOs' current health sector resource allocation. In addition the study team reviewed the tools and methods used by NGOs to identify and target the poor. The study provides an overview of geographic targeting of districts and sub-districts through human development and human poverty indices. Relying on international best practice literature to identify the poor through a PMT, and applying the Bangladesh Bureau of Statistics' Household, Income and Expenditure survey (HIES) 1999-2000 data, a PMT model for Bangladesh has been developed for this study. The model has been field tested with a household sample to assess its stability and robustness. Also presented in this report are policy prescriptions to improve resource allocation, as well as to introduce and implement the PMT formula in order to deliver health subsidies to the poor. The paper comments on the need for greater coordination of programs targeting the poor, as well as more improved and objective methods for spatial resource allocation. 3. Sharif, I. (2009) Building a targeting System for Bangladesh based on Proxy Means Testing. World Bank, Washington DC. This paper comments on the current state of targeting of social safety net programs in Bangladesh before highlighting the key features of targeting using a Proxy Means Test Formula to identify program beneficiaries. Using the latest Bangladesh Household Income and Expenditure Survey (HIES) 2005, the author goes on to discuss the various steps taken to derive the Proxy Means Tests Formula (PMTF) for Bangladesh. Discussions regarding the various checks and balances undertaken to identify the best possible PMTF as well as recommendations on the choice of the cut-off line when determining household eligibility status are included. Comparisons between the targeting accuracy of the PMT model and existing programs are also discussed. The author also highlights key implementation challenges associated with using a PMT- based targeting system in the Bangladeshi context. 4. Ahmed, S. (2004) Delivery Mechanism for Cash Transfer Programs to the Poor in Bangladesh. Social Protection Discussion Paper Series 0520, World Bank. This paper compares the delivery mechanisms of three cash transfer programs in Bangladesh; the Income Generation Vulnerable Group Development (IGVGD), the Primary Education Stipend Project (PESP) and the Rural Maintenance Program (RMP) in terms of practical issues and financial costs. A combination of primary and secondary sources of quantitative and qualitative data is employed to answer questions pertaining to delivery, leakage and targeting. Leakage due to selection failure is high in the case of the PESP while it is low in the other two programs. 5. Matin, I and Hulme, D. (2003) Programs for the Poorest: Learning from the IGVGD Program in Bangladesh. World Development, 2003, vol. 31, Issue 3; Pages 647 – 665. This article examines a program that seeks to reach Bangladesh‘s ―hardcore poor‖ by combining elements of livelihood protection (food aid) with livelihood promotion (skills training and microfinance). Bangladesh Rural Advancement Committee‘s Income Generation for Vulnerable Group Development Program has deepened the outreach of its poverty- reduction activity and achieved impressive results. Detailed local-level fieldwork revealed, however, that program practice differed markedly from program plans. This is found to have important implications for both future program design and the understanding of ―who‖ does not benefit from such innovative programs. The authors conclude that while such programs, mixing livelihood protection and promotion, 68 should be a major focus for anti-poverty strategies there will remain a role for more traditional social welfare schemes. 6. Hossain, N. (2007) The politics of what works: the case of the Vulnerable Group Development Program in Bangladesh. Chronic Poverty Research Centre Working Paper 92. CPRC, Dhaka. This paper explores the political dimensions of the achievements of the Vulnerable Group Development (VGD) program in Bangladesh. The paper is based on a review of the program literature, stakeholder interviews, and on other recent empirical research into the politics of poverty in rural Bangladesh. Contrary to much scholarly discourse, this paper asserts that the VGD works in large part because the program works with existing political interests and imperatives, rather than attempting to work against them. The paper presents evidence and analysis to show that the enduring successes of the VGD program have had clear political dimensions. One reason program resources reach the poorest reasonably effectively is that the program responds to the need for local political elites to continually rebuild their political capital through support for the vulnerable poorest within their communities. At the same time, the local nature of the final selection process leaves it open to some scrutiny and pressure from program partners and community members. It is, in brief, a case in which the imperatives and pressures of local political competition have been supportive of pro-poor outcomes. 7. Webb, P., J. Coates and R. Houser (2002) Does Microcredit Meet The Needs of all Poor Women? Constraints to Participation Among Destitute Women in Bangladesh. Food Policy and Applied Nutrition Program, Discussion Paper No. 3. Tufts University, Boston. Available at: http://nutrition.tufts.edu/docs/pdf/fpan/wp03-microcredit.pdf. This paper seeks to explore the factors that determine the extent of participation among eligible beneficiaries in the Income Generation for Vulnerable Group Development program in Bangladesh. Based on the findings of a 606 households‘ survey, the study confirmed that while many aspects of the program are very valuable to ultra-poor women, the full package of developmental inputs on offer through IGVGD is not taken up by a majority of the programs participants, and many join without full participation in mind. Curtailed participation stems largely from disconnect between ultra-poor women‘s personal motivations, on the one hand, and constraints to participation and the expectations of program planners, on the other. These findings suggest that the needs and goals of ultra poor women may not be appropriately served by the same institutional system that is used to address the needs of women who are merely ‗poor‘. Program modifications are needed to accommodate the unique constraints of ultra-poor women while enabling them to set their sights higher and achieve more in the process. 8. BRAC(2004) Stories of Targeting: Process Documentation of Selecting the Ultra poor for CFPR/TUP Program. CFPR Working paper Series No. 1. BRAC, Dhaka. This paper provides a detailed process documentation of the community-based targeting system used in BRAC‘s ―Challenging the Frontiers of Poverty Reduction – Targeting the Ultra Poor‖ program. Using qualitative research methodology, the authors document the various stages of the targeting process; which include rapport building, Participatory Rural Appraisal (PRA) meeting, Survey and Preliminary Selection and Final Selection, identifying trends and patterns at each stage, and supplementing with valuable first-hand accounts of some of the implementation challenges faced in the field such as positioning, community perceptions and expectations of benefits. The report highlights the importance of establishing a good relationship between the project staff and the communities and could serve as a useful tool for developing similar community-based targeting practice modules. 9. Florent Feulefack, J. and M. Zeller (2005) How accurate is Participatory Wealth Ranking (PWR) in Targeting the Poor? A Case Study from Bangladesh. Conference on International Agricultural Research for Development. Stuttgart-Hohenheim, October 11-13, 2005. This paper assesses the accuracy of Participatory Wealth Rankings (PWR) in predicting absolute poverty, and compares PWR with three other poverty assessment methods. Using a village census in 8 villages located in three of the six divisions of Bangladesh, 1660 households have been scored using the PWR method. A randomly selected subsample of 320 households was interviewed with an LSMS-type questionnaire. The data allow for identification of households that have per- 69 capita expenditures below the international poverty line of 1 dollar a day. Our results show that calibrated PWR scores can achieve an accuracy of 70 to 79 percent, i.e. up to 8 out of 10 households are correctly predicted as to whether they live in extreme poverty or not. As expected, the so-called Total Accuracy of PWR is higher if its scores are calibrated at lower geographical level, and highest if calibrated at the community level. For the case of Bangladesh, the results confirm the accuracy of PWR as a poverty targeting method for development policies and projects if used at the community level. 10. Tietjen, K. (2003) The Bangladesh Primary Education Stipend Project: A Descriptive Analysis. Partnership for Sustainable Strategies for Girls‘ Education. Washington DC. This report primarily provides descriptive and anecdotal information about Bangladesh‘s Primary Education Stipend Program (PESP), drawing heavily from interviews with the national PESP director and headquarters staff, interaction with key actors in the education sector in Dhaka and on reviews of existing literature and program documentation. The author finds that the targeting methodology does not appear sufficiently well defined to ensure that the poorest families in Bangladesh benefit, but only the moderately poor. Costs associated with admission to primary school may prove a barrier to poor families, placing them outside consideration for PESP. Weak management capacity and corruption may distort the pupil selection process. Additional resources: 11. Ahmed, A. U. (2004) Impact of Feeding Children in School: Evidence from Bangladesh. International Food Policy Research Institute (IFPRI): Washington DC. INDIA 1. World Bank (2010) Social Protection for a Changing India. World Bank, Washington DC. This is a comprehensive report on the social protection debate and challenges in India, key safety net programs and their operational and implementation challenges. The report analyses the patterns of fiscal allocations for Social Protection programs in India, the implementation arrangements at various levels and presents micro- level evidence on the performance of various programs. It includes a detailed study of the Below Poverty Line Census system in India, which is the national targeting system used for a number of federal and state- level government assistance programs. It highlights key issues in the reform of the current BPL system and comments on the need for a possible combination of targeting methods. The targeting performance of specific programs is highlighted and can be summarized thus:  BPL Indicator: The BPL indicator is poorly designed and implemented leading to substantial inclusion errors and worrying exclusion among the poorest, albeit overall BPL card-holding appears to be progressive. Targeting effectiveness of the BPL is further compromised by poor program implementation and high leakage rates although there is significant variation in targeting outcomes across programs  PDS: Large diversions of grain, to the extent of 58% are estimated. Considerable leakage of subsidies to the non-poor.  NOAPS: Survey-based estimates coverage rates imply under-coverage. Targeting generally progressive, good performance among SC/ST groups and among the poorest quintile.  Targeted Rural Housing: Large grants under the scheme make it susceptible to capture and leakage as reported amounts received are lower than program-sanctioned transfers. Targeting of rural housing is mildly progressive.  SGRY: Considerable inclusion of the richest quintile among beneficiaries, albeit progressive on the whole, with especially good coverage among ST households and in the lowest quintile.  SGSY: A variety of sources confirm generally low coverage. Progressive coverage rates are more than offset in terms of benefit incidence by the much higher benefits reported by the rich.  School Stipends: Good coverage nationally, with strong inter-state variation. Good performance among SC/ST groups and girls.  Mid Day Meals: Positive impacts on school enrollments reported for girls and lower caste groups. 70 2. Bardhan, P. and D. Mookherjee (n.d.) Decentralizing Anti-Poverty Program Delivery in Developing Countries. The authors study effects on accountability in government service delivery of decentralizing administration of an antipoverty program. While governments at both central and local levels are vulnerable to anti-poor policy biases owing to political capture, centralized delivery systems are additionally prone to bureaucratic corruption owing to problems in monitoring bureaucratic performance. Decentralizing the delivery system promotes cost-effectiveness and improves intra-regional targeting at low program scales. But inter-regional targeting may deteriorate, as central grants to high poverty regions shrink, owing to high capture of local governments by local elites in such regions. 3. Ajwad, M. I. (2006) Coverage, Incidence and Adequacy of Safety Net Programs in India. South Asia Human Development Group, The World Bank, New Delhi. This paper assesses the effectiveness of government safety net programs in all states and Union Territories of India. In this assessment of safety net programs, the author focuses on the following criteria: i) coverage, ii) targeting efficiency; and iii) adequacy and potential impact on household welfare. The primary source of data is the Human Development Profile of India (HDPI) – II 2004-05 household survey collected by the National Council of Applied Economic Research (NCAER). The data canvassed around 41,500 households and 216,000 individuals in India. The data are representative at the state level as well as for urban and rural areas separately. Key results: (i) Overall coverage rates of safety net programs in India are low. Less than 5 percent of all households benefit from 10 safety net programs (out of 14 investigated here), and only 2 safety net programs cover more than a fifth of all households. (ii) Benefit incidence to the poorest households does vary with the type of safety net program. Of 14 safety net programs, 11 benefit households in the poorest quintile more than the overall coverage in India. Only health and life insurance and Kisan card coverage for the poorest quintile is smaller than the overall coverage rate for India as a whole. 4. Nayak, R., N.C. Saxena and J. Farrington (2002) Reaching the Poor: The Influence of Policy and Administrative Processes on the Implementation of Government Poverty Schemes in India. Working Paper 174, Overseas Development Institute, London. The study focuses on factors affecting the implementation of key social protection schemes in India. Specifically four broad types of poverty reduction schemes are examined: (a) those aiming to achieve income transfer to the poorest (including the National Old Age Pension Scheme and the National Housing Scheme); (b) those aiming to enhance the quality of infrastructure, particularly in relation to natural resources (including the Accelerated Rural Water Supply Program and the Drought Prone Areas Program); (c) those having employment creation as a major objective (including the aiming to enhance self-employment possibilities (including the Integrated Rural Development Program and the National Credit Fund for Women). The paper first provides a broad overview of these central (federal government sponsored) schemes, and thereafter examines policy processes and implementation in two states in the country. 5. Mahendra Dev, S., K Subbarao, S. Galab and C. Ravi (2007) Safety Net Programs: Outreach and Effectiveness. Economic and Political Weekly. Issue: VOL 43 No. 35. September 1. Safety net programs, which are designed with three main purposes, protection (ex-post), insurance (ex ante) and poverty alleviation, offer help to households during a period of crisis. This article evaluates the efficiency, awareness, participation, and targeting and distributive outcomes of these programs, based on household/village-level surveys conducted in Orissa, Madhya Pradesh and Karnataka. In addition, the article pays special attention to the functioning of village-level institutions and social capital. Besides giving an overview of the risks and shocks faced by households in these states, the article shows that the current safety net programs do not seriously address the health risk. Comments on the performance of specific programs are as follows: (a) PDS: The program is reasonably targeted to the poor as their share is higher than the top two quartiles. However, there is substantial leakage of benefits to the non-poor; the AAY scheme fares better in terms of benefit incidence (b) SGRY: The program is not attracting the richer 71 quartiles, thus confirming some degree of self-selection, targeting is more effective for females than for males (c) Food For Work: The employment generated under food for work seems to be even more pro- poor than SGRY, however, 28 per cent of the employment is generated by the households belonging to the rich quartile. Additional resources: 6. Supriti, S.M. Barnhardt and R. Ramanathan (2002a) Urban Poverty Alleviation in India Vol. 2: A Landmark Urban Initiatives for the Urban Poor since Independence. Ramanathan Foundation, Bangalore. NREGA 1. GoI (2005) Performance Audit of Implementation of National Rural Employment Guarantee Act, 2005 (NREGA) Office of the Principal Director of Audit, Economic and Service Ministries, Government of India, New Delhi. This is a comprehensive audit conducted by the Comptroller and Auditor General, GoI to evaluate the functioning of the National Rural Employment Guarantee Scheme. The scope of audit was restricted to the initial 200 districts identified for implementation of NREGA with effect from February 2006. The period of audit coverage was from February 2006 to March 2007. The report states the various requirements of the scheme and assesses the current status across states in great detail. 2. CAG Draft Report (2007). Performance Audit of Implementation of National Rural Employment Guarantee Act, 2005 (NREGA). Retrieved from http://groups.google.com/group/rozgar/files?upload=1 3. Jha , R., R. Gaiha and S. Shankar (2008) Reviewing the National Employment Guarantee Program. Economic and Political Weekly. Issue : VOL 43 No. 11 March 15 - March 21. This article presents results on the participation of rural workers in the National Rural Employment Guarantee program based on a pilot survey of three villages in Udaipur district in Rajasthan. Its focus is on participation in the NREG program of different socio-economic groups and the determinants of the participation of these groups. It is found that the mean participation was 59 days and that targeting was satisfactory. The performance of the program has been far from dismal. 4. Mehrotra, S. (2008) NREG Two Years On: Where Do We Go from Here? Economic and Political Weekly. VOL 43 No. 31 August 02 - August 08. This article examines the performance of the National Rural Employment Guarantee program two years since its launch in mid- 2005, drawing from the more detailed Comptroller and Auditor General‘s Report on the performance of the NREGS. Performance is compared across states and districts on indicators such as wage rates, relative to prevailing market rates, types of activities undertaken and the number of person-days generated. The scheme is seen to be targeting SC and ST groups well. Women constitute one-third or more of the beneficiaries in all but a few states. The report identifies key areas for remedial action to address current weaknesses. Some of the challenges identified include the lack of awareness, monitoring and evaluation, schedules of rates and wage rates, norms relating to release of funds and wage payment and the list of permissible activities under the NREGS. Remedies, where available and relevant, are suggested. 5. Dreze, J. and R. Khera (2009). The Battle for Employment Guarantee. Frontline. January 16. This paper assesses the performance of the National Rural Employment Guarantee Scheme. Based on a survey conducted in 100 sample gram panchayats (villages) spread over 10 districts in 6 north Indian states, and drawing on other social audits and informal investigations of the NREGA, the authors assess several operational and implementation aspects. The report sheds light on the level of awareness, profile of participants (with special regard to barriers to women‘s participation), role of contractors, effectiveness of minimum wages, impact on migration, corruption and transparency. Aside from a few indications of corruption and inefficiency, the program is found to have had a beneficial effect on villages by raising wages, reducing migration and through the creation of productive assets. As regards targeting, the program is found 72 to be successful in drawing people from SC/ST groups, illiterate households, those having no electricity and those living in kacchha houses. Women‘s participation varies widely across states, but is generally low. Additional resources: See various field studies produced by the Professional Institutes Network (PIN) for NREGA, available at http://www.nrega.net/pin BPL 1. Jalan, J. and R. Murgai (2006) An Effective ―Targeting Shortcut‖? - An Assessment of the 2002 Below- Poverty Line Census Method. Mimeo – The World Bank, Washington DC. In this paper, the authors use 1999-00 National Sample Survey data to analyze whether the BPL (Below Poverty Line) census (the BPL census is the basis for the PMT-based national targeting system) methodology is effective at distinguishing the poor from the non-poor. Results indicate large targeting losses in moving to a proxy means test method. However, even though the data reject the assumptions implicit in constructing a BPL score, overly stringent assumptions are not the main reason for the poor targeting performance. The authors use an augmented regression model that relies on additional indicators as an alternative that substantially reduces the targeting errors vis-à-vis the BPL method, particularly in the tails of the expenditure distribution. However, under- coverage is still high (34 percent), largely because even an augmented and flexible model performs poorly in sorting the poor from non-poor in the vicinity of the poverty line. With the high density of population with income close to the poverty line, arriving at an effective proxy means test is an inherently problematic and difficult exercise. 2. Ram, F., S. K. Mohanty and U. Ram (2009) Understanding the Distribution of BPL Cards: All India and Selected States. Economic and Political Weekly. Issue: VOL 44 No. 7. February 14. Using the recent National Family and Health Survey-3 data based on a representative sample of 109,140 households, this paper examines the distribution of below poverty line cards. The possession of BPL (Below Poverty Line) cards by the households in various economic and social settings index is analysed. The findings suggest that about two-fifths of the BPL cards in India are with the non-poor households. On the other hand, in many of the states a majority of households in abject deprived groups do not possess a BPL card. The extent of misuse is higher in Andhra Pradesh, Karnataka and Kerala, while it is lower in Tamil Nadu. In economically weaker states like Orissa and Bihar, a higher proportion of non-poor households possess a BPL card. 3. Sundaram, K. (2003) ―On Identification of Households Below Poverty Line in BPL Census 2002: Some Comments on the Proposed Methodology‖, Economic and Political Weekly, March 1. This paper discusses the methodology for the 1997 BPL census and its criticisms as articulated by the Expert Group. This is followed by a discussion of the recommendations of the Expert Group for the 2002 BPL census, including the proposed dropping of the exclusion criteria. The author discusses the ―scorable‖ indicators constituting the BPL score under the 2002 census system and examines the need for and the problems arising in using the ranking of households for program implementation, monitoring and evaluation. 4. Mehtrotra, S. and Mander, H. (2009). How to identify the poor: A proposal, Economic and Political Weekly, Vol. XLIV (19), May 9. This paper first summarizes the main criticisms of the 2002 BPL Census system and proposed a changed methodology for the next round of the quinquennial census scheduled for 2009. The proposed system involves applying a set of exclusion criteria in the first stage, after which the remaining households would be given points on the basis of (i) social group (ii) vulnerability and (iii) occupational status. Criteria within these 3 groups would be used to calculate household scores out of 100 on the basis of which they will be ranked. Thus there would be no single BPL list, but potentially multiple beneficiary lists using the same ordinal ranking of households. This exercise is proposed to be conducted once in 10 years with a review of lists for additions and deletions every two years in the interim period. In a significant departure from the 2002 system, the proposed changes include using external enumerators to conduct the BPL census. 73 5. Saxena Committee Report (2009). ―Identification of BPL households in rural India‖. Draft report from the Ministry of Rural development. Available online at http://rural.nic.in/latest/rpt_bpl_census2009.pdf, last accessed on January 14, 2010. An Expert Committee has suggested significant changes in design of the BPL Census. First the method proposes the automatic exclusion of visibly non-poor households and the automatic inclusion of the most vulnerable households, with a survey and scoring only of the remaining households. Second the new method overlays geographic targeting criteria by specifying district, block and GP-level quotas with respect to the proportion of BPL households. In districts that have a very high proportion of BPL households (over 80%), all households (except the visibly non-poor) would be considered to be poor and automatically included in the BPL list. This design reduces administrative demands and cost to some extent by reducing the scope of the survey. The underlying principle of the proposed methodology is to minimize errors of exclusion, without worrying over-much about errors of inclusion. The recommendations of the report are currently being debated by the Central Planning Commission. Social Pensions 1. Government of India (2006) Report of the Working Group on Social Protection Policy – National Social Assistance Program and Associated Programs, Ministry of Rural Development, Government of India, New Delhi. 2. Dutta, P. V. (2008) The Performance of Social Pensions in India: The Case of Rajasthan, SP Discussion Paper No. 0834, World Bank. 3. Rajan, S. I. (2001) Social Assistance for Poor Elderly: How Effective? Economic and Political Weekly. February 24. 4. Murgai, R. (2007) Do Public Funda Reach the Destitute? Assessment of Leakages and Targeting of Social Pension Programs in Karnataka. Mimeo, World Bank, Washington DC. 5. Operations Research Group (1998) Evaluation of National Social Assistance Program in Selected States. Operations Research Group, New Delhi. PDS 1. GoI (2005) Performance Evaluation of Targeted Public Distribution System. Planning Commission, Government of India, New Delhi. This paper presents findings from an evaluation of TPDS to reflect on the following issues: Efficacy of the delivery mechanism in improving access to PDS for the poor; Off-take by the poor and its determinants; Viability of Fair Price Shops (FPSs) & its implications; Types and magnitudes of targeting errors and their implications on welfare and budgetary consumer subsidy; Extent of leakages and diversions of subsidized food grains; Delivery cost across the States; and Overall performance of TPDS. To generate the required database, the study covered 60 districts, 88 blocks, 16 towns, 176 village panchayats, 240 Fair Price Shops and 3600 households spread over 18 States. Results show that the implementation of TPDS is plagued by large Errors of Exclusion (of BPL families) and Inclusion (of APL), and by the prevalence of ghost BPL cards. Some States have issued more cards than the number of households, while some others have the problem of unidentified households. Though the off-take per household has shown some improvement under TPDS, yet only about 57% of the BPL households are covered by it. The FPSs are generally not viable because of low annual turnover and they remain in business through leakages and diversions of subsidized grains. Leakages and diversions of subsidized grains are large and only about 42% of subsidized grains issued from the Central Pool reaches the target group. Over 36% of the budgetary subsidies on food is siphoned off the supply chain and another 21% reaches the APL households. The cost of income transfer to the poor through PDS is much higher than that through other modes. The performance of TPDS can be considerably improved if some measures are taken to streamline the BPL identification survey and if the delivery system is made effective, efficient and transparent. 2. Dutta, B. and B. Ramaswami (2001) Targeting and Efficiency in the Public Distribution System: Case of Andhra Pradesh and Maharashtra. Economic and Political Weekly. May 5, 2001. This paper compares the public distribution of food in Andhra Pradesh and Maharashtra. Based on the 50th round of National Sample 74 Survey (NSS) household consumption survey data, the authors examine differences in utilization, extent of targeting, magnitude of income transfers and the cost-effectiveness of food subsidies. The findings suggest policy reforms in favour of self-targeting and greater operational efficiency. Additional resources: 3. GoI (2006) Performance Audit of Management of Food Grains. Report No. 16 Ministry of Consumer Affairs, Food and Public Distribution, Department of Food and Public Distribution, Government of India. 4. Kochar, A. (2005) Can targeted Food Programs Improve Nutrition? An Empirical Analysis of India‘s Public Distribution System. Economic Development and Cultural Change. University of Chicago, Illinois. 5. Mahendra Dev, S., C. Ravi, B. Vishwanathan, A. Gulati and S. Ramachander (2004) Economic Liberalisation, Targeted Programs and Household Food Security: A Case Study of India. Markets, Trade and Institutions Division Discussion Paper No. 68. International Food Policy Research Institute, Washington DC. School Feeding/ Mid Day Meals 1. Comptroller and Auditor General (CAG). (2008) Performance Audit on National Program for Nutritional Support to Primary Education (Mid-day Meal Scheme). Report No. PA 13 of 2008. Comptroller and Auditor General, Government of India. 2. Sen, S. (2009) School Feeding Programs in Developing Countries: Summary of Evidence on Impact, Implementation and Costs. Mimeo, World Bank, Washington DC. 3. Drèze, J. and A. Goyal. (2003) Future of Mid-day Meals. Economic and Political Weekly, 1 November 2003 4. Khera, R. (2006) Mid-day Meals in Primary Schools - Achievements and Challenges. Economic and Political Weekly, 18 November 2006. 5. NFI (2006) Evaluation of Mid Day Meal Program in MCD Schools. Scientific Report 18, Nutrition Foundation of India (NFI), New Delhi. 6. Rani S., A. and N. Kumar Sharma. (2008) An empirical study of the Mid-day Meal Program in Khurda, Orissa. Economic and Political Weekly, 21 June 2008. 7. World Bank (2007b) An Assessment of the Mid-day Meal Scheme in Chittorgarh District, Rajasthan, India. Social Accountability Series, Note No 3, August 2007. South Asia Sustainable Development Department, The World Bank. MALDIVES 1. World Bank (2006) Social Protection in the Maldives: Options for Reforming Pensions and Safety Nets. World Bank, Washington DC. This report provides a detailed overview of the poverty and vulnerability situation in the Maldives and underlines the need for developing an effective social protection system for the country. The report contains a detailed discussion on the pension system that covers public sector employees, where the key challenges are in inadequacy of retirement incomes and the lack of safety net for two-thirds of the elderly, not covered by any kind of pension scheme. A reform plan for the same is discussed. Although the Government of Maldives has put in place several income support programs to protect the poor and vulnerable, there are several inadequacies: the safety nets programs are ad hoc and overlapping, they have low coverage and funding, and the distribution of publicly funded programs is inequitable. Reform option suggested include three major components: developing an umbrella poverty targeted program, restructuring 75 the benefits payable to households and reforming targeting. The report explored the potential of using a proxy means test in combination with community-based targeting and recommends a combination of geographic targeting and community-based identification of beneficiaries aided by guidelines that emphasize household characteristics statistically associated with vulnerability. NEPAL 1. World Bank (forthcoming). ―Social Protection in Nepal.‖. Human Development Unit, South Asia Region. World Bank, Washington DC. This report provides a comprehensive framework for the existing social protection programs in Nepal and presents recommendations for policy makers for the future of social protection in Nepal. It examines safety net programs and in-kind transfers, labor market and employment programs, social care services, social security pensions and social insurance; and social protection schemes for disadvantaged groups. A key feature in the institutional architecture of the Nepali Sp system is that policy development and program delivery are fragmented across an array of ministries and agencies. The report notes that presently there is no qualitative/ quantitative evaluation of the effectiveness, efficiency, coverage, administrative structures and fiscal sustainability of Nepal‘s SP programs. In doing so it also notes that safety nets programs in Nepal, and their targeting methods suffer from a number of shortages. Given the large share of pensions and social insurance in SP resources, the non-poor, i.e. those with access to formal employment benefit the most. 2. World Bank (2009b) Targeting Systems for Safety Nets in Nepal: A Review. World Bank, Washington DC. This report takes stock of targeting systems used in safety nets programs in Nepal and provides information on their effectiveness, where possible. The authors examine a number of government and donor-funded cash and in-kind transfers, public works programs. The main messages that emerge are as follows: First, categorical targeting, though easy to administer, has not been effective in reaching the poorest/most vulnerable population in Nepal. Second, in the presence of a tight fiscal envelope, evaluation of the targeting performance of existing safety net programs is needed before making the decision to expand; which would require investing into collecting this information (i.e. adding a SP programs module in the next NLSS). Third, the Government needs to define the objectives and the target audience of the programs, also considering the trade-offs between the costs and targeting efficiency while thinking about the design of an effective safety net system. Finally, fragmentation needs to be addressed with a potentially unified method and piloting would be crucial in determining the important design/implementation features that may be specific to Nepal. 3. Asian Development Bank (2004) Social Protection Study Nepal: Final Report. Asian Development Bank, Manila. Key findings: Safety net programs and the targeting methods utilized in Nepal suffer from a number of shortcomings. The fragmentation of programs across groups and across several public agencies without clear prioritization of services reduces program impact. Furthermore, donor/government harmonization is often lacking in program design and implementation resulting sometimes in multiple small programs running concurrently with varying evaluation rigor, in turn making judgments about the program difficult. The impact of the programs (especially those provided by the Government) is poor due to limited coverage and low benefits. The efficiency of the programs is also limited due to weak administrative capacity. The targeting systems used are fragmented; multiple criteria, multiple implementing organizations, and varying levels of transparency and verification exist. Finally, no systematic evidence on targeting effectiveness of specific systems or programs exists (even when available it is dated), and adequate monitoring and evaluation capacity to make mid-course corrections (or assess program effectiveness) do not exist. 4. UNICEF (forthcoming) Participatory Method for Mapping of Disadvantaged Groups (DAG) and its Application in DACAW Districts: A Summary Report. UNICEF, Kathmandu. This report documents the Disadvantaged Groups (DAG) mapping exercise conducted by the Institute for Integrated Development Studies (IIDS) and commissioned by UNICEF Nepal to study disadvantaged households in all Village Development Committees (VDCs) implementing the Decentralized Action for Children and Women (DACAW) program. The multiple stages of this exercise included a qualitative pre-assessment, developing 76 eight participatory indicators (at the household level) forming a conceptual framework of disadvantage, mapping in all 235 VDCs and eight municipalities of DACAW‘s 15 districts, covering 100 per cent of households and a ranking framework for all VDCs in any district or wards in any municipality. This study uses a definition of DAG-ness that is based on social, economic and empowerment indicators. The report gives details of the methodology used, results of the mapping exercise and comments on issues related to the identification of disadvantaged households and communities. 5. WFP (2000) Food Security and Vulnerability in Nepal – An Issue Paper. World Food Program, Rome. The paper discusses the definition, concepts and status of food security and vulerability, and their forms and determinants. The trend of national level food production, consumption, trades, and food balances is presented along with a discussion of the dimensions of food insecurity and vulnerability covering geographic area, ethnicity and gender relations. This lays down the methodology for a detailed identification of food- insecure areas and groups. The report further identifies macro policies and programs relating to food security, as well as household-level coping strategies and highlights major gaps in knowledge and weakness of policies on food security and vulnerability. PAKISTAN 1. World Bank (2007) Social Protection in Pakistan: Managing Household Risks and Vulnerability. World Bank, Washington DC. This comprehensive report evaluates the social protection system in Pakistan and makes recommendations for improvement. The study employs secondary sources of data as well as original work in presenting a poverty and vulnerability profile of Pakistan, critically assessing existing policies and programs (including issues pertaining to design, institutional structure, financing, coverage, targeting, adequacy and impact), and finally, proposals for a comprehensive social protection policy agenda. With respect to targeting, significant scope for improvement for all programs is reported. The report emphasizes the need for adopting an objective targeting instrument, advocates the adoption of a PMT, and invokes the need for improving the accuracy of household assessments and geographical targeting and promoting transparency in targeting decisions. 2. Yusuf, M. (2006) Poverty Reduction Initiatives in Pakistan: Search for an Optimal Targeting Strategy. TA – 4277 PAK: Support for Poverty Reduction Initiatives in Pakistan. Asian Development Bank – Pakistan Resident Mission, Islamabad. The author reviews the targeting arrangements of key safety net interventions in Pakistan and points to a number of key factors that could potentially ensure success of any targeting mechanism. These include the need for a strong governance structure, presence of political will among government authorities, community participation but without discretion and in the presence of a robust monitoring and evaluation framework, and a strong accountability mechanism. An umbrella approach, using either one or a combination of various targeting techniques is recommended, which entails using a combination of the PWR and indicator approach. The proposed approach builds on the strengths of various analyzed programs. It is hoped that the umbrella nature of the approach and uniform indicators and cut-offs would induce comparability across villages. 3. Shahnaz (2008) Targeting of Social Programs in Pakistan: An Evaluation, manuscript. The author presents a detailed analysis of all major Pakistani poverty reduction programs and identifies the main issues and challenges of each, as well as broader themes pertaining to design and methodology, governance, implementation, transparency and accountability and outcomes of targeted poverty reduction programs. Programs that involve a large element of administrative discretion in beneficiary selection appear to be most ill targeted. Endemic corruption, weak governance structure, lack of transparency and the absence of monitoring and evaluation (especially third party validation of programs) further plague targeting outcomes. The lack of involvement of communities and limited public disclosure are especially highlighted as challenges for most poverty reduction programs. 77 4. CRPRID (2004) Social Protection in Pakistan: Concept, Situation Analysis and the Way Forward. Centre for Research on Poverty Reduction and Income Distribution Working paper No. 5. ILO, CRPRID and UNDP. Available at: http://www.researchcollective.org/Documents/Social_Protection_Way_Forward.pdf This paper highlights conceptual issues with regard to social protection as well as the situation that prevails in Pakistan. Specifically, it provides an overview of the social security schemes available in the formal sector and comments briefly on some of the key social safety net programs (Zakat, Bait-ul-Maal and Public works and microfinance initiatives) on issues pertaining to program design, targeting and governance. A brief overview of informal social security mechanisms is also provided. Recommendations in the last section follow from the concept of social protection outlined, the situation analysis and some basic projections on population, labour force and economic trends. 5. Bari, F., E Hooper, S. Karda, S.N. Khan, I. Mohammad and A. Sayeed (2005) Conceptualising a Social Protection Framework for Pakistan. Pakistan Poverty Assessment Update, Background Paper No. 4. Asian Development Bank – Pakistan Resident Mission, Islamabad. Available at: http://www.researchcollective.org/Documents/Conceptualizing_Social_Protection.pdf This report provides an overview of the safety nets situation in Pakistan and examines some of the larger programs including the Pakistan Bait-ul-Maal, the institution of Zakat (charitable donations), the Workers Welfare Fund, Employees Social Security Institutions, the Workers Participation Fund and the Khushali Bank. The report states that at present, formal social protection measures fall short of providing effective social protection coverage. Apart from this, most schemes are beset by ineffective targeting and resource leakages. Key gaps in the GoP‘s current approach to social protection include: (i) the lack of a clearly articulated social protection strategy; (ii) inadequate government spending on existing social safety nets; (iii) the lack of committed funding for safety net programs; (iv) inadequate current assistance; (v) multiple programs in non-core competence activities; (vi) inadequate targeting mechanisms and weak governance; (vii) the lack of inter-agency coordination; and (viii) weak monitoring and supervision. The report concludes with an identification of the key challenges moving ahead and recommends strategies and interventions to be pursued. 6. Pasha, H. A., S. Jafarey and H. R. Lohano (2000) Evaluation of Social Safety Nets in Pakistan. Research Report No. 32 Social Policy and Development Centre, Karachi. Available at: http://64.224.68.11/pubs/rr/rr32.pdf This paper describes the various social safety net programs in Pakistan and evaluates them on the basis of ten criteria including targeting efficiency, coverage, access, adequacy of support etc. Rankings on different criteria of each scheme given below are based on a subjective assessment and could be considered arbitrary in some cases. It appears that most of the schemes have weak institutional structures, their funding is limited and uncertain, their targeting inefficient and their coverage very small. 7. Kashf Foundation Impact Assessment 2005. The report assesses issues pertaining to identification and selection of clients, loan use and economic and social impacts of the Kashf foundation‘s micro loans on participating women and their families. Information was collected from 250 personal interviews and 7 Focus Group Discussions from urban branches in Lahore. About 90% of all clients were found to be living on less than $1 per day. Significant economic impacts on household poverty were reported but only nascent signs of social impacts viz. decision-making roles of women in the households. 8. Arif, G. M. (2006) Targeting Efficiency of Poverty Reduction Programs in Pakistan. Working Paper No. 4 Asian Development Bank Pakistan Resident Mission, Islamabad. This paper examines the targeting efficiency of three large, nationally implemented poverty reduction programs: (i) the disbursement of zakat (charitable donations), (ii) microfinance, and (iii) the provision of health services through the Lady Health Workers Program (LHWP). Both zakat and microfinance are ―narrow-target‖ programs, incorporating mechanisms to identify the poor, while the LHWP is classified as a ―broad target‖ program. The paper evaluates the targeting efficiency of zakat disbursed through public and private sources, by using data from the Pakistan Socio- 78 economic Survey (PSES) for 2000/01. The findings of this study suggest some success in zakat targeting, particularly in rural areas, but this is marred by the leakage of zakat funds to relatively better-off households, particularly in urban areas. Most microfinance programs were found to fail to target the poorest households, a key reason being that there is no verifiable mechanism for identifying the poor. Finally, the LHWP has not yet reached the most disadvantaged areas or the poorest households. It is the more remote and poorer rural areas where functional health facilities are least likely to exist, and where recruiting suitable educated women to work as LHWs is the most difficult. 9. Oxford Policy Management (2002) Lady Health Worker Program: External Evaluation of the National Program for Family Planning and Primary Health Care. Islamabad: OPM. This evaluation assesses the performance of the LHWP in Pakistan with regard to program delivery and organization, access to the poor, impact on health and costs. This was based on in-depth surveys in each province as well as detailed interviews with 60 key informants. Specifically it finds that the population served by the LHWP is slightly better off than those not served by it, although differences are small. The rich are not served by the LHWP. 10. Montgomery (2005) Serving the Poorest of the Poor: The Poverty Impact of the Khushhali Bank‘s Microfinance Lending in Pakistan. Asian Development Bank Institute, Tokyo. Available at: http://www.adbi.org/files/2005.09.28.book.khushhali.microfinance.study.pdf And 11. Montgomery, H. (2005) Meeting the Double Bottom Line: The Impact of Khushhali Bank‘s Microfinance Program in Pakistan. Asian Development Bank Institute, Tokyo. Available at: http://adbi.adb.org/files/2005.09.28.cpp.khushhali.microfinance.study.pdf These papers examines empirically the poverty impact of Pakistan‘s microfinance sector development program by looking at the impact of Khushali Bank‘s lending program on the welfare of poor households in the country. Using primary data collected from over 2,000 rural and urban households across Pakistan, this study examines whether the Khushali Bank has also had an impact on economic or social measures of poverty, including empowerment of poor women, or income generating activities run by those households. The empirical analysis here demonstrates that participation in the Khushali Bank‘s microcredit program has positive impacts on both economic and social indicators of welfare, as well as income-generating activities, especially for the very poorest participants in the program (those living at less than half the official poverty line). Khushhali has demonstrated that given a supportive regulatory environment, it is possible for commercial microfinance banks to meet a ―double bottom line‖ of simultaneously pursuing profits and a humanitarian social mission. 12. PPAF (2004) PPAF Microcredit Financing: Assessment of Outcomes. Prepared by GALLUP Pakistan for Pakistan Poverty Alleviation Fund. GALLUP, Islamabad. An external outcomes assessment of PPAF's micro credit was carried out by Gallup. Adopting a counter factual "combined approach" (ex ante/ex post and with / without dimensions), it covered 100% of PPAF's Project Offices (as of June, 2001), focused on a 5% sample of 1800 households in 17 districts of the country. Several hypotheses with respect to income, consumption as well as non-income and social indicators were tested. Evidence suggests that lower income households borrowing from PPAF were better off than they would have been in they had not borrowed. On average, their incomes and consumption had increased, there was an improvement in their personal and business assets; their life styles by way of housing facilities was better than before and their social status, especially that of women borrowers, had undergone a positive change. Admittedly, the scale of change was limited, but this was to be expected considering the short time span between intervention and impact evaluation, the small amount of loan and the fact that mostly single cycle loan users were assessed. 13. Toor, I. A. and Nasar, A. (2003) Zakat as a Social Safety Net: Exploring the Impact. Research Report No. 53, Social Policy and Development Centre, Karachi. Available at: http://64.224.68.11/pubs/rr/rr53.pdf This study assesses the performance of Zakat charitable transfers in Pakistan. This is based on 400 detailed interviews with recipients and non-recipients in the field. The study assesses Zakat on the basis of implementation, access and delivery, targeting and monitoring. Many of the households included in Zakat 79 payments were indeed disadvantaged on several indicators of hardship and destitution. Zakat recipients found to be better off than non-recipients after receiving Zakat allowances. The authors conclude that there is a need to improve the targeting of beneficiaries and reduce leakage in the distribution of Zakat. Criteria for eligibility should be carefully thought out and made explicit. Members of Local Zakat Committees should be assisted in identifying individuals. There should be more deliberate efforts to include widows, especially in the rural areas. 14. Lohano, Hari Ram (2001) Credit to the Poor: Consultations with Borrowers. Research Report No. 38 Social Policy and Development Centre, Karachi. The paper reflects the views of loan recipients of the National Rural Support Program and Orangi Pilot Project B two leading microfinance institutions in rural and urban areas respectively. Focus Group Discussion and detailed individual case studies were conducted to evaluate these programs in terms of access, problems and impact of microfinancing. The programs are found to be ineffective in targeting the poorest as credit-worthiness is an eligibility criterion. SRI LANKA 1. World Bank (2006) Sri Lanka: Strengthening Social Protection. Part I. World Bank, Washington DC. This is a comprehensive overview of the poverty, vulnerability and social protection situation of Sri Lanka. The report measures growth, inequality and poverty in Sri Lanka in recent years and identifies key sources of risk and vulnerable groups. It the looks specifically, and in much details at labor market trends and outcomes, as well as social insurance schemes and suggests policy reform options. Chapter 5 of the report provides an overview of safety net interventions and assesses the Samurdhi and Income Support for the Disabled programs, by drawing on the findings of a number of studies. Under policy options for safety nets, the report calls for a reform of the Samurdhi program and the adoption of a PMT-based national targeting system. 2. Glinskaya, E. (2000) An Empirical Evaluation of the Samurdhi Program. World Bank. Background Paper to Sri Lanka Poverty Assessment. SASPR, World Bank, Washington D.C. This paper evaluates the Samurdhi Program, Sri Lanka‘s largest safety net program on the basis of its administrative and distributional efficiency, and impacts on household welfare. The assessment, including of the targeting mechanism (then based on means-testing) is based on a quantitative analysis of SLIS data as well as qualitative tools. Samurdhi does not emerge as an efficiently targeted program as large amounts of its resources reach the non-poor. 3. Salih, R.(2000) The Samurdhi Poverty Alleviation Scheme. Social Security Division, ILO, Geneva. Available at: http://www-ilomirror.cornell.edu/public/english/region/asro/colombo/download/rozscl00.pdf This paper assesses the impact of the protectional element of the Samurdhi poverty alleviation scheme in alleviating hardship among the poorest. The paper looks at the impact of the grant and insurance components in terms of its effectiveness in coverage (number of people), types of contingencies it covers and its effectiveness in providing income support. We will also discuss briefly the other elements (savings and credit scheme and rural infrastructure development) and how these aspects of the Program regards its protection capacity. Much of the information in the paper has been obtained from the 3 publications of the Ministry for Samurdhi and Youth Affairs, personal interviews with officials in the Ministry and from 2 previous studies on the Samurdhi Programs: Gunatilaka et. al (1997) and Gunatilaka and Salih(1999), With regard to targeting the author finds that targeting is politically biased and recommends developing community-based targeting, including non-income dimensions of poverty, formulating an exit strategy. This paper cites useful references on targeting outcomes in Samurdhi in the 1990s. 4. Aryaratne, I., A. Athauda and L. Premarathne (2008) Study on Samurdhi Beneficiary Process: March – June 2008. Mimeo, The World Bank, Colombo. The need to revise / regularize the beneficiary selection process of the Samurdhi program‘s relief delivery, through the adoption of a community-led participatory process has been actively debated in recent years. This Study has focused on studying both adaptability of the (pilot) 80 Family Classification Methodology to local situations and its implementation. It highlights best practices and lessons learnt which inform recommendations for the re-implementation of the methodology. 5. Narayan, A. and Yoshida, N. (2005) Proxy Means Test for Targeting Welfare Benefits in Sri Lanka. Report No. SASPR – 7. World Bank, Washington DC. This paper develops and presents alternate Proxy Means tests models for the Samurdhi program in Sri Lanka. This is based on household data from the Sri Lanka Integrated Survey. Two models are recommended and these include as their set of predictors selected variables of the following types: location, community characteristics, household assets, land and livestock ownership, household demographics and housing conditions. The incidence of targeting by these models is shown to be highly progressive and a large majority of the targeting errors that occur do so relatively close to the poverty line on either side. A few simulations with different payment schemes suggest that replacing the existing system (means-testing) with targeting using one of the recommended formulae would result in welfare gains. GENERAL RESOURCES ON TARGETING 1. Grosh, M., C. del Ninno, E. Tesliuc and A. Ouerghi (2009) For Protection and Promotion: The Design and Implementation of Effective Safety Nets. World Bank, Washington DC. Drawing on research, policy and operational documents from academia and the World Bank on work in over 100 countries, this book provides guidance on how to design and implement safety nets, including useful information on (i) how to design, finance and implement safety nets, (ii) target the poor and disadvantaged and identify beneficiaries, and (iii) set and pay benefits and monitor and evaluate programs and systems. The book synthesizes the literature to date and provides new information on various safety net program options. It concludes with recommendations on how to customize safety nets to meet different needs in six particular contexts. 2. Coady, D., M. Grosh and J. Hoddinott (2004) Targeting of Transfers in Developing Countries: Review of Lessons and Experience. World Bank, Washington DC. This report reviews the lessons learned from 122 antipoverty interventions in 47 transition and developing countries to quantify targeting outcomes and their determinants and to inform the design and implementation of common targeting methods. Key findings: Although targeting has the potential to increase the effectiveness of antipoverty interventions, about 25 percent of targeted programs actually delivered to the poor less than what a universal or random allocation would have. No single preferred method has emerged for all types of programs, or all country contexts. Successful targeting outcomes are dependent on the method of program implementation. 3. van de Walle, D. (1998) Targeting Revisited. The World Bank Research Observer, vol. 13, no. 2 (August 1998), pp. 231-48. World Bank, Washington DC. Public spending programs aimed at alleviating poverty can either be broadly targeted at categories of spending or narrowly targeted at types of people. Each approach has benefits and costs to the poor. It is often claimed that narrow targeting of the poor will allow governments to reduce poverty more effectively and at lower cost. But narrow targeting often has hidden costs, and once these costs are considered, the most finely targeted policy may not have any more effect on poverty than a broadly targeted one. Both approaches also have hidden benefits, although less is known about their impact. Targeting can help, but it is not a cure-all. Reducing poverty calls for broadly targeted social sector spending combined with narrower targeting of cash and in-kind transfers to specific groups. It is also important for governments to experiment with schemes that offer better incentives, to carefully monitor the costs and outcomes, and to be flexible and pragmatic in their policy responses. 4. WFP (2006) Targeting In Emergencies. Executive Board: Policy Issues – Agenda Item No. 5, EB.1/2006/5-A. World Food Program, Rome. This paper establishes basic principles to guide decision- making in a range of emergency situations, drawing on the decades of experience in WFP and its partners in targeting food aid during emergency interventions. Although the WFP does not have a consolidated 81 emergency targeting policy, various policy statements and guidance reflect extensive WFP experience in targeting in emergencies. The paper begins with a review of the definitions of targeting and WFP policies related to targeting in emergencies. Some of the key solutions/ recommendations offered include: Vulnerability analysis and mapping, early warning and emergency needs assessment results are used to establish a basis for food assistance and set initial targeting parameters; Geographic targeting is necessary to confirm the scope of a food crisis and sometimes is the only feasible level of targeting; In most cases, however, geographic targeting must be complemented with household/individual targeting; Partner organizations and local community structures are valuable information sources for developing household- level targeting criteria; Substantial beneficiary participation in defining targeting criteria should be standard practice in responses to slow-onset and recurrent emergencies, and increasingly the practice as sudden-onset crises begin to stabilize. Additional resources: 5. Coady, D., M. Grosh and J. Hoddinott (2002) Targeted anti-poverty interventions: A selected annotated bibliography. World Bank, Washington DC. 6. Van de Walle, D. and K. Nead (1995). Public Spending and the Poor: Theory and Evidence. The World Bank, Washington DC. 82 Table A1: Indicators of program-specific targeting effectiveness Program Type of Type of % Bottom quintile Inclusion Exclusion Benefit Leakage Comments Program Targeting as share of total error error incidence beneficiaries People People Amount Amount Afghanistan N.A. Bangladesh Test Relief Public Geographical 38.9 6.7 - - - Works Primary Education Cash Categorical - - - - - A third of PESP stipends Stipend Program Transfer go to non-poor households; long delivery process Rural Maintenance Public Categorical - - - - - Generally well-targeted, Program Works attributable to setting wages below market rates, and fewer intermediaries Vulnerable Group In-Kind Categorical + 31.7 5.5 - - - 11 percent of participants Development Transfer Means tested of the PESP meet none of the eligibility criteria for program participation while almost none of the beneficiaries meet at least three criteria 27% beneficiaries non-poor Old Age Pension Cash Categorical 31.9 6.5 - - - Transfer Vulnerable Group In-Kind Geographical 36.1 5.2 - - - Feeding Transfer Bhutan N.A. India PDS In-Kind PMT 27.0 10.2 74.4 54.8 6.4 Large leakages in the Transfer program arising from loss of benefits altogether due to high diversion of foodgrains (rather than solely targeting errors) NOAPS Cash Categorical + 32.8 17.9 88.3 50.6 18.4 Targeting progressive, good 83 Program Type of Type of % Bottom quintile Inclusion Exclusion Benefit Leakage Comments Program Targeting as share of total error error incidence beneficiaries Transfer PMT performance among SC/ST groups Targeted Housing In-Kind PMT 28.6 8.96 85.6 43.1 9.3 Large grants under the Schemes Transfer scheme make it susceptible to capture SGRY/ FFW Public Self Targeting 43.4 3.9 98.8 51.7 16.8 Considerable inclusion of Works the richest quintile among beneficiaries, albeit progressive on the whole National Rural Public Self-Targeting - - - - - Good coverage of Employment Works households that live in Guarantee Scheme kacchha houses, have no electricity, are illiterate and belong to SC/ ST groups. Mixed participation of women. School Stipends Cash Categorical 22.6 11.6 88.0 33.3 34.1 Good performance among Transfer SC/ST groups and girls SGSY Credit PMT 32.9 13.7 98.9 23.1 49.8 Good coverage of households that live in kacchha houses, have no electricity, are illiterate and belong to SC/ ST groups. Mixed participation of women. Maldivesa All Government Cash/In- Categorical - - - 26.1 52.2 Over half the government Assistance Kind social assistance goes to the Transfers richest tercile. Nepal Girls‘ Scholarship Cash Categorical - - - 46 3 More than 50% of the girls Transfer who receive scholarship belong to the top 3 income quintiles Food for Work and Public Self Targeted - - - - - Self-targeting systems seem Cash for Work Works not to be reaching the poor and socially excluded 84 Program Type of Type of % Bottom quintile Inclusion Exclusion Benefit Leakage Comments Program Targeting as share of total error error incidence beneficiaries groups Poverty Alleviation Cash - - - - - Only 0.2 % beneficiary Fund transfer/ households categorized as Income non-poor; good coverage of Generation women and socially excluded groups Pakistan Zakat – Guzara Cash - - 33 29 Average expenditure levels Allowance Transfer among beneficiaries is 11% Administrative higher than for non- 21.3b nomination recipient households. Zakat – Rehabilitation Cash - - 47 11 Grants Transfer Pakistan Bait – ul – Cash Administrative - - - 46 17 Maal (PBM) Transfer nomination Lady Health Workers‘ Social Care Self- Targeting - - - - - Tends to serve advantaged Program Services areas over others, also population served is slightly better off than others but not by a huge margin National Rural Micro - - - - - Microfinance services do Support Project finance not reach the poorest as credit-worthiness is an eligibility condition; thus precluding the poorest. Sri Lanka Samurdhi Food In-Kind 31.5 6.9 41.5 60 6 Food Stamp transfers are Stamps Transfers Means tested largely progressive at the (now central level, misallocations community- occur at the local level. Samurdhi Loans based - - - 39 9 Samurdhi Social Micro targeting) - - - 55 13 Insurance Insurance Sources: Bangladesh: World Bank (2006a), World Bank (2010), Ahmed (2004). India: World Bank (2009c); NREG: Dreze and Khera (2009). Maldives: World Bank (2006). Nepal: World Bank (2009b). Pakistan: World Bank (2007); LHWP: Arif (2006), OPM (2002); Others: Shahnaz (2008). Sri Lanka: Glinskaya (2000). Refer to sources cited in the Annotated Bibliography for details of specific programs. 85 Definitions: Inclusion errors - defined as percentage of program beneficiaries who belong to the top income quintile. Exclusion errors - defined as percentage of bottom two income quintile population not covered by program. Note that the extremely high numbers for some programs (esp. in the case of India) are the result of very low coverage (e.g., less than 1 percent in the case of SGSY in India) and targeting errors. Also, for comparability, this definition assumes the target population is the bottom two quintile of the income (or consumption) distribution. However, this is not necessarily the case for all programs (e.g., disability pensions accrue to disabled persons in the bottom quintile). Benefit incidence - defined as percentage of benefits accruing to bottom two income quintiles. Leakage - defined as percentage of benefits accruing to top income quintile. Note that leakage does not include leakage due to loss of benefits altogether due to fraud or corruption. Additional notes: 1\ The above list is only a selection of key safety net programs across South Asia, and is not exhaustive. 2\ Only available figures of targeting performance have been reported. 3\ In some cases, figures reported are authors‘ calculations based on raw data from cited sources. a\ For the Maldives, benefit incidence is defined as percentage of all government assistance on the bottom income tercile and leakage as percentage of benefits accruing to top tercile. b\Figures available for Zakat as a whole, not for constituent components separately. 86 ANNEX 2: LIST OF WORLD BANK PROJECTS SURVEYED Project Project Status Period Country Sample Targeting Code approach Agriculture and Rural Surveyed Included Development in analysis (26) (21) Social Investment Program P053578 A 2003-2007 Bangladesh √ √ CWR Project Andhra Pradesh District Poverty P045049 C 2000-2006 India √ √ CWR/CAT Initiatives Project Madhya Pradesh District P059242 C 2001-2006 India √ √ CWR/CAT * Poverty Initiatives Project Rajasthan District Poverty P010505 C 2000-2005(E) India √ √ PMT (BPL) Initiatives Project Chhattisgarh District Rural P076467 A 2003-2009 India √ √ CWR Poverty Project Bihar Rural Livelihood ―Jeevika‖ P090764 A 2007-2012 India √ √ GEO/CAT Project Orissa Rural Livelihood ―Tripti‖ P093478 A 2008-2013 India √ √ PMT (BPL)* Project Poverty Alleviation Fund Project P105860 A 2008-2012 Nepal √ √ CAT * II Poverty Alleviation Fund P082977 A 2004-2010 Pakistan √ √ CWR Community Development and P074872 A 2004-2008 Sri Lanka √ √ CWR * Livelihood Improvement ―Gemi Diriya‖ Project Community Livelihoods in P086747 A 2004-2011 Sri Lanka √ √ GEO Conflict Affected Areas Project Education Secondary Education Quality P106161 A 2008-2014 Bangladesh √ N.R. PMT and Access Improvement Project Primary Education P074966 A 2004-2010 Bangladesh √ N.A. .. Development Project II Second Higher Education P090967 A 2007-2014 Nepal √ √ PMT Project Fifth Punjab Education P102608 A 2009-2012 Pakistan √ √ CAT * Development Policy Credit (PEDPC – V) Sindh Education Sector Project P107300 A 2009-2012 Pakistan √ √ GEO (SEP) India Elementary Education P102547 A 2008-2010 India √ √ CAT Project (SSA II) Health Health Nutrition and Population P074841 A 2005-2010 Bangladesh √ √ GEO Sector Program Health Sector Emergency P078324 A 2003-2009 Afghanistan √ N.A. .. Reconstruction and Development Project Health Sector Program Project P040613 A 2004-2010 Nepal √ N.A. .. Integrated Child Development P102550 P 2008 India √ √ GEO Services Reform Project Social Protection North-West Frontier Province P097471 C 2007 Pakistan √ N.A. .. 87 Project Project Status Period Country Sample Targeting Code approach Agriculture and Rural Surveyed Included Development in analysis (26) (21) Second Development Policy Credit (NWFP DPC II) Social Safety Net Technical P103160 P 2009 Pakistan √ √ PMT Assistance Project National Social Protection P102542 P 2008 Bangladesh √ √ PMT Project Social Security for the P111867 P 2008 India √ √ PMT (BPL) Unorganized Sector Advisory activities for developing targeting tools TA for Welfare Reform for Sri A Sri Lanka √ √ PMT Lanka (PREM) Social Protection Programmatic A Bangladesh √ √ PMT TA (SP) TA for development of PMT for A Pakistan √ √ PMT Pakistan (PREM/SP) Notes: Status of the project is noted as: A – Active, C – Closed or P – Pipeline, at the time the questionnaires were sent out to project teams (i.e., March 2008). N.A. indicates that information received from the project TTL/team revealed that targeting was not an important element of the project; as a result, this project was not included in the final sample. N.R. indicates no response from an eligible project. CWR – community-based targeting; CAT – categorical targeting; PMT – proxy means testing; GEO – geographical targeting. * Combined with geographic targeting as a first step. 88 ANNEX 3: STANDARDIZED QUESTIONNAIRE FOR TTLS AND PROJECT TEAMS Targeting experiences in South Asia Taking stock of analytical and project work by World Bank teams March, 2008 A. INTRODUCTION Dear TTL: First let us thank you for taking the time to answer this questionnaire and providing us with very valuable, first-hand information on your experience with targeting issues in the context of your broader analytical and/or operational work. Even though (partial) answers to some of the basic questions that we have for you could be extracted from the project documents, most of the information we are asking about cannot be obtained this way for two basic reasons. First, most of our questions (and especially those in sections 4 to 7) focus on implementation issues and could therefore not have been answered during project preparation. Second, it is important for us to tap into the team‘s knowledge and perceptions as much as possible to ensure that we are capturing what actually happened once implementation got underway--hence the need to reach out to you directly. Having said this, we are not only focusing on projects under implementation but are also contacting TTLs who are currently preparing (rather than implementing) a specific project with a targeting element to it. If this is indeed your case, you may not be able to answers all the questions below. It does not matter! We still want to hear directly from you about as many issues as possible to the extent that they have been relevant for you and the team so far. The questionnaire is a combination of structured and open-ended questions. We have also included a final question (question 7) asking for your personal assessment. Answers to all three types of questions are important to us. Finally, before you go into the questionnaire itself, let us explain in a bit of detail what we mean by targeting. Basically we want to understand how beneficiaries of a particular intervention are being identified, both in theory and in practice. For instance, some projects will provide benefits/services to everybody in a particular area, in which case we are interested in learning how those areas were chosen (e.g. using social assessments, poverty maps, other indicators, etc.). Other projects will provide benefits/services to specific groups (maybe within a pre-selected area), in which case we are interested in learning how these groups and/or people were selected (e.g. using proxy-means testing, using categorical variables such as gender, etc.). Still other projects will not explicitly identify beneficiaries but rather put incentives in place for potential beneficiaries to come forward, and so on. It is this selection process and its implementation and monitoring/evaluation that we call targeting. Thanks again! 89 B. QUESTIONS 1. Basic project information Project name: Project code: TTL(s): Sector: Network: Project life (If project is under preparation, please specify expected Board date. If project is under implementation or has already closed, please provide effectiveness date and (expected) closing date) 2. Description of project Please provide a brief description of the program, including: 1. Main objective and basic design 2. Main components 3. Implementation agency(ies) 4. Loan amount 5. Target population 6. Number of beneficiaries/estimated coverage 7. Geographical reach and focus (urban/rural) within country: 8. Expected duration of the program, if different from duration of project 3. Choice and design of targeting tool Please provide information about the following: 3.1 What was the rationale/need for targeting? 3.2 Relation to/interaction with government systems, if any (please mark the appropriate answer below and provide comments as needed): Yes/No Comments Government mandated use of existing government systems. If so, please mention relevant system Project team decided on use of using existing government systems. If so, please mention relevant system Targeting tool developed for purpose of project contributed to and improved upon existing government systems. If so, please briefly explain the nature of improvement Targeting tool developed for purpose of project was different from existing government systems. If so, please mention main reason fro development of different tool Targeting tool developed for purpose of the 90 project in absence of government systems Other (please explain) 3.3 Methodological approach and design  What is the main targeting unit? Please mark relevant answer(s) Geographic (i.e. selected areas) Groups Households Individuals Other (please specify)  Briefly described the methodological approach chosen and provide any existing written references (e.g. PAD, technical background paper, etc.) containing a more detailed description  Why was this targeting method(s) chosen? Please mark all that apply. Need to work within existing system. If so, please mention relevant system Expected effectiveness of selected system Fiscal constraints Cost effectiveness Institutional capacity requirements/constraints Technical capacity requirements/constraints Logistical viability/sustainability Credibility with government Credibility with community Other (please explain)  Who was responsible for the technical design? Please mark relevant answer(s) World Bank staff Local consultant International consultant Government agency (please mention relevant agencies) NGOs Others (please specify)  Were alternative designs considered? If so, briefly explain which ones and why they were discarded. If not, please explain why. 3.4 Does the targeting tool allow for spatial/other comparisons? (i.e. Can targeting outcomes be compared across villages, districts, other geographic units? Can targeting outcomes be compared with/evaluated against those of other programs and/or with other criteria, such as monetary poverty?) 91 3.5 What were the main technical challenges and/or constraints the team faced regarding the design of the targeting tool? 4. Development and implementation arrangements for targeting Please provide information about the following: 4.1. Was the proposed tool piloted prior to implementation/roll-out? 4.2 If a pilot was conducted, please discuss the main implementation arrangements:  Was the targeting tool developed simultaneously for the entirety of the project area or was a phased approach followed? If phased approach, please describe.  Who was responsible for collection of information/implementation of the targeting tool(s?)? Please mark all that apply and provide a brief explanation on main responsibilities of each agent. Yes/No Main responsibility Project team Local government Higher tier(s) of government Independent survey firm/consulting firm Local facilitators NGOs Others (please specify)  Who was responsible for monitoring of targeting tool? Please mark all that apply and provide a brief explanation on main responsibilities of each agent. Yes/No Main responsibility Project team Local government Higher tier(s) of government Independent survey firm/consulting firm Local facilitators NGOs Others (please specify)  Who was responsible for appeals and grievances and how were these handled? Yes/No Main role/responsibility Project team Local government Higher tier(s) of government Independent survey firm/consulting firm Local facilitators NGOs Others (please specify) 92  Was training required for any of these partners and, if so, how was training organized and administered? 4.3 If a pilot was conducted, please explain what the main lessons learned were and how did those influence the final design and recommendations. Also please provide any written references describing the pilot results and conclusions. 4.4 Targeting information and record keeping  How was the information needed for targeting collected?  Where is this information kept and in what format? Please mark all that apply. Centralized database (i.e. digital form) Centralized register (i.e. paper form) Multiple connected databases Multiple connected registers Multiple unconnected databases Multiple unconnected registers Other (please specify)  Who is responsible for maintenance of databases/register? Please mark all that apply and provide a brief explanation on main responsibilities of each agent. Yes/No Main responsibility Project team Local authorities Independent survey firm/consulting firm Local facilitators NGOs Others (please specify) 4.4 Updating of targeting tool  Are there any provisions for systematic update of targeting information? If so, with what periodicity? If not, why not?  Has the targeting tool been applied more than once to the same individuals, communities, etc? If so, please discuss the following: o Was the targeting methodology modified in any manner with respect to the first time the tool was used? If so, why? o Were the implementation arrangements modified in any manner with respect to the first time the tool was used? If so, why? o Were targeting outcomes significantly different across rounds? If so, please discuss potential reasons paying special attention to the following issues: (i) significant change in socio-economic context, (ii) high levels of transitory versus 93 structural poverty, (iii) potential learning about functioning and/or purpose of targeting tool among potential beneficiary households o What were the main lessons extracted from the second targeting round? 4.5. What were the main technical challenges and constraints the team faced regarding the implementation of the targeting tool? 5. Evaluation of targeting tool 5.1 Has an evaluation of the targeting tool been conducted? If so, please describe the main results of this evaluation paying special attention to the following factors: (i) targeting effectiveness (i.e. inclusion/exclusion errors), (ii) costs effectiveness, (iii) administrative sustainability, and (iv) technical sustainability (i.e. potential impact of learning on accuracy of information and targeting outcomes over time). If not, please explain why. 5.2 What is the perception among beneficiaries, communities, local authorities and others of the effectiveness of the targeting tool in identifying the poor—please pay special attention to perceptions of fairness, accuracy, and transparency? In your answer please try to distinguish between perceptions about the targeting tool and the program as a whole (including adequacy of program coverage and other program dimensions). 6. Impact of targeting tool beyond the project 6.1 Has the targeting experience developed under this task influenced (i) the policy dialogue and/or (ii) the government‘s approach to targeting beyond the context of the project? 6.2 Can the proposed methodology be scaled up beyond the scope/coverage area of the project? If so, are there plans to do so? Please explain briefly why, how and in what context. Also discuss who will be responsible and how will this be financed. 7. TTL assessment Please provide a brief assessment of the overall experience with targeting, discussing most important lessons learned, main strengths of proposed approach, possible weaknesses and areas for improvement, any other thoughts. Finally please list any additional references, such as any impact evaluations or publications on program implementation that you may consider relevant for the purpose of this work. Thanks!!!!! 94