69641 Poverty and Social Impact Analysis for Sri Lanka: a Case Study An Analysis of Impact of Welfare Reform Poverty Reduction & Economic Management South Asia Region World Bank June, 2005 Preface and Acknowledgements The policy note on Welfare reform is the first of a series of Poverty and Social Impact Analysis (PSIA) planned for Sri Lanka. With the completion of the Poverty Reduction Strategy Paper (PRSP) in 2003, a set of reforms were chosen for such an analysis in consultation with the Government of Sri Lanka. These include: (a) welfare reform, (b) labor market reform, especially regarding the regulations governing termination of employees in the formal sector, and (c) reform of the power sector. In addition, a PSIA on potential land reform has also been initiated. Since March 2004, with the change in government, the reforms envisaged in the PRS of 2003 including those stated above have undergone some rethinking. The upcoming PRSP will articulate in greater detail the current government’s thinking and policy priorities. In the context of welfare reform, specifically as it pertains to the Samurdhi program, there appears to be no reversal of the intention to improve its targeting – indeed the formula based approach is moving forward in the North and East where the Samurdhi program is being introduced for the first time. On reforming the system in the South, however, the government is yet to reach a decision on when and how to move forward. This note focuses on welfare reform – other areas such as labor and power will be covered at a later stage, depending on the likelihood and timing for the implementation of these reforms. The land PSIA was initiated only recently, and more data and analysis is expected to complete this exercise. Given the evolving nature of reforms, we expect the PSIA to be an ongoing exercise at least for the coming year. This policy note synthesizes much of the technical assistance provided to GOSL under the non- lending TA for welfare reform conducted since FY 03. More specifically, it focuses on all the analytical support and collaboration with the Welfare Benefits board and the relevant ministries to provide the necessary impetus for reform. This note draws on (a) the detailed technical paper on the derivation of the targeting formula (Narayan and Yoshida; 2004), (b) two policy workshops to share the insights from the pilot exercise and to discuss implementation steps and timeline and (c) the briefing notes prepared for Welfare Benefits Board and Ministry of Finance. The TA for Welfare reform also included efforts to support the ministries and build capacity for implementation and the information system to record and track beneficiaries, which are not explained in great detail here because the focus of this note is on impact. The PSIA and TA for Welfare Reform are task managed by Tara Vishwanath (SASPR). In addition, the core team for this policy note consists of Ambar Narayan and Nobuo Yoshida (SASPR). The team for the Welfare Reform TA includes – in addition to the above, Princess Ventura (SASPR), Francisco Ayala (operations consultant), Yoko Kijima (consultant), Hernando Quintero (MIS consultant) and S. Sivakumaran (Welfare Benefits Board). Comments from PREM network (Anis Dani) and Kapil Kapoor (Sector Manager, SASPR) are gratefully acknowledged. We are thankful to Peter Harrold (Country Director, Sri Lanka) for enduring support and optimism in effecting a challenging reform. Several government officials were instrumental in guiding the process – including members of the Steering Committee set up by Ministry of Finance, officials in the Samurdhi Ministry and the members of the Welfare Benefits Board. This policy note will appear in a forthcoming volume "Lessons Learned from Case Studies on Poverty and Social Impact Analysis", to be published by the PREM network next fiscal year. II Table of Contents Executive Summary ......................................................................................................................... V Welfare Reform in Sri Lanka: a PSIA case study ............................................................................ 1 I. Welfare programs in Sri Lanka: the current state ............................................................... 1 II. The impetus for welfare reform.......................................................................................... 3 III. The current Samurdhi program and key principles of the reform .................................. 4 IV. The current status of reforms and the role of PSIA ....................................................... 5 V. Design of Poverty and Social Impact Analysis .................................................................. 6 VI. Results from Poverty and Social Impact Analysis ....................................................... 10 VII. Impact of PSIA............................................................................................................. 27 VIII. Lessons learned ............................................................................................................ 29 References ..................................................................................................................................... 31 Annex ................................................................................................................................................ i A. Expenditure composition of Ministries of Samurdhi and Social Welfare ........................... i B. Characteristics of current Samurdhi transfers ..................................................................... i C. Proxy Means Test Formula (PMTF): rationale and evidence ............................................ ii D. Issues in deriving PMTF for Sri Lanka using SLIS data ................................................... iii E. Additional Tables and Figures ......................................................................................... vii III Tables and figures in main text Table 1: Results from different models ......................................................................................... 14 Table 2: Undercoverage rates ........................................................................................................ 16 Table 3: Leakage rates ................................................................................................................... 16 Table 4: Coverage of population by per capita consumption deciles ............................................ 17 Table 5: Incidence of total budget by decile (%)........................................................................... 19 Table 6: Applying the selected PMTF to the pilot sample: program coverage ............................. 21 Table 7: Coverage rate for likely "vulnerable" groups .................................................................. 22 Table 8: Undercoverage rate among vulnerable groups ................................................................ 22 Figure 1: Coverage of population by consumption decile ............................................................. 17 Tables and figures in Annex Table A- 1: Samurdhi grant amounts (Rs.) and recipients (1999).................................................... i Table A- 2: Samurdhi grant amounts (Rs.) and recipients (2003)*.................................................. i Table A- 3: Deductions from Samurdhi grants in 2003 ................................................................... i Table A- 4: Poverty headcounts for Sri Lanka .............................................................................. vii Table A- 5: PMTF (weight on each variable for the selected models).......................................... vii Table A- 6: Regression results from OLS estimations ................................................................. viii Table A- 7: 95% Conf. Intervals for undercoverage and leakage rates with PMTF (Model III) .. ix Table A- 8: Per capita benefits (1999 Rs./Month)........................................................................... x Table A- 9: Composition of pilot applicants by district .................................................................. x Table A- 10: Coverage rate of pilot areas in the North-East and the rest of the country ................ x Table A- 11: Results with revised eligibility criteria for small households ................................... xi Figure A- 1: Poverty in Provinces - Sri Lanka ............................................................................... xi Figure A- 2: Share of provinces in poor population and Samurdhi budget .................................... xi Figure A- 3: Comparison of distribution of scores (Pilot score, SLIS actual, SLIS score) ........... xii Figure A- 4: Coverage of Samurdhi and PMTF in pilot areas ..................................................... xii IV Executive Summary Sri Lanka has had a long history of social welfare programs, with the most significant among current ones being the Samurdhi transfers program. Expenditure on the Samurdhi transfers program constitutes the highest budget item in welfare spending for poor families. At its peak in 2002, expenditures of the Ministry of Samurdhi consumed close to 1 percent of GDP and 4 percent of the government budget. A number of studies, including an evaluation conducted by the World Bank in 2000 point to large scale mis-targeting and exclusion by the Samurdhi program, with a large share of the expenditure actually going to the top quintiles. Moreover, qualitative results suggest that political factors, including party affiliation or voting preferences appear to influence allocation of Samurdhi grants. This policy note summarizes the efforts to improve the targeting and poverty impact of the Samurdhi program, through a combination of reforms: re-orienting the selection of beneficiaries of the program from the current subjective criteria to a formula-based system that allows for greater objectivity, and creating institutional capacity to support the changes in the selection process. In order to develop such a formula-based system, a careful analysis based on household survey data was conducted to derive a proxy-means test formula (PMTF), which involves using information on household or individual characteristics correlated with welfare levels in a formal algorithm to proxy household income or welfare. The advantage of such an algorithm is that it allows ranking of households based on a measure of welfare using characteristics – such as demographic data, characteristics of dwelling units and ownership of durable assets – that are more observable and verifiable than direct measures of welfare like consumption or income. Success of the reform will also critically depend on institutional arrangements for implementation. While the creation of the Welfare Benefits Board (WBB) represents a useful beginning, more remains to be done. An important part of that would be to institutionalize WBB’s original role as envisaged by the Welfare Act (2002) – to be responsible for selecting beneficiaries and monitoring entry and exit from the program independent of the implementing Ministry. The PMTF must also be complemented by a strong community-based process for independent validation of beneficiary lists and addressing appeals in order to minimize errors of exclusion, which will require setting up community committees with broad representation. Analysis to determine the PMTF, field verification through a series of pilots and a continuous process of consultation that included a series of technical meetings and policy workshops, resulted in a number of recommendations on the target group of the reformed program, the payment amounts and institutional changes necessary to facilitate implementation. Among the most important was the recommendation that the reformed program targets the poorest 30 percent of the population. It was also recommended that the benefit paid to each household include a variable component calculated per “vulnerable� household member (child, disabled, or elderly) – an easily observable indicator that also ensures that the payment amounts are “progressive� (larger amounts for the needier). The reformed system, when implemented, is expected to lead to significant improvements in the distribution and incidence of benefits: the lowest 3 deciles will receive 66 percent of the benefits after the reform, as compared to only 37 percent of the benefits under the current system. Moreover, the poorest 10 percent of the population will be covered more extensively compared to the current system, which is consistent with the fundamental objective of any safety net program. The iterative approach of technical analysis informing policy dialogue, along with support to build capacity for implementation, has enabled this exercise to be effective in terms of creating an impetus for change, and providing the technical underpinnings to inform ex ante the implications of such change. This also necessarily demands a multi-year engagement, especially in the case of V such a reform that is politically difficult to implement because of strong incentives to maintain status quo. Since the beginning of this work, a change in the government in March 2004 has led to some rethinking about the pace and sequence of implementation. Currently, the reformed approach is being implemented in the Northeast, where Samurdhi is being introduced for the first time ever. Extending this effort to the South is necessary and part of the ongoing policy dialogue with the government. The reform in the South, since it involves transforming an existing system, does represent a difficult challenge. The political risks of this process may be mitigated by a concerted effort to develop viable alternatives for livelihood support (programs like microfinance) for those who have to exit out of the reformed transfer program, and the government’s recent development strategy rightly stresses the need to prioritize such programs. VI Welfare Reform in Sri Lanka: a PSIA case study 1. The attempt to transform the social welfare programs in Sri Lanka represents one of the key reform efforts undertaken by the country during the last five years, which can help achieve the objective of providing consumption support to the poor and sustaining expenditures on such programs in the medium-term. The effort at reforming the welfare system of the country also offers a lesson on the difficulties in overcoming the systemic inertia that characterizes a long- running program, especially one that has had a history of politicization. Furthermore, it showcases the complex process through which such reforms evolve, and how the changing political circumstances of a country govern those processes. In the country-context of Sri Lanka, this reform is also a useful test-case for reforms that need considerable political will and commitment to implement, in spite of the potential benefits they promise. The critical nature of this particular reform in the country context and the lessons it offers for other countries with similar entrenched programs are the main reasons why this reform was selected for conducting a PSIA. 2. The engagement of the World Bank in this particular reform has been that of a partner, providing support in developing solutions on how to select beneficiaries of the program in an objective and transparent manner, provide benefits that are meaningful in terms of meeting basic needs and build institutional capacity to implement decisions adopted by the government. The PSIA exercise has thus evolved organically from this partnership, driven by the close collaboration between the Bank team and the government counterparts, and not as a separate activity. In other words, much of the analysis described below has emerged out of the work conducted to support the government counterparts in designing the reform – consultations with the government and other stakeholders, empirical exercises using household data to develop a targeting method, a pilot to corroborate the ex ante analysis of impact, and workshops with stakeholders. 3. While the reform is still ongoing, it is instructive to document this PSIA case study for two reasons: (a) to illustrate how ex ante impact analysis can be used to motivate and provide impetus to the reform process itself; and (b) to underscore the importance of building institutional capacity to enable implementation – key for realizing the benefits of the diagnostic insights in a PSIA. I. Welfare programs in Sri Lanka: the current state 4. To contextualize the role of welfare programs in Sri Lanka, it is useful to briefly refer to the status and pattern of poverty in the country. Over the last two decades, decline in poverty has been modest and marked by rising inequality. Between 1990-91 and 2002, the national poverty headcount ratio fell by around 3 percentage points (from 26.1 percent to 22.7 percent), and was at the same time accompanied by rising inequality – between sectors and provinces, and households (see Table A-4, Annex). Poverty incidence in urban areas was almost halved, while it fell by only 5 percentage points in the rural sector and increased by 10 percentage points in the estate sector; differences across provinces have also been pronounced, with a headcount rate (in 2002) of around 11 percent in Western Province contrasting sharply with one of around 35 percent in Sabaragamuwa and Uva (see Figure A-1, Annex). Rising inequality is reflected by the fact that while mean per capita consumption for the country increased by about 29 percent in real terms between 90-91 and 2002, the increase was around 50 percent for the top quintile (of per capita consumption expenditure) and only 2 percent for the bottom quintile. 5. Poverty in Sri Lanka is thus concentrated geographically, not only in the poorer provinces, but also in pockets of deprivation even in the better-off provinces. The heterogeneity across areas 1 and rising inequality across households make it all the more important for social welfare programs to target government assistance to those in need effectively, ensuring a minimum level of consumption while being consistent with the equity objective of redistributing some of the benefits of growth. The typical clientele for such assistance would consist of the poor, and especially the vulnerable among them – the disabled, the aged, children, and households headed by single women. When measured up against these broad objectives, the social welfare sector in Sri Lanka presents a decidedly mixed picture. On the one hand, a long history of country-wide programs for the poor and vulnerable have created an enabling environment and a consensus around such programs; on the other hand the effectiveness of these programs has been far below potential due to inefficient targeting and inadequate coordination across programs. 6. A multitude of overlapping programs administered by a number of Ministries constitutes the social welfare sector in Sri Lanka. Total expenditure for welfare programs is sizeable – amounting to an estimated 3.2 percent of GDP and 12.2 percent of total government expenditure in 2003, distributed among programs like Samurdhi consumption grants to poor households, pensions to retired government personnel, fertilizer subsidy, expenditures on school uniforms and textbooks, payments to disabled soldiers and dry rations to refugees. 1 A large share of social welfare programs are administered by the Ministry of Social Welfare, which is focused on targeting the disadvantaged sections of society, and Ministry of Samurdhi which also subscribes to development objectives that go beyond that of a welfare/income support program. 2 The Samurdhi transfers program is the most significant of targeted welfare programs, expenditure on which constitutes the second highest budget item in welfare spending (after pension payments to retired public servants). At its peak in 2002, expenditures of the Ministry of Samurdhi consumed close to 1 percent of GDP and 4 percent of the government budget. 3 7. A review of expenditures of the Ministries of Samurdhi and Social Welfare reveals that more than two-thirds of the budgets of the two Ministries are spent on direct transfers to households, although the stated objective of Samurdhi would suggest otherwise (see Annex A). Together, the two Ministries cover as much as half of the population of the country as beneficiaries. In the case of the Ministry of Samurdhi, the transfer goes into the payment of the Samurdhi consumption grant to poor households. In the case of the Ministry of Social Welfare, a large part of the transfer payments goes to disabled soldiers and families of soldiers killed in action (92 percent of transfers in 2003), leaving only a small share for other beneficiaries like vulnerable children, or those affected by natural disasters, disability, or old age. 8. At the same time, there is little coordination between the ministries, with the result that there appears to be considerable overlap between the beneficiaries of the programs administered by the two ministries. Along with expenditures on transfers, the other significant category of recurrent spending is administrative costs, including payments on personal emoluments – especially for the Ministry of Samurdhi which relies on a large cadre of staff to administer the program. 4 The large 1 Other than programs run by different Federal agencies, the Provincial Ministries are also involved in the implementation of social welfare programs, notably the disbursement of the “Poor Relief�, which is financed by the decentralized budgets. 2 An average of Rs. 13.2 bn. was allocated to Samurdhi between 2000 and 2003, which amounts to 1 percent of annual GDP, over 10 percent of social service expenditures and a third of welfare expenditures. The Ministry of Social Welfare and its two associated departments consume only 0.2 percent of GDP, 3 percent of the social services expenditure and 7.5 percent of the spending on welfare. 3 The transfer component of the Samurdhi program had an allocation of Rs. 9 billion (around $ 90 million) in FY 2003, which amounted to 0.75 percent of GDP and about 3 percent of total government spending. 4 For instance in 2003, recurrent spending for the Samurdhi Ministry accounted for Rs. 12.2 bn. out of a total of Rs. 12.5 bn., of which Rs. 9.2 bn. was spent on consumption grants or transfers, and Rs. 2.8 bn. on the salary bill of the Samurdhi Authority (about 22 percent of the total expenditure of the program). 2 salary bill of the Samurdhi program in particular constitutes a drain on the Ministry, and reduces the welfare impact of spending. 9. Efficiency of welfare expenditure in terms of its impact can thus be enhanced by better rationalization between the programs of different Ministries, in terms of their stated objectives, target groups and coverage of beneficiaries. Effectiveness of spending, it also appears, is adversely affected by large-scale errors in targeting of beneficiaries – particularly for the Samurdhi program which by far dominates the other transfer programs in terms of budget and coverage. A number of evaluations in the past have indicated serious flaws with the Samurdhi program that have led to significant mis-targeting, which has in turn led to its impact on poverty being far less than its potential. As described in Section III, this is largely the result of employing highly subjective criteria for identifying households eligible for Samurdhi benefits. II. The impetus for welfare reform 10. The need for reforming the welfare system, to achieve better impact for the considerable expenditure in this sector, was identified as a high priority in its reform agenda by the Government of Sri Lanka in its Poverty Reduction Strategy (PRS) of 2003,5 and this view has subsequently been shared by the World Bank’s Country Assistance Strategy of 2003. Responding to this need, the new Welfare Benefit Act was enacted by the Sri Lankan Parliament on July 12, 2002 to rationalize the legal and institutional framework of all social welfare programs, reduce politicization in the selection of beneficiaries of the program and improve the targeting performance of the Samurdhi foodstamp program in particular.6 11. To help implement the reform along the lines envisaged by this Act, the Government of Sri Lanka requested technical advice and assistance from the World Bank. Responding to this request, a Bank team worked since 2002 in close collaboration with a team comprising of local officials and international consultant; a Steering Committee, comprising of senior officials from key ministries and statistical organizations, provided overall direction during the early stages of the process. The Welfare Benefits Board, set up according to the guidelines provided by the Act, has been the implementing agency in terms of coordinating the exercise, including the pilots that have been completed and the actual implementation process that is currently underway in some parts of the country. While rationalization of targeting and beneficiary selection across all welfare programs is the ultimate objective of this exercise, reforming the Samurdhi transfers program was identified as the first priority. This is because this program accounts for the bulk of spending on transfers (excluding payments to disabled soldiers and families of armed forces), and the most substantive evidence of mis-targeting has been available so far for this program. 12. In accordance with this priority, the PSIA focuses on the Samurdhi transfers program. More specifically the PSIA examines the potential or ex ante impact from the reform that would consist of: (i) transforming the process of selection of beneficiaries from the current system that relies on subjective judgments to one that applies more objective criteria, as well as incorporates institutional mechanisms for community-based validation and appeals process to redress grievances; and (ii) implementing payment amounts that are linked to appropriate and observable characteristics of poor households. This process is expected to yield better targeting that reduces exclusion of the poor from the program, makes greater resources available for poor beneficiaries by reducing leakage of benefits to the non-poor, and lend more objectivity and transparency to the process of selection, validation, appeals, payment amounts and exit from the program. The 5 Regaining Sri Lanka: Vision and Strategy for Accelerated Development (2002) 6 The passage of the Welfare Benefits Act was also one of the prior actions for World Bank’s budgetary support to Sri Lanka (PRSC-I, 2003). Progress in implementing the Welfare Benefits Act was agreed upon with the government as one of the prior actions for a future PRSC-II. 3 impact of the reform will be analyzed in terms of measurable indicators like the incidence of benefits and beneficiaries, coverage of vulnerable groups, and the aggregate impact on poverty/welfare. While the results are based on statistical simulations using household survey data, many of the key findings have also been corroborated with the help of a pilot for the targeting exercise central to the reform, and comparing its results with those from the ex ante simulations. It is also important to realize that the results of this exercise have been instrumental in designing the implementation of the reforms on the ground. III. The current Samurdhi program and key principles of the reform 13. A recent evaluation conducted by the World Bank (2000) suggested that Samurdhi’s targeted foodstamp and cash transfer program, which constitutes 80 percent of the total program budget, misses about 40 percent of households ranked in the poorest consumption quintile, while almost 44 percent of the total budget is spent on households from the top three quintiles. Qualitative results suggest that political factors, including party affiliation or voting preferences influence allocation of Samurdhi grants. Large-scale leakage of benefits has led to the program covering as much as half of the population with the result that the benefits are spread too thinly, with the size of transfers being too small to have a discernible impact on poverty. 7 The high incidence of errors in targeting is also evident from observed large discrepancies between the distribution of Samurdhi beneficiaries across districts (or provinces) with that of poor population across districts (or provinces) available from independent household surveys. 8 Thus it appears quite obvious that an improvement in targeting of Samurdhi by reducing leakage and exclusion can potentially have a sizeable impact on poverty relative to what is seen currently, as subsequent sections of this chapter will show. 14. The errors of targeting of the current Samurdhi program are related to a number of factors, perhaps the foremost among them being the way the program is administered on the ground. The criteria for selection of beneficiaries are a combination of the income of a family and the number of family members; how these criteria are applied is left completely to the program officers on the ground. Since income is generally unobservable and almost impossible to corroborate, this results in a process of selection that is largely subjective and dependent on how the program officers translate these criteria into selection of beneficiary families. The identification of beneficiaries is essentially left to the program officers (more than 26,000 in number), without any process of community validation or formal mechanism for redressing grievances or monitoring of entry and exit. Naturally, this results in a process that is inherently non-transparent to beneficiaries and non-beneficiaries alike, and is also vulnerable to political capture and patronage. 15. There are a number of other factors that further distort the outcomes of this program. The fact that it targets by “families�, rather than the more easily identifiable and objective concept of a “household�, provides incentives to households to split themselves into families to obtain multiple payments. Little or no monitoring of the status of beneficiaries occurs, with the result that once a family enters into the program, there is no transparent or easily enforceable way of “graduating� the family out of the program. Exit from the program, therefore, when it occurs, is imposed rather arbitrarily on families – usually as a result of budget cuts that compel the Samurdhi Authority to eliminate a certain number of beneficiaries from the program without having a clear-cut set of criteria for doing so (this occurred in 2003 when the Samurdhi budget 7 See Glinskaya (2000) for a detailed evaluation of the Samurdhi program. 8 See Figure A-2, Annex for a comparison of share of poor population (using the official survey – HIES 2002) with share of Samurdhi transfer budget received by each province. Uva and Sabaragamuwa account for 29 percent of the poor population, but receive only 22 percent of Samurdhi transfers. Western, North- Central and Northwest provinces together account for 38 percent of the poor, but receive 47 percent of the transfers. 4 was reduced). There are no formal mechanisms for lodging appeals or redress grievances, which adds to the non-transparent nature of the selection process. 16. The problems with the Samurdhi system have also extended to the practices surrounding payment of benefits. A large number of beneficiary families actually end up getting reduced amounts or no transfers at all, primarily because a “compulsory saving� component (along with smaller compulsory contributions for social insurance and a housing lottery) is held back and deposited in the Samurdhi Banking societies in the name of beneficiaries. In 2003, compulsory deductions amounted to an estimated 22 percent of the total value of Samurdhi grants, with the savings component alone accounting for 14 percent (see Annex B for a description of the Samurdhi grant amounts and compulsory deductions). Given that the Samurdhi transfers are intended to augment the consumption of families who are currently at or below the minimum threshold, there seems to be little rationale to justify forced savings to postpone consumption. The rationale is even weaker since – as assessments have shown – households have highly restricted access to these savings and are often unaware of how much savings they possess, how much are they allowed to withdraw, and under what conditions. As highlighted later in Section VI, the Samurdhi Ministry appears to have recognized these problems recently and discontinued the practice of deducting compulsory savings since early 2005. 17. The aforementioned Welfare Benefits Act that set the parameters of the reform seeks to address the key deficiencies of the Samurdhi program summarized above. More specifically, the Act mandated an independent Welfare Benefits Board (WBB) and community level committees to set eligibility criteria, validate entry and exit into the program, and redress appeals. The Act also envisaged setting objective criteria for selection of beneficiaries, to minimize the level of subjectivity in the selection process and lend it greater transparency. The institutional separation between the selection of beneficiaries and the payment of benefits – with the WBB undertaking the former task, and the Ministry of Samurdhi having the responsibility for the latter – was intended to reduce the incentives for distortion of the selection process. The role of the WBB also included monitoring entry and exit into and out of the program, through periodic collection of data, and setting the rules for redressing grievances through an appeals process managed by village or Grama Niladhari (GN) division level committees. 18. While the Act sets the broad institutional parameters of the reform, it is in the process of operationalizing these broad principles that many of the other problems of the Samurdhi program can be addressed. In the course of describing the PSIA exercise below, some of the most important features of the operational design of the reform will be highlighted. This will not only help understand how the reforms address the multiple shortcomings of the current program – which are indeed critical for realizing its potential benefits – but also point to potential areas of risks in implementation on the ground. IV. The current status of reforms and the role of PSIA 19. After the change in government in March 2004, the pace and sequencing of the reform has undergone some re-thinking. Implementation appears to be progressing in phases. An effort, led by the Ministry of Samurdhi with the help of WBB, is underway since December, 2004 to extend Samurdhi to the conflict affected areas of the North and East. This represents the first-ever effort to integrate the North and East into a national safety net program, and also the first attempt in the country to operationalize the formula-based targeting system that constitutes the crux of the proposed reform. It is still unclear however as to when a similar effort will be extended to the South, where the additional difficulty arises from the need to transform an entrenched system. The ongoing effort in the North and East will have key implications for the rest of the country. Successful implementation will demonstrate the benefits from such a program, as well as provide critical operational lessons for future implementation elsewhere. 5 20. In its recent development strategy (also the country’s new PRS) presented at the Sri Lanka Development Forum (2005), the government acknowledges the need to reform targeting of the Samurdhi transfers, and also quite rightly emphasizes the need to develop alternate development programs. 9 Such programs would be especially necessary for those who are not among the poorest – and would thus be left out of the reformed transfer program – but would benefit from credit or other forms of support for livelihood generation. In recent discussions with the Bank, the Ministry of Samurdhi has reiterated the need to provide such alternatives on a large enough scale, which would also likely make the reform of the Samurdhi transfer program more politically feasible in the South. 21. The current Samurdhi infrastructure does provide such options to scale up other livelihood support programs. The existing Samurdhi Banking Societies spread all over the country together have assets of over 13 bn. rupees, out of which only a low proportion is currently being utilized for lending. Ministry of Samurdhi plans to radically expand the microfinance role of the Samurdhi Banks, and have the Samurdhi Development Officers undertake social mobilization to support this initiative. An effort is already underway to train these officers (whose primary role till now was to select beneficiaries of Samurdhi transfers) in their new roles – namely mobilizing communities and coordinating among local officials from different government agencies to support livelihood activities. On being requested by the Ministry, the World Bank is currently considering options for appropriate technical and/or financial support to facilitate this effort. An assessment of Samurdhi Banks, to examine the institutional changes necessary for these to become a viable microfinance network, is also being planned in partnership with the Ministry. 22. The PSIA exercise, given that the government is still in the process of deciding the modalities of extending the reform to the South, takes on added importance. Results from the PSIA, by clearly identifying the impact on poverty and distributional implications of the reform, can potentially play an important role in informing the government’s decision-making process and the World Bank’s policy dialogue with the government on this key reform agenda. 23. The role of the PSIA in moving the reform agenda forward has also assumed added importance due to the challenges facing the welfare system of Sri Lanka in the aftermath of the Tsunami. As a part of its post-tsunami recovery program, the Government introduced a cash grant program in the affected areas, which is being supported by the Bank’s Emergency Assistance project. This program is intended as a short-term livelihood support to the worst- affected households. But as this program winds down, as it is expected to by June 2005, there will be a need to integrate the most vulnerable among the affected into the existing welfare system. 10 The potential entry of this vulnerable population will represent an additional challenge to the current Samurdhi system – in terms of budgetary needs as well as the institutional capacity to identify and target such households. Addressing this challenge will become much easier if the Samurdhi reform is implemented – by increasing the efficiency of spending or targeting, and a targeting mechanism that is able to objectively identify beneficiaries and track and monitor entry and exit into the program. V. Design of Poverty and Social Impact Analysis 24. As mentioned above, the PSIA for Welfare Reforms in Sri Lanka has evolved out of the technical support to the government by the Bank team over the past two years, and is therefore, in 9 New Development Strategy, a Framework for Economic Growth and Poverty Reduction (2005) 10 According to the draft Beneficiary Assessment (May, 2005) of the first two rounds of the cash grants program for tsunami victims, Samurdhi may have to cover an estimated 8-21 percent of households in addition to those already covered in 10 worst-affected districts (the wide range reflects different definitions of “vulnerability� among tsunami-affected households). 6 large part, the product of joint work undertaken by the Bank team and counterparts on the government side. The PSIA exercise incorporates the components listed below, all of which were also central to the process of designing the reform and building institutional capacity for implementation: Technical analysis based on household data: 25. The objective of this exercise is threefold. First, it is intended to develop an objective set of criteria to identify beneficiaries of the Samurdhi program. In principle, conducting a direct “means test� that correctly measures the earnings of a household is the best way to determine eligibility when the poor are the target group, as is the case with Samurdhi. In practice, however, such straightforward means tests suffer from several problems, the most important being that verifying incomes of households is very difficult in developing countries where reliable income records do not exist. Thus the idea of using an econometric exercise to derive a viable Proxy Means Test Formula (PMTF) that avoids the problems involved in relying on reported income is appealing. A PMTF uses information on household or individual characteristics correlated with welfare levels in a formal algorithm to proxy household income or welfare. These instruments are selected based on their ability to predict welfare, using data from an appropriate household sample survey. The obvious advantage of proxy means testing is that good predictors of welfare – like demographic data, characteristics of dwelling units and ownership of durable assets – are likely easier to collect and verify than are direct measures like consumption or income. 11 For these reasons, PMTFs have being widely used around the world for targeting safety net programs. 26. In functional terms, a PMTF is derived from a regression of the measure of welfare (e.g. per capita household consumption) on a set of welfare proxies, using household survey data. The regression is used to identify the combination of indicators that are (a) best able to “predict� the actual welfare of households, and (b) easy to observe and conversely difficult for households to distort; as well as to assign a “weight� to each selected indicator. In order for the PMTF to be applied, information on the welfare predictors must be collected from all applicant households using a simple application form. For every applicant household, the collected information is used to compute a PMTF “score�, which is then used to accept or reject the household from the program based on a pre-decided selection “cutoff� score. 27. The choice of PMTF as the targeting method: The Steering Committee set up by the Government of Sri Lanka to guide the technical aspects of the reform process recommended a PMTF to identify the target group for Samurdhi benefits after careful deliberation, where senior officials from line ministries, the Statistics department of Central Bank and Department of Census played key roles. Initial discussions had made apparent the lack of consensus – even among the Steering Committee members – on an intuitive set of criteria that best identifies poor households without the benefit of supporting statistical analysis and evidence. And introducing such statistical analysis made it apparent that finding the best possible combination of criteria would necessarily require a multivariate regression of a welfare measure on potential “proxies� for welfare – namely a PMTF. 28. Above all, the PMTF methodology was considered to be consistent with the overall objective of the reform in the Sri Lankan context: to replace an entirely subjective and non-transparent process of beneficiary selection (consequently vulnerable to politicization), with one that employs 11 A comparative study of targeting in Latin America (Grosh, 1994) found that among all targeting mechanisms, proxy means tests tend to produce better incidence outcomes in developing countries (for more on the theory and academic evidence on proxy means tests, see Annex B). Coady et al (2004) shows that successful targeting depends critically on how a method is implemented. Both proxy means testing and community-based selection of individuals for instance show good results on average, but with considerable variation across countries. 7 relatively transparent and objective criteria that are able to identify the poor more accurately than the current system. The few alternatives to PMTF that are available for targeting an “unconditional� transfer like Samurdhi were ruled out when their pros and cons were balanced against those of the PMTF method. Direct means-testing, for reasons described above, was not possible. Community-based targeting or relying completely on communities to select beneficiaries, while an option in some countries, was not considered appropriate in Sri Lanka, given the past history of distorting such mechanisms – Samurdhi and its predecessor, Janasaviya being the recent examples. In these cases, the absence of a set of objective and observable criteria was the obvious handicap that the PMTF was expected to address. Geographic targeting, i.e. saturating specific geographic areas with transfers, was considered infeasible primarily because it would be politically unacceptable to transform a national program into one that benefits entire populations in certain areas to the complete exclusion of others. Given that poverty in Sri Lanka occurs in small isolated pockets even in better-off areas, such an approach would also run the risk of completely eliminating groups like the poor in Colombo city, which would further reduce its acceptability. 12 29. Along with the decision to adopt a PMTF, it was also agreed that this method cannot be a solution in isolation, and must be complemented by a community-based mechanism that acts as an independent check to the selection process. Like all statistical exercises, a certain degree of errors must be expected from the PMTF – arising from imperfections in the predictions of the model/algorithm itself as well as errors in the information on which the formula is applied. While these cannot be eliminated completely, a transparent community-based mechanism for validating the list of beneficiaries selected and evaluating appeals can help reduce such errors. The appeals mechanism specifically is critical to correct errors of exclusion arising from a variety of sources like incorrect information on households, improper application of formula, or special (and thus rare) circumstances of specific households that no statistically derived formula can take into account. In order for the PMTF to be effective, it is also equally important to have an institutional setup to implement the system – including the community-based processes for validation and appeals. 13 30. Second, once the PMTF is derived from a regression exercise using suitable household survey data, recommendations can be made on (a) the cutoff point in terms of the “score� computed by the PMTF that defines the target group, which in turn determines what proportion of the population is targeted; and (b) appropriate payment schemes that optimize impact on welfare and take into account distributional concerns within the target group, for a given budget constraint. These in turn make it possible to ex ante simulate the impact of targeting by PMTF (with the selected cutoff and payment scheme) on targeting accuracy, the distribution of beneficiaries across income/consumption groups and the consequent impact on selected poverty measures, and coverage of vulnerable groups of interest, relative to the existing system employed by the Samurdhi program. 14 12 Geographic targeting also places enormous premium on the ability to estimate accurately highly geographically disaggregated poverty rates – which poses a technical challenge under any circumstances. A recently completed poverty mapping exercise has yielded poverty estimates at the level of DS division and below (see World Bank, 2005), which can in future be used a reference point for validation of targeting outcomes – for existing Samurdhi as well as post-reform. 13 See Coady et al (2002), who argue that successful targeting is critically dependent on implementation. 14 This is possible since the household survey data used to derive the PMTF for Sri Lanka also contains information about who receives Samurdhi benefits. 8 31. Third, the above ex ante simulation exercise using household data can also be used to identify the implications of different budgetary allocations for the program, by clearly measuring the outcomes or impacts corresponding to different levels of allocation. 15 Corroboration from a pilot: 32. In order to validate the results of the simulations using household sample survey data, a pilot targeting survey covering 48,000 households in 114 GN (Grama Niladhari) divisions was conducted by the Welfare Benefits Board with technical assistance from World Bank in June- August, 2003. The pilot’s primary objective was to test the proxy-means tested application process for selection of beneficiaries of the Samurdhi program, by conducting an application process using an application form that collects information on all household characteristics that constitute the welfare predictors of the PMTF, and comparing the results from applying the PMTF on the pilot data with those predicted by simulations using household data. Results from this analysis were used to fine-tune the selection cutoff and to make some adjustments to the criteria for selection, to correct for shortcomings of the PMTF in certain cases. Subsequently, for a sub-sample of pilot GN divisions, a field validation pilot was conducted, whereby the list of households selected as eligible by applying the PMTF was validated through visits to households, consultations with community elders and GN division level officials. The primary objective of the validation pilot was to gauge how far the identification of eligible households using the formula resonates with perceptions of poverty on the ground. 33. The results and insights from the pilot, on technical aspects of using the PMTF as well as the application process and design of application forms, constituted a key input into the PSIA of the reform, which in turn influenced the design of the reform. By allowing corroboration of the ex ante simulations with results from an actual application process, along key dimensions like coverage rates among the population, specific vulnerable groups and in areas with special characteristics like those affected by conflict, the pilot greatly enhances the reliability of the analysis to identify the poverty and distributional impact of welfare reform. Consultations with stakeholders: 34. A key element of the PSIA exercise – and more broadly the entire process of supporting the reform – was consultations with stakeholders. Formal forums for consultation consisted of two workshops with wide participation that were conducted at different phases of the pilot exercise. This was supplemented with numerous consultations and field visits with Samurdhi officers on the ground and program beneficiaries, as well as regular meetings and brainstorming sessions with the Steering Committee and the Welfare Benefits Board. Consultations with stakeholders were thus not limited to the formal setting of organized workshops, but took the form of an evolving partnership with senior officials from Treasury and line ministries that implement social welfare programs, government agents or district secretaries (especially in the North) and technical experts from the Central bank and Department of Census & Statistics. 35. To understand the process of consultations, it is useful to start with the institutional setup to support and implement the reform. At the onset of the process in late 2002, a Steering Committee (as mentioned above) was set up to provide direction to the process. This Committee was constituted with representation from relevant line ministries (Samurdhi, Social Welfare) and senior officials from the Treasury, Central Bank and Department of Census & Statistics (DCS). The Committee was assisted by a small group of technical staff, and worked in partnership with the World Bank team and two international consultants – one with extensive experience in the implementation of safety net programs and the other in Management Information Systems required to administer and monitor such programs. 15 For some of these projections, data from the Census of Population (2001) is used. 9 36. The Steering Committee’s role was to coordinate across all stakeholder departments and guide (with policy decisions as necessary) the work to develop the PMTF, design and implement the pilot; the process of wider consultation with insights from the pilot; and the development of an MIS for program management and monitoring. Much of the work that forms the core of the PSIA described here – namely the econometric work to derive the PMTF, the subsequent analysis of impact using household data along the dimensions described above, and the analysis of the pilot data – were conducted jointly by the World Bank team and local statisticians and other technical staff appointed by the Steering Committee. With time, as the Welfare Benefits Board was constituted as a formal body in accordance with the guidelines of the Act, it gradually adopted the role of overseeing and facilitating the work as well as that of coordinating across different line ministries, while the Steering Committee continued providing guidance on technical issues. 37. Consultations with the results and insights from the pilot were held in two workshops organized jointly by the Welfare Benefits Board and Ministry of Samurdhi. The first workshop was held just after the fieldwork for the pilot was completed (August, 2003) where the main focus was on learning from the experiences of field staff who conducted the process of inviting applications from households and suggesting solutions for logistical problems on the ground. 38. The second workshop was held after the pilot analysis was complete (November, 2003), with wide participation from all relevant line ministries and departments, district secretaries of some districts and field-based program staff. The broad objectives were to discuss the implications of the pilot results and identify ways forward. A number of issues surrounding key policy recommendations were discussed in this workshop – including eligibility cutoffs and size of target group, schedule of benefits (how are they determined taking into account household characteristics?), kind of benefits (cash, checks or foodstamps?), modalities of payment (commercial banks, Samurdhi banking societies or post offices?), and the critical question of whether forced savings and the provision of microfinance should be linked to the cash transfers program as they are currently. The workshop participants also focused on the question of how the targeting method and application form should be modified for conflict affected areas of the country – a discussion that benefited strongly from the active participation of experienced government officials from those parts of the country. 39. As the next section will elaborate, the discussions and feedback from the workshop were instrumental in formulating the final targeting criteria and schedule of benefits – particularly in the context of certain special cases where the PSIA showed that applying the PMTF directly may lead to higher rates of error. These adjustments were made in specific cases like small households, households with certain categories of vulnerable members, and for conflict-affected areas. 40. In the section that follows, the main results of the PSIA, drawing from the different components described above, are elaborated. VI. Results from Poverty and Social Impact Analysis 41. As apparent from the description above, the technical analyses of poverty and social impact of welfare reform are built around the econometric exercise for deriving the proxy means test formula (PMTF) for targeting of Samurdhi. A discussion of the main findings of the analyses must therefore be preceded by a description of the PMTF exercise, which will then lead into the results from the simulation exercises using household survey data, and that from the pilot data. 10 Developing a Proxy Mean Testing Formula (PMTF) for Sri Lanka 16 42. As mentioned previously, deriving the PMTF involves defining an indicator for welfare of a household – which in this case was per capita household consumption expenditure (monthly) – and then identifying a set of easily observable variables/predictors that together serve as the best proxy for the welfare indicator.17 To derive the PMTF, the consumption variable is regressed, using Ordinary Least Squares (OLS) method, on different sets of explanatory variables. The case for using OLS as the model for predicting welfare is driven primarily by convenience and ease of interpretation – especially convenient when a large numbers of predictor variables (including continuous variables) are available; and amenable to intuitive interpretation of the coefficients of the welfare predictors, which is more easily understandable to a policymaker (see Annex C for a fuller discussion). 43. The primary source of data for this exercise was the Sri Lanka Integrated Survey (SLIS), conducted by the World Bank in collaboration with local institutions in 1999-2000 – designed to be representative at the national and provincial levels for all of Sri Lanka. However, due to the conflict in the North-East at that time, the sample from those areas did not turn out to be reliable. Thus for the purpose of this exercise, the North-East sample was excluded, leaving a sample of around 5600 households, which by design is representative for the country excluding the North- East (using sample weights). On all accounts, the SLIS appears to be well suited to the purpose of developing the PMTF. Being a multitopic household survey in the style of a Living Standard Measurement Survey, it has rich and detailed information on most correlates of poverty, along with information on the benefits received from the existing Samurdhi program. At the time of the exercise, the SLIS was also the most recent source of representative household data available for Sri Lanka. Exclusion of the North-East, while unfortunate, should be seen in the context that no survey has managed to cover the region during the past decade of conflict. 44. The Household and Income Expenditure Surveys (HIES) conducted by the DCS – another potential source for evaluating the coverage of Samurdhi or conducting a PMTF analysis – was found to be unsuitable for this purpose. The HIES has limited information on poverty correlates (or welfare proxies), which takes it out of the contention for a PMTF analysis. While the most recent HIES round (2002) does attempt to measure coverage of Samurdhi, the only question on this topic – that asks whether the household received any Samurdhi foodstamp in the last month – is not sufficient the gauge actual coverage or transfer amounts. Samurdhi payments are made for months together at a time to beneficiaries, and sometimes at irregular intervals, which makes this question ambiguous and open to different interpretations by the respondent. Quite naturally as a result, the transfer amounts reported in HIES are highly inconsistent with the actual monthly transfer amounts reported by the Ministry, and the overall coverage of 27 percent is substantially lower than what is implied by Ministry figures on beneficiaries and the country’s population as measured by the Census of 2001. Coverage rates and benefit amounts from SLIS, in contrast, are quite consistent with the Ministry information on number of beneficiaries and transfer amounts, which is why SLIS was found suitable for the analysis in this section. 45. That said, for future updating of the PMTF and monitoring of Samurdhi coverage and targeting, a regularly repeated, official survey like HIES should be the preferred source. A 16 For a more detailed description, see Narayan and Yoshida (2004). 17 In development literature, consumption expenditure is generally considered a more accurate measure of welfare than income for several reasons. First, consumption expenditures are more likely to indicate the household’s “true� economic status, as a result of households with sporadic incomes smoothing their consumption patterns over time. Second, consumption is generally measured with far greater accuracy than income in a household survey, primarily because households’ sources of income may include home-based production and own farms, calculating the flow of net incomes from which is problematic. 11 planned expansion of the HIES is expected to yield a survey that incorporates information on welfare indicators other than consumption and income, as well as questions necessary to accurately measure incidence of social welfare programs including Samurdhi. 46. As an independent validation of the statistical exercise conducted using SLIS, a parallel analysis to develop a PMTF was also undertaken using the Consumer Finance Socio-Economic Survey (CFSES) conducted by the Central Bank, which is the official source for socio-economic statistics reported by the Central Bank. A PMTF was derived using the latest available round of the CFSES (1996-97), adopting a methodology identical to the one described below (using SLIS). The “alternate� CFSES-based PMTF served a number of useful purposes. First, it indicated the kind of welfare predictors – independent of the source of data – that are likely to work well in a PMTF. Second, it afforded a comparison of results from applying two PMTFs derived from completely unrelated data sources on the pilot data, so that the better-performing one could be chosen as the final PMTF; although, for reasons described above, SLIS was ex ante expected to yield a more reliable PMTF. Third, it also facilitated capacity-building – the work to develop the CFSES-based PMTF was conducted primarily by local statisticians, informed by the parallel work being conducted for the SLIS-based PMTF. 47. The process of identifying potential variables to predict welfare (measured by per capita consumption) took into account two separate criteria: correlation between the welfare measure and the predictor, which will determine accuracy of the prediction, and verifiability of the predictor, which will determine the accuracy of information used to impute welfare. There is often a trade-off between these two criteria, which in turn calls for a certain amount of subjective judgment in the final choice of the model. The types of predictors used for this exercise constituted the following broad categories: location variables, community characteristics, housing quality, household characteristics, ownership of durable goods or farm equipment and ownership of productive assets (including land). 18 48. Very briefly, the steps in the procedure for arriving at the PMTF run as follows. After identifying the original set of potential variables belonging to the six broad categories, the set of selected predictors are introduced in a weighted OLS regression of (log of) per capita monthly consumption expenditure. Different subsets of variables are checked for possible multicollinearity, and adjustments made accordingly. A stepwise regression is then used with the remaining set of variables to eliminate from the regression variables that are not statistically significant and do not increase the model's overall explanatory power. From this process, different models evolve based on the subset of variables entering into the regression, which are evaluated against each other to decide on the final model for the PMTF. Determining eligibility using PMTF 49. Each model predicts a certain level of welfare, as measured by (log of) per capita monthly consumption expenditure. These predicted welfare levels are used to assign individuals to eligible or ineligible groups, based on an eligibility cutoff point. The eligibility cut-off point is determined by the welfare level of a certain percentile of the individual welfare distribution, using “true� welfare as measured by actual per capita consumption. For the purpose of analysis, a range of cutoff points were considered, defined by specific percentiles of actual/true per capita consumption expenditures (e.g. 25th, 30th, 40th). The eligible group identified by the PMTF constitutes all households whose predicted consumption (or PMTF score) falls below the cutoff point. 50. It is important to note that the selection of the cutoff point is essentially a policy, and not a technical decision – dependent on the proportion of the population that the government would 18 See Annex C for a more detailed discussion on each category of variable vis-à-vis the two main criteria. 12 like to target with the program. The role of the analysis was to inform this decision by simulating a wide range of scenarios corresponding to different cutoff points for the selected model. In this context, the analysis had two objectives: firstly, to show the sensitivity of the model and its attendant errors in targeting to changes in cutoff points; and secondly, to help the government decide what the cutoff point should be, taking into account the tradeoffs inherent in choosing a relatively higher cutoff vis-à-vis a low one. 19 Criteria to evaluate a targeting formula 51. As with all regression analyses, different specifications of the model and different samples of the population yield different results and it is not always easy to determine which specification is superior. However, a variety of tests can be conducted, which, taken together, can be used to select one model over another, as well as to compare the selected model with the current targeting of Samurdhi. The first criterion, used to evaluate alternate options for the PMTF, is the regression’s R2, which is the proportion of the variation in consumption that is explained by the regression model. Higher the R2, the better are a particular set of variables in predicting welfare. The second criterion involves looking at measures that indicate the ability of various models, and the existing Samurdhi system, to identify the poor properly. Following standard practice with PMTFs, targeting accuracy is evaluated using Type I and II errors from which rates of undercoverage and leakage are derived, and incidence of benefits across consumption groups. 20 52. To apply this criterion, certain concepts are important. The target population is defined as a group of individuals whose actual per capita consumption expenditure is below a selected threshold (say “a poverty line�), while the eligible population comprises of individuals whose PMTF score is below the selected cutoff point for eligibility. A Type I error or error of exclusion refers to an individual incorrectly excluded by the formula (i.e., belongs to the target group but not the eligible group), and conversely Type II error refers to a person incorrectly identified as being eligible (i.e. belonging to the eligible group and not the target group). Undercoverage rate is calculated by dividing the number of Type I errors by the size of the target group. Leakage rate is calculated by dividing the number of Type II errors by the size of the eligible group. Undercoverage reduces the impact of the program on the welfare level of the intended beneficiaries, but carries no budgetary cost. Leakage increases program costs and can also be welfare reducing in the presence of a budget constraint – higher the leakage of benefits to the non-deserving, lower would be the amount available for transfers to those who truly deserve. 53. It is worth noting that undercoverage and leakage rates are useful in evaluating targeting accuracy only if the targeted population and the eligible population are defined consistently. It is easy to see that for a given target group, as the eligible group expands (i.e. eligibility cutoff is raised) undercoverage falls while leakage increases, and conversely as the eligible group shrinks. 21 This represents the tradeoff between undercoverage and leakage for a given target group, as the eligibility cutoff is shifted. 54. For the sake of simplicity and to ensure consistency for comparison, the analysis presented here are for cases where eligibility cutoff is held identical to the threshold that defines the target 19 A useful reference point here is that the headcount poverty rate for Sri Lanka was estimated at 25 and 22 percent, using HIES data from 1995-96 and 2002 respectively. One must however keep in mind that there is no way to compare the nth percentile from SLIS data with the poverty line computed from HIES; the former can, at best serve as an indicative poverty threshold for the SLIS. 20 See Grosh and Baker (1995) and related literature for other countries. 21 Two extreme examples help illustrate this point clearly. If a program selects the entire population as eligible, undercoverage rate is zero irrespective of how the target population is defined. Conversely, if the targeted population is the entire population, the leakage rate is zero, no matter what the selection criteria which determines the eligible population. 13 population – and these cases turn out to be enough to highlight the main results. Note that when these two lines move together, raising the eligibility cutoff (and thus also the “poverty line� or threshold to define the target group) would lead to lower undercoverage and leakage – simply because the “burden� on the formula to accurately predict welfare is reduced. But given the program’s budget constraint, these come at a cost, namely that of reducing the amount available for each beneficiary, which has important welfare implications, particularly for the poorest. This represents the key tradeoff that policymakers need to take into account before deciding on the eligibility cutoff. 55. The last criterion to evaluate targeting efficiency is by looking at how a specific PMTF allocates potential beneficiaries across the expenditure distribution. It is preferred that a model has good incidence, i.e. most of the identified beneficiaries belong to the bottom of the consumption distribution, and relatively few, if any, from the top of the distribution. Selection of a PMTF model 56. A large number of specifications were tried out and evaluated according to the criteria described above. All specifications involve OLS regressions of (log of) per capita monthly consumption measured in Sri Lankan rupees on a set of predictors, with only those variables retained whose statistical significance is 80 percent or above. It was found that a model that includes province and rural/urban location dummies – along with a list of variables belonging to other categories as described above – yields the highest R2 and the lowest rate of undercoverage and leakage for cutoffs set at 25th, 30th and 40th percentiles of actual per capita consumption (model I in Table 1). 22 However, for a number of reasons, it would have been difficult to incorporate province-specific and rural/urban location variables into the final PMTF: not only would this test the limits of what is politically unacceptable, but it is also the case that the Census definitions for rural/urban sectors is out of date and do not conform with people’s perceptions. Furthermore, it made sense to restrict the set of predictors further – to reduce the information burden in applying the PMTF – as long as it did not lead to significant increases in targeting errors. Table 1: Results from different models Undercoverage Rate for Leakage Rate for different Models different cutoff percentiles cutoff percentiles R-square 25th 30th 40th 25th 30th 40th I. Full model 0.51 0.42 0.30 0.39 0.35 0.29 0.58 II. Restricted and excl. province dummies 0.52 0.43 0.28 0.39 0.36 0.31 0.56 III. Restricted and excl. province and 0.53 0.43 0.28 0.39 0.35 0.31 0.56 rural/urban dummies Note: The 25th, 30th, 35th, and 40th percentiles of actual consumption amount to Rs. 1129, 1201, 1270 and 1347 monthly per capita (at 2000 prices) respectively The target group in every case is defined by the cutoff point for the eligible group – i.e. when the cutoff is the 30th percentile of consumption, the target group is the bottom 30 percent of the population by consumption 57. Dropping the province and rural/urban location dummies, and retaining only the set of highly significant variables (with significance level of 99 percent or higher) did not lead to an appreciable change in the results. This is seen from comparing models I and III in Table 1 above. More disaggregated analysis also showed that although model I yields aggregate results that are slightly superior, the undercoverage rate varies widely across provinces, which is problematic. Eliminating province weights from the PMTF, as in models II and III, reduces such variations 22 Note that as mentioned previously, all calculations are made for the case where the eligibility cutoff is equal to the threshold to define the target group. 14 considerably. Similarly, a comparison between models II and III in Table 1 shows that excluding the rural/urban location variable leads to little or no change in targeting errors and R-square. Analysis disaggregated by sectors also showed that III has lower undercoverage for urban areas than II, which results in a much smaller gap in undercoverage rates between rural and urban areas – a feature that is appealing to policymakers. 23 58. Model III was thus the final choice for the PMTF (for a systematic description of pros and cons of each model, see Annex C). The set of predictors with appropriate weights comprised of: community characteristics (presence of a bank or divisional headquarters in the community); household assets (consumer non-durables, farm equipment); household’s ownership of land and livestock; characteristics of household head (age, education, main activity, marital status); household demographics (household size, whether all children attend school); and housing characteristics (owned housing or not, type of wall and latrine, ratio of rooms to number of household members). 59. The results compare well with those from similar exercises conducted for other countries. For an eligibility cutoff equal to the 30th percentile of actual per capita consumption, the selected model III yields an undercoverage rate of 43 percent and a leakage rate of 35 percent. For the same cutoff in percentile terms, a similar exercise using Jamaica data for 1989 yields undercoverage and leakage rates of around 41 and 34 percent respectively, and using data for 2000 the rates are 69 and 44 percent; the corresponding rates are 39 percent and 24 percent for urban Bolivia, and 54 and 35 percent for urban Peru (1990 data for both cases). 60. The selected PMTF (i.e. Model III) is presented in detail in Table A-5, Annex (along with the closest alternative: Model II), based on the regression results listed in Table A-6, Annex. The formula is more likely to assign benefits to larger households; households where all children do not go to school; households with few durable goods and amenities, little land and livestock, and poor housing; households with older heads; and where the head is a female widow, has lower levels of education and does not work as a salaried employee. The weights thus seem to be appropriate in terms of increasing the chance of households who are more likely to be poor or with vulnerable members to be selected. 61. To check whether the methodology used for the estimation can be improved upon in terms of impact on targeting efficiency, two alternative methods of estimating the PMTF were explored – both involving using a subset of the sample for estimation (see details in Annex C). These exercises did not yield results that were significantly better in terms of targeting outcomes, or insights that would introduce doubts about the robustness of the selected model and its predictions. 62. The parallel exercise – as mentioned at the beginning of this section – to derive a PMTF using the CFSES (1996-97) as a data source yielded a formula that confirmed the main characteristics of the model developed using SLIS, and thus provided additional confirmation of the suitability of the selected model. At the same time, the SLIS-based model was found to clearly outperform the CFSES-based one on the pilot data, primarily because the CFSES had a fewer number of poverty predictors among those that are most likely to be important, and was an older survey. 24 The latter was a particularly important reason why the CFSES of 96-97 was not the ideal data source for this exercise – consumption patterns had changed over the years that have elapsed, which in turn reduced the reliability of many of the welfare predictors, especially those related to the ownership of household durable goods. 23 For example, for the 30th percentile cutoff point, undercoverage rates of model II were 41 and 71 percent for rural and urban areas respectively; while those of model III were 42 and 53 percent respectively. 24 For example, the gap between the predicted rate of coverage and the actual rate of coverage on the pilot data was much higher with the CFSES-based PMTF than with the SLIS-based PMTF. 15 Impact of implementing the PMTF on targeting efficiency: ex ante results 63. One of the important questions to address is how the new selection criteria based on the PMTF compare with the current Samurdhi program in identifying the poor. Table 2 and Table 3 illustrate how undercoverage and leakage rates of the selected PMTF and the current Samurdhi program vary as the cutoff line (equal to the threshold to define the target group) increases from 25th percentile to 40th percentile of actual per capita consumption expenditure distribution.25 64. A fair comparison between the PMTF and the current Samurdhi program can only be conducted for a cutoff set at 40th percentile of the actual per capita consumption expenditure, which implies that the targeted population is the bottom 40 percent of consumption distribution and the program coverage is around 40 percent. 26 This decision is justifiable for two reasons. Firstly, coverage of the current Samurdhi program is around 40 percent of entire population. Second, given Samurdhi’s coverage of 40 percent and its clear intention to cover the poorest section of the population, the assumption that the targeted population is the poorest 40 percent of the consumption expenditure distribution is perfectly reasonable. 65. When the 40th percentile is chosen as the threshold, both Table 2: Undercoverage rates undercoverage and leakage Total Rural Urban rates of the PMTF are found to cutoff a → 25 30 40 25 30 40 25 30 40 be substantially lower than PMTF 0.53 0.43 0.28 0.51 0.42 0.27 0.67 0.53 0.35 those of the current Samurdhi Samurdhi NAb NA 0.42 NA NA 0.40 NA NA 0.62 program – nationally and separately for urban and rural Table 3: Leakage rates sectors. The difference in Total Rural Urban undercoverage rate is 14 cutoff → 25 30 40 25 30 40 25 a 30 40 percentage points nationally PMTF 0.39 0.35 0.31 0.39 0.35 0.30 0.47 0.38 0.36 and that in leakage is 12 Samurdhi NA NA 0.43 NA NA 0.42 NA NA 0.57 percentage points; and the gaps Source:a Sri Lanka Integrated Survey 1999-2000 indicates a percentile of the actual per capita consumption expenditure are much higher for urban Notes: Cutoff"NA" refers to "Not Applicable" since the current Samurdhi covers 40 % distribution. b areas. These results indicate of population. that one should expect a All point estimates have a standard error; for confidence intervals and standard errors significant improvement in of undercoverage and leakage using PMTF, see Table A-4, Annex targeting accuracy by replacing the current Samurdhi selection criteria with the PMTF. 66. However, having a safety net program intended to target the poor cover 40 percent of the population seems excessive – taking into account the poverty rate of Sri Lanka as well as the welfare tradeoff in terms of lower amounts available to poor households given the existing budget constraint. A cutoff equal to the 30th percentile of actual per capita consumption expenditure appears to be more reasonable in terms of program coverage (program coverage of around 26 percent of the population is predicted), while ensuring tolerable levels of undercoverage and leakage. This was the recommendation for eligibility cutoff presented by the technical team at the Workshop, and was accepted by the policymakers and other Workshop participants. 67. Interestingly, it also turns out that the targeting errors from using the PMTF for a cutoff equal to the 30th percentile of consumption is superior to those of the current Samurdhi even when the 25 Note that this means that if the threshold is the 25th percentile, the target population is the bottom 25 percent of the actual consumption distribution, and the eligible/beneficiary population is a group of individuals whose per capita consumption expenditure is below the 25th percentile. 26 Setting the cutoff line at 40th percentile of actual per capita household consumption expenditure does not guarantee the program coverage also becomes 40 percent. This is a sort of coincidence. For example, if the cutoff is set at 30th percentile, the program coverage becomes 27 percent (see Table 4). 16 cutoff is set at the 40th percentile (see Table 2 and Table 3). Undercoverage rates are very close for the two cases and leakage rates are higher by 8 percentage points for Samurdhi. This indicates that even when the target and eligible groups for the Samurdhi program are allowed to be larger than that of the PMTF, which should naturally lead to lower targeting errors for the former, the PMTF comes out as clearly superior. Moreover, this gain is just on the dimension of targeting efficiency and does not account for the welfare gains arising from the larger benefits received by the beneficiaries under the PMTF regime, for a given budget of the program. 68. The last measure to evaluate targeting efficiency of Table 4: Coverage of population by the PMTF is the incidence of coverage across the per capita consumption deciles distribution of actual per capita consumption Cutoff 30th a 40th pctile expenditure (Table 4). Again, a fair comparison with line pctile the current Samurdhi would be for 40th percentile as the Decile PMTF Samurdhi PMTF cutoff line and the threshold to define the target group. 1 0.91 0.69 0.74 If the PMTF is used for selection of beneficiaries with 2 0.75 0.60 0.53 the cutoff at the 40th percentile, more than 20 percent of population in the poorest decile and 15 percent of the 2nd 3 0.69 0.54 0.44 decile will be added as beneficiaries compared to the 4 0.55 0.49 0.29 current Samurdhi, while the proportion of coverage of 5 0.47 0.47 0.24 the top 3 deciles will be reduced by between 5 and 13 6 0.34 0.43 0.18 percent. Even when the cutoff point is set at the 7 0.24 0.36 0.11 recommended 30th percentile level for the PMTF, the coverage of the bottom decile is significantly higher than 8 0.16 0.25 0.06 that for Samurdhi and leakage to the richest 30 percent 9 0.05 0.18 0.03 of population is dramatically reduced in comparison 10 0.01 0.06 0.01 with the current Samurdhi program. 27 Total coverage 0.42 0.41 0.26 69. Table 4 also shows that targeting using the PMTF, Source: Sri Lanka Integrated Survey 1999-2000. irrespective of which eligibility cutoff is used, is highly Notes: a. Numbers in the cutoff line denotes a progressive, which is consistent with the distributional percentile of the actual per capita consumption expenditure distribution. objectives of an income transfer program. Moreover, as shown in Figure 1, the degree of progressiveness in coverage is much higher for the PMTF, for either eligibility cutoff, than that for the current Samurdhi. Figure 1: Coverage of population by consumption decile 70. These results also indicate which segments of the population (distributed by consumption) will gain from the PMTF regime, and conversely who will lose – just in terms of the proportion of individuals receiving benefits. The biggest winners will obviously be the poorest group, namely the bottom decile, whose coverage will increase by 5 percentage points post-reform. Also clearly, the richer groups will lose the most, with the proportion of individuals in the top 4 deciles who receive benefits 27 Note that the total coverage if 30th percentile is used for the cutoff line is less than 30 percent. This is related to the fact that predicted per capita consumption expenditures tend to be larger than actual consumption for poor households, and households are defined eligible if their predicted consumption is lower than 30th percentile of actual consumption. 17 expected to fall from 21 percent pre-reform to less than 5 percent post-reform. The reduction in coverage will be less for the middle group (4th to 6th deciles), from 46 percent pre-reform to 24 percent post-reform; and it will be almost negligible for the bottom 3 deciles, from 61 percent covered pre-reform to 57 percent covered post-reform. These statistics however do not reflect the full extent of gains and losses for each group, since they do not take into account the changes in amounts received. One of the important benefits from the reform is expected to be the significant increase in transfers for those selected by the PMTF, since the same budget is now being distributed among 26 percent of the population (and not 41 percent as under the current Samurdhi). The next section attempts to quantify the net welfare gains from the reform, using an appropriate distribution-sensitive welfare measure. Selection of payment scheme for eligible individuals, and its welfare impact 71. The analysis so far has examined the PMTF exclusively from the point of view of targeting efficiency. A related, and equally important question is how to set the amounts for benefits to eligible beneficiaries, for the selected eligibility cutoff (equal to 30th percentile of actual per capita consumption) – where the challenge is to identify a payment scheme that maximizes the welfare objectives of the program. 72. In order to conduct this analysis, it is necessary first to define appropriate welfare measure(s) that represent the main objectives of the program. While Samurdhi’s objective is to reduce consumption poverty of the beneficiaries, the simplest measures of poverty – like headcount poverty or the poverty gap – do not take into account the distribution of transfers among the poor. Therefore, to evaluate the welfare impact of different payment schemes Sen’s poverty index and Squared Poverty Gap are considered. Both these measures reward progressiveness in distribution of benefits, in other words register an increase in welfare when the marginal rupee goes to the neediest. 73. For operational purposes, feasibility of a payment scheme is as important as its welfare implication. The most ideal payment scheme from the point of view of distributional impact also turns out to be almost impossible to implement. 28 Moreover, analysis also shows that the welfare levels attained by the “ideal� redistributive scheme are achievable to a large extent by employing simpler schemes – for example, one which provides each eligible household a sum of the following two components: a fixed component that is an equal amount to all eligible households, and a variable component that has a fixed amount per member or appropriately defined subsets of members. Within this class of payment schemes, one that turned out to be most appealing to policymakers and stakeholders was where the variable amount was defined per vulnerable member of the household, where the vulnerable are children aged between 0 and 15, elderly aged 66 and above, and those who are disabled or permanently ill. 74. Intuitively, such a payment scheme is easy to justify since such individuals can not, or should not earn income for the household, and are also the ones most affected by shocks to the livelihood of the household. And the simple reason why such a scheme turns out to be progressive is the high correlation between the ranking of households by consumption and the number of such vulnerable individuals in the household. Moreover, such a scheme is not difficult to implement with a well designed Management Information System (MIS) – creating which has been an important part of building institutional capacity for implementing the reform. 28 For example, one of the most customized payment schedules is to determine the level of payment to eligible households sequentially: the poorest household receives enough to achieve the consumption of the next poorest, and next, the two poorest households receive enough to attain the consumption of the next poorest, and so on, till the budget is exhausted. This type of payment scheme has the most egalitarian or redistributive impact, but it is almost impossible to implement on the ground. 18 Optimal payment scheme – impact on welfare and benefit incidence: 75. The exercise thus becomes one of finding the optimum value of parameters a and b for a monthly payment level of (a+bXk) to household k, where a is the fixed component, b is the payment per vulnerable member, and X is the number of vulnerable members in household k, subject to a budget constraint (equal to the recent annual budget for Samurdhi transfers). This also allows for a high degree of flexibility in choice, since varying degrees of progressiveness can be induced by selecting different levels for a and b. 29 The search for an optimal combination of (a, b) was conducted using simulations from SLIS (1999-2000) data, taking Sen’s index and Squared Poverty Gap as the relevant aggregate welfare measures. Two other features of the analysis are important: first, the simulations attempt to measure the impact of the replacement of the existing system of payments with a new one; second, the budget for the program was taken as Rs. 9 bn. at 2004 prices (the budgetary allocation for 2004), adjusted to 1999-2000 figures by appropriate inflation rates. The simulations reveal (a,b)=(80,197) in 1999 Rs, or (115, 284) in 2004 Rs. to be the most desirable payment scheme, i.e. one for which both Sen’s index and Squared Poverty Gap are minimized. Furthermore, for this payment scheme, around 15 and 17 percent reduction is achieved in Sen’s Index and Squared Poverty Gap respectively in comparison to the existing Samurdhi scheme – which represents the aggregate welfare impact of the reform, when the optimum level of payments are implemented.30 76. Using the above optimum payment Table 5: Incidence of total budget by decile (%) scheme, simulations on SLIS show the Decile of per PMTF1 incidence of benefits to be far more capita (optimal progressive than under the existing Samurdhi consumption payments) Samurdhi2 scheme. This is reflected by the amount of 1 29.5 12.9 benefits per capita per month for every decile 2 20.6 12.3 (Table A-8, Annex), as well as the share of 3 15.7 11.4 every decile in the total amount of benefits 4 11.8 11.2 (Table 5). As Table 5 shows, the share of 5 9.7 13.3 benefits for the bottom 3 deciles under PMTF 6 6.2 11.2 is much higher (by 29 percentage points) – and 7 3.6 10.4 conversely the share of top 3 deciles is much 8 1.9 8.7 lower (by 14.5 percentage points) – than under 9 0.9 6.3 existing Samurdhi. While there is little 10 0.1 2.4 progressiveness in the distribution of benefits Total 100 100 under the existing Samurdhi scheme (share of Source: SLIS 1999-2000 benefits is almost the same for the deciles 1 to Notes: 1: The share of budget for each decile if 8), the PMTF optimal payment scheme yields a beneficiaries are chosen by the PMTF (with cutoff at the 30th percentile of actual per capita consumption) and declining share for higher deciles of per capita each household receives an amount given by the optimal consumption. These results show clearly that payment scheme 2 in terms of benefits received, the winners from : The share of Samurdhi budgets received by each decile this reform will be the poorest 40 percent of the population, at the cost of losses suffered by the 29 There can be tradeoffs: by increasing b, progressiveness is increased on the average; however for a limited budget, this implies a lower a, which affects eligible households who are poor but have few or no vulnerable members. At the margin, i.e. when b is high enough such that a has to be set to zero, it turns out that as many as 5.5 percent of households belonging to the poorest decile of actual per capita consumption fall below the PMTF eligibility cutoff but do not receive any benefits. 30 These gains are somewhat overstated, since the simulations use a budget constraint of Rs. 9 bn. for the optimum payment scheme under PMTF, which is higher than the total budget implied by the existing Samurdhi benefits received by households in SLIS. However, separate simulations with the budget held constant yield comparable results: for example, squared poverty gap shows a reduction of at least 10 percent from the existing Samurdhi payments, when the PMTF is used to distribute payments. 19 top 60 percent of the population, with the losses being especially large (100 percent or more) for the top 4 deciles. 77. As an operational recommendation for the program’s payment scheme, the following ranges are prescribed: a= (Rs. 100-110) and b= (Rs. 270-280) at 2004 prices. Both the fixed and the variable components are recommended to be slightly lower in comparison with the optimal amounts since it is advisable to keep some budgetary margin – to account for the fact that these simulations are rough calculations using a number of simplifying assumptions, and the possibility of hidden implementation costs. 78. It is also useful to reflect on the implications of changes in program budget for welfare. A reduction of budget from the current level of Rs. 9 bn. (at 2004 prices) would necessarily imply a decrease in the number of beneficiaries or payment amounts. The analysis showed that current payment amounts are the minimum necessary to yield meaningful benefits, which implied that any budget reduction would have to be accommodated by a reduction in the number of beneficiaries (lowering of cutoff point below the current level). This will lead to an increase in the error of exclusion and therefore a reduction in overall welfare. On the other hand, if there were an increase in the budget, significant welfare gains would be obtained by increasing the payment amounts, while keeping the number of beneficiaries unchanged (maintaining the recommended cutoff point). Higher payment amounts to eligible households would ensure that beneficiaries are able to reach consumption levels that are closer to that of the poverty line. Evaluation of PMTF using data from the “targeting pilot� 79. The analysis so far has been limited to simulating impact on welfare, targeting efficiency and distribution using household survey data – which is what similar exercises in many countries are limited to. However, there are at least three important limitations on this type of analysis in the Sri Lankan context. First, since SLIS was conducted more than five years ago, it is important to ascertain whether the analysis is applicable to current patterns of household consumption. Second, it is an important policy question as to whether the information necessary for the PMTF will be reported accurately by households during an application process that is inherently different from the data collection for a household survey like SLIS. Third, SLIS is not appropriate for analyzing the targeting impacts of the PMTF at a more disaggregated level due to its limited sample size. 80. The pilot targeting exercise was conducted over 12 districts and 114 GN divisions, where each GN division was covered in its entirety, to simulate as far as possible the application process for the actual reform when it is implemented. 31 Enumerators distributed application forms including variables for the PMTF to all households in selected GN divisions, and explained to households that the information drawn from the application forms could be used for future selection of welfare beneficiaries. The household was left to decide whether to return the application form to the enumerator. In many cases, the enumerators helped the households to fill up the forms, and in most cases the information filled in the forms were checked before they were accepted and signed by the Samurdhi Development Officer of the area. 81. The sample GN divisions were drawn by the DCS, based on criteria for selection of pilot areas developed jointly with the Steering Committee. The sample was not designed to be statistically representative for the country, but since the sample was very large and distributed 31 The pilot covered each GN division in its entirety. Total population of pilot areas (from Population Census) is 199,151 (48,501 households), out of which around 26,703 households (with population of 115,580) applied. Out of a total of 114 GN divisions, 23 were from the North-East, with an estimated population of 42,568 (10,367 households), from which 6.876 households (with population 29,271) applied. 20 across different districts (see Table A-9, Annex), it was expected that the results would largely reflect the country as a whole and its different regions. 82. The pilot data addresses many of the limitations of SLIS data discussed above—it was recent, simulated the expected real application process, and around 27,000 households filled out and sent back their application forms. Nevertheless, by the very nature of the exercise, this data also has important limitations. First, it does not include per capita household consumption expenditures, making it impossible to compare actual consumption with that predicted by the PMTF (i.e. the PMTF score), and therefore to measure undercoverage and leakage. Secondly and most importantly, households in the pilot sample were self-selected, and around 45 percent of households did not return the application form. However, from the way it was conducted, it is reasonable to presume that non-respondents are largely those who are well-off and not interested in receiving benefits from the welfare program. A comparison of the distribution of PMTF scores from the pilot with those from SLIS confirms that the universe of pilot respondents are poorer on the average than the survey sample (see Figure A-3, Annex); it does not however rule out the possibility of some households who did not or could not fill the application even if they had wanted to participate in the program. 83. Among the numerous questions that can be addressed by the pilot data in conjunction with SLIS, three were considered most important from the perspective of a PSIA: (i) how does the coverage (percentage of population identified as beneficiaries) of the selected PMTF for different eligibility cutoffs measure up to what was simulated from SLIS? (ii) what can one say about coverage of groups that are likely to be especially vulnerable? (iii) what additional insights are available for the North-East, which was excluded from the SLIS sample used for PMTF analysis? Aggregate coverage of the PMTF 84. Addressing question (i) above serves as a basic check of the validity of the prediction of the simulations using household data. To compute coverage rates from the entire population of pilot GN divisions, it is assumed that all non-respondents would not be eligible for the program even if they had turned in their application forms – reasonable in theory since the pilot sample is expected to be a self-selected sample of poorer households. 85. Coverage rates predicted Table 6: Applying the selected PMTF to the pilot sample: program from SLIS turn out to be coverage reasonably close to the Percentile of actual Coverage (%) from Cutoff Coverage (%) coverage rates from the pilot, per capita pilot GN divisions scores for predicted from especially if the cutoff scores consumption All pilot PMTF SLIS NE are low. Table 6 shows that (SLIS) areas th when the cutoff score is set at 20 695 12.1 13.4 11.6 the 20th or the 25th percentile 25th 703 19.6 21.0 18.5 of actual consumption 30th 709 26.4 27.7 24.6 distribution, the coverage rate 40 th 721 41.6 42.2 37.0 for pilot areas are within 1-2 Source: SLIS 1999-2000 and Data set from the pilot (2004). percentage points of those Note: Number of population in non-Northeast is census population in 2001. When calculating coverage, it is assumed that all non-respondents would not have been eligible predicted by SLIS. Although even if they had applied. this gap expands for higher cutoff points, for the cutoff of 30th percentile the gap is still less than 2 percentage points – indicating that the SLIS predictions of coverage are not off the mark. For obvious reasons, the coverage rate is much higher among pilot households who had turned in their application forms. 86. It is useful to explore why the difference between coverage rates predicted from SLIS and that from the pilot increases as the cutoff rises. One plausible reason has to do with the assumption about pilot non-respondents described above. To understand why, one should accept 21 that this assumption is less likely to be perfect as the cutoff is raised – this is equivalent to saying that most of the non-respondents would be ineligible at the cutoff of 25th percentile even if they applied, whereas some of them may have actually been eligible when the cutoff is raised to the 40th percentile. If this is indeed the case, as the cutoff rises, the coverage rates of pilot areas are more likely to be an underestimation of true program coverage due to false exclusion of some of non-respondent, and as a result, the difference between the two coverage rates should be expected to widen. This would imply that the pilot coverage rates should be referred to as the lower bound of expected program coverage. 32 87. Thus the pilot results appear to be consistent with the expected coverage of the PMTF, which indirectly enhance credibility of other analyses (such as undercoverage and leakage rates) based on SLIS data. Coverage of vulnerable groups 88. Whether the PMTF “discriminates� against certain vulnerable categories of people (question (ii) above) is an important policy question, even if the overall targeting of poor households is satisfactory. In order to examine this question, coverage rates of the certain categories of households likely to be vulnerable are looked at. Consultations with government counterparts also revealed a keen interest in these categories, which are households with: (a) disabled household heads; (b) single female household heads; (c) cultivable land less than 1 acre; (d) household head aged 70 or above; and (e) very large number of members (7 or more). 89. Table 7 presents the results, only for the recommended cutoff Table 7: Coverage rate for likely "vulnerable" groups th (eligibility cutoff: 30th pctile of actual per capita consn.) of 30 percentile. The question of whether a vulnerable group of SLIS SLIS Pilot SLIS SLIS Pilot actual (PMTF) (PMTF) actual (PMTF) (PMTF) households are inadequately covered by the PMTF is easily a. disabled head d. age 70+ head answered by comparing coverage 0.37 0.34 0.43 0.34 0.32 0.42 rates for each category with that b. single female head e. household size 7+ of the general population (the 0.35 0.31 0.39 0.49 0.58 0.75 shaded cells). The pilot coverage c. land owned <1acre GENERAL POPULATION rates here simply refer to calculate 0.31 0.29 0.41 0.30 0.26 0.39 coverage from the pilot data, and Source: SLIS 1999-2000 and data from Pilot (2004). many percent do not make any adjustment for Note: Figures show howPilot scores areof each vulnerable group is coveredsample. different cutoff points. calculated by using the entire pilot at non-respondents, so that (as The results under “GENERAL POPULATION� include coverage rates for the expected) coverage rates of the entire population for various cutoff levels. pilot sample are higher than those of the SLIS data for all categories of vulnerable households. 90. The results of Table 7 – from the SLIS as well as the pilot – provide no indication of any bias in the PMTF against the likely vulnerable groups identified here. Interestingly, the coverage from PMTF predictions in SLIS data is quite close to the target group defined by actual SLIS consumption data for all vulnerable categories except large households. For example, actual SLIS consumption data identifies 37 percent of individuals in households with disabled heads to be in the poorest 30 percent of the population, and the PMTF selects 34 percent of such individuals as eligible. The PMTF however covers a significantly larger proportion of large households than what is reflected by actual consumption figures, indicating that the formula tends to favor large households. Table 8: Undercoverage rate among vulnerable groups 32 Experiences from pilots in other countries also show a similar pattern – that of coverage rate of pilot areas being lower than that predicted by household data, which gives credence to the argument made here. 22 91. Finally Table 8 also Cutoff Dis- Single Land Age House- General shows that the (pctile) abled female owned 70+ hold popu- undercoverage rate head head <1 acre head size 7+ lation among these potentially 25 th 0.51 0.50 0.52 0.42 0.28 0.53 vulnerable categories of th 30 0.40 0.41 0.41 0.33 0.20 0.43 households is in fact th lower, than that among 35 0.32 0.35 0.34 0.32 0.17 0.37 the general population, 40 th 0.26 0.27 0.26 0.25 0.12 0.28 for different cutoffs between 25th and 40th Source: SLIS 1999-2000 percentile of consumption (and target group taken to be equal to the cutoff percentile). Undercoverage is particularly low among large households and those with head of age 70 or more. This is further evidence that the PMTF covers the poor among these vulnerable groups as well as, or even a little better, than it does the poor among the rest of the population. Pilot insights for the North-East 92. Addressing question (iii) from above is critical in view of the impact of long-standing conflict in the North-East and the lack of reliable information on this region from other surveys including SLIS. Table 6 indicates that there are systematic differences in coverage rates between the pilot areas in the North-East and in the rest of the country, with the coverage in North-East being consistently higher for different cutoff points. This is completely explained by a much higher response rate to the application process in the North-East – 66 percent of households (69 percent of the population) applied in the North-East, compared to 52 percent of households (55 percent of population) in the South. Among those who applied, coverage was in fact lower in the North- East than in the rest of the country. For example, for the 30th percentile cutoff, coverage rate among those who applied was 38 percent for the North-East GN divisions, and 39 percent for other GN divisions (see Table A-10, Annex). 93. There are a number of potential explanations for the higher response rate in the North-East. One, offered by people and agencies familiar with the area, is that the people of this region have grown more responsive to an application process for welfare – since they have been receiving assistance from a number of relief agencies due to the war conditions, and are more used to the idea of receiving benefits by filling up forms. Another possible explanation is that the application process in the South was not able to extend opportunities to all those who would have liked to apply, or because some of the needy people have received nothing in the past and therefore see no point in filling an application form. A third explanation is simply that the people of North-East are worse off due to the conflict, and therefore apply in greater numbers for such programs. 94. While there is no way to pick one of the explanations as more plausible than the others, they raise important questions that the actual program must take into account. The most important issue it raises is the need to ensure, with strong outreach efforts and information campaign if necessary, that all who would like to receive welfare payments are actually able to apply. This may particularly be a challenge in areas where the existing program is visibly present – those who are currently excluded may not be hopeful enough about receiving benefits to take time to fill up the application form. 95. The pilot data also revealed that most household characteristics in North-East pilot districts are not significantly different from those in the other pilot districts. One difference that does emerge is a higher incidence of single female headed households in the North-East – 14 percent of all households compared with 12.6 percent of households in the other pilot areas. 23 96. Thus on the whole, the pilot did not suggest any major systematic difference between household characteristics in the North-East and the rest of the pilot districts that would bring into question the use of the same PMTF for the North-East. The only large difference in the data, namely in the response rates, clearly has no bearing on the applicability of the PMTF-based selection criteria. These conclusions must however remain tentative at this stage – partly because the pilot was conducted in just two districts from this region; and also because in the absence of consumption data from these areas, there is no proper benchmark against which the predictions from the PMTF model can be compared. Implementation of the program in the North-East must therefore be tempered with caution. PMTF beneficiaries and existing welfare recipients from the pilot sample 97. The distribution of Samurdhi beneficiaries was found to be somewhat correlated with PMTF scores in the pilot areas of North-East where Samurdhi was operating; there was however little correlation between the two in non-N&E districts (see Figure A-4, Annex). It is instructive to see how the beneficiaries from the pilot areas look like if the PMTF is applied with cutoff set at the 30th percentile (which covers about 25 percent of the pilot population): 63 percent of beneficiaries are current Samurdhi recipients, 6 percent benefit from other welfare programs (but not Samurdhi), and 31 percent currently receive no benefits. Moreover, in these 114 GN divisions from 12 districts, 53 percent of current Samurdhi beneficiaries will not receive benefits under the PMTF regime with the selected cutoff. On the other hand, 35 percent of those who receive no benefits now will do so, and so will 45 percent of those who currently receive benefits (likely to be much smaller) only from other programs. Clearly, these different groups represent the potential winners and losers of the reform in the pilot areas. Addressing special cases: small households, homeless households and conflict-affected areas 98. The scrutiny that the PMTF and the pilot were subjected to, in various forums including the workshops and other consultations, has led to a few important adjustments in the selection criteria and the way the PMTF would be applied in practice. A large number of questions were raised by field workers, government officials and other participants in the process. Some of these, based on perceptions and imperfect understanding of the idea behind PMTF, were resolved without significant changes in the selection criteria. 33 These interactions in fact played a valuable part in the process of improving the analysis and building consensus around the notion of introducing objectivity into selection of beneficiaries through a targeting formula– a novel idea in the context of most countries where it has not been tried before. 99. Such an interactive process also helped first identify, and then resolve a particular concern under the selected eligibility rule, namely a high rate of exclusion of small households (household size less than or equal to 3), which is critical from an equity point of view. It turns out that for this special group, almost 80 percent of the target group (bottom 30 percent of the consumption distribution from SLIS data) are not identified as eligible by the PMTF. The pilot sample also confirmed the PMTF’s apparent “bias� against such families: most households with household sizes of three or less had higher scores than the cutoff, including those for whom all other poverty predictors suggested they are very likely to be poor. 100. This results from a structural problem of the PMTF approach. Since poor households on the average are more likely to be larger in size, it is but natural that PMTF puts a large negative 33 For example, a typical concern was one that having positive weight for the ownership of radio might penalize the poor since a radio is so cheap that everyone including the poor can own it. This perception was however not supported by either SLIS or the pilot data—both showed a significant number of poor households do not own a radio, which explained why radio ownership turns out to be a good predictor of poverty in combination with other predictors. 24 weight on household size to improve prediction of consumption expenditure for the whole sample. In other words, for the aggregate population, having large negative weights in the PMTF for household size is useful, but also implies that the relatively low proportion of households that are small and poor are much less likely to be selected. In other words, what works well for the population as a whole (the PMTF), strongly biases it against a small sub-group, that of poor households of size 3 or less. 101. The only practical way of addressing this problem is to devise some additional criteria for the special case of small households of size three or less. Based on careful simulations, where a large number of possible criteria were considered, adjustments to the eligibility criteria were made to (a) maximize the coverage of the poor small households; (b) minimize leakage to non- poor small households; and (c) involve simple, transparent and easily implementable criteria. The adjusted criteria involved two cutoff scores for eligibility – the original one (corresponding to the 30th percentile) and a higher one (75th percentile). 34 All households above the higher cutoff are eliminated and all below the lower cutoff are selected. Those between the two cutoffs are selected if the following are fulfilled: (i) household size less than or equal to 3; (ii) number of rooms (excluding kitchen, bathroom, garage, verandah) equal to 1; 35 and (iii) at least one “vulnerable� (i.e. single female, age 15 years or less, disabled or permanently ill, age more than 60) household member. 102. Using the revised selection criteria, substantial improvements in results were found among small households (Table A-11, Annex) – nationally, as well as for urban and rural/estate regions separately. Without using the criteria described above, the coverage rate among households of size 3 and less was about 7 percent in SLIS data, compared to a target group of 17 percent. In contrast, with the revised criteria the coverage of this group increases to 18 percent. Also, substantial reductions in undercoverage rates among small households are achieved by using the new criteria. Furthermore, the new revised criteria induced little change in the aggregate results for the population – including coverage, undercoverage and leakage rates, and incidence of benefits. This is expected since the proportion of small households in the total population is quite low, and the eligible group is but a small proportion of such households. 103. Homeless households: The homeless posed some issues, since the SLIS data on which the PMTF was estimated, like most household surveys, did not include homeless households. The paucity of data makes it difficult to even estimate the proportion of homeless in the total target population. A consensus was reached at the workshop that all homeless households should be included in the program automatically, without applying the PMTF, which would be difficult to calculate in any case. This does not lead to any significant change in the coverage and other estimates – homeless households are only about 1.3 percent of all applicant households in pilot areas, and the average household size is much lower for homeless households, which translates to relatively low amounts per household using the payment scheme described previously. 36 104. Applying PMTF in conflict areas: As described above, the pilot results did not uncover any evidence to suggest that the PMTF should not be applied in the conflict areas. However, a number of issues were raised by workshop participants on the need to change the way certain variables were defined and interpreted for the North-East, which would in turn imply a change in the instructions for the application form for this area. These included – for instance – that of the 34 These cutoffs translate to PMTF scores of 709 and 767 respectively. 35 As an alternative for the number of rooms criterion, it is possible to use a floor condition – type of floor: unprepared earth or prepared clay. However, the number of rooms criterion is slightly superior than the floor type criterion, especially for urban areas. 36 24% of homeless households comprise of a single member, compared to 1% of those living in homes. 5% of homeless households comprise of a 6 or more members, compared to 37% of those living in homes 25 definition of cultivable land, which is a variable in the PMTF. In the North-East, the definition of this variable will have to be adjusted to a possible impact of conflict – namely land which was cultivable in the past, but cannot be cultivated next season because it may now have mines. Similar adjustments may also have to be made to the definitions of other variables, like ownership and quality of dwelling. A number of these changes were incorporated in the questionnaire manual for the North-East when implementation started in that region. These adjustments are expected to improve the welfare impact of the PMTF in the North-East, by taking into account special conflict-related circumstances that may have unusual impact on certain variables in the formula. Field validation of eligibility of pilot households 105. A field validation exercise, conducted in four GN divisions taken out of the 114 pilot GN divisions, showed encouraging results. The objective of the exercise was to validate the list of selected households in each GN division with local officials and elders, to test how the formula’s predictions based on the information that was collected resonate with the subjective perceptions about poverty on the ground. A second objective was to visit households that GN level officials believe to be cases of targeting error – i.e., selected by the PMTF but did not deserve it, or not selected by the PMTF even though they are actually poor – to ensure that their information was recorded correctly, and if so, the precise reasons why the PMTF score has come out the way it has. 106. Not only does this exercise help correct errors in information and disseminate knowledge about the way the PMTF works, it also provides valuable clues about why perceptions may depart from statistics, which in turn is critical for anticipating future problems and devising strategies to address them. The field validation exercise also helped design the rules and mechanisms for the appeals process, which is the only way to ensure that (i) targeting errors arising out of incorrect or distorted information are corrected; and (ii) there are some clear mechanisms to make allowances for special circumstances – however these need to be appropriately defined, so that they are not overly broad and cannot be applied too often; allowing too many such special criteria and applying them too make for an arbitrary and subjective process, which takes away the entire point of applying the PMTF. Institutional reform to support implementation 107. In order to implement the PMTF-based selection mechanism as described above, the Act and the WBB already provide the broad institutional framework. The stipulations in the Act imply that the responsibilities for selection of beneficiaries according to the eligibility criteria set by the formula, monitoring of entry and exit into the program and setting the rules of the appeals process will reside with the Board. The existing program officers of Samurdhi will continue to be responsible for administering the application process under the supervision of the WBB and district and divisional administrators, and facilitating the necessary information campaigns and the community validation process. The separation of roles between the program officers and WBB is fundamental to implementing an objective selection mechanism. Furthermore, community validation will also build in a mechanism to enforce accountability of the program officers, and minimize the likelihood of political influence in the selection process. 108. Given that the reform will result in paring down of the list of current beneficiaries, an appeals process with clear and transparent guidelines will be necessary. The participants at the policy workshop of November, 2003 came up with a number of recommendations to clarify the objectives and independence of the appeals process. It was recommended that the appeals should be handled by community-level committees with broad participation (including representatives of ethnic groups, community elders like priests and teachers, and government officials like the Grama Niladhari). The Samurdhi Development Officer can play the role of facilitating the 26 formation of the committee and assisting it in its functions, but should not be in a position to influence its judgments. The appeals should also be subject to guidelines that ensure consistency in redressing grievances across communities and minimizing arbitrary judgments. The draft of a detailed operational manual that specifies the roles, responsibilities and guidelines – reflecting the recommendations made by the policy workshop and agreements reached on subsequent meetings with the Steering Committee and WBB – has been prepared and will need to be finalized by the government before the reform is implemented island-wide. 109. A number of changes in the modalities of payments have also been recommended by the policy workshop participants. The two most important among them are: first, to distribute the cash transfers through accounts created in post offices (instead of Samurdhi banking societies as is the current practice), preferably in the name of the female member of the household; second, to eliminate the current practice of compulsory withholding of savings and insurance premiums from the benefits. The de-linking of transfers from savings and insurance and using post offices for distributing benefits will ensure that access to insurance, saving and credit services from the Samurdhi banks are not contingent upon selection for Samurdhi payments. This is desirable because the transfers serve objectives that are completely different from those that are served by microfinance/insurance programs. Recent discussions with the Samurdhi ministry have revealed encouraging developments in this direction. It appears that withholding of forced saving has been discontinued from early 2005, which sets the stage for complete de-linking of Samurdhi transfers from other financial services provided by the Samurdhi banks. Importance of a communication campaign 110. Implementing the reforms as envisaged here would require a carefully conceived information campaign. A campaign that aims to create confidence among stakeholders about the transparency and fairness of the process, as well as the credibility of the promises made, may also go a long way towards mitigating the possible political risks of implementing such a reform in the South. While the details of how such a campaign should be conducted must await formal decisions on how and when the reforms will be implemented island-wide, the broad objectives of such a campaign are important to stress at this stage. 111. The primary objective of the campaign should be to effectively communicate the potential gains from such a reform: better coverage among (and more sizeable benefits for) the poor and vulnerable, at the cost of eliminating benefits for those who are relatively better-off; and a more objective and transparent criteria for selection of beneficiaries and assigning payment amounts. The campaign must also clearly communicate the new set of rules and institutional structure that will govern the process of selection of beneficiaries, to lend transparency to the entire process. This will also entail assuring stakeholders about the checks and balances that will operate through the community committees with broad representation who will play an important role in validating beneficiary lists, redressing grievances and handling appeals. Finally, in order to mitigate the possible political fallout from those who stand to lose benefits, it will be important to credibly communicate the range of alternative programs – like the previously-described livelihood support initiatives being considered by the Ministry of Samurdhi – that will be available, independent of whether a household is selected for the transfers program. VII. Impact of PSIA 112. At various stages along its path, the PSIA informed the design and preparation for implementation of the reform, because it was closely integrated with the Bank’s support and policy dialogue for the reform. The PSIA-type of analysis, initiated with the exercise to derive a PMTF for Sri Lanka, was also supplemented by other, equally important activities that added up to the full range of support necessary to bring about the institutional capacity to undertake the 27 reform of a program as large as Samurdhi. The previous sections have described – sometimes in detail – the analytical part of the support, and the specific elements of the exercise that influenced key changes in the design of the reform, or led to introduction of compensatory measures to mitigate negative impacts. Some of these are summarized below. 113. To start with, the basic policy decision to go forward with a PMTF approach – which was the precursor to all the other decisions that followed on eligibility, payments and so on – was greatly influenced by the analysis using SLIS and CFSES data to explore the feasibility of a PMTF, and the gains that can be expected from adopting such an approach. At a later stage, the results from the analysis of ex ante impact and the subsequent analysis with the pilot data were the moving force behind the discussions and agreements reached at the Policy Workshop In December 2003. Based on many elements of the analysis described in the previous section, as well as the field experiences of the pilot, the discussions at the workshop led to decisions on a number of key elements of the reform – including the choice of the PMTF, the eligibility cutoff for selection, and the schedule of payments for beneficiaries (each of these choices are identified and incorporated in the analysis in the previous section). In addition, a number of recommendations emerged from the workshop on the composition of the appeals committee, enrollment of beneficiaries into the program, and the steps necessary to build a functioning MIS database. 114. There are a number of examples of the analysis informing adjustments in design of reforms – to mitigate potential adverse impacts or induce additional benefits. One example is the exercise described in Section VI, undertaken to mitigate the potentially high likelihood of exclusion of small, poor households (of size 3 or less) by the PMTF-based selection criteria. Since this group represents a small share of the population, this exclusion has only negligible impact on aggregate targeting errors; however, the adjustment was necessary to minimize adverse impact on a potentially vulnerable section of the population who are likely to rely on social assistance (e.g. aged people or couples living by themselves, or with a grandchild). 115. As mentioned in Section IV above, the most recent policy impact of the work on designing and preparing for the reform – which includes the technical analysis that is the subject of this paper, as well as institutional capacity-building, policy dialogue and MIS development – has been the decision to introduce the Samurdhi program targeted by the PMTF in the North-East. The analytical work, which is still ongoing, will be critical in implementing the scheme in the North-East. The PSIA thus remains an ongoing exercise, as it must if it is to continue providing solutions to potential problems, which are quite likely to arise given the challenging environment in this region. 116. The current status – the program being introduced in the North-East, but uncertain future for reform in the South – also puts the onus on the PSIA’s potential role, namely that of informing policy debates with a view of generating consensus around the need to undertake such reform for the South also. Section IV refers to the additional strain expected for the welfare system of the country due to the imminent phasing out of the cash grant program for tsunami- affected families; and accommodating the additional demand for benefits generated will require exploring options to make Samurdhi much more efficient in targeting the poor. The proposed Samurdhi reform has identified a way to achieve precisely that objective, and more importantly one that appears to work – in pilots, and now hopefully with similar success in actual operation in the North-East. The PSIA is in the unique position to bring these facts and its implications out clearly, to inform and influence the policy dialogue that can revive the impetus for this reform. 28 VIII. Lessons learned 117. As the discussion above has shown, the PSIA for welfare reform in Sri Lanka has played an integral part in motivating the reform process on the one hand, and influencing its design on the other. The lessons gained from such an exercise are numerous – ranging from the technical issues relevant for similar PMTF exercises or the design of safety nets, to broader lessons about the process of PSIAs and the role they can play in facilitating reform. 118. On the technical side, the PSIA exercise has generated a number of lessons that are highly relevant for designing safety net interventions, and especially for deriving a PMTF. Two are considered the most important. The first key lesson is the importance of a pilot, primarily due to the fact that household surveys, which are inherently different in nature from the exercise of actual collection of data from applicants to the program, cannot replicate the incentives on the ground and logistical difficulties that may affect the quality of data. A pilot carefully designed to replicate the actual exercise is therefore critical for testing the validity of the targeting formula, as well as to indicate whether the process is adequate in terms of (a) collecting the relevant information accurately, and (b) including all potential applicants for the program – especially those in vulnerable groups and remote areas. Moreover, as was the case in Sri Lanka, the pilot is also valuable for checking whether certain specific groups are being excluded by the PMTF, and how consistent the selected list of beneficiaries are with perceptions on the ground. Even when the pilot cannot involve the first-best option of actual payment of benefits, as was the case in Sri Lanka, it can still be of enormous value along the above dimensions, if designed appropriately. 119. The second lesson relates to the strong link between a country’s statistical systems for monitoring poverty and social indicators and measure poverty on the one hand, and the feasibility of conducting a proxy means testing exercise on the other. In Sri Lanka’s case, while the one-off exercise of SLIS turned out to be a boon, using this survey to derive the PMTF however raises the question of how the PMTF will be updated in future, since the SLIS in its current form will never be repeated. The future plans of Sri Lanka’s official survey agencies however offer viable options – DCS’s plan involves expanding their current Household Income and Expenditure Survey (HIES), which has little information on welfare predictors, into one that is closer to the form of an LSMS. 37 As such a survey becomes available, there is now requisite capacity among the technical staff within the WBB to update the PMTF, based on the experience they gained from the collaboration with the Bank team on developing and piloting the current PMTF. 120. The recent resolution of the poverty line and measurement issue, by ensuring a consensus about poverty incidence in the country, has also helped provide a clear benchmark for the size of the target group for the PMTF. The Bank’s efforts in supporting these activities have been able to exploit synergies with the progress on the welfare reform front. The externalities flow both ways: resolution of poverty measurement and survey plans are essential to make the estimation and the future updating of the PMTF feasible; while the adoption of the PMTF has in turn generated the impetus for appropriate surveys, as the problems from not having such surveys have become apparent. 121. On the broader issue of the role of the PSIA in facilitating the reform process, an important lesson pertains to the links between the analytical work on the one hand, and institutional capacity building and legal framework on the other. In the case of Sri Lanka, the passage of the Welfare Act by the country’s parliament provided the critical legal framework for the reform. Equally critical was the work to build institutional capacity, which helped in 37 The Central Bank has also recently completed its own Consumer Finance and Socio-Economic Survey (CFSES) of 2003-04, which contains information on many of the welfare predictors. The future surveys by DCS and the Central Bank will both afford opportunities to evaluate or even update the PMTF using more recent data, when the data become available. 29 internalizing the insights from analytical work into the design of the reform and enabling an institution like the Welfare Benefits Board (WBB) to be in a position to implement the design on the ground. 122. Related to above is a lesson for international agencies like the World Bank on the role of PSIAs: an effective PSIA must take the form of a continuing engagement rather than a one-off exercise. This is all the more crucial in the case of reforms like the one discussed here, which involves changing the institutional mindset of a well-entrenched program where there are strong political incentives to maintain status quo. Such reforms are also intrinsically different, in terms of design and implementation challenges, from stroke of the pen policy reforms like removing certain distortions in trade policy or reducing fuel subsidies. In this case, even when the enabling legal and institutional framework is in place (as with the Act and the WBB), there was a need for continuing technical assistance that integrates PSIA-type analysis into its broader objectives. Furthermore, in such cases, it is also important to recognize that PSIAs may need multi-year engagement, to provide not just analytical solutions but also help create the enabling environment for implementing these solutions on the ground. 123. Another lesson is the importance of embedding reform in the context of the broader policy framework of the country. In the case of Sri Lanka, the broader policy dialogue on the evolution of safety net programs in the country also needs take into account the potential for moving from an exclusive reliance on unconditional transfers to a role for conditional transfers, including public works programs that would also address the long-standing problems of mis- targeting by inducing an element of self-selection into the program. The future of this policy dialogue will be critical for implementing the welfare reform countrywide – easing the transition into a more selective Samurdhi program by clarifying the role and clientele of such programs, and providing alternative opportunities for those who do not qualify for unconditional transfers like Samurdhi. Currently, the country is poised for serious consideration of this issue, especially in the aftermath of the tsunami that has created additional challenges for the welfare system and the strategy outlined in the PRSP progress report endorsing the role of programs like public works. The work for this PSIA and other related exercises – including the Bank’s support for tsunami related safety net programs – can potentially play an important role in informing this important debate. 30 References Coady, D., M. Grosh, J. Hoddinott (2004) Targeting of Transfers in Developing Countries: Review of Lessons and Experience. Regional and Sectoral Studies. World Bank. Washington DC Glewwe, P. (1990). “Efficient Allocation of Transfers to the Poor.� Living Standards Measurement Study Working Paper No. 70. World Bank, Washington, D.C. Glewwe, P. and O. Kanaan. (1989). “Targeting Assistance to the Poor: A Multivariate Approach Using Household Survey Data.� Policy, Planning and Research Working Paper No. 225, World Bank, Washington, D.C. Glinskaya, E. (2000). “An Empirical Evaluation of Samurdhi Program�. Background paper to Sri Lanka Poverty Assessment. SASPR, World Bank, Washington, D.C. Government of Sri Lanka (2005). “Sri Lanka New Development Strategy, a Framework for Economic Growth and Poverty Reduction� Government of Sri Lanka (2002). “Regaining Sri Lanka: Vision and Strategy for Accelerated Development� Grosh, M. (1994). Administering Targeted Social Programs in Latin America: from Platitudes to Practice. Washington, D.C. : The World Bank, c1994. 174 p. Grosh, M., and J. Baker. (1995). “Proxy Means Tests for Targeting Social Programs: Simulations and Speculation.� LSMS Working Paper No.118, World Bank, Washington, D.C. Grosh, M and E. Glinskaya. (1997). “Proxy Means Testing and Social Assistance in Armenia.� Draft. Development Economics Research Group, World Bank. Hentschel, J., J. Lanjouw, P. Lanjouw and J. Poggi. (1998). “Combining Census and Survey Data to Study Spatial Dimensions of Poverty: a Case Study of Ecuador�. Policy Research Working Paper No. 1928, World Bank. Narayan, A. and N. Yoshida. (2004). “Proxy Means Test for Targeting Welfare Benefits in Sri Lanka�. mimeo. Forthcoming South Asia Policy Working Paper Series. Ravallion, M. and K. Chao. (1989). “Targeted Policies for Poverty Alleviation under Imperfect Information: Algorithms and Applications.� The Journal of Policy Modeling, 11(2): 213-224. World Bank (2003). Sri Lanka Country Assistance Strategy. Report No. 25413-CE. Washington D.C. World Bank (2005). “A Poverty Map for Sri Lanka—Findings and Lessons�. Policy Note (draft). South Asia Region. 31 Annex Table A- 3: Deductions from Samurdhi grants in 2003 A. Expenditure composition of Ministries of Samurdhi and Social Welfare An average of Rs. 13.2 bn. was allocated to Samurdhi between 2000 and 2003, which amounts to 1 percent of annual GDP, over 10 percent of social service expenditures and a third of welfare expenditures. Total Samurdhi expenditures (nominal) increased from around Rs. 10 bn. in 2000 to Rs. 15.2 bn. in 2002, and then declined to around Rs. 12.2 bn. in 2003 – largely due to changes in the size of the transfer component. Recurrent spending typically accounts for about 98 percent of total expenditure, and about 97 percent of that consists of consumption grant to households and salaries paid by the Samurdhi Authority to a 26,000+ cadre of staff that administer the program. For instance in 2003, recurrent spending accounted for Rs. 12.2 bn. out of a total of Rs. 12.5 bn., of which Rs. 9.2 bn. was spent on consumption grants or transfers, and Rs. 2.8 bn on the salary bill of the Samurdhi Authority (about 22 percent of the total expenditure of the program). In contrast to the Ministry of Samurdhi, the Ministry of Social Welfare and its two associated departments, consume only 0.2 percent of GDP, 3 percent of the social services expenditure and 7.5 percent of the spending on welfare. The breakdown of expenditures of the three institutions shows that recurrent expenditure accounts for about 98 percent of the total. Mirroring the pattern seen for Samurdhi, there was a steady increase in total expenditures from 2000 to 2002, from around Rs. 2.5 bn. to Rs. 4.4 bn., followed by a decline to around Rs. 4 bn. in 2003. The increase is primarily explained by an increase in the size of the transfer component; the share of this component in the total budget has also significantly increased over time, from 79 percent in 2000 to 94 percent in 2003. As much as 92 percent of transfers in 2003 went for social security for disabled soldiers (including families of soldiers who have lost their lives), leaving only 8 percent for beneficiaries within other vulnerable groups. B. Characteristics of current Samurdhi transfers Tables A-1 and A-2 show the grant amounts made by the Table A- 1: Samurdhi grant amounts existing Samurdhi system in 1999 – when the SLIS data (Rs.) and recipients (1999) was collected – and 2003. There were 5 different grant Official Grant Size Share of amounts ranging between Rs. 100 and Rs. 1000 monthly (1999) Recipients (SLIS) in 1999. A large majority of the beneficiaries were 100 3.2 getting Rs. 200, 250 or 500, with the largest proportion 200 17.4 (59 percent) receiving Rs. 500. The official figures from 250 14.6 2003 show a largely similar picture. Benefits of Rs. 1000 500 58.5 remain extremely rare in both years; in 2003 a majority of 1000 1.2 beneficiaries received between Rs. 250-600, with the Note: 5.2 percent of recipients reported Table A- 2: Samurdhi grant amounts largest proportion grant amounts not listed officially (Rs.) and recipients (2003)* (31 percent) receiving Rs. 600. Official Grant Share of Recipients One significant change in 2003 is that a sizeable Size (Samurdhi Ministry) proportion of the population received the minimum 140 21.1 amount, i.e. Rs. 140. Furthermore, as seen in Table A-3, 250 12.7 these beneficiaries in effect got nothing at all in hand, 350 17.0 since the entire amount was deducted as compulsory 400 17.8 saving, social security and housing lottery fund 600 31.2 contributions. In addition, deductions totaling Rs. 40 1000 0.2 were also made from each grant of Rs. 400, Rs. 140 from * : Estimates as of May 2003 Note: Total no. of beneficiaries = 1,892,842 each grant of Rs. 600, and Rs. 240 from each grant of Rs. 1000. i Therefore not only is it the case Payment Goods Cash Compulsory Social Lottery that a significant proportion of the Size stamps stamps Savings Security Fund beneficiaries receive very small 140 0 0 100 30 10 amounts (less than Rs. 400 per 250 240 0 0 0 10 family per month), but also that 350 340 0 0 0 10 many of them receive nothing for 400 360 0 0 30 10 current consumption after the 600 250 210 100 30 10 compulsory deductions. Even for 1000 400 360 200 30 10 those who receive more sizeable amounts of Rs. 600 and above, the amounts available for consumption are reduced substantially by the compulsory contributions. In the aggregate, our calculations show that compulsory deductions amounted to an estimated 22 percent of the total value of Samurdhi grants in 2003, i.e. the actual consumption grant to beneficiaries amounted to only 78 percent of what was allocated. The compulsory saving component accounted for about 14 percent, while social security and housing lottery contributions accounted for 5.5 and 2.6 percent of the total value of the grants respectively. C. Proxy Means Test Formula (PMTF): rationale and evidence Rationale for Proxy Means Test Formula to identify beneficiaries of a transfer program Targeting benefits to the poor first requires a precise definition of the target group. Once the target group is established, a methodology must be found for identifying individuals or households that are in that group and for excluding those who are not. For instance, if the poor are identified as a target group for a program, one must be able to make a precise judgment about the level of welfare or the means of the recipient. In principle, conducting a means test that correctly measures the earnings of a household is the best way to determine eligibility when the poor are the target group, as is the case with Samurdhi. In practice, however, such straightforward means tests suffer from several problems. First, applicants have an incentive to understate their welfare level, and verifying that information is difficult in developing countries where reliable records typically do not exist. Second, income is also considered an imperfect measure of welfare in developing countries, since it is unlikely to measure accurately imputed value of own-produced goods, gifts and transfers, or owner-occupied housing. Incomes of the poor in developing countries are also often subject to high volatility due to factors ranging from seasonality of agriculture and sporadic nature of employment in the informal sector. Since adjustments for such volatility are hard to make in practice, actual welfare from income measures are likely to be highly distorted. In the light of these difficulties, rigorous means tests are largely reserved for industrialized economies where a well-educated labor force is concentrated in jobs in which cash is paid regularly and payments are reported to tax or welfare authorities. Where means-testing is used in developing countries, it is greatly simplified, at a considerable cost to accuracy. 38 Given the administrative difficulties associated with sophisticated means tests and the inaccuracy of simple means tests, the idea of using proxy means tests that avoid the problems involved in relying on reported income is appealing. Proxy means test involves using information on household or individual characteristics correlated with welfare levels in a formal algorithm to 38 Simple means tests are performed as part of the food stamp programs in Jamaica (prior to 2002), Honduras, Sri Lanka and Zambia. In Jamaica and Sri Lanka, this evaluation has been largely subjective and does not contain any systematic examination or weighting of certain factors. Evaluations reveal that the two programs delivered only 56 and 57 percent of its benefits respectively to those in the poorest 40 percent of the population. ii proxy household income or welfare. These instruments are selected based on their ability to predict welfare as measured by, for example, consumption expenditure of households. The obvious advantage of proxy means testing is that good predictors of welfare – like demographic data, characteristics of dwelling units and ownership of durable assets – are likely easier to collect and verify than are direct measures like consumption or income. The efficacy of proxy means testing is indicated by a recent comparative study of targeting in Latin America (Grosh, 1994), which has found that, among all targeting mechanisms, proxy means tests tend to produce the best incidence outcomes in developing countries. Academic evidence and practical experience with Proxy Means Tests A number of simulations in academic papers by various authors show how proxy means test could work, and the welfare gains likely produced by implementing such a targeting system. Haddad, Sullivan and Kennedy (1991) used household survey data from Ghana, the Philippines, Mexico and Brazil to show that some variables that would be very simple to collect could serve as good proxies for the measures of caloric adequacy that are usually used as the standard measures of food and nutrition security, which are harder to collect as they rely on the memory of individuals and on the anthropometric indicators of pre-school children. Glewwe and Kanaan (1989) have used regression analysis on a data from Cote d'Ivoire to predict welfare levels based on several combinations of variables that are fairly easy to measure. The paper demonstrated that simple regression predictions could improve targeting markedly over untargeted transfers.39 In a recent study, Grosh and Glinskaya (1997) used regression analysis with data from Armenia to show how the targeting outcomes of a current cash transfer program can be improved by using a suitable proxy mean test formula. Grosh and Baker (1995) carries out simulations on Living Standards Measurement Survey data sets from Jamaica, Bolivia and Peru to explore what kind of information can best be used in a proxy means test and how accurate such tests might be expected to be. Their results show that more information is generally better than less for a targeting formula, though there are diminishing returns. The proxy systems all have significant undercoverage, but they cut down leakage so much that the impact on poverty is better with imperfect targeting than with none. While academic exercises have been useful in developing such a proxy mean test system, more insights on the implementation of such programs can be gained by looking at actual experiences on the ground, in Chile where it has existed since 1980, and more recent programs in Costa Rica, Colombia and Jamaica. The Ficha CAS in Chile uses a form filled out by a social worker that collects information on household characteristics such as location, housing quality, household composition and education and the work done by the household members. Scores are then assigned using a complicated algorithm and then used to determine eligibility for two large cash transfer programs and for water and housing subsidies, and if so, the level of subsidy. D. Issues in deriving PMTF for Sri Lanka using SLIS data Predicting welfare: the choice of Ordinary Least Squares (OLS) To derive the PMTF, the consumption variable is regressed, using OLS method, on different sets of explanatory variables. The case for using OLS as the model for predicting welfare is driven primarily by convenience and ease of interpretation. The first problem with using an OLS model is that many of the explanatory variables are likely to be endogenous to (and thus not independent 39 Glewwe (1990) took the same basic approach of predicting welfare. Instead of using regressions, he solved a poverty minimization problem to derive weights for each household variable. While theoretically more appropriate, the poverty minimization technique is much more difficult to compute, and produces results not dissimilar from those based on regression analysis. iii explanators of) household welfare. This problem is however is of less concern to us, since our objective is solely to identify the poor and not to explain the reasons for their poverty. Second, Grosh and Baker (1995) points out that strictly speaking, OLS is inappropriate for predicting poverty since the technique minimizes the squared errors between the "true" and the predicted levels of welfare, which is a different theoretical problem from that of minimization of poverty. That said, OLS is considered convenient and useful by these authors when a large numbers of predictor variables, including continuous variables, are available. Incidentally, an algorithm that does solve the problem of minimizing poverty is found in Ravallion and Chao (1989), and could be a better tool for designing a transfer scheme than the OLS model. However this algorithm is very difficult to use when a large number of predictive variables are available, and is difficult to interpret for policymakers. 40 Moreover, using OLS has the advantage of being able to intuitively interpret the coefficients of the predictors on welfare – a feature that is likely to appeal to a policymaker and more amenable to achieving political consensus in the country. As mentioned previously, the PMTF exercise involves defining an indicator for the welfare of a household – which in this case was chosen to be per capita household consumption expenditure (monthly) – and then identifying a set of easily observable variables/predictors that together serve as the best proxy for the welfare indicator. 41 To derive the PMTF, the consumption variable is regressed, using Ordinary Least Squares (OLS) method, on different sets of explanatory variables. The case for using OLS as the model for predicting welfare is driven primarily by convenience and ease of interpretation – especially convenient when a large numbers of predictor variables (including continuous variables) are available; and amenable to intuitive interpretation of the regression coefficients of the welfare predictors, which is more easily understandable to a policymaker (see Annex for a fuller discussion on the choice of OLS). Choice of variables to consider for PMTF Selection of variables to predict welfare as measured by per capita consumption should take into account two separate criteria: correlation between the welfare measure and the predictor, which will determine accuracy of the prediction, and verifiability of the predictor, which will determine the accuracy of information used to impute welfare. The types of predictors used for this exercise, discussed below, were arrived at after judging all possible predictors on the basis of these two criteria (including trade-offs that at times exist between them). Location variables are obviously the most easily verifiable, and the same is true for characteristics of the community, when it is defined in simple terms like the presence of a bank or administrative offices. Housing quality may also be easily verified by a social worker visiting the home. Household characteristics, such as the number of members and dependents, and age, education and occupation of the household head, are less easy to verify. However, it is generally felt that these information, firstly, are not overly difficult to verify, and secondly, that households are less likely to misrepresent such information. Using program officers who live in the same community as the applicant households to collect the information – as is envisaged for Sri Lanka – also makes it more likely that such information will be reported correctly. 40 See Grosh and Baker, Annex I for a fuller discussion 41 In development literature, consumption expenditure is generally considered a more accurate measure of welfare than income for several reasons. First, because consumption expenditures are more likely to indicate the household’s “true� economic status, as a result of households with sporadic incomes smoothing their consumption patterns over time. Second, consumption is generally measured with far greater accuracy than income in a household survey, primarily because households’ sources of income may include home-based production and own farms, calculating the flow of net incomes from which is a big problem. iv Ownership of durable goods or farm equipment is verifiable by inspection – however they can be misrepresented by the household removing the goods from the home during an expected visit by the social worker, which is easier to do with small or mobile items than for items such as stoves or refrigerators. The general presumption in the literature is also that people are more willing to lie about ownership of such items than they are about household characteristics. However, these variables tend to have high predictive power for welfare, and therefore including them can reduce mis-targeting substantially. Ownership of productive assets is again not easy to verify. The presence of livestock is verifiable to some extent. As for land ownership, while it may not be measured perfectly, one can reasonably expect that program officers who belong to the community will have local knowledge about whether a household owns a large amount of land or not, which will deter misrepresentation. The fact that these variables are likely to have high correlations with poverty in rural areas makes a strong case for including them as predictors of welfare. Recommendations for formulas for PMTF: comparison of models I, II and III While the final choice was model III, it is useful to summarize the pros and cons of each model. • Model I is the most comprehensive model – incorporating province dummies and variables from all categories mentioned above. - Yields the best fit and the lowest error rates on the aggregate - Province weights: (a) may be hard politically to incorporate in a formula (b) some weights are not intuitive, which reduces their acceptability - Because of the weights, rates of undercoverage vary widely – some provinces covered far better than others • Model II omits province location variables, and restricts the set to variables that are highly significant (99 percent level and above) - Fit and error rates are close, but not identical to those for model I - Avoids the problems in Model I due to the use of province weights - Undercoverage rates are more uniform across provinces than for model I – which is desirable - Fewer number of variables reduces the information requirement to apply the model • Model III is identical to model II, with urban location variables omitted. Therefore all the pros and cons of model II vis-à-vis Model I apply. - Yields overall error rates very similar to those for model II - Reduces urban undercoverage as compared to model II, at the cost of slight increase in rural undercoverage (this is possible since the number of urban poor is very low compared to that of rural poor) - Should be selected over model II since the gap in undercoverage rates between rural and urban areas is much smaller. Comparison of selected models with those using alternative methods of estimation To check whether the methodology used for the estimation can be improved upon in terms of impact on targeting efficiency, two alternative methods of estimating the PMTF were implemented. The first consisted using the poorer segment of the population to derive a PMTF, which may lead to better results since it puts more emphasis on accurately predicting the welfare of those near the bottom of the distribution, where the improvements are most relevant to the goal v of poverty reduction. 42 Applying this approach, a model estimated on the bottom 80 percent of the population performs marginally, but not significantly, better than model III above. However, this improvement is not large enough to justify its choice over model III for a number of reasons. Firstly, the improvement is not unambiguous, since lower undercoverage from using the new model vis-à-vis model III has to be weighed against the lower leakage rates from the latter. Secondly, even the improvement in undercoverage is not very high (between 2 and 5 percentage points) for the cutoff points likely to be the most relevant (25th and 30th percentiles). Thirdly, the method of estimating the poverty predictors using only a certain part of the sample, depending on the poverty ranking of households, appears to be rather arbitrary. For these reasons, model III is easier to understand and explain – and given that the advantages of the alternate model are marginal, model III still appears to be the best choice. The second alternative method consists of using half the sample to run the regressions to predict welfare, and testing the predictions from this model by calculating undercoverage and leakage rates on the other half of the sample. The utility of this lies in reducing the likelihood of “overfitting� the sample. 43 By separating the estimation and the testing of the model between two non-overlapping parts of the sample, the model will likely be subjected to a harder test – by minimizing the bias in favor of the model that may occur when the predictions from the model are used on the same observations that were used to derive the coefficients. This is important to consider, so as to mimic as far as possible the real-world situation where our models will be applied to impute the welfare of households who will not be the same set of households for whom the formula is estimated. This exercise is conducted as a test for the sensitivity of the existing models in two ways: firstly, to see whether the coefficients of the model using this method are significantly different from those derived using the whole sample; secondly, to see how the targeting errors are when the new method, involving a “harder test� of accuracy in targeting, is used. This exercise is conducted for the two sets of variables used in model II and III, and the two new models are called IIa and IIIa respectively. Firstly, the coefficients from model IIa turn out to reasonably close to those from the original model II, and similarly those from model IIIa to the original model III. This suggests that the original models are quite robust to adjustments for overfitting. Secondly, the error rates using the new method are close to those of the original models for various cutoff points and poverty lines, with the former being usually higher by 2-3 percentage points. Thus subjecting the methodology to a harder test does not lead to significant increases in targeting errors; moreover, the coefficients or weights of the variables after adjusting for overfitting are similar to those of the original models. These results essentially validate the OLS models, the methodology underlying them, as well as the results from the simulations of targeting errors with these models. 42 Grosh and Baker presents a case where they use only the poorest half of the population as the basis for building the targeting models, and show that such an approach leads to significantly lower undercoverage. 43 See Grosh and Glinskaya (1997), Hentschel et al (1998) for applications of this method vi E. Additional Tables and Figures Table A- 4: Poverty headcounts for Sri Lanka 90-91 95-96 2002 National 26.1 28.8 22.7 Urbana 16.3 14.0 7.9 Rurala 29.4 30.9 24.7 Estate 20.5 38.4b 30.0 Source: HIES data for relevant years, using official poverty lines (DCS) Notes: a: The classification of urban and rural areas is different between HIES 90-91 and HIES 95-96 onward as discussed below. The changes do not alter the arguments on the poverty trend, although the levels of headcount ratios (urban and rural) need to be cautiously interpreted. b: Comparability of estate headcount for 95-96 with that for other years may be affected by the fact that HIES in 95-96 was sampled differently for the estate sector Table A- 5: PMTF (weight on each variable for the selected models) Variables Dummy Model II Model III (selected as PMTF) Location Rural/Estate * -10 0 Community characteristics Public/Private bank in community * 7 8 Divisional Secretariat in community * 8 9 Household assets Car/van * 40 40 Cooker (kerosene/gas/electric) * 15 17 Bicycle/Tricycle * 4 4 Fan * 11 11 Refrigerator * 11 12 Motorcycle/Scooter * 9 8 Radio/CD/Cassette player * 4 4 Sewing Machine * 7 7 Tractor * 15 15 TV/Video player * 7 8 Land and livestock Cultivable land owned by household : 14 * 17 16 Livestock (any) * 8 8 Household head Not a female who is widowed/separated/divorced * 6 5 Age: 70-79 * -6 -5 80 and above * -13 -13 Education: Passed OL or Grade 11 * 7 7 Passed AL/GAQ/GSQ * 10 10 Has Degree/PG/Diploma * 17 16 Work: Salaried employment or in business * 5 5 Household demographics Household size: 3-4 members * -22 -23 vii 5-6 members * -39 -39 7-8 members * -51 -52 >8 members * -59 -59 All children age 5-16 attend school * 7 6 Housing characteristics Dwelling owned by hhold * 4 4 Fuel for cooking: Gas/electricity * 12 13 Toilet: Private and flush type * 16 16 No. of Rooms (excl. kitchen/bath) per hhold member 17 16 Walls: Not cabook/mud/plank/cadjan * 6 6 Constant 715 707 Notes: 1) All scores are derived from regressions of (log of) per capita consumption expenditure on a set of variables 2) The score for each variable is its coefficient in the regression, multiplied by 100, and rounded to the nearest integer 3) The aggregate score for each household is calculated as constant +/- the weight on each variable • For each dummy variable (indicated by *), multiply the score by 1 if true for household, by 0 if not true • For each continuous variable, multiply the score by the value of the variable for the household 4) Regressions include only variables with significance level of 99 percent and above Table A- 6: Regression results from OLS estimations (Dependent variable: Log of actual per capita monthly consumption expenditure of household) Variable Description of variable Model II Model III non_urban 1= lives in Rural/Estate -0.098 0= lives in urban area (6.11)** car_van 1= hhold has Car/van 0.402 0.403 0= otherwise (16.33)** (16.32)** cooker 1= hhold has Cooker (kerosene /gas/ 0.147 0.166 electric); 0= otherwise (7.58)** (8.68)** cycle 1= hhold has Bicycle/Tricycle 0.040 0.036 0= otherwise (3.68)** (3.27)** fan 1= hhold has Fan 0.108 0.114 0= otherwise (6.77)** (7.15)** fridge 1= hhold has Refrigerator 0.112 0.118 0= otherwise (5.92)** (6.20)** m_cycle 1= hhold has Motorcycle/Scooter 0.089 0.083 0= otherwise (5.39)** (5.01)** radio 1= hhold has Radio/CD player/Cassette 0.044 0.043 player; 0= otherwise (3.21)** (3.12)** sew_mach 1= hhold has Sewing Machine 0.073 0.073 0= otherwise (5.93)** (5.95)** tractor 1= hhold has Tractor 0.149 0.149 0= otherwise (3.87)** (3.88)** tv_vcr 1= hhold has TV/VCR 0.072 0.075 0= otherwise (5.68)** (5.87)** bank_com 1= Public/Private bank in community 0.073 0.083 0= otherwise (5.59)** (6.46)** ds_com 1= Divisional Secretariat in community; 0.083 0.091 0= otherwise (3.88)** (4.24)** wid_f 0= Head is female and widowed /separated 0.056 0.053 /divorced; 1= otherwise (3.49)** (3.25)** ageHcat4 1= hhold head Age: 70-79 -0.055 -0.054 0= otherwise (2.69)** (2.61)** ageHcat5 1= hhold head Age: 80 + -0.134 -0.133 0= otherwise (3.95)** (3.89)** viii edulevH4 1= hhold head Passed OL or Grade 11 0.065 0.065 0= otherwise (4.16)** (4.16)** edulevH5 1= hhold head Passed AL/GAQ/GSQ 0.103 0.103 0= otherwise (4.32)** (4.33)** edulevH6 1= hhold head Has Degree /PG /Diploma; 0.169 0.163 0= otherwise (3.86)** (3.71)** activH34 1= hhold head in Salaried employment or 0.049 0.050 business; 0= otherwise (3.80)** (3.89)** landown2 1=Cultivable land owned by household: 0.075 0.069 14 0.166 0.159 0= otherwise (3.74)** (3.58)** lstk 1=hhold has Livestock (any) 0.084 0.082 0= no livestock (4.39)** (4.24)** dhsize2 1= hhold size: 3-4 members -0.220 -0.227 0= otherwise (7.49)** (7.71)** dhsize3 1= hhold size 5-6 members -0.387 -0.393 0= otherwise (12.67)** (12.84)** dhsize4 1= hhold size 7-8 members -0.512 -0.516 0= otherwise (15.30)** (15.36)** dhsize5 1=hhold size 8 + members -0.587 -0.586 0= otherwise (15.02)** (14.94)** rsch5_16 1=All children in hhold of age 5-16 attend 0.065 0.062 school; 0=otherwise (3.67)** (3.45)** dwellten1 1=Dwelling owned by hhold 0.038 0.035 0= not owned by hhold (3.09)** (2.82)** fuel1 1=Fuel for cooking: Gas/electricity 0.122 0.126 0= other (5.37)** (5.53)** latrtyp1 1=Toilet: Private and flush type 0.163 0.162 0= other (9.93)** (9.80)** rmsmem No. of Rooms (excl. kitchen/bath) per 0.165 0.159 hhold member (11.50)** (11.05)** walltyp137 0=Walls: cabook/mud/plank/cadjan 0.062 0.063 1= other (4.77)** (4.80)** Constant 7.145 7.071 (175.16)** (181.02)** #Observations 5257 5257 R-squared 0.56 0.56 Source: SLIS (1999-2000) Note: ** denotes significant at 99% level Table A- 7: 95% Conf. Intervals for undercoverage and leakage rates with PMTF (Model III) Cutoff percentile Undercoverage Leakage 25 [0.48 , 0.57] [0.34 , 0.44] (0.021) (0.027) 30 [0.39 , 0.46] [0.31 , 0.39] (0.019) (0.022) 35 [0.33 , 0.40] [0.30 , 0.38] (0.018) (0.020) 40 [0.25 , 0.31] [0.27 , 0.34] (0.017) (0.018) Notes: All calculations taking threshold for target group same as the eligibility cutoff; confidence intervals are in square brackets; standard errors are in parentheses ix Table A- 8: Per capita benefits (1999 Rs./Month) PMTF with optimal Decile payment scheme Current Samurdhi 1 89.5 53.4 2 63.1 48.4 3 48.3 43.3 4 36.1 42.1 5 29.7 45.0 6 18.9 39.2 7 11.1 33.0 8 5.8 24.7 9 2.9 16.1 10 0.2 5.5 Total 30.6 35.1 Note: calculations are made taking all households for a decile as the denominator, not just the households receiving (or designated to receive) benefits. Table A- 9: Composition of pilot applicants by district Table A- 10: Coverage rate of pilot areas in the North-East and the rest of the country Percentile of Cutoff Coverage Coverage (%) in Coverage (%) in pilot actual per capita scores (%) population in pilot sample consumption from SLIS predicted districts (SLIS) model from SLIS NE non NE NE non NE 20th 695 12.1 18.4 18.2 13.4 11.1 25th 703 19.6 28.9 29.3 21.0 17.8 30th 709 26.4 38.2 39.0 27.7 23.7 35th 715 33.5 48.3 48.8 35.1 29.6 40th 721 41.6 58.1 58.5 42.2 35.5 x Table A- 11: Results with revised eligibility criteria for small households Sample with household size<=3 Facility Criterion coverage Proportion of poor Undercoverage Leakage No. of rooms<=1 0.18 0.17 0.53 0.56 Original 0.07 0.17 0.77 0.47 Sample with household size<=3 (Rural/ Estate) Facility Criterion coverage Proportion of poor Under-coverage Leakage No. of rooms<=1 0.18 0.18 0.56 0.57 Original 0.08 0.18 0.79 0.51 Sample with household size<=3 (Urban) Facility Criterion coverage Proportion of poor Under-coverage Leakage No. of rooms<=1 0.15 0.08 0.10 0.49 Original 0.05 0.08 0.46 0.13 Note: Original refers to the original PMTF being used to identify eligible households. Coverage refers to % of population in households of size<=3 selected by the criteria. Proportion of poor refers to the true % of poor in households of size<=3 (using SLIS data). Figure A- 1: Poverty in Provinces - Sri Lanka 40 35 30 % values 25 20 15 10 5 0 North-Central Central Uva Western Southern Sabaragamuwa Northwest % of population % of poor headcount (%) Source: HIES (2002) Figure A- 2: Share of provinces in poor population and Samurdhi budget 25 20 % Values 15 10 5 0 North-Central Central Uva Western Southern Sabaragamuwa Northwest Share of poor population (2002) Share of Samurdhi budget (2004) Source: HIES (2002); Ministry of Samurdhi xi Figure A- 3: Comparison of distribution of scores (Pilot score, SLIS actual, SLIS score) 1 .8 .6 .4 .2 0 600 700 800 900 1000 score Pilot SLIS SLIS_actual Note: Scores are log points times 100. SLIS_actual is log per capita expenditure times 100, SLIS refers to predicted score from SLIS data, and Pilot score is calculated by the PMTF using the entire sample including the Northern and Eastern provinces. Figure A- 4: Coverage of Samurdhi and PMTF in pilot areas North & East districts O the r pilot districts 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10 Deciles by pilot scores using SLIS formula No welfare Samurdhi Other Welfare program Source: targeting pilot (2003) xii