100200 AUTHOR ACCEPTED MANUSCRIPT FINAL PUBLICATION INFORMATION A Retrospective Impact Evaluation of the Tamil Nadu Empowerment and Poverty Alleviation (Pudhu Vaazhvu) Project The definitive version of the text was subsequently published in The Journal of Development Studies, 51(9), 2015-07-01 Published by Taylor and Francis and found at http://dx.doi.org/10.1080/00220388.2015.1028538 THE FINAL PUBLISHED VERSION OF THIS ARTICLE IS AVAILABLE ON THE PUBLISHER’S PLATFORM This Author Accepted Manuscript is copyrighted by the World Bank and published by Taylor and Francis. It is posted here by agreement between them. Changes resulting from the publishing process—such as editing, corrections, structural formatting, and other quality control mechanisms—may not be reflected in this version of the text. You may download, copy, and distribute this Author Accepted Manuscript for noncommercial purposes. Your license is limited by the following restrictions: (1) You may use this Author Accepted Manuscript for noncommercial purposes only under a CC BY-NC-ND 3.0 IGO license http://creativecommons.org/licenses/by-nc-nd/3.0/igo. (2) The integrity of the work and identification of the author, copyright owner, and publisher must be preserved in any copy. (3) You must attribute this Author Accepted Manuscript in the following format: This is an Author Accepted Manuscript of an Article by Khanna, Madhulika; Kochhar, Nishtha; Palaniswamy, Nethra A Retrospective Impact Evaluation of the Tamil Nadu Empowerment and Poverty Alleviation (Pudhu Vaazhvu) Project © World Bank, published in the The Journal of Development Studies51(9) 2015-07-01 CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo http:// dx.doi.org/10.1080/00220388.2015.1028538 © 2015 The World Bank A Retrospective Impact Evaluation of the Tamil Nadu Empowerment and Poverty Alleviation (Pudhu Vaazhvu) Project Madhulika Khanna, World Bank* Nishtha Kochhar, World Bank** Nethra Palaniswamy, World Bank*** Draft December, 2013 Abstract Community based livelihood interventions, which focus directly on increasing income and employment, have become an increasingly important component of large-scale poverty reduction programmes. We evaluate the impact of a participatory livelihoods intervention- the Tamil Nadu Empowerment and Poverty Reduction (Pudhu Vaazhvu) Project (PVP) using propensity score matching methods. It explores the impact of PVP on its core goals of empowering women and the rural poor, improving their economic welfare, and facilitating public action. We find significant effects of PVP on reducing the incidence of high cost debt and diversifying livelihoods. We also find evidence of women’s empowerment, and increased political participation. *Corresponding Author E-Mail Address: madhulika.khanna28@gmail.com Address: Room number 474, World Bank, Hindustan Times Building, 18-20 Kasturba Gandhi Marg, New Delhi-110001, India. ** E-Mail Address: nishtha.kochhar89@gmail.com Address: Room number 474, World Bank, Hindustan Times Building, 18-20 Kasturba Gandhi Marg, New Delhi-110001, India. *** E-Mail Address: npalaniswamy@gmail.com Address: Room number 474, World Bank, Hindustan Times Building, 18-20 Kasturba Gandhi Marg, New Delhi-110001, India. This research paper is an output of the Social Observatory Team of the World Bank and Pudhu Vaazhvu Project (PVP). Discussions with the PVP project team, led by the Additional Project Director RV Shajeevana, were critical to the design of this evaluation. Support from all Project Directors of PVP; and from Kevin Crockford, Samik Das and Makiko Watanabe from the World Bank task team is gratefully acknowledged. We also thank PVP for support during survey implementation and GfK Mode for implementing the survey. The authors have benefitted from guidance, discussions and detailed comments from Upamanyu Datta, and Vijayendra Rao. The usual disclaimer applies. 1 1. Introduction Participatory projects that are driven by the hope that greater engagement with local actors will lead to faster and more equitable development have been a popular way of delivering foreign aid. Over the last decade, the World Bank alone invested 85 billion USD in participatory community driven development projects (CDD) (Mansuri & Rao, 2012). Based on the idea that multiple interventions are needed to address the several problems of development, the typical CDD project deploys some form of citizen involvement to implement this set of multiple interventions. Taken together, these interventions – that range from the delivery of local infrastructure and public services to household targeted programmes that involve credit and skills – define a multi-dimensional approach to improving welfare and reducing poverty. Despite the popularity of these projects, the empirical evidence on whether they can in fact affect the wide-ranging changes that they propose is unclear. This evidence, that comes from impact evaluations of different types and aspects of CDD projects, finds that (i) the poverty impact of CDD projects that focus more on local infrastructure ranges from no impact to limited impacts for certain groups (see- Arcand & Bassole, 2007, Chen, Mu, & Ravallion, 2008, Park & Wang, 2010 and Voss, 2008), and (ii) individual components of CDD projects that focus more on livelihoods (credit, skills) show some positive potential (see- Gine & Mansuri, 2011 and Blattman, Fiala, & Martinez, 2011). Each of these evaluations examines the poverty, participatory, or component-specific impacts of these projects, rather than the impact of CDD programmes in and of themselves. The multidimensional approach of the standard CDD programme is typically operationalised through a set of inter-related programme interventions that are implemented at the same time. Whether such projects, that implement multiple and interrelated interventions, can in fact succeed in affecting the range of socio-economic and political change associated with their ambitious goals, however, remains largely unknown. This limited literature has two strands. First, 2 Casey, Glennerster, & Miguel, 2012 use randomised programme assignment to evaluate, GoBifo- a CDD project in post-conflict Sierra Leone. They find positive impacts on the provision of public goods and on economic effects at the household and village level. They find no effects on measures of collective action and local decision-making. The second strand of this literature focuses on livelihoods focussed CDD projects in India. Deininger & Liu, 2009 exploit the time of entry into the programme and use a combination of difference in difference and propensity score matching (PSM) to assess the impact of a project in the state of Andhra Pradesh. While this study finds empowerment effects on the full sample, any impacts on measures of economic welfare of households are confined to the members of Self-Help Groups (SHGs). Although SHG members were a key target group for the project, their key identification assumption- that the self-selection into SHGs across project and non-project area is based on the same mechanism- is problematic. Datta, 2013 uses retrospective PSM to evaluate a similar project Bihar. The study finds that the project leads to positive impacts on some measures of economic welfare (reduced the incidence of food shortages and high cost debt), and indicators of women’s empowerment. The location of the GoBifo randomised evaluation-in post-conflict Sierra Leone- defines a very particular political and social context. In particular, a CDD programme in post-conflict regions faces fundamentally different political, economic and social challenges. These challenges can define very different mechanisms to potential impacts. The two livelihoods focussed evaluations on the other hand have methodological limitations. These limitations stem from the context in which these evaluations were designed. In particular, they were designed when there was a need for evidence on such programs, even though the on-going phase of implementation was largely complete. Despite their limitations therefore, these evaluations were important as they contributed to the limited evidence on multi-dimensional CDD projects. The need for evidence on such projects is still relevant, and it is particularly pressing in the case of India where the National 3 Rural Livelihoods Mission expects to invest six billion USD in this project approach, and reach 600,000 villages over the next decade (World Bank, 2011). Against this background, we use retrospective PSM methods to evaluate a livelihoods focussed CDD project in India- the Tamil Nadu Empowerment and Poverty Reduction Project or the Pudhu Vaazhvu Project (PVP). PVP is a multi-dimensional project that works with the poorest households in the poorest regions of the Indian state of Tamil Nadu. This project implements a set of interrelated interventions that target the various underlying causes of poverty. The core objectives of this project are to improve household well-being; empower women; and to make local governments more inclusive by investing in the social capital of communities. We evaluate the impact of this inter-related set of interventions on the measures of women’s empowerment, political participation and local civic action, and household well-being and indebtedness. In particular, we ask if the first phase of PVP- which was implemented over the period 2005-2011- was successful in improving broad-based measures of socio-economic welfare. In doing so, we contribute to the limited evidence on the core impacts of CDD projects that emphasise a multidimensional approach to improving welfare. We also provide first time evidence on the impact of a unique model of participatory development within the Indian context, wherein it attempts to improve accountability by working with local governments rather than in parallel to them. While acknowledging the limitations of cross sectional PSM in identifying the causal impact, the timing and context of this evaluation meant that this was the only feasible identification strategy to learn anything at all about this project. The primary aim of this evaluation, therefore, is to inform the project about its successes and failures in its five-year effort that covered one million households. Such impact evaluations are a vital part of learning system for complex participatory projects, which need a fundamentally different approach to development in that they need to continuously learn by doing through monitoring, tracking and 4 evaluation (Mansuri & Rao, 2012). This impact evaluation was designed as part of such a process of learning, and hence is a first step that will inform learning from evaluations in PVP. Besides the timing related constraint, designing impact evaluations for demand driven and multi-dimensional projects is challenging for other reasons. First, ex-ante these projects can affect multiple outcomes, which in turn can be measured in different ways, lending itself to the possibility of data mining. We address the plausible concerns on data mining by demonstrating transparency on our choice of outcome variables and their measures (see Casey, Glennerster, & Miguel, 2012 on these concerns and Section 2.2 for how we address them). Second, the core participatory intervention of creating networks of SHGs often has a long implementation history through different donor and state funding modalities making the task of identifying a valid counterfactual difficult. We address this problem by imitating the project roll-out strategy as closely while identifying our control group (see Section 3). Our results suggest that PVP had significant and positive impacts across the broad spectrum of outcomes that it targeted. Women’s empowerment and agency- both in the sphere of local public action, and intra-household decision making- were a key area of impact. Impacts on women’s public agency include a 25 per cent higher tendency in reporting issues of local service delivery and women’s public safety; a significant increase in propensity to approach the local government to solve these problems; and 64.5 per cent increase in their participation in Grama Sabha, which is the deliberative forum of village government. Women in PVP areas also report a significantly greater agency in key intra-household decisions that range from the purchase of durable assets to decisions on children’s education and their own occupational choices. We also find a positive and significant impact of PVP on its core credit and livelihoods related interventions. In particular, households in PVP project areas report a greater increase in assets over the recall baseline values, lower high cost debt (23.45%), and an increase in skilled employment. 5 The rest of this paper is organised as follows. Section 2 provides background on livelihoods focussed CDD projects- it describes PVP and its institutional context Section 2.1 and lays out the main hypothesis that we examine Section 2.2. Sections 3 and 4 present our estimation strategy and data. Section 5 presents our key results. Section 6 demonstrates the robustness of these results, and section 7 concludes. 2. Background Livelihoods focussed CDD projects are complex and multi-sectoral interventions that are frequently implemented in collaboration with different government line ministries and or local governments. The core intervention of such projects often involves facilitating participation in women’s groups that focus on credit and savings. Credit and savings are, however, the first of the several interventions that follow. Almost all projects also include a strong training component, which supports a wide array of productive activities that include productivity improving investments, private transfers and marketing support. Several projects also implement a set of agriculture, food security, and health and nutrition related interventions. To illustrate, let us consider a typical livelihoods project. This project first forms community based groups, often SHGs, in each village. Bringing these groups under a village organisation, which is usually federated from these groups, follows. With its core implementing body, village organisation, in place, the project then rolls out its core credit intervention. Once this roll out is complete, the project then offers the village organisation a set of three interventions: a producer group intervention that helps forge better market linkages for a range of producers spanning livelihoods that range from dairy to the production of local garments; an intervention that attempts to improve youth employment, and an agriculture intervention that targets productivity. 6 Under the demand driven design of this project, let us now consider how these various interventions are implemented. Village A, for instance, is located close to a peri-urban area, and has a large number of high school educated but unemployed youth, and women work in the garment industry. In contrast, Village B is more remote, with agriculture being the primary livelihood activity. Given these very different local contexts, the village organisation in these two villages will choose different types and intensities of the different interventions offered by the project. In particular, Village A is likely to focus more on the youth employment and producer interventions; while village B would benefit more from focusing on agriculture. Indeed, with multiple interventions that seek to affect a set of multidimensional outcomes, livelihoods projects are very diverse in how they are implemented and in what they implement. Because of their demand-driven design, they vary in important ways across different socio-economic contexts despite being implemented under a single overarching programme design. 2.1 PVP: The Institutional Context PVP was launched in 2005 in 2300 Village Panchayats (VPs) drawn from 70 blocks (a sub-district administrative unit that is made up of a cluster of VPs) in 16 selected districts of Tamil Nadu i. In 2012, the project expanded to 46 new blocks with additional financing. This evaluation covers the first phase of the project. Like other livelihoods projects, PVP’s core intervention involves providing credit and livelihoods support for women that belong to project- facilitated SHGs. Working in partnership with local governments (VPs), PVP then facilitates access of the rural poor to its various benefits, and attempts to improve local accountability. The districts that were to be covered by PVP were chosen using a combination of objective poverty related criteria, as well as other factors that were to reflect the state of development of the district. The total number of blocks that would be covered was defined by the 7 available budget, which was allocated during the design phase of the programs. Blocks within districts where the programme would be implemented were supposed to be chosen on the basis of a poverty ‘backwardness’ score, that would equally weight (i) the population of the historically disadvantaged social groups- the Scheduled Castes and Tribes (SCs and STs), and (ii) and the number of below poverty line households in the block. This block selection rule is henceforth referred to as the population criterion. All VPs within a project block were eligible to receive the programme, and the take-up of the programme was universal. Within each VP, a set of households identified through a participatory identification process formed the core target population for the project. Different programme interventions within PVP are however, targeted differently. While interventions that involve cash grants and credit are exclusively for the core target population, livelihoods focused interventions are primarily, though not exclusively, targeted to the poor. Village wide efforts to improve access to and accountability of the local state are extended to both target and non-target households. Although PVP currently reaches one million households, it was not the first attempt to promote SHGs in Tamil Nadu. The SHG movement in Tamil Nadu was first initiated in the early 1990’s, by the National Bank for Agriculture and Rural Development and other donor initiatives. These SHGs were then consolidated in 1997-1998 under the ‘Mahalir Thittam’ initiative of the state government of Tamil Nadu. Mahalir Thittam then, grew to cover around 200,000 SHGs and reach out to three million women. Despite the success in scaling up this initiative, there remained challenges of exclusion of the poor from these SHGs along with the ability of these SHGs to sustainably reduce debt, and to support diversification in livelihoods portfolios. In addition, there was an overarching question of whether these institutions could serve as a participatory forum that could support civic action in the absence of any linkages both among themselves and to local governments (World Bank, 2005). 8 PVP was designed to address these challenges. At the village level, it was envisaged that this would be done by making SHGs more inclusive, by shifting the focus of SHGs from group formation to livelihood creation, and by supporting the institutional development of SHGs through a village organisation that would link them to credit and other resources. This village organisation- the Village Poverty Reduction Committee (VPRC) - is the core institution through which PVP implements its various interventions. The membership of VPRC is drawn from the core target population. Each habitation (a sub unit of the village) chooses its VPRC representative, and a typical VPRC has 10 to 15 members. A Social Audit Committee (SAC), which comprises three to five people and is nominated by the village, monitors all activities of the VPRC. In an attempt to promote civic engagement of the VPRC, it was intended that they would work in close partnership with the elected village government (World Bank, 2005). The VP president plays an important role in this project intervention. Formally, the VP president submits a memorandum requesting PVP to implement its interventions in its panchayat, signs this memorandum, and agrees to serve as the ex-officio president of the VPRC. The VP then initiates project activities by facilitating a participatory process wherein the target poor are identified. As a constitutional authority, the VP ratifies both this list of the identified poor and the selection of VRPC and SAC members in the Grama Sabha. Project activities are then implemented through the VPRC. A three-tiered project structure- with staff at the levels of the state, block and cluster (of villages) - supports the implementation of this project. All activities implemented by this project are monitored by the SAC. In an attempt to further strengthen local accountability, monitoring reports (which are supplemented by an annual external audit) also have to be presented in and passed by the Grama Sabha. 9 Although the VPRC’s core mandate includes credit and livelihoods, it also places significant emphasis on several other activities. First and foremost among these activities is that of facilitating public action. In this role, it facilitates the access of the poor to available safety nets and social services ii; and implements a grant intervention for the target population. The VPRC also implements two other interventions: (i) it facilitates access to skilled employment by organising training and placement of the village youth with the private sector, and (ii) and it provides differently abled persons with need-based grants iii, and matches them with appropriate livelihoods activities and credit. Using data from our sample of VPRCs, we describe what this multiple intervention programme looks like in an average VP. Each VPRC received its total allocation (adjusted by population) of about 16,000 USD on average, in three instalments, over the sample period of six years. While almost all VPRCs in our sample implement at least one of the four main PVP interventions, there is variation in the intensity with which different programme components are implemented. 97.6 per cent of VPRCs implemented the core loans and targeted grants intervention; and 88 per cent also report having received training on this core mandate. There was larger variation in the implementation of two other components. About 85 per cent of VPRCs report implementing the disability intervention and over 40 per cent implement the skills intervention. 2.2 PVP’s Impact: Hypotheses and Measures of Outcomes Interventions that target a set of interrelated and multidimensional outcomes lend themselves to the possibility of data mining in order to cherry pick results. We demonstrate that our results are not cherry picked through a two-step procedure. First, we trace the broad categories of outcomes that we assess in this evaluation to an official World Bank document, the Project Appraisal Document (PAD) that was written before the project started in 2005. Second, 10 we show that both our hypotheses and outcome measures follow established norms in the current literature on these types of projects. The PAD is a summary of project related information that the board of the World Bank approves when clearing a project for assistance. A PAD includes the Project Development Objectives (PDOs), specifies indicators that will be used to measure these objectives and implementation arrangements, and details on how results and outcomes will be monitored and evaluated. While the outcomes that we use in this evaluation are the PDOs that are specified in the PAD, we cannot always use the PAD’s indicators for these outcomes. This is because indicators focus on outputs rather than their intended outcomes. The connection between outcomes and outputs in developing country contexts is, however, often weak and tenuous. Impacts, therefore, need to be assessed on the ultimate outcomes. Since the data that we use were collected explicitly for this evaluation, our surveys were designed to measure outcomes that the project sought to affect. To identify appropriate measures for these outcomes, we draw on findings and hypotheses in the literature that have been used to articulate theories of change for very similar interventions elsewhere. Hypotheses and outcome measures that we use are summarised below, and they are sorted by the two broad areas that PVP seeks to influence. i. Economic welfare: This set of indicators measure the direct impact of the credit and livelihoods component of the programme on the economic welfare as measure by household indebtedness, savings, incomes and livelihoods, consumption expenditures and asset portfolios. The hypothesis that access to group-based credit can have positive effects on these commonly used metrics of household welfare is both straightforward, and widely used (see for instance- Banerjee, Duflo, Glennester, & 11 Kinnan, 2013, Datta, 2013, Deininger & Liu, 2009, Khandker & Pitt, 1998). A summary of these hypotheses and measures, and the PDOs that they measure are in Table 1, Appendix B (see PDOs 3 and 4). ii. Women’s empowerment: We measure the impact of PVP’s women centred and group based approach on the empowerment of women both within the household and in public life. The hypotheses on the effects of SHG centred credit interventions on women’s collective action in the public sphere draw on Sanyal, 2009. Hypothesis on the individual empowerment effects are now fairly established in evaluations of women centred SHG interventions (see for instance- Cartwright, Khadker, & Pitt, 2006, Banerjee, Duflo, Glennester, & Kinnan, 2013, Khandker & Pitt, 1998 and Datta, 2013). The hypothesis that types of programs can empower women in the private and public spheres is also used in Blattman, Fiala, & Martinez, 2011. The measures used to examine these empowerment hypotheses follow established norms as summarized in Table 1, Appendix B (see PDOs 1 and 2). The impacts of CDD projects tend to vary across different socio-economic groups. This then suggests that there is a need to look at heterogeneous impacts, especially for social groups that are specifically targeted by the project (see for instance- Chen, Mu, & Ravallion, 2008 and Deininger & Liu, 2009). We therefore examine the heterogeneous effects of PVP on two key target groups for the project: Scheduled Caste (SC) and landless households. 12 3. Identification Strategy We use retrospective PSM methods in order to identify a counterfactual that allows us to estimate the impact of PVP. Ibanez & Rao, 2005 examine the social impact of a CDD project in Jamaica using a similar approach. Blocks within the chosen districts were selected into the PVP programme based on the population criterion- a backwardness score which equally weighted the SC and the ST population proportions and the number of below poverty line (BPL) households. If this rule had been followed, a regression discontinuity design could have been used to estimate the impact of PVP; and this was the original evaluation design plan. However, discussions with the project staff revealed that this score was not strictly followed in some districts. The key reason, according to them, that led to deviation from the population criterion was that it did not always identify the truly disadvantaged blocks within the district. Therefore, in some districts, the project took a policy decision to implement the programme in blocks that were not identified as the most backward by the population criterion, but were instead defined to be backward using other measures. These other measures included poor infrastructure, poor public services, and industrial backwardness. Our sampling strategy tries to replicate the final block selection as closely as possible. We do this by matching project and non-project blocks within a project district on the two factors that determined program’s choice of which blocks it would enter: (i) the population criterion, as measured by total population and SC/ST populations, and (ii) a set of block level infrastructural variables that would capture the reasons for deviation from the original rule. This additional set of variables includes all available census data that could measure broad based disadvantage- number of villages in the block, average distance of the village to the nearest town, total population; percentage of villages in the block which had primary and middle schools, commercial banks, cooperatives, agricultural and non-agricultural societies, medical facilities and drinking water facilities. 13 We adopt a two-step matching procedure. First, we generate propensity score matched blocks using a standard probit model that uses the variables listed above. We use census data from 2001. Within each district, a PVP block was matched to the non-PVP block with the closest propensity score. This ensured that the chosen non-PVP block was as likely to receive the intervention as its matched PVP block. Second, we match VPs within each matched pair of blocks on village level infrastructural data. The variables used for this VP level matching are the same as those used for the block matching. In each matched block pair, 12 to15 of closest matched pairs of VPs were chosen into the sample. Thus, the finally selected VPs from PVP and non-PVP blocks were ex-ante, equally likely to receive the programme. This two-step sampling strategy ensures pre-treatment similarity on observable covariates of treatment across treatment and control areas. Figure 1 summarises the two-step sample construction in a PVP district. Figure 1: Illustrative example of the identification strategy A PVP District Block A (PVP) Block B (PVP) Block C (Non-PVP) Block D (Non-PVP) (pscore-0.11) (pscore- 0.07) (pscore- 0.12) (pscore- 0.13) AVP 1 CVP 1 (pscore-0.56) (pscore-0.67) AVP 2 CVP 2 (pscore- 0.39) (pscore- 0.55) AVP 3 CVP 3 (pscore- 0.53) (pscore- 0.57) Consider Figure 1. In the sample PVP district shown here, Block A and Block C are the closest project - non-project matches, and are therefore chosen as the sample block pair in step 1. 14 Now within this pair of matched blocks, AVP 1 and CVP 3 and AVP 3 and CVP 2 are the closest matched pair of VPs. The final sample will, therefore include AVP 1, AVP 3, CVP 2 and CVP 3. Table 2 (see Appendix B) shows that by following this matching procedure, in 2001, the sample villages were indeed similar on all relevant observables that possibly determined selection into the programme. Indeed, this is the requirement of the conditional independence assumption of the PSM methodology. The assumption of conditional independence in PSM implies that programme outcomes must be independent of treatment status prior to treatment, given a vector of observable covariates X, or that (Y1, Y0) ⫫ D|X Here, Y1 and Y0 are the observed potential outcomes for treatment and control groups, respectively, and D takes value one if the comparison unit belongs to the treatment group, and takes the value zero if the comparison unit belongs to the control group. With conditional independence, the average difference between the matched comparison units from treatment and control groups will yield a consistent estimate of the Average Treatment Effect on the Treated (ATT). Since all households were ex-ante eligible to receive PVP’s multiple interventions, we consider a household to be treated if it is situated in PVP area. Admittedly, this is a weak definition of treatment which will underestimate the true impact of the project. As discussed, PVP puts special emphasis on poor and very poor households, identified through a participatory identification process. Since this exercise could not be replicated in the non-project villages, SC and ST households- which are overrepresented in the PVP’s target population- were oversampled in both project and non-project areas. The ATT estimates, hence calculated, are under the ‘intent to treat’ framework. 15 To estimate outcomes at the household level within treated and comparison VPs, we match households in PVP and non-PVP areas using a kernel matching technique. Following Rosenbaum & Rubin, 1983, we match households using a propensity score generated by a logit regression of treatment status on: (i) retrospective household level wealth indicators (access to electricity, sanitation, type of dwelling, water access, number of rooms and first component of principal component analysis on consumer asset ownership in 2006), (ii) time invariant household characteristics (caste), and (iii) and time deterministic household characteristics (total number of adults in the household, total number of female adults in the household, age of the household head and its square, education of the household head and education of the highest educated women) iv. This matching is done using survey data, collected as part of this evaluation. In addition, a set of village level variables from 2001 census are also included. Details can be seen in Table 3 (see Appendix B). Table 3 (see Appendix B) shows that almost all pre-programme village level variables do not significantly affect the probability of treatment. The one exception is the proportion of cultivated land that is irrigated, which is significantly lower in treatment areas. Insofar as the latter measures disadvantage in development potential, it supports the project’s contention that deviations from original population criterion in the selection of programme blocks intended to target more disadvantaged blocks, and therefore villages. This lends further support to our identification strategy. A key operational requirement for PSM is the existence of a region of common support, that is, for each value of a vector of observables X (or propensity score generated using X), there is a positive probability of finding a comparison unit in both treatment and control groups. That is, 0< P(D=1|X)<1 16 This implies that the probability of being treated, which in our case is the probability of being in a PVP village, lies between zero and one. Figure 2 (see Appendix A) shows that there is a good overlap in the propensity score distribution across project and non-project areas. To impose common support, we limit the comparison to a sub-sample of observations where propensity score is more than the minimum value in treatment group and is less than the maximum value in control group. For our data, the region of common support is given by (.074, .86). We only lose seven observations by imposing this common support. We use kernel matching to match households in treatment and control areas. This method compares outcomes for each household in PVP areas with a weighted average of the outcomes for all the households in non-PVP areas, with the highest weight placed on that household whose propensity score is closest to the household in PVP area. Thus, ATT is estimated as [� − � ]/ =1 =0 N is the total number of surveyed households in the PVP areas. Households in treatment areas are indexed i, and households in control areas are j. To estimate the weights, we used the Epanechnikov kernel function with bandwidth parameter of 0.06 units. As discussed earlier, both in its design and implementation, PVP places special emphasis on targeting the disadvantaged social groups. Therefore, a priori, we expect PVP to have an additional impact on these groups. Following Chen, Mu & Ravallion, 2008, we estimate heterogeneous impacts on the landless and the SC households using propensity-weighted ordinary least squares. We estimate the following regression: = + + + ∗ + + where δ is the parameter of interest; is the treatment status for the th household; and is the dummy indicating that the th household belongs to the vulnerable group (for example, 17 some caste group) for which we estimate heterogeneous impact. is a vector of pre-programme or time deterministic variables, and is the random error term. Observations in treatment areas are assigned a weight equal to unity and those in control � ( )) where � ( )/(1 − areas are weighted by ( � ( ) is the probability of treatment predicted using . This weighting technique gives efficient estimates of treatment effects (Hirano, Imbens, & Ridder, 2002). Standard errors are clustered at village level. 4. Data Our data come from a survey implemented by PVP, in collaboration with the World Bank during the period December 2012-March 2013. This survey covered ten districts, out of the 16 total districts where PVP had implemented its interventions over the period 2006-2010. The sample districts were chosen to ensure representation from different geographic regions of PVP’s operation v. Since this survey was designed and implemented after this evaluation was designed, our data was collected from households in matched project and non-project block pairs in these districts. Within each district, the survey covered the matched block pair, and matched VPs within these blocks. As mentioned earlier, 12 to16 VPs that had the closest match on propensity scores were sampled. The lower bound of this range was defined at the number of VPs at which our sample would in effect have picked a census of VPs within the block, that is, we saturate the treatment VPs within a block. In each VP, we sample two villages, at random. In the case of VPs with only one village, our sample covers that single village. In each village, a household questionnaire was administered to a sample of 12 households; and to the elected president of the VP. In order to measure the impact of the project, which targets the disadvantaged poor, the household sample was drawn using stratified random 18 sampling. Stratification was, therefore, used to oversample SC/ST households; and this was based on their population proportions within the village. With this oversampling, SC/ST households comprise a third of the sample. In all, we administered the household questionnaire to 3,692 households, drawn from 268 VPs. The household questionnaire had two components: (i) a general household module that included an LSMS vi type consumption module; and detailed information on the livelihoods portfolio and debt profile of the household, and (ii) a woman’s module that was administered to an adult married woman in the household, and measured different metrics of women’s empowerment. These measures included questions on decision-making within the household, and on women’s participation in local government and civic action. At the household level, we also collected retrospective data on assets and housing quality. Retrospective data on other outcomes, such as mobility, intra-household decision-making and public action- were not collected due to a higher likelihood of recall error on these measures. In addition to this household module, two other modules were administered. A village focus group discussion collected information on key infrastructure facilities in the village, and public good preferences. A VP president survey collected information on his/her political backgrounds and preferences. In PVP areas, we also collected data on the key activities of VPRC. We use data on 3,678 households, almost equally split between PVP and non-PVP areas in our final analysis. The caste composition of the sample is similar in PVP and non-PVP areas (see Table 4, Appendix B). Women headed households are 14.46 per cent of the sample. SHG membership is high across both project and non-project areas, reflecting the long history of SHG movement in the state. 51.69 per cent of the sample households in projects areas are members of SHGs, while this proportion is 44.41 per cent in non-project areas. 19 5. Results Our results indicate that PVP had significant and positive impacts across the multiple outcomes. Women’s empowerment and agency- both in the sphere of local public action and intra-household decision making- is a key area of impact. We also find that PVP has been successful in reducing the debt burden facing households, and in improving measures of economic welfare at the household level. In line with PVP’s focus on targeting the excluded poor, we find evidence that SC households in PVP areas have more diverse livelihoods portfolio, participate more in the local deliberative forums, and report greater change on some measures of women’s empowerment. We discuss the empowerment (intra-household and collective empowerment) and economic, effects of PVP, as well as heterogeneous impacts in greater detail below. 5.1 Impact on Economic Welfare The results suggest that there is a large and significant reduction in high cost debt, and in the number and size of loans taken out by households in PVP areas vii. The results also suggest a movement towards more productive loans, with more households reporting borrowing for non- farm livelihood purposes in PVP areas. We also find evidence of more general changes in economic welfare at the household level as measured by the livelihoods portfolios and asset holdings of households. Our results suggest a shift in livelihoods portfolios towards more skilled employment in PVP areas. Specifically, there is an increase in household members working in relatively more skilled jobs as their primary occupations (31.57% higher in PVP areas). At the same time, fewer household members report working in low security jobs as their primary livelihoods activity in PVP areasviii. These results are for rainy agricultural season. They persist for the non-rainy agricultural season. 20 For asset holdings, we use the first component of principal component analysis (PCA) on indicator variables for 29 consumer durables to construct an asset index. This index was 0.23 units higher in PVP areas. The change in this index, calculated using retrospective values from the year before PVP interventions started, is also significantly higher for PVP areas. However, there is no significant difference in the monthly per capita total expenditure between PVP and non-PVP areas, though the direction of change is positive. Finally, we also find evidence of higher spending on house construction and repairs within the last year in PVP areas. Households in PVP areas spent 25.35 per cent more on house construction and repair in the past one year. Since these expenses typically represent relatively large outflows of cash and or credit, they might also be indicative of higher economic welfare. However, there was no significant difference either in housing quality indicators or in change in housing quality over past five years ix. These results can be seen in Table 5 (see Appendix B). 5.2 Women’s Empowerment: Intra-household impacts We measure women’s empowerment within the household through a metric of female respondent’s ability to influence decisions within the household. To understand changes in women’s agency, women respondents were administered a series of vignette-based questions that relate to intra-household decision-making. On multiple dimensions of women’s agency within the household, we see that women in PVP areas report having a greater say in intra-household decisions. Women’s agency is defined as decisions taken by women, either by the respondent or by other female members in the household. In particular, we find that more women reported making decisions on purchase of household durables in PVP areas (9.96% higher) and making decisions on their children’s education (9.84% higher). More women in PVP areas also reported making their own decisions on the livelihoods activities they may choose to pursue (21.01% higher in PVP areas). Puzzlingly 21 though, fewer women reported making independent decisions on whom to vote for in national and state level elections in PVP areas (3.54% lower in PVP areas). 5.3 Women’s Empowerment: Collective Empowerment impacts We find large and significant effects of PVP on women’s participation in and interaction with local government. The percentage of women who attended last Grama Sabha was 19.38 per cent, which was 65.48 per cent higher than the proportion for non-PVP areas. To measure the likelihood of taking any action in response to civic problems, we described hypothetical problems that are typically faced in a village. We designed a set of vignettes that would capture responses to problems related to public service delivery x, village level infrastructure xi, family disputes xii and local law and order conditions xiii. For problems with the delivery of public services, a higher percentage of women in PVP areas said that they would approach the VP president, or the Grama Sabha as the first port of call. The likelihood of inaction on public service problems was also over 25 per cent lower in PVP areas. The response to extreme instances of domestic violence, and on law and order situations related to the public safety of women elicited similar findings both on the likelihood of public action. Notably, 16.87 per cent more respondents in PVP areas said that they would approach the police in case of a law and order situation. For more general problems with village infrastructure, the local state was again seen as the first point of contact. 4.88 per cent more respondents in PVP areas said that they would approach the VP president or raise the issue in Grama Sabha. PVP’s effects on political participation extended beyond its core target group of women. 30.62 per cent of households in PVP areas reported attending the last Grama Sabha meeting (31.43% higher). 22 5.4 Heterogeneous Effects Both in its design and implementation, PVP has placed special emphasis on targeting the historically disadvantaged, and the rural poor. In this section, we estimate the impact of PVP on the identifiably poor households in rural areas- SC households, and the landless. SC households are historically disadvantaged and typically dominate the rural poor category. Landless households are also similarly poor, and particularly vulnerable due to their limited and uncertain livelihoods. While we do not have retrospective data on land ownership, we use current landownership data as a proxy for the landless status of the household in 2005. This could be argued to be unbiased, given the well-known rigidity of rural land markets in South Asia xiv. We find some evidence of the targeting of PVP to these categories of the rural poor. In particular we find a larger reduction in high cost debt for SC households in PVP areas, alongside higher per capita consumption expenditures for these households. In addition, landless households in PVP areas reported an increase in the household members working in skilled jobs. Some of PVP’s impacts on public action also extend to these social groups. Women in SC and landless households exhibited a higher likelihood to approach the VP president or the Grama Sabha for problems related to public services. We also find an additional seven per cent increase in Grama Sabha attendance for SC households in PVP areas. However, we do not find additional impacts on measures of women’s empowerment within the household for these social groups. These results can be seen in Table 6 and Table 7 (see Appendix B). 23 6. Robustness Checks We use different matching specifications to check for robustness of our results. Most results persist across different specifications. When using nearest neighbour matching with five neighbours, all our results still hold (Table 8, Appendix B). When using radius matching with the radius set at 0.001 units however some impacts from the kernel matching estimates did not persist. First, the impact on the asset holdings did not persist. Second, some impacts on indicators of women’s empowerment also did not persist. In particular, there was no significant difference in women’s agency as measured by decisions on the purchase of household durables and education of children; and the result on women approaching the local government for instances of extreme domestic violence did not persist. Reports on intended action in response to the local public service delivery problems were also no different. Results on all other outcomes hold with this specification. It should be noted that 130 observations lie off-support with this matching technique, which reduces precision in estimating impacts. The directions of the effects that lose significance however remain the same, and the magnitude of these effects is also similar across specifications. These results are presented in Table 9 (see Appendix B). In summary, all our results hold across two commonly used matching methods. With the third method-that uses the strictest matching criteria -we lose some common support, and therefore a few results lose precision, though the direction and magnitude of the effects do not change. Measures of women’s empowerment within the household, and the first component of the asset index– which are the hardest changes to obtain- are the only results that do not maintain significance in the third matching method. While the results on women’s public action become weaker in this last specification, measures of this metric remain significant on multiple (but fewer) dimensions of public action. 24 7. Discussion Our results suggest that PVP has had an impact on a range of measures of economic welfare. In addition to a reduced debt burden, which is a key constraint faced by poor households in rural India, PVP also appears to have led to an increase in asset holdings of households. Importantly, our results also point to a shift in the livelihoods portfolios of households towards more skilled jobs in PVP’s target areas. Our results also suggest that some of these impacts extend to a key target group for PVP- that of the rural poor. In particular, relatively disadvantaged sections of the rural poor in the PVP areas have higher consumption expenditures, and demonstrate a shift towards more skilled livelihoods. We find strong and significant effects on women’s empowerment both within and outside the household. In particular, we find that women in PVP areas are more likely to report having a greater say in key intra-household decisions; and that they are more likely to report taking action on public problems. Women in PVP areas are also more likely to approach the local state- as manifest in the office of the VP president and the Grama Sabha- in order to seek a solution for these problems. In addition, we find strong evidence of increased participation in the deliberative forums of local government. While this increased participation also extends to other households members, the increase is particularly high for women. This result on women’s participation in Grama Sabhas is particularly interesting given previous empirical evidence from South India which raised concerns about the lack of representation of women and their issues in these meetings (Besley, Pande, & Rao, 2006). Our results on women’s participation in public action and participation, the relevance of the local state in addressing public problems, and the increased participation of non-female household members point to the potential of local accountability in successfully implementing community driven interventions. Whether this increased participation, or the greater likelihood of public action can actually influence the quality of deliberation and 25 public action, however, remains an open question. This question will be the focus of a forthcoming qualitative study that will use a counterfactual design to answer these questions. While we cannot claim the same standard of causal evidence as that of evaluations that use randomisation or a discontinuity design, programme selection on observables make a strong case for a robust PSM and therefore for these results. Moreover, this evaluation is only one component of a comprehensive learning system in PVP. In particular, this retrospective evaluation will be complemented by an ongoing qualitative examination of the mechanisms and pathways of success and failure. In addition, an evaluation of the next phase of the same project that uses a more rigorous regression discontinuity design is currently on going, along with five other impact evaluations that focus on various sub-interventions within PVP. Taken together, these evaluations and qualitative analyses will contribute to filling the gap on the welfare effects of these types of large-scale, multi-dimensional programs. 26 Appendix A Figure 2: Area of common support 27 Appendix B Table 1: Hypotheses and outcomes used to measure Project Development Objectives (PDOs) PDO 1: Developing, strengthening and synergizing pro-poor local institutions and groups (Including Village Panchayats) Hypothesis: Group-based credit interventions can promote women’s social capital and facilitate their collective empowerment for public action (Sanyal, 2009); also tested in (Blattman, Fiala, & Martinez, 2011). Outcomes Similar outcomes used in the literature Any member in the household attended • Political participation of women has previously a the last Grama Sabha been measured both by female Grama Sabha Women respondents voted in the last attendance (Besley, Pande, & Rao, 2006). b Grama Sabha elections • Participation in civic action has also been Women respondents attended the last measured through complaints lodged with the c Grama Sabha Grama Panchayat and or by raising issues in Responses to public action village Grama Sabha (Chattopadhyay & Duflo, 2004) d level issue vignettes- women respondents approaching the VP president or the Grama Sabha Responses to public action village e level issue vignettes- women respondents approaching village SHGs PDO 2: Enhancing skills and capacities of the poor (especially the women and the vulnerable) Hypotheses: (a) Group-based credit interventions empower women by improving their access to and control over resources; and therefore improve their capacity to influence intra household decisions (Cartwright, Khadkar & Pitt, 2006; Banerjee et al., 2013) (b) The observed impacts also hold for target social groups (hetereogenous impacts for SC and landless households Outcomes Similar outcomes used in the literature Responses to intra-household decision • Cartwright, Khadkar & Pitt, 2006 construct a making vignettes on decisions being an index of women’s empowerment using and taken by the women in the households its proxy indicators using a large set of qualitative responses to questions that Vignettes included questions on purchase characterise women’s autonomy and gender of durable assets; education of children; relations within the household. and decisions on own occupation. • Banerjee et al., 2013 measure the impact of a microfinance project on women’s empowerment , where empowerment is measured by their capacity to influence intra- household decisions 28 PDO 3: Financing productive demand driven sub-project investments related to the livelihoods of the target poor Hypotheses: Improving household access to cheaper credit can have several independent effects on household indebtedness: (a) It can reduce the need for high cost informal loans. This hypothesis is currently being used in an on-going RCT of a microfinance intervention xv. (b) It can result in debt-swapping where new credit is used to retire higher cost debt loans. This hypothesis is tested in Datta, 2013. (c ) Combining credit and livelihoods focussed interventions could lead to a greater demand for more productive loans (Datta, 2013) Outcomes Similar outcomes used in the literature • Reduced need for high cost informal loans a can be measured by a decline in the number Proportion of number of high cost loans in total loan portfolio of forma loans, and by a decline in the proportion of informal loans in the loan b portfolio of the household Proportion of amount of high cost loans • Debt swapping is measured by the stated purpose of the loan being taken is reported as c for the purpose of repaying an old, high cost Proportion of number of loans taken for loan (Datta 2013) non-farm livelihood purposes • Productive loans are measured by the stated purpose of the loan being for an d Proportion of amount of loans taken for investment/livelihoods related purpose. non-farm livelihood purposes e Proportion of number of loans taken for repaying old loans Proportion of amount of loans taken for f repaying old loans PDO 4: Improve livelihoods and incomes of the target poor Hypothesis: (a) Access to group-based credit can have positive effects on these commonly used metrics household welfare. This hypothesis on household welfare is both straightforward, and widely used (Banerjee et al 2013, Datta, 2013, Deininger and Liu 2009, Khandker and Pitt 1998). (b) Khandkar and Pitt, 1998 estimate the impact of microfinance on livelihoods patterns. Outcomes Similar outcomes used in the literature a First component of asset dummies • Standard measure of household welfare used principal component analysis household consumption expenditures, and b Consumption expenditure xvi asset indices. Consumption expenditures are c Housing expenditures also the most reliable way of measuring Proportion of pucca households economic welfare, since income data is Proportion of households who spent any typically unreliable in rural developing 29 amount on household repairs country contexts Average amount spent on repairs • In addition to these, we also examine house d Livelihoods construction measure as these can be an Proportion of skilled labourers within a important expenditure for households. household • Livelihoods patterns are measured by Proportion of individuals in business reported occupations (animal husbandry, petty shop, handloom, fishing, any other business) 30 Table 2: Village level balance in the surveyed sample -1.095 Approach Road- paved road (-0.139) 0.0352 Approach Road- mud road (-0.934) 0.08 Approach Road- footpath (-0.844) 0.00266 Distance from the nearest town (-0.787) -0.0472 Number of agricultural credit societies (-0.887) -0.0447 Number of non-agricultural credit societies (-0.878) -0.0211 Bus services (-0.958) -0.0419 Primary school (-0.689) -0.229 Middle school (-0.283) -0.223 Secondary school (-0.529) -0.311 Senior secondary school (-0.542) 0.437 Proportion of SC population in the village (-0.496) 2.093 Proportion of ST population in the village (-0.81) 0.00048 Total number of households in the village (-0.275) 0.932 Constant (-0.267) p-values in parenthesis * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% Source: Authors calculations, based on Census (2001) data 31 Table 3: Logit results- generating propensity score -0.14 ST (-0.826) 0.02 Marginally Backward Caste (MBC) (-0.878) 0.25*** Backward Caste (BC) (-0.007) 0.7 Forward Caste(FC) (-0.128) -0.16 Total number of female adults in the household (-0.2) 0.02 Age of the household head (-0.285) 0 Education of the household head (-0.896) 0.02* Education of the highest educated woman (-0.072) Total number of adults in the household 0.13* (-0.08) 0 Age of the household head^2 (-0.124) -0.24* Private water access (-0.07) -0.85*** Electricity (0) -0.2 Semi-pucca house (-0.225) -0.11 Kuccha house (-0.359) -0.45 Public sanitation (-0.244) 0.15 Private sanitation (-0.221) 0.03 Number of rooms in the household (-0.331) -0.03 PC1 (-0.219) 0.84 Proportion of SC population in the village (-0.199) 6.85 Proportion of ST population in the village (-0.442) 32 0 Total number of households in the village (-0.227) -1.45*** Proportion of cultivated land that is irrigated (-0.001) -0.93 Approach Road- paved road (-0.253) -0.25 Approach Road- mud road (-0.57) 0.26 Approach Road- footpath (-0.556) 0 Distance from the nearest town (-0.793) -0.01 Number of agricultural credit societies (-0.987) -0.42 Number of non-agricultural credit societies (-0.181) 0 Bus services (-0.991) -0.06 Primary school (-0.546) -0.25 Middle school (-0.33) -0.17 Secondary school (-0.665) -0.34 Senior secondary school (-0.508) 1.76 Constant (-0.103) p-values reported in parenthesis. Most values have been rounded off to nearest two decimals. Only household level covariates reported. * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% Source: Authors’ calculations from impact evaluation survey Table 4: Caste distribution in the sample- percentage of households Non-project Project Sample SC 34.25 33.02 33.63 ST 1.04 0.93 0.98 MBC 34.36 29.48 31.91 BC 29.15 34.93 32.05 FC/OC 1.21 1.63 1.42 Source: Authors’ calculations from impact evaluation survey 33 Table 5: Results- Kernel Kernel matching Treatment Control Percentage Standard increase/decrease error in project areas when compared to control areas Debt Proportion of number of high cost loans 0.0766 0.1082 (-)29.26*** 0.0111 Proportion of amount of high cost loans 0.0657 0.0859 (-) 23.45** 0.0099 Proportion of number of loans taken for non-farm 0.0645 0.0421 53.37*** 0.0074 livelihood purposes Proportion of amount of loans taken for non-farm 0.0656 0.0467 40.39** 0.008 livelihood purposes Proportion of number of loans taken for repaying old 0.0343 0.0299 14.46 0.0054 loans Proportion of amount of loans taken for repaying old 0.0336 0.0292 14.94 0.0057 loans Assets First component of asset dummies principal component 0.1099 -0.1153 (-)195.33** 0.0905 analysis Housing Quality Proportion of pucca households 0.7353 0.7253 1.37 0.0172 Proportion of households who spent any amount on 0.1919 0.1531 25.35*** 0.0145 household repairs Average amount spent on repairs 11,586.61 10,188.01 13.73 3,180.75 Livelihoods Rainy season Proportion of skilled labourers within a household 0.0866 0.0658 31.57*** 0.0077 Proportion of individuals in business (animal husbandry, 0.0739 0.0776 (-) 4.82 0.0082 petty shop, handloom, fishing, any other business) Proportion of people in low security (MGNREGA, casual 0.4743 0.523 (-) 9.31*** 0.0158 unskilled and agricultural labour) for primary income generation Political Participation Any member in the household attended last Grama Sabha 0.3062 0.233 31.43*** 0.0168 Woman respondent voted in the last Grama Sabha 0.9387 0.9329 0.62 0.0101 elections Woman respondent attended last Grama Sabha 0.1938 0.1171 65.48*** 0.0141 Intra-household decision making Proportion of women reporting that females in the 0.5281 0.481 9.79** 0.0198 household take decisions regarding purchase of household durables Proportion of women reporting that females in the 0.4835 0.4366 10.74** 0.0198 household take decisions regarding education of 34 Kernel matching Treatment Control Percentage Standard increase/decrease error in project areas when compared to control areas respondent’s children Proportion of women reporting that females in the 0.4557 0.3766 21.01*** 0.0196 household take decisions regarding respondent’s livelihood activity Proportion of women taking their own voting decision 0.7728 0.8012 (-)3.54* 0.0163 Village level issues Ration shop does not open regularly. People in the village often have to buy food grains from market Approached Grama Sabha or village president 0.6855 0.6407 7.00** 0.0189 Approached SHG 0.0153 0.0091 69.22 0.0047 No action taken 0.0521 0.0702 (-)25.77* 0.0093 A woman is beating her daughter in law in the village Approached Grama Sabha or village president 0.2437 0.1994 22.25*** 0.0164 Approached SHG 0.0318 0.0276 15.3 0.0069 Approached police 0.1947 0.1802 8.07 0.0151 No action taken 0.1157 0.1564 (-)26.02*** 0.0135 A man is your village drinks and creates a ruckus in the village Approached Grama Sabha or village president 0.2341 0.1931 21.21** 0.0162 Approached SHG 0.019 0.0254 (-)25.13 0.0061 Approached police 0.3278 0.2805 16.87*** 0.018 No action taken 0.1158 0.1563 (-)25.91*** 0.0135 There are insufficient public water sources in the village, making water availability difficult Approached Grama Sabha or village president 0.7781 0.7419 4.88** 0.0156 Approached SHG 0.0146 0.0072 101.92* 0.0039 No action taken 0.0205 0.0221 -7.26 0.0052 Expenditure Share of food expenditure .6132 .6266 (-)2.14** .0059 Log per capita consumption expenditure 10.2212 10.2057 0.15 .0213 Per capita daily Kcal intake 2506.8871 2297.097 9.13** 93.7343 * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% 35 Table 6: Heterogeneous impact on SC households Treatment SC SC*T status Debt -0.02 0.05*** -0.05* Proportion of number of high cost loans (-0.17) (0) (-0.05) Proportion of loans taken for non-farm livelihood 0.03*** 0.01 -0.02 purposes (0) (-0.22) (-0.13) Assets 0.16*** -0.40*** 0.14 PC1 (-0.04) (0) (-0.26) Political 0.05*** 0.05* 0.07*** Participation Any household member attended last Grama Sabha (-0.05) (-0.06) (-0.05) Livelihoods- 0.02** -0.04*** 0 rainy season Proportion of skilled labourers in the household (-0.05) (0) (-0.82) Proportion of individuals in the household involved 0.07** 0.06 -0.02 in NREGA work as their secondary income (-0.02) (-0.14) (-0.69) generating activity Proportion of individuals in the household involved -0.05*** 0.12*** 0 in low paying work as their primary activity (-0.01) (0) (-0.86) Intra- Proportion of women reporting that females in the 0.01 -0.07* 0.11** household household take decision on purchase of durables (-0.8) (-0.09) (-0.01) decision Proportion of women reporting that females in the 0.04 -0.07* 0.12** making household take decision on respondent's livelihood (-0.18) (-0.07) (-0.01) activity Public Action 0.03 -0.02 0.09*** Ration shop- Raised the issue in Grama Sabha (-0.42) (-0.49) (-0.04) 0.02 -0.02 0.07*** Domestic violence- Raised the issue in Grama Sabha (-0.3) (-0.34) (-0.03) Expenditure Share of food expenditure excluding top 1 percentile -0.01 0 0 annual expenditure observations (-0.33) (-0.76) (-0.98) Log per capita consumption expenditure excluding 0.02 -0.11*** 0.09*** top 1 percentile annual expenditure observations (-0.35) (0) (-0.02) Other variables controlled for in the regression- total number of adults, total number of adult females, age of the household head, age of the household head squared, education of the household head, education of the highest educated woman, caste, land ownership, first component of the principal component analysis of assets, electricity, private water access, type of sanitation facility, type of dwelling and number of rooms. p-values reported in parenthesis. Most values have been rounded off to nearest two decimals. Only household level covariates reported. * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% 36 Table 7: Heterogeneous impacts on landless households Treatment Landless Landless*T status households Debt -0.01 0.04** -0.03 Proportion of number of high cost loans (-0.32) (-0.03) (-0.19) Assets 0.21*** -0.11 -0.02 PC1 (-0.03) (-0.21) (-0.88) Political 0.05* -0.07*** 0.03 Participation Any household member attended last Grama Sabha (-0.09) (-0.02) (-0.41) Livelihoods- 0 0.05*** 0.04*** rainy season Proportion of skilled labourers in the household (-0.84) 0 (-0.01) Expenditure Log per capita consumption expenditure excluding 0.02 -0.13*** 0.06 top 1 percentile annual expenditure observations (-0.49) 0 (-0.17) Other variables controlled for in the regressions- total number of adults, total number of adult females, age of the household head, age of the household head squared, education of the household head, education of the highest educated woman, caste, land ownership, first component of the principal component analysis of assets, electricity, private water access, type of sanitation facility, type of dwelling and number of rooms. p-values reported in parenthesis. Most values have been rounded off to nearest two decimals. Only household level covariates reported. * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% 37 Table 8: Nearest neighbour matching (five neighbours) Treatment Control Percentage Standard increase/decrease error in project areas when compared to control areas Debt Proportion of number of high cost loans 0.0766 0.1111 (-)31.08*** 0.0123 Proportion of amount of high cost loans 0.0657 0.0844 (-)22.05* 0.0109 Proportion of number of loans taken for non-farm 0.0645 0.0438 47.29*** 0.008 livelihood purposes Proportion of amount of loans taken for non-farm 0.0656 0.0499 31.48* 0.0086 livelihood purposes Assets First component of asset dummies principal component 0.1099 -0.0744 247.71* 0.1002 analysis Housing Quality Proportion of households who spent any amount on 0.1919 0.1507 27.39*** 0.0157 household repairs Livelihoods- Rainy season Proportion of skilled labourers within a household 0.0866 0.0667 29.91** 0.0083 Proportion of people in low security (MGNREGA, casual 0.4743 0.5115 (-)7.27** 0.0172 unskilled and agricultural labour) for primary income generation Political Participation Any member in the household attended last Grama Sabha 0.3062 0.2329 31.5*** 0.0182 Woman respondent attended last Grama Sabha 0.1938 0.1253 54.68*** 0.015 Intra-household decision making Proportion of women reporting that females in the 0.5281 0.4703 12.3*** 0.0215 household take decisions regarding purchase of household durables Proportion of women reporting that females in the 0.4896 0.4441 10.26** 0.0215 household take decisions regarding education of respondent’s children Proportion of women reporting that females in the 0.4692 0.3902 20.23*** 0.0213 household take decisions regarding respondent’s livelihood activity Proportion of women taking their own voting decision 0.7544 0.8028 (-)6.026*** 0.0178 Village level issues Ration shop does not open regularly. People in the village often have to buy food grains from market. Approached Grama Sabha or village president 0.6855 0.6477 5.83* 0.0205 A woman is beating her daughter in law in the village Approached Grama Sabha or village president 0.2437 0.2091 16.58** 0.0177 A man is your village drinks and creates a ruckus in the village Approached Grama Sabha or village president 0.2341 0.2023 15.69* 0.0175 38 Treatment Control Percentage Standard increase/decrease error in project areas when compared to control areas Approached police 0.3278 0.2821 16.2** 0.0195 No action taken 0.1158 0.1532 (-)24.4*** 0.0147 There are insufficient public water sources in the village, making water availability difficult Approached Grama Sabha or village president 0.7847 0.7488 4.78* 0.0186 Expenditure Share of food expenditure 0.6132 0.6284 (-)2.42** 0.0064 Per capita daily Kcal intake 2506.8871 2276.7633 10.11** 101.473 * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% 39 Table 9: Radius matching (radius = 0.001 units) Treatment Control Percentage Standard increase/decrease error in project areas when compared to control areas Debt Proportion of number of high cost loans 0.0794 0.111 (-)28.5** 0.0124 Proportion of amount of high cost loans 0.0681 0.0864 (-)21.16* 0.0111 Proportion of number of loans taken for non-farm 0.0663 0.0443 49.52*** 0.008 livelihood purposes Proportion of amount of loans taken for non-farm 0.0671 0.0504 33.14* 0.0087 livelihood purposes Housing Quality- Proportion of households who spent 0.199 0.1541 29.11*** 0.0159 any amount on household repairs Livelihoods- Rainy season Proportion of skilled labourers within a household 0.0862 0.063 36.91*** 0.0083 Proportion of people in low security (MGNREGA, casual 0.4764 0.5197 (-)8.34** 0.017367 unskilled and agricultural labour) for primary income generation Political Participation Any member in the household attended last Grama Sabha 0.3006 0.2246 33.81*** 0.0183 Woman respondent attended last Grama Sabha 0.1904 0.1186 60.48*** 0.0153 Intra-household decision making Proportion of women reporting that females in the 0.4723 0.4144 13.97*** 0.0218 household take decisions regarding respondent’s livelihood activity Proportion of women reporting that they take their own 0.7478 0.8048 (-)7.08*** 0.0183 voting decisions Village level issues Ration shop does not open regularly. People in the village often have to buy food grains from market Approached Grama Sabha or village president 0.6923 0.6443 7.45** 0.0208 A woman is beating her daughter in law in the village No action taken 0.1119 0.1478 (-)24.29** 0.0151 A man is your village drinks and creates a ruckus in the village Approached Grama Sabha or village president 0.2313 0.2015 14.79* 0.0179 Approached police 0.328 0.2795 17.34** 0.0198 No action taken 0.1166 0.1503 (-)22.39** 0.0151 There are insufficient public water sources in the village, making water availability difficult Approached Grama Sabha or village president 0.7896 0.7522 4.98** 0.0189 Expenditure Share of food expenditure .6121 .6289 (-) 2.67** 0.0066 Per capita daily Kcal intake 2498.341 2303.6375 8.45* 101.6953 * indicates significant at 10% ** indicates significant at 5% *** indicates significant at 1% 40 Endnotes i Coimbatore, Cuddalore, Kancheepuram, Nagapattinam, Namakkal, Ramanathapuram, Salem, Theni, Thiruvannamalai, Thiruvalur, Thiruvarur, Thoothukudi, Tirrupur, Tirunelveli, Vellore and Villupuram. ii Examples of these safety nets and services include India’s National Rural Employment Guarantee scheme, old age and widow’s pensions, and housing schemes that are implemented by both the state and central governments. iii For their specific disability related needs such as hearing aids, crutches, and so on. iv Rosenbaum and Rubin, 1983 show that instead of matching on the covariate vector X if households are matched using a linear projection of X, outcome is still independent of the treatment status. The linear projection we use is the propensity score generated using logit regression. v Kancheepuram, Thiruvallur, Thiruvanamalai and Villupuram from north; Namakkal and Tiruppur from west; Thoothukudi and Tirunelveli from south; and Cuddalore and Nagapattinam from the coastal region. vi Living Standard Measurement Survey vii High cost debt is defined by loans with an annual interest rate of more than 50 per cent. viii Low security jobs include the Mahatma Gandhi National Rural Employment Guarantee Act (MGNREGA), agricultural labour and unskilled casual labour as primary income generation activities. ix Our measure of housing quality focused on the materials used for the roof, wall and floors of the house, and the addition of rooms to the house. Expenditure on repairs and could include the latter, as well as more minor repairs. x ‘Ration shop does not open regularly. People in the village often have to buy food grains from market’ xi ‘There are insufficient public water sources in your village, making public water availability difficult’ xii ‘A woman in the village is beating her daughter-in-law’ xiii ‘A man drinks and creates a ruckus in your village’ xiv The low market turnover of land in South Asia has been well documented in literature (see for instance- Rosenzweig & Wolpin, 1985, Rosenzweig, 1980 and Biswanger & Rosenzweig, 1986). xv http://www.ifmrlead.org/cmf/wp-content/uploads/2014/03/KGFS-Policy-Brief-on-Informal-Lending- Publication.pdf xvi It is difficult to get reliable estimates of income in developing countries because a large proportion of labour force is engaged in self-employment in small business and agriculture. In order to deal with this problem, the standard best practice is to use consumption expenditure of a household as a proxy for its income. Measuring economic welfare of households therefore requires consumption expenditure data. Consumption is being used as a proxy for income in all the leading surveys, like the National Sample Survey (NSS) and LSMS. 41 References Arcand, J.-L., & Bassole, L. (2007). Does Community Driven Development Work? Evidence from Senegal. CERDI-CNRS , Université d’Auvergne, France. Banerjee, A., Duflo, E., Glennester, R., & Kinnan, C. (2013, April). The miracle of microfinance? Evidence from a Randomized Evaluation. MIT, Department of Economics, Working Paper eries, Working Paper13-09. Cambridge, Massachusetts, United States. Besley, T., Pande, R., & Rao, V. (2006). Participatory Democracy in Action: Survey Evidence from South Asia. Journal of the European Economic Association, 3(2-3), 648-657. Biswanger, H. P., & Rosenzweig, M. R. (1986). Behavioral and Material Determinants of Production Relations in Agriculture. The Journal of Development Studies, 22(3), 503- 539. Blattman, C., Fiala, N., & Martinez, S. (2011, September). Employment Generation in Rural Africa: Mid-term Results from an Experimental Evaluation of the Youth Opportunities Program in Northern Uganda. SP Discussion Paper Number- 1120. Social Protection and Labour, World Bank. Cartwright, J., Khandekar, S. R., & Pitt, M. M. (2006). Empowering Women with Micro Finance: Evidence from Bangladesh. Economic Development and Cultural Change, 54(4), 791-831. Casey, K., Glennerster, R., & Miguel, E. (2012). Reshaping Institutions: Evidence on Aid Impacts Using a Preanalysis Plan. Quarterly Journal of Economics, 1755-1812. Chattopadhyay, R., & Duflo, E. (2004). Women as Policy Makers: Evidence from a Randomized Policy Expriment in India. Econometrica, 72(5), 1409–1443. Chen, S., Mu, R., & Ravallion, M. (2008). Are There Lasting Impacts of Aid to Poor Areas? Evidence for Rural China. World Bank Policy Research Working Paper 4084. Datta, U. (2013). Economics and Empowerment: Catalysts or Competitors? Unpublished doctoral dissertation. Deininger, K., & Liu, Y. (2009, March). Economic and Social Impacts of Self Help Groups in India. Policy Research Working Paper 4884. Gine, X., & Mansuri, G. (2014, June). Money or Ideas? A Field Experiment on Constraints to Entrepreneurship in Rural Pakistan. World Bank Policy Research Working Paper (6959). Hirano, K., Imbens, G. W., & Ridder, G. (2002). Efficient Estimation of Average Treatment Effects using the Estimated Propensity Score. Econometrica, 71(4), 1161-1189. 42 Ibanez, A., & Rao, V. (2005, December). The Social Impact of Social Funds in Jamaica: A “Participatory Econometric” Analysis of Targeting, Collective Action, and Participation in Community-Driven Development. Q-Squared Working Paper No. 16. Jalan, J., & Ravalliom, M. (2003). Does piped water reduce diarrhea for children in rural India? Journal of Econometrics 112, 112(1), 153-173. Khandekar, S. R., & Pitt, M. M. (1998). The Impact of Group Based Credit Programs on Poor Households in Bangladesh: Does the Gender of Participants Matter? Journal of Political Economy, 6(5), 958-996. Mansuri, G., & Rao, V. (2012). Localizaing Development: Does Participation Work? Washington DC: World Bank. Park, A., & Wang, S. (2010, April). Community-based Development and Poverty Alleviation: An Evaluation of China’s Poor Village Investment Program. Journal of Public Economics, 94(9), 790-799. Rosenzweig, M. R. (1980). Neoclassical Theory and the Optimizing Peasant: An Econometric Analysis of Market Family Labor Supply in a Developing Country. The Quarterly Journal of Economics, 31-55. Rosenzweig, M. R., & Wolpin, K. I. (1985). Specific Experience, Household Structure, and Intergenerational Transfers: Farm Family Land and Labor Arrangements in Developing Countries. The Quarterly Journal or Economics, 961-987. Sanyal, P. (2009). From Credit to Collective Action: The Role of Microfinance in Promoting Women’s Social Capital and Normative Influence. American Sociological Review, 74(4), 529–550. Voss, J. (2008). Impact Evaluation of the Second Phase of the Kecamatan Development Program in Indonesia. Jakarta: World Bank. World Bank. (2005). Project Appraisal Document: Tamil Nadu Empowerment and Poverty Reduction "Pudhu Vaazvhu". World Bank. (2011). Project Appraisal Document for the National Rural Livelihoods Project . 43