77390 the world bank economic review, vol. 16, no. 2 219–240 Impact Evaluation of Social Funds Supporting Communities in Transition: The Impact of the Armenian Social Investment Fund Robert S. Chase The Armenian Social Investment Fund supports communities’ efforts to improve local infrastructure during Armenia’s economic transition away from central planning, fi- nancing community-designed and -implemented projects to rehabilitate primary schools, water systems, and other infrastructure. This article considers the targeting, household impact, and community effects of the social fund’s activities. It relies on a nationally representative household survey, oversampled in areas where the social fund was active. Using propensity and pipeline matching techniques to control for community self-selection into the social fund, it evaluates the household effects of rehabilitating schools and water systems. The results show that the social fund reached poor households, particularly in rural areas. Education projects increased households’ spending on education significantly and had mild effects on school attendance. Potable water projects increased household access to water and had mild positive effects on health. Communities that completed a social fund project were less likely than the comparison group to complete other local infrastructure projects, suggesting that social capital was expended in these early projects. By contrast, communities that joined the social fund later and had not yet completed their projects took more initiatives not supported by the social fund. In centrally planned economies, national governments exerted tremendous eco- nomic control. This control extended to investment in local infrastructure, in- cluding building and maintaining roads, schools, and water systems. When these economies collapsed, governments became bereft of resources. Systems for main- taining local infrastructure began to fail; as deep economic recession took hold, schools and water systems fell into disrepair. Local public services deteriorated, compounding other hardships for people living in postcommunist conditions. But because communities were accustomed to relying on central authorities to meet local needs, they often were unable to address their problems. Robert S. Chase is an assistant professor of International Economics at the School of Advanced Inter- national Studies of Johns Hopkins University; Senior Economist and Social Capital Coordinator in the World Bank’s Social Development Family; and Research Associate of the William Davidson Institute. His e-mail address is rchase@worldbank.org. The research underlying this article progressed thanks to sup- port from the Armenian Social Investment Fund, the State Department of Statistics of Armenia, the Social Funds 2000 study, Caroline Mascarell, and Lynne Sherburne-Benz. The author benefited from extensive assistance in data collection and analysis from Ghislaine Delaine, Julia Grutzner, Kalpana Mehra, Mirvat Sewadeh, and Sylvia Zucchini, and from helpful comments from Julia Magluchants and Amalia Poghossian. © 2002 The International Bank for Reconstruction and Development / THE WORLD BANK 219 220 the world bank economic review, vol. 16, no. 2 The World Bank supported social funds in postcommunist transition economies as part of a strategy to improve this situation. These flexible financing instruments provide resources for community initiatives to improve local infrastructure. In the former Eastern Bloc countries social funds sought to provide temporary employ- ment, alleviate local public service hardships, and mobilize communities to ad- dress local needs. This article investigates the household and community impact of one such project, the Armenian Social Investment Fund (asif). It seeks to ascertain whether asif resources reached poorer households, whether the social fund altered the behavior or welfare of those in asif communities, and how the fund related to communities’ ability to act collectively. There have been many investigations of social funds since their introduction in Latin America at the end of the 1980s (see, for example, Goodman and others 1997, Khadiagala 1995, and Marc and others 1993). These have adopted many different evaluative approaches, including analyses of fund disbursement patterns, institutional studies of operating procedures, and sociological studies of benefi- ciaries’ attitudes. But by the mid-1990s household data and quantitative tech- niques had been used only in evaluating the Bolivian social fund. To provide deeper, more diverse evidence on the impact of social funds, in 1997 the World Bank initiated a multicountry analysis, Social Funds 2000. This article comes out of that research program. Social Funds 2000 used household survey data to isolate statistically significant social fund effects in Armenia, Bolivia (Newman and others 2002), Honduras (Walker and others 1999), Nicaragua (Pradhan and Rawlings 2002), Peru (Paxson and Schady 2002), and Zambia (Chase and Sherburne-Benz 2001). This article on Armenia is the first quantitative impact evaluation to consider a social fund in a postcommunist economy. I. The asif Though social funds share common characteristics, each is designed to fit the objectives and institutions of the country operating it. The asif began as a pilot project, funded by the U.S. Agency for International Development, to provide employment, support community initiatives, and enhance civil society during the postcommunist transition. In January 1996 the first World Bank loan for asif became effective, providing US$12 million in concessional financing for a $20 million project. Although some of this funding helped develop institutions nec- essary to administer the social fund and monitor household welfare, most went to support projects that communities designed and implemented. Between 1996 and December 2000, when the first World Bank loan for asif closed, the social fund received proposals for 726 projects, of which 334 were approved and 259 completed. Over the course of the loan the average project size was $50,000. As a direct result of World Bank financing, 178 contractors implemented projects, providing jobs to 5,000 people. The infrastructure improve- ments reached an estimated 640,000 beneficiaries. Chase 221 Like other social funds, asif offered a menu of projects from which commu- nities chose. It specified types of projects that would meet the country’s pressing needs in local infrastructure and would likely be interesting only to communities in difficult circumstances, thus self-targeting asif resources to the poor. Of the 259 completed projects in 1996–2000, 35 percent were small-scale school reha- bilitations, 32 percent potable water projects, 11 percent minor irrigation works, and 5 percent rehabilitations of health facilities. The other 17 percent included initiatives for community centers, pension homes, roads and landscaping, and sewage and waste management. The focus here is on primary school rehabilita- tions and water projects. These two groups not only are the largest but also con- sist of projects that are relatively homogeneous, allowing easier comparisons within each group. Though communities stepped forward to participate in asif, the social fund administration also targeted resources to areas of the country with the most press- ing need for small-scale infrastructure improvements. From 1996 to 2000, 38 percent of projects were in marzes (regions) where the 1988 earthquake caused the most devastation—Aragosotn, Lori, and Shirak. Marzes that suffered most from the Karabakh conflict—Sunik and Tavush—implemented 21 percent of the projects. Yerevan received 25 percent. Fifteen percent was spread among the remaining five marzes. As will be discussed, though this regional distribution of projects focused resources on marzes in the most difficult circumstances, it cre- ated technical challenges for evaluation. Notably, the targeting of resources to specific areas makes it difficult to identify control communities that did not par- ticipate in the asif project but otherwise had characteristics similar to those of participating communities. II. Data and Methodology To analyze changes in household behavior and outcomes, the article relies prima- rily on an integrated household survey. This comprehensive, nationally repre- sentative data source allows in-depth analysis of the welfare of the Armenian population. Among other topics, the core survey instrument includes information on household composition, income, expenditures, education, and health. Conducted from July 1, 1998, to June 30, 1999, the survey includes roughly 3,600 households in its basic sample. Enumerators visited 20 households per sample cluster. To allow impact evaluation, asif staff and the State Directorate of Statistics added a module to the survey instrument that posed questions about asif activi- ties and community organizations and initiatives. It asked households to report changes to community infrastructure that had taken place in the previous five years. It also asked whether they had taken part in the effort to repair or upgrade the infrastructure and what their attitudes were toward the resulting infrastructure. To ensure adequate coverage of asif treatment areas, the survey oversampled households in areas where the social fund was active. Beyond the base sample 222 the world bank economic review, vol. 16, no. 2 representative of the Armenian population, survey enumerators visited an addi- tional 2,260 households in 113 clusters where the social fund was active. Within this group of oversampled communities, the survey collected data on two groups of households: those where projects had been completed and those where asif had approved a project but the project had not yet been completed. As the evaluation literature has long emphasized, it is difficult to isolate the effects of an intervention, particularly when potential participants chose to in- volve themselves in the intervention (for an overview of key evaluation issues, see Moffitt 1991). Fundamentally, impact evaluation compares outcome indica- tors for a group that completed a project—the treatment group—with those for a comparison group. If the comparison group is correctly identified, the differ- ence between the treatment and comparison groups isolates the effect of the in- tervention. But in many cases, including that of social funds, identifying appro- priate comparison groups can be difficult. Because of the way social funds operate, communities wanting to participate must organize themselves to earn funding for their initiatives. Before the social fund has disbursed any resources, a treatment community distinguishes itself from its neighbors by assembling a project committee and proposal. Thus a simple “with-and-without� comparison for social fund participants and nonparticipants is biased. It mistakenly attributes to the social fund the community selection effects that encourage participation. Randomized control design avoids these selection problems in creating a com- parison group by randomly selecting parts of the country where the social fund can and cannot operate. But like most other social funds (the Bolivian social fund being the exception), asif did not randomly choose where it would operate, pre- cluding this robust evaluative approach. But alternative techniques allow the impact evaluation to generate treatment and comparison groups for asif. Information collected about households in communities where asif had ap- proved a project but the project had not yet been completed makes it possible to establish a pipeline comparison group. These communities have demonstrated that they can organize themselves for social fund projects, so there is no selec- tion bias. But they have not yet gained the benefits of the projects. So a compari- son of these pipeline communities with those that have completed projects of- fers insight into the effects of those projects, abstracting away from characteristics that led communities to participate in asif. In addition to pipeline matching, this evaluation also uses propensity score matching to correct for selection biases.1 To create a comparison group, the traits of communities that participated in asif are analyzed, and then a propensity func- 1. Several studies have used the propensity score matching approach to evaluate impact. Some have used it for individual-level interventions (Heckman and others 1997, 1998). Among those applying it to community-level interventions are several contributions to the Social Funds 2000 study: Newman and others (2002) look at Bolivia, Pradhan and Rawlings (2002) consider Nicaragua, and Chase and Sherburne- Benz (2000) analyze Zambia. Chase 223 tion is generated that links the characteristics of a community to the likelihood that it will submit a successful proposal for a social fund project. In Armenia geography was a crucial determinant of which communities par- ticipated in the social fund. Because asif focused resources on the earthquake and conflict zones, communities in these areas were more likely to participate in asif, and the basis for their inclusion in the program differed from that in other areas. For these reasons three separate propensity functions are estimated to stratify the sample by the earthquake zone, the conflict zone, and nontargeted zones. The propensity functions isolate the effects of community means for house- hold expenditures, share spent on food, female headship, and education levels in each of these zones. Estimates of propensity function parameters for each of the subsamples are used to predict the probability of program participation for all community clus- ters, by pooling those that participated and those that did not (see appendix). These probabilities are propensity scores. To create a comparison group of communities whose propensity to partici- pate in the social fund was comparable to that of treatment communities, each community that completed an asif project is matched with a community in the same zone that did not participate but that had an equivalent propensity score. This procedure creates a comparison group of communities just as likely to participate in the social fund as the treatment group. The difference between treatment and comparison thus isolates the effect of implementing the social fund project, ab- stracting away from traits that led communities to work with the social fund. Although the idea of the propensity score matching procedure is clear, its application in Armenia is challenging. Notably, for treatment communities with very high propensity scores, it is not always possible to find a control commu- nity equally likely to participate in the social fund. The distribution of propen- sity scores before matching for asif communities (those that participated) and comparison communities (those that did not) shows that proportionately more asif communities had high propensity scores in nontargeted zones as well as in the conflict and earthquake zones (figures 1–3). Thus, treatment communities differ from randomly selected communities in their likelihood to participate in asif. Because of this difference, randomly selected communities are an inadequate comparison group. After communities are matched by propensity score, the treatment and com- parison groups would have the same distributions of propensity scores if the matching procedures had worked perfectly. Although the distributions do be- come more similar after the matching procedure, in each case they are still dis- tinct (figures 4–6). For many of the communities that completed asif projects estimated propensity scores are very close to one, signifying that the communi- ties were almost sure to participate. For these communities there are no matches in the same zone—that is, communities that almost assuredly should have par- ticipated but did not. A comparison group created from each of these zone-specific propensity matches does not adequately correct for selection bias. 224 the world bank economic review, vol. 16, no. 2 Figure 1. Pre-Match Propensity Score Distributions for asif versus Non-asif Communities in Non-Targeted Zones Source: Author’s calculations. Figure 2. Pre-Match Propensity Score Distributions for asif versus Non-asif Communities in Conflict Zones Source: Author’s calculations. Chase 225 Figure 3. Pre-Match Propensity Score Distributions for asif versus Non-asif Communities in Earthquake Zones Source: Author’s calculations. Figure 4. Post-Match Propensity Score Distributions for asif versus Matched Communities in Non-Targeted Zones Source: Author’s calculations. 226 the world bank economic review, vol. 16, no. 2 Figure 5. Post-Match Propensity Score Distributions for asif versus Matched Communities in Conflict Zones Source: Author’s calculations. Figure 6. Post-Match Propensity Score Distributions for asif versus Matched Communities in Earthquake Zones Source: Author’s calculations. Chase 227 III. Effectiveness of Targeting Social funds support project options that more less well-off communities find attractive. They also use administrative efforts to target poorer areas. Through a combination of these two strategies, social funds purport to focus resources on a country’s poorer communities. Household expenditure data from commu- nities where asif was active and from randomly selected communities give in- sights into whether asif resources reached relatively less-well-off communities. The household expenditure data show that people in asif communities are poorer (on average) than other Armenians (table 1). To a statistically significant degree, asif households across the country spent less per capita than randomly selected households. Furthermore, households in asif communities devoted a larger share of their expenditures to food (82 percent) than did non-asif house- holds (80 percent), a robust indicator of higher relative poverty. These differences remain when the country is divided into urban and rural areas. In urban areas asif households spent significantly less per capita (11,800 drams/month) than non-asif households (13,100 drams) and directed a sig- nificantly larger share of their spending to food. Similarly, in rural areas asif households spent less than non-asif households and allocated significantly more of their money to food (85 percent, compared with 83 percent for non-asif households). Thus households in social fund communities are poorer on average than other Armenian households. But did the social fund reach the poorest Armenians? Concentration curves showing the distribution of household per capita expendi- tures in asif communities and in the entire Armenian population offer an answer to this question. If the distribution of poverty among asif households was the same as that among the rest of the Armenian population, asif targeting would be neutral with regard to poverty, and the concentration curve would correspond to the 45° line. But if, say, the 20th percentile of asif households had the same income as the 10th percentile of all Armenian households, that would show that asif resources were being allocated progressively, targeting the relatively poor. Concentration curves above the 45° line indicate propoor targeting. Concentration curves for the country as whole, for urban areas, and for rural areas are all fairly close to the 45° line, showing that asif targeting is relatively neutral (figure 7). Urban spending appears slightly progressive because the con- centration curve is above the 45° line for all parts of the household expenditure distribution. By contrast, at lower parts of the distribution rural spending is slightly regressive. These findings are notable and somewhat surprising. Studies of social funds in other countries have generally found rural spending to be more progressive and urban spending generally more regressive (see Chase and Sherburne-Benz 2001, Newman and others 2000, and Pradhan and Rawlings 1999). asif targeted areas with poor infrastructure, such as the conflict and earthquake zones, where reha- bilitating schools and water systems could have a large direct effect. But it did not 228 Table 1. Monthly Household Expenditures in asif and Non- asif Communities, 1998–99 Total Urban Rural Non-asif asif Non-asif asif Non-asif asif Indicator communities communities t-statistic communities communities t-statistic communities communities t-statistic Total expenditures (drams) 51,654 48,814 1.40 47,934 46,233 0.67 57,364 51,259* 1.86 Per capita expenditures (drams) 13,268 12,554* 1.66 13,144 11,762** 2.33 13,495 13,305 0.25 Share spent on food (%) 79.8 82.1** 6.03 77.8 78.8* 1.93 82.9 85.2** 4.45 *Significant at the 10 percent level. the world bank economic review, vol. 16, no. 2 **Significant at the 5 percent level. Source: Author’s calculations. Chase 229 Figure 7. Concentration Curves of asif Targeting in Relation to Armenian Population as a Whole Source: Author’s calculations from household data. explicitly target areas with low household expenditure. In Armenia poor infra- structure does not appear where household expenditure is lowest. The regressive rural targeting may result from the 10 percent contribution that asif requires of communities. According to asif staff, in rural areas this require- ment selects against the poorest communities. Households there are unwilling or unable to contribute for community public goods, such as schools or improved water systems. Though relatively progressive urban targeting is also unusual for social funds, asif focused its activity on the capital city, Yerevan, one of the least well-off areas of the country. Here, asif activities reached the poorest communities. With many asif projects in Yerevan, progressive targeting there implies that asif reached Armenia’s poorest households. IV. Impact of Education and Water Projects To investigate the household impact of asif school rehabilitation projects, the impact evaluation compares education outcomes for households in communi- ties where asif school projects were completed, households in propensity matched communities, households in pipeline communities, and an unmatched set of ran- domly selected households. The results show that across the country as a whole, 230 the world bank economic review, vol. 16, no. 2 asif generated few significant differences in how much households with primary- school-age children spent on schooling (table 2). In earthquake zones, however, households in asif communities spent 22 percent more than those in the matched comparison group and 27 percent more than those in the pipeline control group. The opposite appears to be the case in conflict zones, where asif households spent significantly less than either of the control groups. Expenditures by the treat- ment group averaged 2,125 drams/month compared with 4,062 drams in the matched comparison group and 3,600 drams in the pipeline comparison group. These findings point to different social fund effects in earthquake and conflict zones. There is also evidence that school enrollments were higher in communities in which asif supported school rehabilitation. In treatment communities 87 per- cent of primary school–age children were in school. This is significantly higher than the 79 percent in school in communities where an asif project had been approved but not yet completed. In earthquake zones households near asif -supported school rehabilitation projects not only spent more on schooling but also were more likely to have their children attend school. Together, these pieces of evidence suggest that asif in- creased demand for education. If the quality of school facilities in the earthquake zones had been low, renovations financed by asif would make the schools more attractive, increasing demand for primary education. Subjective measures of changes in the quality of school services provide inter- esting insight into the impact of asif. Although 8 percent of households in asif communities reported that school services had improved in the previous 12 months, fewer (6 percent) in matched comparison communities said that schools had recently improved, suggesting that people in asif communities had more positive impressions of their schools. In pipeline comparison communities, how- ever, 12 percent reported improvements, significantly more than in the treatment communities. What explains this inconsistency? It is possible that asif projects had been completed some time ago, so that impressions of improvement were remote, and there had been recent discussion of school improvements in pipe- line comparison communities. Overall, however, few households in Armenia believed that schools were improving. Besides school rehabilitations, asif also supported community projects to improve local water supply. For the impact analysis of these projects, the pri- mary indicators of interest are household access to water and sanitation. House- hold data offer evidence that the projects improved access. For example, in asif treatment communities 93 percent of households had access to cold running water, compared with 85 percent in matched communities and 72 percent in pipeline communities (table 3). Further, 92 percent of households in asif communities had central water systems, significantly more than in both matched commu- nities (83 percent) and pipeline communities (68 percent). Finally, compared with pipeline communities, more households had an indoor tap in asif commu- Table 2. Household Effects of asif-Supported School Rehabilitation Projects Treatment Matched Pipeline Unmatched Indicator communities communities t-statistic communities t-statistic communities t-statistic Monthly household expenditures 3,105 3,627 1.32 2,640 1.44 3,967** 2.06 on schooling (drams) Earthquake zones 3,873 3,391 0.69 2,808** 1.98 3,790 0.12 Conflict zones 2,125 4,062** 2.13 3,600** 2.18 5,105 1.56 Log monthly household expenditures 7.84 7.90 0.91 7.76 0.92 7.95* 1.71 on schooling Earthquake zones 8.02 7.80* 1.94 7.75** 2.04 7.84 1.65 Conflict zones 7.62 8.18** 3.42 7.98 1.36 8.20** 2.90 Proportion of 7- to 12-year-olds 0.87 0.83 1.52 0.79** 2.30 0.83 1.59 attending school 231 Earthquake zones 0.86 0.80 1.65 0.76** 2.15 0.80* 1.64 Conflict zones 0.93 0.84 1.04 0.83 0.75 0.83 1.24 Proportion of households reporting 0.08 0.06** 2.17 0.12* 1.84 0.07 1.44 that school service improved in previous 12 months Earthquake zones 0.11 0.10 0.30 0.21** 3.16 0.08 1.28 Conflict zones 0.13 0.01** 4.52 0.00* 1.66 0.03** 3.46 Number of observations 232 646 148 1,298 Earthquake zones 113 150 80 247 Conflict zones 20 87 5 208 *Significant at the 10 percent level. Chase **Significant at the 5 percent level. Source: Author’s calculations. 231 Table 3. Household Effects of asif-Supported Potable Water Projects Treatment Matched Pipeline Unmatched Indicator communities communities t-statistic communities t-statistic communities t-statistic 232 Proportion of households with Indoor water tap 0.68 0.69 0.46 0.43** 7.12 0.67 0.32 Earthquake zones 0.81 0.71** 2.53 0.90* 1.75 0.69** 3.20 Conflict zones 0.90 0.38** 7.92 0.28** 9.74 0.50** 6.10 Central water system 0.92 0.83** 4.17 0.68** 8.28 0.84** 4.05 Earthquake zones 0.94 0.74** 5.32 0.99* 1.75 0.72** 5.93 Conflict zones 0.85 0.58** 4.04 0.49** 4.93 0.71** 2.31 Flush toilet 0.52 0.58** 2.22 0.33** 5.29 0.58** 2.20 Earthquake zones 0.65 0.55** 2.08 0.71 0.97 0.55** 2.21 Conflict zones 0.50 0.25** 3.94 0.21** 4.17 0.38* 1.88 Cold running water 0.93 0.85** 3.95 0.72** 7.67 0.78** 2.33 Earthquake zones 0.93 0.80** 3.75 0.96 0.97 0.76** 4.91 Conflict zones 0.92 0.78** 2.49 0.53** 5.63 0.82* 1.83 Proportion of households reporting (for previous 12 months) Water service improvements 0.34 0.22** 4.78 0.28* 1.81 0.21** 5.83 Earthquake zones 0.52 0.30** 5.13 0.60 1.19 0.24** 7.25 Conflict zones 0.083 0.14 1.20 0.15 1.26 0.15 1.40 Sanitation improvements 0.09 0.09 0.16 0.06 1.57 0.08 0.31 Earthquake zones 0.18 0.13 1.46 0.26 1.46 0.11** 2.48 Conflict zones 0.02 0.00** 2.01 0.00 1.42 0.03 0.50 Illness 0.13 0.18** 2.68 0.17** 1.99 0.20** 4.07 Earthquake zones 0.16 0.17 0.45 0.06** 2.86 0.20 1.57 Conflict zones 0.09 0.20** 2.38 0.24** 2.81 0.20** 2.52 Inactivity due to illness 0.15 0.17 0.69 0.25** 2.80 0.20* 1.87 Earthquake zones 0.15 0.15 0.00 0.18 0.42 0.20 1.49 Conflict zones 0.19 0.27 0.76 0.34 1.33 0.26 0.66 Ill children 0.03 0.03 0.06 0.03 0.13 0.04 0.95 233 Earthquake zones 0.05 0.03 0.12 0.01 1.49 0.04 0.61 Conflict zones 0.02 0.02 0.07 0.03 0.52 0.04 0.59 Number of observations 340 1,740 380 3,600 Earthquake zones 160 420 80 700 Conflict zones 60 240 120 580 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. 234 the world bank economic review, vol. 16, no. 2 nities. Thus, households in asif communities had greater access to water by sev- eral measures—across Armenia and in its earthquake and conflict zones. Subjective measures of changes in water service indicate that households in asif communities were more likely to report improvements in the previous 12 months than either of the comparison groups. Within the treatment group 34 percent said that water service had improved, compared with 22 percent in the matched comparison group and 28 percent in the pipeline comparison group. But asif communities were no more likely to report improvements in sanitation. Although this impact evaluation focuses on output indicators, comparing changes in facilities and their availability to households, ultimately asif projects seek to improve the welfare of people. Few indicators are available to measure these final outcomes, though there is some weak evidence that asif potable water inter- ventions improved beneficiaries’ health. Fewer households (13 percent) in treat- ment communities reported illness than in both matched (18 percent) and pipe- line (17 percent) comparison groups. asif households were less likely to report inactivity due to illness (15 percent) than were those in the pipeline comparison group (25 percent). But there was no statistically significant effect on the propor- tion of households reporting ill children in asif communities compared with either of the comparison groups. Thus, although asif water supply interventions may have had some impact on these health indicators, the data show few strong effects. V. Community Participation Social fund support to local initiatives is often described as having important community impacts that may not be reflected in the welfare of individual house- holds. In post-Soviet societies, where many look reflexively to central govern- ments to solve local problems, a program demonstrating that the community can act collectively might have a positive impact on that institution. Indeed, asif beneficiary assessments show that community participation in the social fund can change community attitudes, encouraging people to solve local problems through their own efforts. Further, as the literature on social capital suggests (see, for example, Grootaert and van Bastelaer 2001), communities that act collectively may be better en- dowed with positive structural or cognitive social capital. Though there is ex- tensive debate about how to define and measure social capital (see, for example, Grootaert 1997, Knack and Keefer 1997, and Woolcock 1998), this analysis focuses on collective action, looking at the likelihood that communities that completed an asif-supported school rehabilitation or potable water project also undertook other recent community initiatives. Information about asif education activities is not universally shared. In com- munities where asif financed the rehabilitation of a school, 49 percent of house- holds reported that this change had recently taken place (table 4). In those where asif financed the rehabilitation of the water system, 46 percent were aware of it (table 5). These results could mean that the rehabilitations had taken place some Table 4. Likelihood That Communities Completing asif-Supported School Rehabilitation Projects Undertake Other Community Infrastructure Projects Treatment Matched Pipeline Unmatched Variable communities communities t-statistic communities t-statistic communities t-statistic Build new school 0.21 0.15** 3.48 0.38** 6.15 0.11** 7.46 Build health facility 0.08 0.09 1.05 0.18** 4.92 0.06** 2.04 Rehabilitate health facility 0.11 0.14* 1.75 0.24** 5.79 0.11 0.09 Build new road 0.02 0.04** 2.07 0.05** 2.79 0.04** 2.10 Rehabilitate road 0.24 0.29** 2.46 0.35** 4.15 0.25 0.93 235 Build or rehabilitate piped water 0.07 0.15** 5.12 0.30** 10.37 0.17** 6.47 Build or rehabilitate reservoir 0.04 0.03 1.24 0.04 0.46 0.03 0.50 Build or rehabilitate sanitation 0.03 0.01** 3.80 0.01 –0.07 0.03** 3.86 Proportion of households reporting 0.49 0.25** 11.64 0.51 0.49 0.22** 14.26 school rehabilitation Number of observations 620 1,740 439 3,600 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. Chase 235 236 Table 5. Likelihood that Communities Completing asif-Supported Potable Water Projects Undertake Other Community Infrastructure Projects Treatment Matched Pipeline Unmatched Variable communities communities t-statistic communities t-statistic communities t-statistic Build new school 0.11 0.15** 2.28 0.21** 3.93 0.11** 0.14 Rehabilitate school 0.28 0.25 1.44 0.36** 2.32 0.22 2.47 Build health facility 0.07 0.10* 1.77 0.06 0.53 0.06 0.41 Rehabilitate health facility 0.18 0.14* 1.79 0.19 0.36 0.11** 3.67 Build new road 0.08 0.04** 3.10 0.05* 1.80 0.04** 3.44 Rehabilitate road 0.16 0.29** 5.08 0.36** 6.46 4.05 0.93 236 Build or rehabilitate reservoir 0.20 0.03** 13.3 0.18 0.44 0.032** 14.28 Build or rehabilitate sanitation 0.03 0.03 0.04 0.07** 2.45 0.033 0.04 Proportion of households reporting 0.46 0.15** 13.5 0.72** 7.51 0.17** 12.91 building or rehabilitation of the world bank economic review, vol. 16, no. 2 piped water Number of observations 340 1,740 380 3,600 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. Chase 237 time in the past, so that households were unaware of them or did not think of the changes as recent. There is evidence that the community effort required to complete an asif school rehabilitation project displaces effort on other local infrastructure projects. To a statistically significant degree, communities that rehabilitated a school are less likely to have also built or rehabilitated a road or piped water system or rehabilitated a health facility. Further, communities in the asif pipeline to do an education project were also more likely to have carried out other types of infrastructure projects than were those that had completed an asif education project. Communities in the pipeline for an asif education project were more likely than propensity-matched communities to take other initiatives. This suggests that communities whose par- ticipation in asif came later had greater social capital. Where asif water projects were completed, collective action does not appear to be as uniformly weakened as with education projects. Completing an asif water project reduced the likelihood that a community would build a health facility or school, rehabilitate a road, or build or rehabilitate a reservoir. But it increased the likelihood that a community would rehabilitate a health facility or build a new road. As with education projects, communities that had not yet completed their asif-supported water initiatives were more likely to have undertaken other infrastructure projects, such as building or rehabilitating a school. In general, if undertaking other community initiatives is used as an indicator of social capital, the communities that completed asif projects had less social capital than propensity score–matched comparators. But the asif communities that had not yet completed their projects—those whose participation began after the social fund had been in place for some time—had significantly higher social capital. VI. Conclusion This article offers several insights into how the Armenian social fund affected households and communities during the postcommunist transition. It provides evidence about the degree to which asif reached poor Armenian households, the effects of the infrastructure projects on households within the projects’ catch- ment area, and the effects on community collective action. asif was not specifically designed to reach poorer communities. Instead, it sought to reach areas with poor infrastructure, where primary schools and water systems were in particular disrepair. Nonetheless, by some robust measures, asif resources reached less well-off parts of the population. Across Armenia and in both urban and rural areas, asif households are on average less well off than other Armenian households. But when concentration curves are used to consider the entire distribution of household expenditures, the story becomes less clear. The targeting of asif resources was relatively neutral with regard to poverty— slightly progressive in urban areas and slightly regressive in rural areas. One explanation for the progressive urban targeting is asif’s focus on Yerevan, whose population suffered acutely from economic dislocation. The regressive rural tar- 238 the world bank economic review, vol. 16, no. 2 geting may result from the difficulties rural communities faced in coming up with the required 10 percent community contribution, which could have excluded poorer communities. Using propensity score and pipeline matching of household data, the analysis demonstrated several impacts of asif-supported projects to rehabilitate schools and water systems. In the earthquake zones household spending on primary education and primary school attendance both rose in communities that had completed school projects, suggesting increased demand for education. In com- munities that had undertaken water projects, households reported improvements in access to water and in water services. But there were few robust indicators of improvements in health in these communities. One of the central objectives of the asif was to increase community involve- ment. The evidence suggests that the communities that completed a social fund project were less likely than comparison groups to complete other local infra- structure projects, suggesting that social capital was expended in these early projects. By contrast, communities that joined asif later and had not yet com- pleted their projects reported more collective action. Although further research is needed to improve on evaluative approaches, this analysis provides substantial evidence from Armenia that social funds do reach communities in difficult economic circumstances. Furthermore, social funds affect the services available to households. Finally, at least in the communities that be- came involved more recently, the social fund bolsters their ability to address local needs. These effects suggest that social funds are a useful tool for improving pub- lic services in Armenia. As social funds begin to operate in an increasing number of countries undergoing transition from central planning, more opportunities will emerge to learn whether it is appropriate to generalize from the Armenian evidence analyzed here to social funds supporting the transition in other countries. Appendix. Community Determinants of Likelihood of Participating in Social Fund Table A-1. Propensity Score Probits for Nontargeted Zones Variable Coefficient z-statistic Community mean per capita expenditure –0.00009** 2.70 Community mean share of food in expenditure 0.021 0.97 Community share of female household heads –0.0093 0.01 Community mean of household head’s education 0.25* 1.65 Constant –2.07 0.98 Number of observations 145 Chi-squared (4) 11.7 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. Chase 239 Table A-2. Propensity Score Probits for Earthquake Zones Variable Coefficient z-statistic Community mean per capita expenditure 0.0002** 2.21 Community mean share of food in expenditure –0.076 1.21 Community share of female household heads –4.77* 0.01 Community mean of household head’s education 0.79** 2.08 Constant 2.15 0.37 Number of observations 41 Chi-squared (4) 15.9 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. Table A-3. Propensity Score Probits for Conflict Zones Variable Coefficient z-statistic Community mean per capita expenditure –0.00003 0.52 Community mean share of food in expenditure 0.162** 2.61 Community share of female household heads 6.06** 1.97 Community mean of household head’s education –0.217* 0.63 Constant –14.6 2.30 Number of observations 43 Chi-squared (4) 15.0 *Significant at the 10 percent level. **Significant at the 5 percent level. Source: Author’s calculations. References Chase, Robert, and Lynne Sherburne-Benz. 2001. “Impact Evaluation of the Zambia Social Fund.� World Bank, Social Protection, Washington, D.C. Goodman, Margaret, Samuel Morley, Gabriel Siri, and Elaine Zuckerman. 1997. “So- cial Investment Funds in Latin America: Past Performance and Future Role.� Inter- American Development Bank, Social Programs and Sustainable Development Depart- ment, Evaluation Office, Washington, D.C. Grootaert, Christiaan. 1997. “Social Capital: The Missing Link?� In Expanding the Measure of Wealth: Indicators of Environmentally Sustainable Development. Envi- ronmentally Sustainable Development Studies and Monographs Series No. 17. Wash- ington, D.C.: World Bank. Grootaert, Christiaan, and Thierry van Bastelaer. 2001. “Understanding and Measur- ing Social Capital: A Synthesis of Findings and Recommendations from the Social Capital Initiative.� Social Capital Initiative Working Paper No. 24. World Bank, Washington, D.C. Heckman, James, Hidehiko Ichimura, and Petra Todd. 1997. “Matching as an Econo- metric Evaluation Estimator: Evidence from Evaluating a Job Training Programme.� Review of Economic Studies 64(4):605–54. 240 the world bank economic review, vol. 16, no. 2 ———. 1998. “Matching as an Econometric Evaluation Estimator.� Review of Economic Studies 65(2):261–94. Khadiagala, Lynn. 1995. “Social Funds: Strengths, Weaknesses and Conditions for Suc- cess.� World Bank, Environment and Social Protection Department, Washington, D.C. Knack, Stephen, and Philip Keefer. 1997. “Does Social Capital Have an Economic Pay- off? A Cross-Country Investigation.� Quarterly Journal of Economics 112:1251–88. Marc, Alexandre, Carol Graham, Mark Schachter, and Mary Schmidt. 1993. “Social Action Programs and Social Funds: A Review of Design and Implementation in Sub- Saharan Africa.� World Bank Discussion Paper 274. World Bank, Washington, D.C. Moffitt, Robert. 1991. “Program Evaluation with Nonexperimental Data.� Evaluation Review 15(3):291–314. Newman, John, Menno Pradhan, Laura B. Rawlings, Geert Ridder, Ramiro Coa, and Jose Luis Evia. 2002. “An Impact Evaluation of Education, Health, and Water Supply Investments by the Bolivian Social Investment Fund.� World Bank Economic Review 16(2):241–274. Paxson, Christina, and Norbert R. Schady. 2002. “The Allocation and Impact of Social Funds: Spending on School Infrastructure in Peru.� World Bank Economic Review 16(2):297–319. Pradhan, Menno, and Laura B Rawlings. 2002. “The Impact and Targeting of Social Infrastructure Investments: Lessons from the Nicaraguan Social Fund.� World Bank Economic Review 16(2):275–295. Walker, Ian, Roberto del Cid, Francisco Ordoñez, and Francisco Rodríguez. 1999. Ex- Post Evaluation of the Honduran Social Investment Fund (FHIS 2). Produced by esa Consultants, Honduras, for the World Bank, Latin American and Caribbean Region. Woolcock, Michael. 1998. “Social Capital and Economic Development: Toward a Theo- retical Synthesis and Policy Framework.� Theory and Society 27:151–208.