54510 v2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 VOLUME 2 RURAL POVERTY REDUCTION in NORTHEAST BR A Z I L An Evaluation of Community Driven Development Hans Binswanger Fátima Amazonas Túlio Barbosa Alberto Costa Naércio Menezes Elaine Pazello Claudia Romano ii © 2009 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 Telephone 202-473-1000 Internet www.worldbank.org/rural Email:feedback.worldbank.org All rights reserved. The findings, interpretations, and conclusions expressed herein are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The World Bank encourages dissemination of its work and will normally grant permission promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978- 750-4470, www.copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank. org. Cover illustration:Tomoko Hirata iii Contents Abbreviations And Acronyms............................................................. vii Executive Summary .............................................................................xi 1. Introduction .................................................................................1 2. Methods for Evaluating Programs ................................................5 Standards and methods for impact evaluation ..............................6 Making up for lack of random selection .......................................8 Making up for a missing baseline .................................................9 A classification of available data and studies on the PCPR program ...................................................................................................10 The number of studies and results in each class ..........................13 3. Critical Review of Evaluation Studies of Rural Poverty Alleviation Projects ..............................................19 The PCPRs are operating on a large scale ...................................19 The history of the PCPRs in Northeast Brazil .............................20 Results: the expanding scope of the PCPRs.................................27 Focus: PCPRs benefit rural communities living in poverty ..........31 Social Capital: the PCPRs as an incentive for the development of social capital ...............................................................................35 The PCPR has a positive effect on gender issues and on quilombola and indigenous populations .....................................55 Sustainability, quality and the satisfaction of the beneficiaries ....63 Effects of the PCPR on income, health, quality of housing and access to services.........................................................................72 Sustainability of productive subprojects..................................... 76 How to build the performance of productive subprojects ...........81 Summary and Main Conclusions ................................................84 4. Impact Evaluation of the PCPR ..................................................89 Introduction ...............................................................................89 Sample design .............................................................................92 iii iv Data description .........................................................................94 Methodology ..............................................................................99 Results on infrastructure, health and physical capital ...............103 Summary of main results ..........................................................124 The social impact of the PCPR: Building institutions, new patterns of social relations and social capital..........................................125 Conclusions ..............................................................................155 Appendix A (Chapter 3)­ Results for household assets .................157 Appendix B (Chapter 3) - Results of propensity score used for analysis of social capital ...............................................................163 References ....................................................................................167 Tables Table 2.1 Classification of the 73 reviewed studies .........................17 Table 3.1 Aggregate Results, Northeast Program (1993-present) ....28 Table 3.2 Overview of Project Beneficiaries RNRDP+RPAP+RPRP (February 1993 to November 2005) ..............................................30 Table 3.2.A Overview of Project Beneficiaries RNRDP+RPAP+RPRP (January 2006 to April 2008) .........................................................30 Table 3.3 Logistic Regression..........................................................50 Table 3.4 Physical Performance Studies carried out, 1998 - 2005 ...70 Table 3.5 Some results of the Physical Performance Studies, 1998/1999 carried out in the States of Sergipe, Ceará, Bahia, Piauí, Rio Grande do Norte and Pernambuco ..........................................71 Table 3.6: Some results of the Physical Performance Studies, 2004/2005, carried out in the States of Pernambuco, Sergipe, Rio Grande do Norte and Piauí.............................................................72 Table 3.7 ­ Evolution of indicators of quality of life .......................76 Table 3.8 Result of Financial Analysis of Selected Productive Subprojects .....................................................................................81 Table 3.9 Results of Sensitivity Analysis, Productive Subprojects ....83 Table 4.1 Demographic characteristics of households in 2002 ........95 Table 4.2 Characteristics of Households and Access to Services in 2002 and 2005 ...............................................................................96 Table 4.3 per capita value of household assets, in 2002 and 2005 (in Brazilian R$ of 2005) .....................................................................96 Table 4.4 Characteristics of the communities ..................................98 Table 4.5 Incidence of diseases and infant mortality: % of communities with a rise or with a fall in incidence between 2002 and 2005 ...............................................................................................99 Table 4.6 Types of projects funded by the PCPR ­ Nº of Communities ................................................................................103 v Table 4.7 Variables representing the social capital of the community before the community association was formed .............................104 Table 4.8 Variables representing the infrastructure conditions in the community in 2002 ......................................................................105 Table 4.9 Characteristics of head of household and household infrastructure in 2002 ...................................................................105 Table 4.10 Propensity Score ­ Households....................................106 Table 4.11 Weighting of control units resulting from the nearest neighbor matching ........................................................................108 Table 4.12 Impact of PCPR on access to water .............................110 Table 4.13 Impact of PCPR on access to electricity .......................112 Table 4.14 Impact of PCPR on variation in total per capita assets (not including land)- 2002-2005 ...................................................114 Table 4.15 Comparison between Different Estimates of Impact ....116 Table 4.16 Variables that represent the infrastructure conditions of the community..............................................................................117 Table 4.17 Propensity Score for the Community ...........................118 Table 4.18 Weighting of control group units from community matching.......................................................................................119 Table 4.19 Impact of PCPR on variation of community assets ......121 Table 4.20 Impact of the PCPR on infant mortality ­ 2002-2005 .122 Table 4.21 Impact of the PCPR on the incidence of diseases ­ 2002 - 2005 ..............................................................................123 Table 4.22 Results Obtained at Municipal Level...........................129 Table 4.23 Results Obtained at Community Level ........................129 Table 4.24 Results Obtained at Household Level..........................131 Table 4.25 Distinctive Characteristics of Community Leaders ......134 Table 4.26 Indicators of stock of the social capital and its four basic components ..................................................................................136 Table 4.27 Measurement of Impact of PCPR on Social Capital ....137 Table 4.28 Estimates of the time trend of social capital ................138 Table 4.29 Stocks of Social capital according to time period and type of community ...............................................................................138 Table 4.30 Measurement of Impact of PCPR on Social Capital ....139 Table 4.31 Measurement of the Impact of the PCPR on Social Capital Inputs.....................................................................140 Table 4.32 Measurement of Impact of PCPR on Social Capital Outputs ........................................................................................141 Table 4.33 Measurement of the Levels of Community Participation ... .....................................................................................................143 Table 4.34 Measurement of the Impact of the PCPR on Community Participation at Community Level ................................................144 Table 4.35 Indicators of Perception of Change .............................145 Table A1: Impact of PCPR on assets (electrical appliances) - 2002-2005..................................................................................157 Table A2: Impact of PCPR on assets (automobiles) - 2002-2005 .158 Table A3: Impact of PCPR on assets (agricultural equipment) - 2002-2005..................................................................................159 vi Table A4: Impact of PCPR assets (animals) ­ per capita figures: - 2002-2005..................................................................................160 Table A5: Impact of the PCPR on financial assets: 2002-2005 ......161 Table B1:Propensity Score at household level for Social capital exercise .........................................................................................163 Figures Figure 4.1 Distribution of propensity-score before matching ........108 Figure 4.2 Distribution of propensity-score after matching ...........109 Figure 4.3 Propensity Score for communities before matching ......119 Figure 4.4 Propensity score for communities after matching .........120 vii Acronyms, Abbreviations, and Glossary AACC Associação de Apoio às Comunidades do Campo [Rural Community Support Association] ADESE Agência de Desenvolvimento do Seridó (Rio Grande do Norte) [Seridó (Rio Grande do Norte) Development Agency] APCR Apoio às Pequenas Comunidades Rurais [Support for Small Rural Communities] ASPEC Associação de Pesquisa e Estudos Científicos [Association for Research and Scientific Studies] BA State of Bahia BNB Banco do Nordeste do Brasil [Bank of Northeast Brazil] CDD Community Driven Development CE State of Ceará CEPLAN Consultoria Econômica e Planejamento [Economic and Planning Agency] CF Projeto Crédito Fundiário [Land-Based Rural Poverty Alleviation Program] CODEVASF Companhia de Desenvolvimento do Vale do São Francisco [São Francisco Vally Development Company] CONTAG Confederação Nacional dos Trabalhadores na Agricultura [National Confederation of Agricultural Workers] CPI Community Participation Index CT Projeto Cédula da Terra [Land Reform and Poverty Alleviation Project] vii viii EDF Estudo de Desempenho Físico [Physical Performance Study] FATRES Fundação de Apoio ao Trabalhador Rural da Região do Sisal [Support Foundation for Rural Workers in the Sisal Region] FECAMP Fundação de Economia de Campinas, Universidade de Campinas [Economics Foundation of Campinas, University of Campinas] FETARN Federação dos Trabalhadores na Agricultura do Estado do Rio Grande do Norte [Federation of Agricultural Workers of the State of Rio Grande do Norte] FLACSO Faculdade Latino Americana de Ciências Sociais [Latin American Faculty of Social Sciences] FUMAC Fundo Municipal de Apoio Comunitário [Municipal Community Support Fund] FUMAC-P Fundo Municipal de Apoio Comunitário ­ Piloto [Municipal Community Support Fund--Pilot] FUNAI Fundação Nacional do Índio [National Indian Foundation] FUNDEC Fundo de Desenvolvimento Comunitário (Bahia) [Fund for Community Development (Bahia)] IBGE Instituto Brasileiro de Geografia e Estatística [Brazilian Insitute of Geography and Statistics] IDE Instituto de Desarrollo Económico (Banco Mundial) [Institute for Economic Development (World Bank)] HDI Human Development Index HDI-M Human Development Index - Municipal IPI Institutional Performance Index IICA Instituto Interamericano de Cooperação para a Agricultura [InterAmerican Institute for Cooperation in Agriculture] INCRA Instituto Nacional de Colonização e Reforma Agrária [National Institute for Settlements and Agrarian Reform] INTERCOOP Cooperativa Interdisciplinar de Serviços Profissionais Ltda [Interdisciplinary Professional Services Cooperative Ltd.] IRR Internal Rate of Return MA State of Maranhão ix MCPI Municipal Council Performance Index MDA Ministério do Desenvolvimento Agrário [Ministry of Agrarian Development] NRDP Northeast Rural Development Project (NRDP) OED Operations Evaluation Department, World Bank OPI Operational Performance Index NGO Non-government Organization PAC Programa de Apoio Comunitário [Community Support Program] PAPP Programa de Apoio aos Pequenos Produtores Rurais do Nordeste [Support Program for Small Rural Producers in the Northeast Region of Brazil] PE State of Pernambuco PI State of Piauí PIDL Projeto Integrado de Desenvolvimento Local [Integrated Local Development Project] PNAD Pesquisa Nacional de Domicílios [National Households Survey] PNV Present Net Value POA Plano Operativo Anual [Annual Operational Plan] PROÁGUA Programa de Desenvolvimento de Recursos Hídricos [Water Resources Development Program] PRONAF Programa Nacional de Fortalecimento da Agricultura Familiar [National Program for Strengthening Family Farming] PRONESE Empresa de Desenvolvimento Sustentável do Estado de Sergipe (ex-Projeto Nordeste do Estado de Sergipe) [Sergipe State Sustainable Development Company (formerly, Sergipe State Northeast Project] PRORURAL Projeto Renascer do Estado de Pernambuco [Pernambuco State Rebirth Project] RPAP Rural Poverty Alleviation Project PCPR I Rural Poverty Alleviation Project--First generation PCPR II Rural Poverty Alleviation Project--Second Generation RPRP Rural Poverty Reduction Project RMT Revisão de Meio Termo [Mid-Term Review] RNRDP Reformulated Northeast Rural Development Project (Reformulated NRDP) x SANA Feira Internacional de Produtos Naturais [International Natural Products Fair] (Bologna, Itália) SEAGRI Secretaria de Agricultura do Estado da Bahia [Bahia State Department of Agriculture] SETHAS Secretaria do Trabalho, da Habitação and de Assistência Social (Rio Grande do Norte) [Rio Grande do Norte State Department of Employment, Housing and Social Services] TU Unidade Técnica Estadual do PCPR [State PCPR Technical Unit] xi Executive Summary 1. This study is being published as a companion volume to the book Rural Poverty Reduction in Northeast Brazil: Achieving Results through Community-driven Development (Coirolo and Lammert, 2009). Background on the Rural Poverty Reduction Program (RPRP or as it is known in Brazil by its Portuguese acronym PCRP, Programa de Combate a Pobreza Rural), its history, objectives, targets, opera- tional features, results and lessons learned are provided in the main book, to which the reader may wish to refer for supplemental infor- mation. The study was carried out by a World Bank team led by Hans Binswanger and comprising Alberto Costa, Claudia Romano, Elaine Pazello, Fátima Amazonas, Naércio Menenzes and Túlio Barbosa. 2. The study evaluates the impact of the PCPR1, based on a critical review of the literature on the Program through the beginning of 20062 and a new study using a "quasi-experimental" design and methodology to rigorously evaluate the direct impact of the Program on infrastructure, health, and physical and social capital, which other studies have not rigorously analyzed. In addition, the study analyzes commonly occurring problems in impact evaluations, along with a classification of evaluation methods. The latter is used to analyze the available literature and classify the results of each of these studies ac- cording to the methodology utilized. 3. The PCPR operates on a large scale, funding various kinds of invest- ments and setting up institutional mechanisms with an enormous capacity for implementing community projects in an efficient manner and at a relatively low cost. Community demand has focused during 1 Throughout the report the program will be referred to as PCPR. Nevertheless, as will be seen, the Program went through three phases and for clarity each phrase will be referred to by its proper name (for a detailed description of the phases, see Coirolo and Lemmert, 2009). 2 Some data/information,, particularly in Chapter 3, have been updated to reflect the status of the PCPRs implementation as of beginning of 2008. xi xii the period 1993-2005 on water supply and electricity projects, and preliminary estimates suggest that the program has had a significant impact in terms of access to these services in the Northeast region. It is estimated that of all rural households in the Northeast that gained access to water and electricity between 1992 and 2003, 59 and 60 percent, respectively, obtained it through the PCPR. The PCPR has also implemented projects that cost on average 30% less than similar programs run by the public sector. The results of the quasi-exper- imental study carried out for the purposes of this evaluation, also suggest that households that gained access to water and electricity through the PCPR would not have gained access to such services through other programs. 4. The PCPR benefits poor rural families. Buainain & Fonseca (2005b) suggest that 75 percent of the beneficiaries have a per capital income of less than US$1 a day prior to the implementation of the projects. Furthermore, the results of the quasi-experimental study suggest that the Project has improved targeting over time: the families benefiting more recently have a standard of living somewhat lower than those who benefited in the past. This contradicts a common criticism of the Program. Various studies suggest that the Municipal Councils and their participatory form of organization constitute one of the main reasons for the continuing improvement in targeting by the Project. 5. The PCPR has also adopted various measures to benefit women, and in- digenous and quilombola (runaway African-slave descendents) groups. Around 30 percent of the PCPR's beneficiaries are women. Water supply systems promote huge economies of time previously wasted by women in activities related to fetching water to their households and free them to leisure, study and other income related activities. Women are becoming increasingly active in community associations and activi- ties, municipal councils and civil participation. With the exception of Minas Gerais, all other States with indigenous populations already have plans specifically designed for them, and Minas has also developed an indigenous plan for the proposed follow-up project. Quilombola people have been singled out for special attention in the States where they exist. 6. The quality of the projects and the level of satisfaction of the ben- eficiaries are high. More than 90 percent of the beneficiaries are satisfied with the quality of construction materials and the overall quality of the projects. The sustainability of the projects is also high. On average, 80 percent of them, with the exception of certain kinds of productive project, are still running three to five years after imple- xiii mentation. Only more complex productive projects, which depend on markets outside the communities, have low indices in terms of sustainability, when not linked to markets. Various studies have concluded that, to improve the rate of success of these productive projects, access to markets should be reinforced and more emphasis should be laid on prior analysis of the technical, economic, and envi- ronmental viability of projects3. 7. In terms of the effects of the PCPR on quality of life, there have been significant improvements in health, housing conditions, and access to services. Access to water and electricity has had various positive effects on health and on the cost of obtaining water. In the case of health, in particular, the quasi-experimental study suggests improve- ments in infant mortality and the incidence of various common diseases, such as diarrhea, intestinal parasites, asthma, hepatitis, and Chagas' disease. 8. Various studies suggest that the PCPR has had a positive impact on income and the accumulation of physical capital, although the results are not conclusive. The results of the quasi-experimental study also suggest that the impact of the PCPR on physical capital is positive, but these results are not statistically significant. It will be necessary to investigate this matter further before arriving at definite conclusions regarding the impact of the Project on income and the accumulation of physical capital, which will only be possible by returning to the sample investigated in the baseline study done by FECAMP in 2004 (Buainain & Fonseca, 2005b). 9. The PCPR has had a positive and sustainable impact on social capital. Institutional arrangements and the transparency of the operation mechanisms of the Project promote social control of the public sector and minimize political interference, corruption and that "local elites" take advantage of the programs. The PCPR uses the social capital already existing in rural communities and transforms it into actions in new social and political spheres, such as local development, and relations between communities and with the local government. The results of the quasi-experimental study suggest that the PCPR is em- powering its beneficiaries and significantly increasing social capital within the communities and municipalities. In addition, the results 3 Recommendations derived from this study (see pages 73 and following) are currently being implemented by many PCPRs, particularly exploring opportunities for access to national and international markets by small farmers. For a detailed description of the approach (strategy) being adopted, see World Bank, 2007 and Coirolo and Lammert (2009). It should be also noted that the share of productive subprojects, in relation to in- frastructure and social subprojects had increased, from 20% by 2005 to 35% as of April 2008. xiv show that the social capital generated continues to increase even sub- sequent to project implementation, thereby demonstrating the sustain- ability of the impact caused. 10. The PCPR has established wide-ranging and effective institutional mechanisms which are already being used in various States to channel funding from other local and community development programs. The PCPR mechanisms have the potential to be used even more widely and to be incorporated into public policy at local, State and Federal level. 1 1. Introduction From 1993-2005 the Rural Poverty Reduction Program (PCPR) has funded more than 50,000 community subprojects in almost 38,000 poor rural communities in the Northeast region of Brazil4. It has used the ap- proach known as Community Driven Development (CDD), according to which the communities themselves request and administer financial resources and are responsible for the execution of subprojects. This CDD approach constitutes a radical change in relation to previous rural develop- ment programs in the Northeast, including those financed by the World Bank. From the outset, the Program has been the subject of various studies and critical analyses, and this literature will be discussed in detail in the second chapter of this report. The dialectical interaction between critiques and evaluations has gradually contributed to improving the design and implementation of the Program, which has progressively incorporated the lessons learned. This literature was first summarized in a report elaborated by Johan van Zyl, Loretta Sonn and Alberto Costa (2000), which also included a discussion of the political and economic situation in Northeast Brazil, the history of programs that preceded the PCPR, and the evolution of the Project over time. The report concluded that the Program · was well-targeted; · built good quality infrastructure at a low cost that kept functioning in the long term; · significantly increased the community's and the municipality's social capital; and · had a positive impact on the health of the beneficiary population due to the emphasis on water supply subprojects. 4 Since then and until April 2008 additional 4,906 subprojects have been financed ben- efiting 200,000 families in 3,617 communities, leading to a total of 54,184 subprojects, 2,740,283 beneficiary families in 40,907 poor rural communities (without repetition between and within phases of the PCPR). 1 2 A report by van Zyl et al also showed that the productive subprojects, taken as a whole, were less sustainable than those involving infrastructure, but pointed out that, even so, there was a positive return. However, this report, which contains an excellent analysis of the program, has never been subjected to a formal critical review or published, and, therefore, its results have not been authenticated and they remain unknown. One weak aspect of the report is that there is no information on the methods used for the studies that served as the basis for the analyses it contains. One of the greatest shortcomings of the literature on the PCPR, up to the present study, is the absence of rigorous impact assessments to provide reliable data on the social and economic impact of the Program. However, at the beginning of 2005, work on a series of new studies was undertaken precisely for the purpose of complementing the information that was still unavailable. These studies aimed to: · produce a critical review of all the studies of the PCPR hitherto car- ried out in such a way as to review the methods and data used in each of these studies; · prepare a summary of the methods used and of the conclusions drawn based on rigorous scientific methodology; and · carry out a rigorous study of the impacts of the PCPR on social and economic factors, and on health, comparing the situation before and after the Program for the group of beneficiaries with a control group. The authors of this report formed the team that carried out these studies. A public opinion polling company, Sensus, collected the data used in the impact study in September and October of 2005. Analysis of the data was completed at the end of February 2006 and a shortened version of the results has been included in a chapter on evaluations of the project in a companion volume on the PCPR (Coirolo & Lammert, 2009). This report presents a fuller and more detailed account of the results. The following chapters are organized as follows: Chapter 2 analyses common problems encountered in evaluations of programs with a special emphasis on impact studies. It was written for a lay public with no special knowledge of evaluation methodologies. In this chapter, a classification of methodologies is developed and this classification is used to analyze the available literature on studies of the PCPR and to classify each result according to the methodology used. Chapter 3 evaluates the existing literature using the classification of methodologies developed in Chapter 2 and summarizes those conclusions, which are based on appropriate methodologies. Chapter 4 reports the results of the impact study carried out in the States of Ceará, Piauí, and Rio Grande do Norte. In this chapter, 3 the methodology used is presented in mathematical language, the data and the processing of data are also presented, and the results for infrastructure, health, and physical and social capital are discussed. 4 5 2. Methods for Evaluating Programs Hans Binswanger and Claudia Romano This chapter is a methodological introduction to the next two chapters, which in turn review the existing literature about the PCPR program, and provide a new impact evaluation of the program. The chapter is written for readers without strong background in evaluation methods and statis- tics. There are many different types of evaluation studies and they have many different purposes. They range from measures of project expenses and the number and type of projects that have been funded and completed, to case studies of projects, supervision reports and impact evaluation studies. Different types of studies have different purposes, but at times some studies are being used for purposes they were not designed for, or which they are unable to answer. In this chapter, we start out by discussing impact evaluation, and the exacting design standards they have to fulfill. Unfortunately the existing studies about the PCPR do not contain many impact evaluations, and those they have do not fulfill the appropriate stan- dards. For this reason, we developed an entirely new impact evaluation study of the program, which meets the standards set out below for impact evaluation studies and is presented in a following chapter. However, there is a wealth of other studies available about the PCPR program. In order to provide a map to this literature, we have classified these studies according to the methodology used. In the second part of this chapter, we present this classification and provide guidance on what types of conclusions and inferences each of these types of studies can reli- ably support. Each of the studies reviewed was also classified according to this typology and where studies used different methodologies, we classi- fied specific results according to the methodologies used. The numbers of studies and results under each classification are provided. In the following chapter, the major conclusions about the PCPR are reviewed based on this 5 6 classification and only conclusions that can be reliably supported by the methodologies used are discussed. Standards and methods for impact evaluation The gold standard of impact evaluation In order to rigorously evaluate the impact of a program (or treatment) such as the PCPR we need to compare a group of households or associa- tions that benefit from the program (the treatment group) with a group of households or associations which did not benefit from the program (the control group). And we need to observe both of these groups before and after the date of the start of the program, that means that we need both a baseline survey and at least one follow up survey including both groups. Moreover, we need to ensure that the treatment and the control group do not have different characteristics that would influence the way in which the treatment would affect them. The classic way to ensure that there are no significant differences between treatment and control group is an experiment, in which individuals are randomly assigned to the treatment and the control group. The gold standard of impact evaluation, therefore, is an experiment with random assignment of individuals to treatment and control groups, and with a baseline survey and follow up surveys of the variables that the program could impact.5 Both random selection and base- line surveys are critical to meet the gold standard. Random selection: The PCPR program design invites people who are members of an association or who want to form an association to apply for the program benefits. The reason is that willingness to work together in an association is a necessary condition for success. Therefore benefi- ciaries and their associations select themselves into the program, i.e. into the treatment group. Without violating one of the basic features of the program it is not possible to take a sample of individuals and randomly assign them to the treatment or the control group.6 The lack of random assignment (or of an appropriate correction for it in the analysis) means that there are likely to be systematic differences between the members of the control and the treatment group. For example, the members of the treatment group could be better educated, more enterprising, or wealthier 5 Even experimental studies which satisfy these exacting conditions are subject to problems (see Ravallion, 2005). 6 Random assignment also means that some individuals or associations are denied the benefits of the program until the end of the evaluation period. This is a condition which program sponsors usually find hard to accept. 7 than those in the control group. Even in the absence of the project or treat- ment, they would therefore have a higher income or asset growth than the control group. When we measure the difference in income growth between the two groups, the treatment effect, we would not know whether it arose because of the treatment, or because the treatment group was better equipped to improve its income to start with. The observed difference in performance therefore would overestimate the treatment effect. This error in the estimation of the impact is called a positive sample selection bias.7 In the next sub-section we will discuss ways of making up for the lack of random selection. Baseline data. The need for baseline data is universally acknowledged. It allows for the measurement of the treatment impact between the baseline period and the resurvey period(s). Various PCPR studies have collected baseline data, including the Buainain and Fonseca (2005a) study and var- ious random baseline studies carried out in 1999, such as FETARN/IICA (1999) and Fonsêca and Melo (1999). If later the sample units in the base- line are resurveyed, we end up with before and after data for both a treat- ment group and for a control group. With such data we use the difference of difference method to estimate the treatment effect. If, for example, we want to measure the treatment impact on income, we subtract the income gain of the control group from the income gain of the treatment group. If incomes of both groups grew over the period, for reasons unrelated to the program, but that of the treatment group grew faster, this method cleans out the underlying income trend. Unfortunately, few evaluations are based on such "before and after" data. Consequently, many investigations are based on comparison of project beneficiaries and non-beneficiaries long after the treatment has been implemented. Of course, at that time it is im- possible to assign randomly individuals to treatment and control groups. Moreover, the investigator can only do "with and without" comparison of the treatment groups at the same point of time. However, these compari- sons are inevitably plagued by sample selection biases. Again, there are a number of methods to overcome partially or fully such biases, and they will be discussed in the next section. 7 Sample selection biases can also be negative, as when sicker people are attracted to enter a treatment because they hope that the treatment would improve their health. 8 Making up for lack of random selection The basic idea is to match the beneficiaries with non-beneficiaries who are as similar to the beneficiaries as possible. For example selecting beneficia- ries from the same village, the same occupation, and/or the same income group as non-beneficiaries would make them more similar to each other and eliminate parts of the sample selection bias. Individuals and communities differ in many respects, and therefore an index has to be constructed out of the observed variables that allows for the matching of individuals via a single variable. The most commonly used technique to do this is called Propensity Score Matching (PSM). The tech- nique involves collecting good data on many characteristics of the mem- bers of the treatment and control groups, which we call the z variables. A variable x is then constructed which has the value of 1 for the treatment group and the value of zero for the control group. The following multiple logit regression is then run to explain the probability of an observation being a member of the treatment group or a member of the control group: Z = bX + e Where: b is the coefficient explaining how a variable impacts probability and e is an unexplained error. The regression can then be used to predict the Propensity Score (the probability of being in the treatment group) for each of the observations. Observations in the treatment group can then be compared to observations in the control group with similar propensity scores. We will use this tech- nique extensively in the quasi-experimental study presented in Chapter 4, where it is explained in much greater detail. The problem with this method is that unobserved or unobservable variables can have an impact on the probability for becoming a member of the treatment group. For example, individuals and associations with higher ability or better quality education could be more likely to apply, but we have no data on ability or quality of education. Therefore PSM cannot correct for the impact of unobserved variables, and an unobserved variable bias remains. The difference of difference method can partially overcome the hetero- geneity between treatment and control groups associated with unobserved variables. Suppose the unobserved ability increases the baseline income of an individual, but does not affect the subsequent income growth. Then the gain of income of any individual during the treatment period is 9 independent of his or her ability. When we subtract the income gain of the treated group from the income gain of the control group, the resulting difference correctly measures the treatment effect. However, if ability also affects the growth of income, an unobserved variable bias remains. It is for this reason that the difference of difference method is also called the "fixed effects method." An additional way to partially overcome the sample selection bias is the "pipeline" method. Under this method all individuals or communities in the sample selected themselves into the program, and therefore are more similar to each other than individuals not interested in the program would be. But the treatment group entered the program shortly after the baseline, while the control group entered the program shortly after the resurvey. For example a program could have been rolled out in phases and became avail- able to the control later for reasons which had nothing to do with their observable or unobservable characteristics. The pipeline method reduces sample selection biases to the extent that this assumption is correct. As will be seen in Chapter 4, the quasi-experimental impact study of the RPAP combines all three methods to reduce remaining sample selection biases to a minimum. Making up for a missing baseline The main method to make up for a missing baseline is to use recall data that asks the respondents to provide information from before the time the project was started. But human brains are limited and cannot reliably rec- ollect huge amounts of information. Therefore recall data has to be used with great caution. To anchor people's memories, recall data are best col- lected for dates when important events occurred, such as a major political event, or a major drought. The quasi-experimental study of the PCPR, for example, uses the date of the last time Brazil won the World Cup in football (in 2002). Recall data should be confined to important and easily recollected information: Most people will have no problem recalling where they lived five years ago, how many children they had, whether they lived in their own house or rented one, the characteristics of the house, whether and what kind of car or other assets they owned. But it would be very difficult to ask a farmer how much income he got five years ago, or even how much maize he produced. This study, therefore, focuses primarily on the asset and debt data, rather than income and consumption data. We will use these data to reconstruct the net worth of households before and after the treatment. If a program or treatment has a significant impact on incomes, it should ultimately reflect itself in the accumulation of assets. 10 In addition to economic variables, recall data can also be used for social and health variables. Most people would remember which asso- ciations they belonged to five years ago, whether the number of friends increased over the period or decreased, and whether they could have relied on friends or family members for support in an emergency. On the other hand remembering how much money they contributed to an association five years ago would be difficult to recollect. They may also be able to state whether health conditions and infant mortality have improved, or have de- teriorated. We therefore also collected data on easily recalled social capital and health variables. Despite the opportunity to use recall data, few of the evaluation studies of the PCPR used them. Instead most studies relied on with and without comparisons. A partial way to overcome the biases which will arise in such comparisons is to use regression analysis, in which the impact variable of interest (such as income) is regressed on observable charac- teristics of the treatment and control group members. In order for this method to fully eliminate any biases, all variables which affect income growth must be included in the regression. But many such variables are not observed, or may be intrinsically unobservable. For example it may be possible to observe years of education of an individual, but not the quality of the education, or his or her ability. The omission of such relevant variable leads to left out variable biases (also called unobserved variable biases). Experience from many studies which allowed for the evaluation of left out variable biases suggests that they tend to be large, and that even the inclusion of many well measured variables will not eliminate them. Increasingly, therefore, the use of with and without comparisons is being abandoned as a technique for impact evaluation. Of course such studies can provide other useful information, a topic which is discussed below. A classification of available data and studies on the PCPR We will now discuss the broad purposes of each type of data and studies and provide examples of the conclusions that can reliably be drawn from them. At the end of this section we have a table which summarizes the number of studies we found in each class of studies. For each type or class of study we have also developed an abbreviation, and the references will list these abbreviations for each individual study. The classification of all relevant PCPR studies is indicated in the list of references, at the end of this report. In the next chapter we will use the insights from this section and the classification of the studies to make sure that we only report find- 11 ings from the pre-existing literature which can actually be supported based on the methodologies and data collection methods used in the studies. The Management Information System (MIS) Ever since the reformulation of the Program in 1993 the technical units in each state developed strong management information systems which are all similar to each other. The states use this MIS system for their own man- agement purposes, and on a monthly basis they send a set of common and comparable data to the World Bank, thereby enabling it to monitor the progress of the overall RPAP across states. The RPAPs MIS systems and data have never come under criticism. The MIS system traces the progress of each sub-project, from the date an association submits a proposal until the completion of the project, the acceptance of the communities' financial accounts by the technical unit, and finally the signature of a document between the association and the technical unit acknowledging that the project execution has been con- cluded at the satisfaction of both the community and the technical unit. The MIS system is therefore the source of data on the expenditures and the outputs of the program in terms of number, size, and types of projects, the number of benefited families, the location of the subproject, the delays between each project steps, budgets and expenses of each sub- project, etc. The MIS system monitors whether the legal status of each association and the expenditure reporting by them is up to date. For ex- ample the system checks whether the term of their presidents have expired and they have been reelected. The system has a cadastre of the potential technical assistance providers. Two states also monitor the political party of the mayor of each municipality. Finally the MIS also contains all the communications between the technical units, the communities, and the municipal councils, the supervision reports by the technical unit, as well as the agreements among the different actors. All the project sampling ever conducted, for both the World Bank su- pervision mission and for all the research studies ever conducted, is based on the MIS system. The MIS information is used in this chapter to describe the overall evolution of the program, the types of projects implemented, their cost and geographic distribution, the number of beneficiaries, the rate of project repetition within the same associations, and the speed of sub- project implementation. 12 World Bank reports Supervision reports (PSR) Project supervision is a management tool of the World Bank to ensure that projects achieve their intended objectives. Each project in each state is supervised at least twice a year by the World Bank. Project supervision focuses on whether the project is progressing well or not, whether it is achieving its objectives, whether financial, procurement, and monitoring and evaluation systems are working properly, and whether the legal project covenants are being adhered to. Supervision missions also identify any problems encountered in project execution and agree on remedial actions. A supervision mission therefore involves both the study of the project re- cords at the state level, as well as field visits to communities and municipal councils, where the project processes, financial accounts, and quality of the subprojects are evaluated. Subprojects to be visited have always been selected at random. At the conclusion of the supervision, the findings are summarized in an Aide Memoire and in the Project Status Report (PSR). In the latter the project implementation and the various project processes are classified into whether they are satisfactory or not. Project supervision reports are a rich source of insights into how a project is proceeding and whether it is achieving the intended outputs. They are not independent evaluations, because they are often being car- ried out by people who have been involved in the project development or its subsequent evolution. A constant fear therefore is that the supervising World Bank staff members are no longer independent observers, and that they may be over-optimistic in their assessment of the project. The supervi- sion reports summarize expert opinion, not results from systematic sur- veys. Nevertheless the project supervision provide the first assessment by those responsible for the project on many of the issues which are also the topic of independent evaluation studies, ranging from the targeting of the program to the quality of subprojects, their costs, their impacts on produc- tion and welfare. Quality Assurance Reviews (QAR) The Quality Assurance Group (QAG) is under the control of World Bank management and reports to Management in order to enable it to improve project performance while the project is still under preparation or under implementation. As in the case of OED reports, QARs may be based on desk reviews of project documents, or on field visits. QARs may be on the preparation of the project, or on specific topics, such as the quality of supervision, financial management, or procurement. In this chapter we 13 cite the findings of a QAR on procurement in Paraíba, on water resource management in Bahia and Pernambuco, and on the quality of financial management of the associations in Bahia. Expert Opinions (S) The analysis of previously existing information by an expert, sometimes supported by a few visits to projects in the field. Research studies Many research studies, in particular those which involved large surveys, were commissioned and financed by the state technical units out of project funds. Terms of references were often designed with the involvement of World Bank staff, either in the writing of the terms of reference, or via comments made during the process of providing no objection for the fi- nancing of the studies from project funds. Most studies were then subject to competitive selection of independent consultants, whose proposals both covered the design of the study and the data collection, as well as the analysis. The consultants with the winning bids then carried out the studies independently until the first draft stage, at which project units and World Bank staff would comment on the studies. A typical example of these processes was the development of the Estudos de Desempenho Fisico (EDF) carried out in seven states. These are the main source of findings presented in this chapter on physical quality of the subprojects and on the procurement procedures adopted by the associations. Some of the studies were also directly commissioned by the World Bank, such as the large review of the program by Johan van Zyl, Loretta Sonn, and Alberto Costa.8 Their large and comprehensive study included review of existing studies, analysis of existing MIS and survey data, as well as additional data collection on selected topics, such as the performance of productive subprojects, and the development of social capital. Other studies were initiated by independent researchers themselves, often in the context of masters and doctorate dissertation. A typical ex- ample is the doctorate dissertation by João Matos Filho (2002), which was the best example of a study of sub-project sustainability. Classification of the research studies Case studies (C) They involve one or several community projects, and or the work of the municipal councils. Most are descriptive of processes, perceptions and 8 van Zyl, Sonn and Costa (2000). 14 results, but there are also some which quantify the results. Case studies are particularly useful as early warning systems of problems which may arise to generate hypothesis which can later be tested in more representative studies, or to obtain reliable interpretation of quantitative results obtained in surveys. On the other hand they are not statistically representative so that no definitive conclusions can be drawn from them, and in particular not in terms of impacts.9 A good example of a study that served as early warning was Nichter (2003). An example of a study that generated hypotheses, which were later tested in larger surveys is the study by Kahn and Silva (2002) that pointed out more emphatically the health effects arising from the water supply and sanitation projects. An example of a case study that helped interpret quan- titative results was Instituto Civitas (2004). Random cross section surveys of beneficiaries, associations, and subprojects These surveys are carried out before, during, or after the implementation of community subprojects. They differ among each other depending on whether or not they include a control group or not. Random baseline surveys (RB) Random surveys of communities and beneficiaries that so far had not received any projects. For example Sampaio et al. (1999) carried out such a survey for Pernambuco. If the random sample had ever been resurveyed, it could have formed the basis of a rigorous impact analysis, using some of the techniques discussed above to correct for sample selection bias. As it stands, the study can only discuss the characteristics of the rural popula- tion. Without any control group (WO) Such studies can provide information about project beneficiaries, their satisfaction with projects, the processes used to carry them out, and the direct expenditures and outputs of the projects. The Buainain and Fonseca (2005a) baseline studies, for example, report about the percentage of bene- ficiaries below the poverty line. The EDF studies discussed above analyzed project quality and user satisfaction. Both these sets of studies include con- trol groups, but the data for the control groups were not analyzed. 9 Some interesting case studies have been prepared for the PCPRs of Paraiba, Bahia, Sergipe and currently are under way for the States of Ceará, Maranhão and Minas Gerais. For the specific case of Paraíba, which methodology is similar for the other states, see World Bank, 2007, in particular annex 6. 15 With comparisons to state means (WWOSM) Some studies did not collect data from any control groups but instead compared the characteristics of beneficiaries to the mean characteristics of the rural population of the state, or with the rural poor population. For example the Buainain and Fonseca (2005b) study showed that the benefi- ciaries of the RPAP had a slight advantage in terms of education over the mean of the poor population in Ceará, Bahia and Pernambuco. With comparison to a control group within the same survey (WWOSS) Most of these studies were designed to measure impacts. As discussed above, these comparisons will suffer from self selection bias, and can therefore not be used for an analysis of impact of the programs. None of the existing studies used propensity score matching to try to reduce such biases. Before and after studies As discussed before, the best way to collect before information is to carry out a baseline survey, followed by one or several resurveys. There is only one such study, by Kahn and Silva (2002) in Ceará, which has already been mentioned. A number of studies used recall data, including the Buainain and Fonseca (2005a) study for immediate impact. Before and after studies of beneficiaries, associations, and projects (BAWO) The classic economic return studies of productive projects carried out by the World Bank for decades use this approach. Van Zyl, Sonn and Costa. (2000) carried out such a study in the context of the program, and calcu- lated rates of return and net present values. In these studies the additional income of the beneficiaries arising from the project is analyzed, and the foregone income of the beneficiaries who gave up some productive activi- ties is collected on a recall basis. The Kahn and Silva (2002) study on project impacts and sustainability discussed before, and some of the social capital studies also fall into this category. Before and after, with comparison to state trends (BAWWO ST) Kahn and Silva (2002) in Ceará studied a sample of projects which in- cluded PCPR projects and the pilot project of the Cédula da Terra program in which the beneficiaries also acquired land. They compared and found that the income of the beneficiaries in these projects grew faster than that of the mean rural population. Unfortunately, the results are not reported separately for the PCPR and Cédula da Terra projects. 16 Before and after, with comparison to a control group within the same sample (BAWWO SS) While a number of studies were designed as such studies, they sometimes did not report the before and after comparisons with the control group, as in Buainain and Fonseca (2005a and 2005b), or they only carried out a baseline survey which was not followed up by a resurvey. As discussed before, such studies should also use propensity score matching, difference- in-difference analysis, and/or the pipeline method to correct for sample selection bias, but none has done so. The number of studies and results in each class The exercise classified 73 studies, some of which use only one method- ology and, therefore, fall into a single classification. Other studies use different methodologies for different results. A first way to look at these studies is to classify them by the highest level of methodology used. Another way is to classify separately each result as a separate study. By looking at the two different ways of counting, we will get a column with the net number of studies, and another column with the gross number of results. Table 2.1 reports both of these findings. As discussed, some studies do not report results on data of comparators they had actually collected. We do not classify these studies by their study design, but by the methods used to report results. The bibliography lists each study of the PCPR and includes a classification of their results. It is disappointing that the data collection effort and survey quali- ties often far exceeded the quality and depth of data analysis. Another disappointment is that several of the available baseline studies were never followed up by resurveys. Moreover, the data had been lost in some of the cases. 17 Table 2.1 Classification of the 73 reviewed studies Number of studies whose highest level methodology falls Number of results that fall into into classification classification PSR 9 9 QAR 2 2 S 10 10 C 16 16 RB 4 4 WO 19 20 WWOSM 1 2 WWOSS 2 3 BAWO 5 7 BAWWOST 2 2 BAWWOSS 3 3 18 19 3. Critical Review of Evaluation Studies of Rural Poverty Reduction Program Túlio Barbosa, Hans Binswanger, Fátima Amazonas, Alberto Costa, Claudia Romano State Projects comprising the Rural Poverty Reduction Program (PCPRs)10 started in the Brazilian Northeast in 1993 with the reformulation of the Northeast Rural Development Projects (NRDP),11 also inaugurating the Community Driven Development (CDD) approach in the Northeast, that is, development driven by the demands of the communities themselves. The list of studies that have the PCPRs and the CDD approach as subjects of study are also long and this chapter presents a summary of the principal results derived from these studies.12 The PCPRs are operating on a large scale Since the reformulation of the NRDPs in 1993, when the CDD approach was adopted, until December 2005, the program funded 50,120 com- munity subprojects based on the demands of around 38,000 communities, benefiting a total of 2.54 million rural families living in poverty across 10 States in the Northeast region.13 In total, US$ 1.4bn were invested up to 10 In this Chapter, PCPR designates the set of State projects to reduce rural poverty, which can be divided up according to time into three generations: the Reformulated NRDP, the Rural Poverty Alleviation Projects and, finally, the current Rural Poverty Reduction Projects. The latter, in turn, were divided into two phases. See Coirolo and Lammert (2009) 11 For a detailed history of the Reformulated State Projects comprising the NRDP see Coirolo and Lammert (2009). 12 More than 100 studies and documents were consulted, 73 of which were reviewed and classified according to the methodology used. A description of the characteristics used to classify them can be found in the previous chapter. The bibliographical citations include the corresponding classification. 13 Without repetition, within and between the Reformulated NRDP phases, the Rural Poverty Alleviation Project and the Rural Poverty Reduction Project. 19 20 2006, US$ 952 million in the form of World Bank loans.14 To support de- centralized implementation, the project help establish 1,505 participatory municipal councils in 1,686 municipalities in the Northeast region and these are still functioning today.15 These councils, which have the majority participation of representatives of beneficiary associations, take decisions regarding which community demands to prioritize for financial purposes. These figures clearly demonstrate the enormous dimensions that the PCPRs have assumed over the last 12 years, both in terms of geographical coverage and in terms of reaching out directly to focus on rural people living in poverty16. The history of the PCPRs in Northeast Brazil The first statement in the literature under review is that, as of the pilot experience of the "Support for Small Rural Communities" (APCR) com- ponent of the NRDPs between 1985 and 1992, the PCPRs, as they are now known, would pass through a process of improvement based on the "learning by doing approach", and move towards greater decentraliza- tion. The PCPRs have gone through three generations since 1993: (a) the reformulation of the State NRDPs, lasting around two years; (b) the Rural Poverty Alleviation Projects (RPAP) generation, in which World Bank loaned directly to State governments for the first time; and (c) the present generation of Rural Poverty Reduction Projects (RPRPs),17 which are being implemented in two phases.18 In the first generation, that of the reformulation of the NRDP, the main implementation mechanism for the PCPRs was the Community Support Program (PAC), according to which there was a direct relation between the communities and the Technical Units (TUs) of the PCPRs. In 14 As of the end of 2005, new loans, totaling US$260.25 million, were in an advanced stage of being contracted and /or negotiated and prepared/to be appraised . As of May 2008, the RPRP portfolio comprised of 7 active projects (Bahia, Pernambuco, Rio Grande do Norte, Ceará, Maranhão, Piaui and Minas Gerais, with loans amounting to US$ 231.85 million. Additionally, new operations for Bahia, Sergipe, Paraiba, and Rio Grande do Norte, (amounting to US$94.2 million) were in various stages of negotiations, appraisal and signing. 15 Counting municipal councils functioning in the States of Bahia, Sergipe, Pernambuco, Paraíba, Rio Grande do Norte, Ceará, Piauí and Maranhão. Since then, a new PCPR for Minas Gerais has been approved and is under implementation, in 188 municipalities and with equal number of municipal councils. Therefore, the total number of municipal coun- cils rises to 1,623 in the 9 States. 16 See footnote 1 for aggregated numbers of subprojects, beneficiary families and communi- ties for the period 1993-2008 (April) 17 Both the RPAPs and the RPRPs have used different names in different States. The RPAPs were implemented at different times in the eight States between 1996 and 2004, and the RPRPs between 2002 and 2008 (first phase, in 7 States). As indicated earlier, Reformulated-NRDP, RPAP and RPRP are indistinctly denominated PCPR., except in this historical sketch 18 For details see Coirolo and Lammert (2009) 21 other words, rural communities, organized into associations, submitted their proposals for subprojects directly to the TUs. At this point, the project established the Municipal Community Support Fund (FUMAC) pilot, through which FUMAC Councils analyzed, prioritized, and sent to the TUs for final approval proposals submitted by community associa- tions. The pilot started with five FUMAC Councils in each participating State (50 in total). All the other municipalities continued to use the PAC. The whole Reformulated RNDP generation was dominated almost exclu- sively by demands for basic infrastructure (water and electricity supply) from the communities.19 The studies and field observations collected over the years record tan- gible results including infrastructure investments, high beneficiary satisfac- tion, and improved living conditions. However, as an approach that broke with tradition, CDD has not been without its critics. These criticisms, however, should be evaluated in the context of the period in which they were voiced. Those, for example, made at the time of the Reformulated NRDP20 ceased to be relevant at the time of the PCPRs, as many, if not all, of the main causes of such criticisms were removed with the improvements to the design and the operational mechanism.21 The criticisms typical of the RPAP stage likewise lost much of their relevance in the present Rural Poverty Reduction Projects stage (RPRPs).22 Which criticisms, then, are 19 In the second generation, that of the RPAPs, in the State of Ceará, in 1996 a pilot access to land scheme was introduced (market-oriented agrarian reform), which was known locally as Solidarity Agrarian Reform, and which also adopted the CDD approach. Later, between 1997 and 2002, this pilot experiment was expanded to cover five States (Minas Gerais, Bahia, Pernambuco, Ceará and Maranhão), under the aegis of the Cédula da Terra Project (Land Reform and Rural Poverty Alleviation Pilot Project), which, in turn, was continued between 2001 and 2006 on a larger scale in 14 States through the Crédito Fundiário (Land-Based Rural Poverty Alleviation Project). In view of the similarity in terms of design and approach (CDD) and their geographical proximity in the Northeast through the Technical Units, these projects, in some cases, will also be covered here. The target public of Land Credit is similar to that of the RPAP and RPRP 20 The review showed criticisms and/or supposed statements that take various lines, such as (i) the distance between the communities and the Technical Units (TUs) was great and tended to be occupied by intermediaries--especially politicians--who were seen as engaging in "clientilistic" activities. (ii) the deficient initial information campaign did not provide enough information for the communities to know before hand about the existence of a rules regarding the RPAPs, leaving them at the mercy of intermediaries; (iii) the illegitimate nature of choices made by the communities, in so far as these were "suggested" or "imposed" by third parties, interested in gaining political advantage--i.e. intermediaries would be seen as doing favors for poor communities and, therefore, sub- projects were used to exercise "clientilistic" practices and the benefits reaped by the elites were more political than economic; (iv) rural communities that were already organized-- and not always the poorest ones--were first in line to receive resources; (v) the sustain- ability of subprojects could be negatively affected by the original illegitimate nature of the demand and (vi) the FUMAC Municipal Councils could be dominated by the Town Hall or by the Mayor in person. See: Garrison et al. (2000); Garrison (2000); Couto Soares et al (2000). 21 See the following section in which the improvements and changes are outlined. 22 See the following section. 22 relevant today and require special attention from those responsible for the design and implementation of the PCPRs? This question will receive due attention in the course of this document. Taking on board the many lessons learned Subsequent generations of the program (Rural Poverty Alleviation Projects - RPAP and Rural Poverty Reduction Projects - RPRP)23 gradually and consistently incorporated lessons learned from prior experience,24 in- cluding: · Extending coverage of the municipalities through FUMAC (leading to a scaling-down of the PAC, the centralized original version of the programs, and its virtual disappearance in the present first phase of the RPRP generation); · the flexibility of the operation of the councils, when it became clear that the board of directors would be elected by peers, i.e. mayors would not be considered automatically president of the council; · broadening of the publicity campaign, along with communication by word of mouth, regarding the existence of the Projects and its rules, reaching a greater number of potentially eligible communities25; · increasing efforts to organize communities26; · the introduction of the pilot FUMAC-P,27 which represents a higher degree of decentralization; · An attempt to introduce the notion of the annual indicative budget for the councils; i.e. the adoption of a formula for allocating resources for funding subprojects among the municipalities participating in FUMAC and FUMAC-P. The Councils are supposed to be informed annually regarding resources allocated in advance of the beginning of the financial year; 23 Eight States participated in the RPAP generation, while six are now participating either in the first or in the /second phase of the RPRPs. Paraiba, Rio Grande do Norte and Sergipe will soon join the group as they are close to negotiate/signing new operations. 24 Including responses to the criticisms leveled at that time. See Footnote 17. 25 The Project's publicity plan became a condition for the effectiveness of World Bank loans. During the RPAP generation and the first phase of RPRP, various TUs launched internet sites with broad access to the public 26 See, for example, the State of Bahia, where, between 1993 and 1995, 3,792 community associations submitted proposals for funding for subprojects. By 2002-2005 the number of associations had risen to 4,462 27 This project implementation mechanism is a variant FUMAC, with a higher level of decentralization. Financial resources, the quantity of which is decided in advance in the Annual Operational Plan (AOP), are transferred by the TUs to the FUMAC-P Municipal Councils, and the latter are responsible for receiving, analyzing, setting priorities, ap- proving and transferring resources to the associations whose subprojects have been ap- proved. For a specific evaluation of the performance of the FUMAC-Ps in Pernambuco, see the Renascer Project (2002); in Sergipe, see PRONESE (2004). 23 · improvement of the mechanisms for targeting the projects--i.e. giving priority to serving the poorest municipalities and rural communities;28 · introduction of the notion that the RPAPs and RPRPs should integrate with other projects and programs with a view to realizing the full poten- tial of outcomes and impacts, especially through the mechanism of the municipal councils;29 · The idea has begun to take shape and gained impetus that, although very important per se, physical investment does not fully achieve the principal explicit objectives of the Projects as a means of overcoming poverty, and, consequently, generating a (sustainable) improvement in the quality of life of families. In other words, investment is explicitly seen as a means to reduce poverty and improve the quality of life of families; and · at the same time, the idea has become clearer that the development of social capital is both a form of investment and a specific objective to be achieved by the Projects.30 The review shows that, during the RPAP and RPRP generations, there has been a more fruitful and sophisticated discussion of the issues that have, directly or indirectly, contributed to improving the design and implementa- tion of the Projects, including the following: · universality versus contextual;31 28 Seven States (Bahia, Sergipe, Pernambuco, Rio Grande do Norte, Ceará, Piauí and Minas Gerais) which are participating in the first or second phase of the RPRP generation have introduced, in their respective Operational Manual, the directive of establishing priorities for the allocation of resources (specifying the sums involved) to municipalities with the lowest Municipal Human Development Indices (IDH-Ms) (below a certain value, which varies from State to State). By the (recently started) second generation of the RPRPs, al- location of resources for funding of subprojects have already become sub-categories of World Bank spending on loans, according to the level of IDH-M, with disproportional al- location to favor the poorest municipalities. In turn, the Municipal Councils will continue to be responsible for identifying the poorest rural communities and their respective areas of jurisdiction. 29 When the RPAP was still running and in the first phase of the RPRP generation, the States made a point of seeking, on their own initiative, some kind of integration of Project ac- tions with other projects and programs. Among various examples, see the experiences in Piauí (Civitas Institute, 2004b) and Maranhão (Argôlo de Souza, 2004). The idea of inte- gration has become a trademark of second generation RPRP projects in the second phase. See, for example, the Project Appraisal Documents for the Projects for the States of Bahia and Maranhão, Project #7327-BR and #4735, respectively. 30 See the section which treats the RPAPs as a way of encouraging the development of social capital. 31 Given the size of the target public to be served with poverty alleviation/reduction mea- sures, and its notably rural nature, and budget restrictions, the option of focusing was reinforced and has continued to be the preferred and most efficient approach adopted by State governments. The debate on this was most intense in 2003, with especially relevant work on this issue produced by Rede Nós/BNB (2003). 24 · the need for "structural" investment that remove structural factors that cause and perpetuate poverty32; · The best balance between demand-driven and supply-driven ap- proaches in the context of the RPAPs and RPRPs.33 Although all states have clearly opted for a demand-driven approach, in practice, the choices made by the States vary in terms of the level of "purity" of the demand-driven approach. They combine to a greater or lesser extent the principle of meeting the demands of the communities with their sector policies, encouraging certain kinds of subprojects. The combination of demand-supply driven approaches has gained a privi- leged forum for its implementation: the municipal councils--which gradually transform into municipal development councils. In this way, the States have begun to realize that they can expand the reach of sector policies aimed at development for low-income farmers and/ or poor rural communities, at the same time as applying the principle of respecting the choices of the communities. In Pernambuco, where 70% of the population lives below the poverty line, study estimates show that the Project served at least 10% of the rural population in 2002 and 2003. In Ceará, estimates show that RPAP I and RPAP II served about 60% of the rural population living in poverty by the end of 2003. In Bahia, the project served 45% of rural dwellers living in poverty.34 The combination of demand and supply approaches, by in- tegrating the RPAPs/RPRPs with other sector programs, allows them to reach an even larger portion of the population living in poverty. · Although the demand for basic infrastructure has continued to pre- dominate, there is a growing demand both from the communities and from State governments to fund subprojects involving "income and employment generation". As explained below, the program has 32 Frequently cited are the need for access to land and the appropriateness of investing in infrastructure and larger scale production. Access to land has become possible with the Solidarity Agrarian Reform and Cédula da Terra Projects, and, more recently, Land Credit and Rural Poverty Alleviation. The explicit integration of the RPRPs with other projects and programs, on the initiative of both State and Federal governments, has been an alternative way of putting together larger investments funded by other sources with the smaller community investment funded by the Projects (for example; water mains funded by PROÁGUAS and community systems for supplying water obtained from these water mains). Some RPRPs, such as that of Pernambuco, and Land Credit, have been using the alternative available for funding a number of somewhat larger-scale subprojects, with a total cost in the range of >US$ 50,000 50 327,366 90,781 0.9 Small-scale irrigation 2 28,311 28.1 23,573 15,238 3.3 Goats 43,346 16.5 16,925 12,180 5.8 Animal feed production 6,730 15.4 1,831 1,149 5.9 Honey production 26,102 16.8 9,112 7,252 5.5 Bakery 43,352 35.0 56,868 29,575 2.8 Fishing 46,104 15.6 13,739 13,162 5.6 Cashew nuts 64,964 >50 288,790 72,195 1.1 Sweets 41,123 41.7 69,062 35,008 2.4 Dairy 62,583 19.7 30,223 34,591 4.7 conventional credit systems, whether traditional or based on land credit.100 For this reason, the PCPRs work by supplying seed money to rural families that are living in poverty but have the potential to take on and manage productive undertakings, on the assumption that, with the success of the initial undertaking, they will be able to stand on their own two feet and have access to the traditional credit system. There is much evidence that this strategy is acceptable: the cashew nut producers of Serra do Mel, in Rio Grande do Norte; the honey producers in Simplício Mendes, in Piauí, and many other producers who participated in the Fair Trade pilot, who are now "liberated"--i.e., they can now do business without the financial assistance of the PCPRs. How to build the performance of productive subprojects Data on the profile of demand in the communities, in the context of the PCPR, show a tendency for the demand for directly productive investment to increase, as basic infrastructure needs have now been met (especially water and electricity supply). It is, therefore, now critical that measures be taken by TUs to seek to improve the performance of productive subproj- ects that are being demanded by communities and funded by Projects. The review of the studies hitherto carried out clearly shows that there are not 100 This was even more the case prior to the expansion of the National Program for Strengthening Family Farming and the Agrarian Reform Projects. 82 only problems and challenges, but that there are viable alternatives that could or should be expanded. These are outlined in this section. The analysis and verification on location of the causes of failure of productive subprojects revealed that little investment had been made, by community associations, by those who provide them with technical as- sistance and by the TUs themselves in prior technical, financial, economic and environmental analysis of proposals for productive subprojects. The examples analyzed by van Zyl, Sonn and Costa (2000) show the adverse impact of inadequate knowledge (or a high level of error) regarding rev- enue, prices and costs on the viability of productive subprojects. Therefore, the first measure needed to improve performance would be to invest in prior viability analysis, particularly of subprojects that are more exposed to competition and market forces. As the authors examined here show, prior analysis is necessary but not sufficient to bring about the success of productive subprojects, which also need to be accompanied and provided with assistance throughout the operational phase. Intimately linked to the question of viability are the recommendations that stronger ties should be developed with the market and communities and producers should organize themselves into networks of associations. The first experiments in both of these areas are already being made in some States. In the case of Pernambuco, for example, a strategy is being adopted of including productive subprojects in a network of solidarity. Since December 2002, 11 productive subprojects that are part networks have been approved: agro-industry (8 communities, mainly in Mata Sul and two awaiting approval) and fish processing (3 communities in Mata Norte). These subprojects are attracting more attention from the Project or other partners, and, because of this assistance, the subprojects that form part of a network run less of a risk of running into the kind of problems frequently encountered by other isolated productive subprojects. The ques- tion remains as to whether the concept of networked subprojects is repli- cable on a wider basis among beneficiaries. In other States, such as Piauí, a similar approach has been adopted that uses the concept of "territories". Another path to success has been, as shown above, exporting through the Fair Trade scheme. 83 Table 3.9 Results of Sensitivity Analysis, Productive Subprojects Scenario 1 - 20% Scenario 2 Scenario 3 Type of Sub-project Case base production - 20% price + 20% costs Granary IRR (%) 15.8 11.5 10.1 14.4 PNV (R$) 5,377 1,385 132 4,124 Tractor IRR (%) 37.6 17.2 <0 16.8 PNV (R$) 35,141 10,836 -23,602 10,221 Small-scale irrigation 1 IRR (%) >50 >50 39.4 >50 PNV (R$) 327,366 163,375 77,015 154,646 Small-scale irrigation 2 IRR (%) 28.1 7.4 4.6 13.0 PNV (R$) 23,573 -2,965 -6,057 3,546 Goats IRR (%) 16.5 12.8 11.5 15.2 PNV (R$) 16,925 7,097 3,847 13,395 Animal Feed IRR (%) 15.4 11.2 10.2 14.4 PNV (R$) 1,831 397 62 1,496 Honey IRR (%) 16.8 10.0 7.2 11.6 PNV (R$) 9.112 42 -3,635 2,162 Bakery IRR (%) 35.0 20.1 <0 15.8 PNV (R$) 56,868 22,451 -21,559 12,859 Fishing IRR (%) 15.6 9.3 <0 <0 PNV (R$) 13,739 -1,732 -39,886 -26,024 Cashew nuts IRR (%) >50 > 50 48.3 >50 PNV (R$) 288,790 210,642 132,561 201,015 Sweets IRR (%) 41.7 24.0 <0 12.7 PNV (R$) 69,062 29,690 -33,720 5,652 Dairy IRR (%) 19.7 2.0 n/a (*) n/a (*) PNV (R$) 30,223 -23,493 -138,697 -84,981 (*) IRR not established; net benefits not negative every year 84 From an operational point of view, whether by way of Fair Trade or tradi- tional markets at regional or national level, the preparation and approval of productive subprojects should move in the opposite direction. In other words, once the project identifies clear market opportunities for products, investment proposals should be designed (that are viable from a technical, financial, economic and environmental point of view) in order to take advantage of these opportunities. This was successfully achieved in the case of the Fair Trade pilot experiments in the States of Piauí, Ceará, Rio Grande do Norte and Bahia. When addressing the question of productive subprojects both aca- demics and the beneficiaries themselves frequently point to the need for access to land as a precondition for implementation of various kinds of productive sub-project, normally involving agricultural and livestock farming. This question becomes especially pressing when the PCPRs change their focus, from rural small-producers (with direct and/or in- direct access to land) to whole communities and their needs. Although small-scale producers continue to make up an important part of the target public, the explicit incorporation of other community groups (salaried workers, landless people, tenants, partners, women, young people, ethnic minorities and others) leads to a higher demand for access to land as a way of making productive subprojects viable. This challenge could partly be faced by adopting productive subprojects that do not require land or require only a small plot (small industrial processing units, handicrafts and so forth) and the consolidation and expansion of the successful ex- perience with the Land-Based Rural Poverty Alleviation Project (Crédito Fundiário). Summary and Main Conclusions On one hand, the ambitious changes introduced by adopting the CDD ap- proach through the reformulated NRDP, the RPAPs, and the RPRPs--al- beit gradually (1993-2005)--inevitably leads to criticisms and skepticism. On the other hand, these changes also led to a desire on the part of those responsible for implementing the projects and some organized social move- ments to persevere, given the expectation that results in the field would demonstrate their quality. This review of studies carried out during the period demonstrates that criticisms made at one point in time are constantly being followed by later improvements. It has also shown that the some of the issues that criti- cisms address where already known when the Projects were designed, i.e. prior to implementation. Nevertheless, it is understood that some changes 85 require new cultural attitudes that take time to be internalized and ac- cepted. Numerous examples exist. For instance, the project designers under- stood that decision-making mechanisms built into municipal FUMAC councils could bring countless benefits in contrast to the risks inherent in the PAC. The real state of affairs, however, suggested the need to organize and mobilize rural communities living in poverty as a whole and to bring traditionally important players--such as, community leaders, municipal authorities, state authorities, and, in particular, politicians--to understand and accept that decentralization means transferring power to traditionally excluded segments of society before installing the councils. This takes time, but the designers always expected the integration of the PCPR to bring more benefits and lead to more efficient investment. However, overcoming corporate resistance requires time and considerable powers of persuasion. Doubt always existed as to whether wishful thinking would be enough to ensure the sustainability of direct investment in subprojects; however, the designers knew that "benevolent dictators" could not be discouraged simply by fiat. Studies have demonstrated increased participation in community associations and in contributions to their operation and maintenance. Community associations have become an important social player in the rural parts of the Northeast and social capital has been growing among beneficiaries and members of associations. Even OED (now IEG) agrees that there is a difference in social capital between members and non-mem- bers of associations. Some studies confirm a growing confidence in community associations and community leaders and in their capacity to represent the interests of the community and resolve its problems on the part of (a) residents of the community; (b) local authorities and (c) State authorities. They have also confirmed that decentralized mechanisms for the provision/delivery of services and investment have improved both in terms of checking the inter- ference of traditional political players and in terms of sharing information and selecting communities. Municipal councils that are more inclusive perform better and have more control over political advantage-taking in decision-making. Better-organized communities entered the program and benefited from it first. However, the PCPR not only uses the social capital that already exists in the communities, it puts the communities' social capital to work building relationships and opening avenues yet untried. This helps point the way towards decentralization, citizen participation, and the obligation 86 to be accountable, in a context in which decentralization and citizenship have not been common features. The implementation of subprojects raises the levels of social capital and the confidence of communities in the capacity of community associa- tions to represent their interests and resolve community problems. The analysis of the aggregate results showed that there is a clear ten- dency for the PCPR, especially once the municipal councils have spread, to increase the physical range of implementation, with the result that little more than 10% of communities have benefited from two or more sub- projects, or rather, around 90% of communities had only one sub-project. One corollary of this analysis is that the social capital built up has been underused. However, nowadays, there is growing involvement of mu- nicipal councils in local decision-making regarding investment being made by other federal and State government programs, such as in electricity, housing, sanitation, water supply and land credit, and thereby making better use of the social capital that has been generated. The review of various studies showed that families benefiting from the PCPR are genuinely poor. However, the studies do not allow us to establish, beyond doubt, that most of the beneficiaries are the poorest of the poor. The vast majority of the studies reviewed show that the most obvious benefits (including from the point of view of the beneficiaries) are to be derived from subprojects involving investment in infrastructure (such as bringing electrical energy to rural areas and supplying water) and from those that are clearly social in nature (improved housing and sanitation). Many studies show significant improvements in well-being and quality of life resulting from these subprojects, especially for women. There are some indications of a favorable effect, albeit a small one, on income generation in some specific cases; the FECAMP studies have demonstrated that there has been an increase in income in Ceará and Pernambuco, in the case of produc- tive subprojects. According to various studies, the beneficiaries consider the quality of works to be good or excellent, regard the scale of the subprojects to be suffi- cient for the needs they were designed to meet, and use the services provided by the sub-project on a regular basis. On the other hand, the studies reviewed are almost unanimous in pointing out the difficulties encountered by communities that opted for more sophisticated productive subprojects, which depend on market forces for their viability, in contrast to simpler productive subprojects, such as grana- ries, simple processing centers, and tractors, which are generally successful. Insufficient prior (technical, economic and financial) viability analyses un- derlie the difficulties such subprojects encounter. Successful experiences, such 87 as the use of Fair Trade mechanisms, show that there are viable alternatives for overcoming the difficulties of productive subprojects. The review showed that there are a large number of disagreements and criticisms regarding the sustainability of subprojects, especially alleging that associations are incapable of collecting user fees to meet the cost of operation and maintenance and the future repayment of the investment. Many of these analyses, however, did not take into account the nature and type of subprojects. As is shown here, in the case of many subprojects, such as those involving rural electricity supply, roads, bridges, fords, water tanks, improvements to housing and sanitation (septic tanks and bath- rooms), the cost of operation and maintenance is covered without needing to charge beneficiaries. Other subprojects, such as the simpler productive ones and many water supply systems, have a high level of sustainability. It is estimated that at least 80% of subprojects funded do not have serious problems with sustainability. It is impossible to separate the view of the world and of time that a researcher has from the analysis he or she makes of the phenomenon under observation. It is therefore impossible to uncover the ideological facets of the point of view of researchers and analysts. The old adage applies here: some see the cup as half empty, while others see it as half full. One example is a study, whose original version was analyzed here. This study laments the fact that a certain rural community has only 12 telephones, when the vast majority has none! Therefore, the review of the literature has brought these "private" views regarding the world and the PCPR in particular to light. This is not problematical or improper; we should realize that any study which seeks to evaluate a Program as objectively as possible sets off alarm signals for those responsible for that Program. The improvements seen in the design of PCPR in the course of the transition from Reformulated NRDP to RPAP and RPAP to RPRP are the fruit of these evaluations that were considered to be objective, both by the States and by the World Bank itself. The (real) persistence of some prob- lems and/or difficulties, on the one hand, and criticisms, on the other, has concentrated attention on the quality of the various evaluations that have been or are being carried out. The most recent examples are the OED/IEG evaluation of the CDD experiment in Rio Grande do Norte and the studies conducted by FECAMP in the States of Bahia, Pernambuco and Ceará. The question now is whether these studies have solid and unquestionable methodological bases and are sufficiently reliable to justify their conclu- sions? Some say yes; others have their doubts. What then is to be done? A new approach should be tried, and such an attempt is made with the quasi-experimental study, to which the following chapter is dedicated. 88 89 4. Impact Evaluation of the PCPR Hans Binswanger, Alberto Costa, Naércio Menezes, Elaine Pazello, Claudia Romano Introduction None of the studies of the PCPR carried out up to 2005 provided reliable information of the impact of the project in terms of income and some other aspects relating to the quality of life of the beneficiaries. This was because the methodologies used in these studies were not appropriate for impact assess- ment. Therefore, a new study was carried out to obtain this information, using techniques of analysis and sample design appropriate for impact assessment studies. This chapter presents the results of this study.101 The study carried out research covering 864 households in 108 com- munities and 90 municipalities, half comprising project beneficiaries and half comprising a group of control communities, in the States of Ceará, Piauí, and Rio Grande do Norte. The study was conducted in 2005, with the field research carried out from September to October. The researchers collected data on demographics, health, education, and social and physical capital at household, community, and municipal level. The study is based on data on the current situation (as of 2005) compared with 2002, the latter collected using recall, that is, questions asked in 2005 on the pre- vious circumstances of those interviewed. This chapter describes the results and the methodology used in this study. The authors have no knowledge of other studies that have used as many methodologies appropriate for reducing the problem of selection 101This study was drawn up under the technical guidance of Hans Binswanger. Claudia Romano was responsible for the overall supervision and coordination of the study and for writing the introductory section and description of data; Naércio Menezes and Elaine Pazello were responsible for the econometric analyses and for writing the section of physical capital, health and infrastructure; Alberto Costa was responsible for the analysis and writing of the section on social capital. The team also received valuable contributions from Túlio Barbosa. 89 90 bias as have been used here. Apart from this, the study presents a seminal attempt to analyze in quantitative terms the impact on levels of social capital in communities and municipalities. The research team considered a number of options in terms of study design and the methodological structure. As it is an impact assessment study, the first possibility considered was returning to the sample used by a study carried out by Fecamp in 2004 (Buainain & Fonseca, 2005a and 2005b), which is the only study available that collected baseline data not only from beneficiaries but also from a control group. This study inter- viewed 1,940 beneficiaries whose PCPR projects were being implemented at the time or had been concluded recently. It also interviewed 1,250 control households in three States and collected data at household and community level. The greatest advantage of this approach was the baseline data collected at the time the projects were implemented, since there is no other available study of the PCPR that contained detailed data on a con- trol group with baseline data. However, the researchers rejected this option because the baseline study included only projects implemented between the end of 2003 and 2004, so the time between implementation and impact assessment was too short to provide results on the medium- and long-term impact. In the future, however, a return to this sample would allow for the analysis of various medium- and long-term impacts, including income impacts. The second possibility considered was to take advantage of delays in project implementation caused by factors exogenous to the community, in order to select the control and treatment groups. The treatment or beneficiaries sample would be selected from the group that requested a project at a certain time and obtained them more promptly. The control sample would be selected from those groups of communities that requested projects at the same time as the beneficiaries' sample, but whose project implementation was delayed. This would basically be a quasi-experimental design structure where the control group is selected from the lists of com- munities that have requested and been approved for PCPR projects, but where implementation has not yet occurred or occurred later than ex- pected. While such examples existed, mainly between Stages I and II of the RPAP, cases of delay for "exogenous" reasons did not occur in sufficient quantity to put together a sample that would have the minimum size to be statistically representative. The team finally decided on a third option, which is described in the next section. 91 Some issues about the methodology The overall design of the study used a quasi-experimental structure, where the control sample was selected from communities that requested and were awarded PCPR projects, but where these projects had either not been implemented or had been implemented only very shortly prior to the period during which the field research was carried out. Thus, communities whose projects were approved in the first semester of 2005 made up the control group. The treatment group (beneficia- ries) consisted of communities whose projects had been approved in the first year of RPAP II, in 2002. The two groups should be similar in most of the features that make a community sufficiently organized to be successful in receiving a Project, thereby reducing the possibility of serious problems with selection bias in the analysis. However, since the control group only obtained a PCPR project three years after the treatment, there may still be differences between them which may cause selection bias. The team used four different methods to deal with the problem of selection bias.102 These are described briefly here, but will be described in more detail in the section below that describes the methodology used in the econometric analysis. The first method, known as the pipeline sampling, basically involves selecting a control sample from among communities that have received preliminary approval for the requested projects but have not yet been financed or implemented them. However, even after applying pipeline sampling, problems of selection bias could remain since the treatment group requested and obtained projects three years prior to the control group. This probably means that the control group had less capacity and fewer opportunities to request projects earlier, at the time the treat- ment group did. One method for dealing with this potential selection bias was to use propensity score matching, which uses information on observable features of communities and households to select the most similar pairs from the treatment and control groups from the samples under study. This method was applied in two distinct phases. The first used existing data on features of the communities, col- lected at the time they requested a subproject along with features of their munici- palities.103 This dealt with the problem of selection bias arising from the observable community and municipal variables. Upon completing the field research, the study could also use the more detailed data collected from households and communities to carry out a second stage propensity score matching exercise. This second stage 102 In the first chapter, on impact analysis methodologies, all these methods were described in detail. 103 The variables used for the first stage propensity score varied a little from State to State, owing to the kind of information available, but always included a variant of the following: distance from the municipality's main town; presence of health clinic and schools in the community; traffic conditions on the main access road; number of inhabitants per health clinic for the municipality, population density of the municipality; relation between the population of the community and that of the mu- nicipality; average level of education of the population of the municipality; the various indices that make up the municipal HDI; log of municipal GDP. 92 approached the problem of selection bias based on a larger number of ob- servable features. Finally, the team used difference-in-difference analysis to compare the difference between the situation in 2002 and in 2005 of the control and treatment groups, thus adjusting for the additive effects of non-observable features, which may also result in selection bias. The authors have no knowledge of other studies that have used all these methods together to reduce selection bias.104 The econometric results indicate that a combination of the four methods for reducing the problem of selection bias was effective: the second stage propensity score matching and difference-in-difference methods can reduce the selection bias that re- mains once the pipeline sampling and first stage propensity score matching methods have been applied. Detailed results are presented later in this chapter. Another important feature of the study design is that information about the baseline situation were obtained through questions about the past situation, or recall questions, asked about July 2002. Since in that period the finals of the 2002 World Cup was taking place, when Brazil won for the fifth time, this presumably helped people to remember their situation at that time. Sample Design This study used a sample of 864 households in 108 communities and 90 municipalities, divided equally between the three States of Ceará, Piauí, and Rio Grande do Norte.105 The control makes up half of the sample and the treatment group the other half. The team interviewed eight households from each community. The sample was designed to be statistically repre- sentative of the beneficiary population in the three States together.106 The treatment communities were selected from those that had a project approved between April and September 2002. This meant that no effects of the projects would have occurred in July 2002, the pe- riod to which the recall questions were directed. Notably, the average time between project approval and completion of implementation is 104 Even so, there can be no guarantee that there are no problems with selection bias in this analysis, as the effect of non-observable features that vary over time, i.e. those which are not additive, is not reduced by the difference of difference method. However, it should be noted that there is no known method of analysis that deals with this problem. 105 These three States were chosen for practical reasons, as they were the only States that kept good records of the profiles of communities, which are registered when projects are requested. This information was used for the first stage propensity score. 106 The household sample is statistically representative, with an error of 5% and 95% con- fidence level, while the community sample was drawn to give an error of 10% at 95% confidence level. 93 approximately six months, although this varies from state to state and according to the type of project. Limiting the sample selection to projects approved in this period also avoided another potential problem relating to the time passed between July 2002 and project implementation. This problem could arise from the fact that factors exogenous to the projects could influence the situation of the beneficiaries between July and comple- tion of project implementation, and the impact of these exogenous factors would be counted erroneously as if they were effects of the PCPR. Thus, limiting the sample to approvals occurring at the latest in September 2002 reduced this problem. The control communities were selected from those that had proj- ects approved between March and July 2005, so that by September and October 2005, when the field research was carried out, no impact would yet have been felt. The sample included both infrastructure and productive projects. In Ceará, the sample included only infrastructure projects, mainly involving water supply, as no productive projects were approved during the period covered by this study. In Piauí, practically all the productive projects ap- proved over the period covered, which were few in number, were included in the sample and the remainder of the sample is made up of infrastructure projects. Rio Grande do Norte has a better balance between productive and infrastructure projects. It is worth noting that the communities considered for sample selec- tion that requested infrastructure projects included only those that had not received other PCPR projects prior to the project implementation investigated by this study (including during the period prior to the current version of the project, that is, before 1993). This means that the treat- ment communities had received no other PCPR project before 2002 and the control communities had received no PCPR projects prior to 2005. The communities with productive projects considered for selection could include communities that had previously received an infrastructure project, but no other type of PCPR project. Apart from this, if a community with a productive project had already received an infrastructure project, it could have a greater or lesser likelihood of receiving another project, and the econometric analysis therefore controlled for the fact of a community having received a PCPR project in the past. The first stage propensity matching score was calculated separately for each state, and, in Rio Grande do Norte, which had a large number of projects of both types, was also calculated separately for communities with infrastructure and productive projects. 94 Note on the methodology used for analyzing social capital This study constitutes a pioneering attempt to use econometric methods for quantitative analysis of changes in the levels of social capital in com- munities and municipalities. Different from the analyses of physical cap- ital and other indicators of well-being, the analysis of social capital used different methods to control for effects external to the PCPR. These will be described in more detail in the next section, which discusses impacts on social capital. The sample for the present study was designed to create a control group for the analyses of physical capital and health, since in September and October 2005, when the field research was conducted, impacts of the PCPR on these indicators would not have taken place yet for the control group. Nevertheless, the social capital of these same control communities would already have been impacted by the PCPR when field work took place since, by then, the associations were already actively preparing or coordinating the initial steps of implementation of the projects. Since the PCPR is demand-driven and requires communities to legally constitute associations in order to apply for projects, the sample of control com- munities do not constitute a "control group" for the social capital analysis given the PCPR already impacted on these communities social capital. Another strategy had to be sought to control for factors external to the PCPR that impact on social capital. The solution was to estimate a time trend of the level of social capital in the period before the community as- sociations were formed. This methodology will be described in detail in the section on the results of the social capital analysis. Data description This section presents the characteristics of the treatment group (benefi- ciaries) and the control. Tables 3.1 to 3.3 sum up the characteristics of households, comparing treatment and control groups, covering a total of 864 households, divided equally between the two groups. Table 4.1 describes the demographic features of households in 2002. The age and sex of heads of households is similar in the two groups: the majority are men of an average of 48 to 49 years of age. The average number of indi- viduals per household is approximately 4.5 for the two groups. The dif- ference between the groups appears in the level of education of the heads of household, with the beneficiaries having a higher level of education in comparison with the control. The percentage of heads of households who have never attended school is lower and the proportion that attained 95 higher levels of schooling is higher in the treatment group. Nevertheless, both groups, generally speaking, have low levels of schooling. Table 4.1 Demographic characteristics of households in 2002 Characteristic Control Treatment Head of household is a man (%) 93 93 Age of head of household (years) 48 49 Level of education of head of household: Never attended school (%) 17* 12 Completed primary school (4 years) (%) 20 23 Completed high school (%) 8 10 Attended university (%) 0.5* 1.6 Number of individuals per household 4.7 4.6 * if the difference between the control and the treatment group is statistically significant at least at 10% Table 4.2 describes the level of access of households to various services in 2002 and in 2005. In 2002, the communities from the treatment and con- trol groups were similar in various ways, including access to water--which was relatively low for both groups--having an indoor bathroom, and sewage treatment. However, in 2002 a greater proportion of households from the treatment group had access to electricity and a higher percentage of families from this group also owned their own home. In 2005, following the implementation of the PCPR projects in the treatment communities, access to water had risen considerably from 31% to 43% of households, while, in the control group, there was an increase of only 1%. Housing conditions also improved in the treatment group as indicated by having an indoor bathroom: in 2002 the proportion of house- holds with an indoor bathroom was similar for both groups, while in 2005 the treatment group had better conditions than the control group. The total value of household's durable goods in 2002 was a little higher for the treatment group, as can be seen in Table 4.3, but this dif- ference is not statistically significant. In 2002, the beneficiaries had more electrical appliances, animals, tools and agricultural machinery than the control group, although the value for automobiles107 was a little higher 107 `Automobiles' includes various types of vehicle, including private cars, motorbikes, trucks and bicycles. 96 in the control group. Financial assets, which are the sum of debits and credits, were similar for the two groups. . Between 2002 and 2005 the value of all types of assets considered in this analysis rose a little in both groups (with the exception of financial assets, which were already nega- tive and decreased further), and the difference in the total per capita value between the control and the treatment group fell a little, despite continuing to be statistically insignificant. Table 4.2 Characteristics of Households and Access to Services in 2002 and 2005 Control Treatment 2002 2005 2002 2005 With ownership of house (%) 78 * 85* 89 90 Access to piped water (%) 29 30* 31 43 Access to sewerage system or septic tank (%) 30 37 34 40 Rubbish collection or rubbish burning (%) 65* 69* 74 75 Access to Electricity (%) 60* 68* 80 89 Indoor bathroom (%) 59 70* 63 75 *indicates that the difference between the treatment and the control groups, for the same year, is statistically significant at least at 10% Table 4.3 per capita value of household assets, in 2002 and 2005 (in Brazilian R$ of 2005) Control Treatment 2002 2005 2002 2005 Electrical appliances 227* 335* 348 441 Automobiles 405 568 350 450 Tools and Agricultural Machinery 21* 27* 42 47 Animals 328* 345* 562 566 Financial Assets -60 -236 -50 -247 Total per capita value 921 1039 1256 1257 * indicates that the difference between the treatment and the control groups is statistically significant at least at10% Analysis of these tables suggests that the treatment group was a little better-off than the control group in the period prior to project imple- 97 mentation in 2002. In specific terms, the level of education of heads of households, access to electricity, home ownership, and the total value of electrical appliances, agricultural tools, and animals was higher for the treatment group. As the control group for this study was selected from among recent PCPR beneficiaries (2005), it can be concluded that the PCPR, which was already reaching poor communities in 2002, is focusing better on poor communities over time. Tables 4.4 and 4.5 supply a description of the characteristics of the communities for the control and treatment groups. The sample covers 108 communities, 54 in the treatment group and 54 in the control. Table 4.4 describes the characteristics of these communities, including population and access to services in 2002 and 2005. In our sample, the control and treatment communities are located at similar distance from the main town of the municipality and are more or less the same size in terms of popula- tion. Generally, the treatment communities, in 2002, had slightly better access to services than the communities in the control group, but only the differences in access to secondary school, health clinics and electricity are statistically significant. The two groups were similar only in terms of access to water in 2002. This changed significantly in 2005, with a great improvement for the treatment group. These data corroborate the analysis of household data on access to services and show that the treatment group was slightly better-off in 2002. The value of community assets, that is, assets belonging to and used by the community, such as churches, soccer field and casa de farinha (manioc flour mill), for example, is similar for the treatment and control groups, both in 2002 and in 2005, although it rose in both groups over the period. It should be pointed out that community assets directly funded by the PCPR are not included in this figure. The analysis of the characteristics of the communities, taken as a whole, suggests that communities from the treatment and control groups were not much different in 2002, although the treatment group was slightly better-off in terms of infrastructure. The difference was greater in 2005 in terms of access to services. The health of the population was analyzed only in terms of changes occurring between 2002 and 2005 and the analysis suggests that there was significant improvement in both groups, although this was greater in the treatment group, as can be seen in Table 4.5. The infant mortality rate fell by 38% and 25% for the communities in the treatment and control groups, respectively. Incidences of dengue fever, worm infection and diar- rhea all fell significantly and in similar proportions in the two groups. The incidence of diarrhea, for example, was down 70% in the sample commu- nities from both groups. 98 In the case of other diseases, the communities interviewed presented changes between 2002 and 2005 only in the incidence of hepatitis, asthma, and Chagas' disease, the incidence of all of these decreasing in both groups, although more significantly in the communities from the treatment group than in those from the control. The economic analysis described in the fol- lowing sections confirms these results. Table 4.4 Characteristics of the communities Control Treatment 2002 2005 2002 2005 Distance from the main town (km) 19 18 Number of houses inhabited 55 ? 60 ? Estimated population 263 ? 245 ? Church (%) 30 32 37 44 Rural workers' union (%) 28 37 30 39 Police station (%) 6 4 4 6 Farmers' Association (%) 30 32 20 20 Post Office (%) 4 4 6 7 Public Telephone (%) 20 35 30 37 Primary School (%) 63 57 74 70 Secondary School (%) 9* 7* 24 24 Health Clinic (%) 11* 13* 33 32 Houses with access to piped water (%) 31 33* 45 63 Houses with electricity (%) 48* 56* 70 81 Houses connected to a sewerage system or with a septic tank (%) 38 44 51 58 Total value of community assets (in Brazilian R$ of 2005) 65451 68810 86384 73513 1 Variables for access to services indicate the percentage of communities, or of houses in the communities that have access to each service. * indicates that the difference between the treatment and the control group is statistically significant at least at 10% 99 Table 4.5 Incidence of diseases and infant mortality: % of communities with a rise or with a fall in incidence between 2002 and 2005 Type Control Treatment Infant Mortality Rose 0% 2% Fell 25% 38% Diarrhea Rose 9% 7% Fell 69% 70% Worms Rose 19% 17% Fell 43% 50% Dengue fever Rose 13% 13% Fell 41% 43% Hepatitis Rose 13% 4% Fell 33% 41% Asthma Rose 14% 9% Fell 27% 42% Chagas' Disease Rose 4% 8% Fell 17% 33% * indicates that the difference between the treatment and the control group is statistically significant at least at 10% The analysis of the data comparing the treatment group and the control group suggests that the focus of the PCPR is improving over time, as the communities that received projects later are initially worse off than those that benefited earlier in terms of: infrastructure, education, access to health clinics, and some kinds of household assets. This suggests that over time the project is reaching even poorer families, exactly the opposite conclu- sion to that of some frequently aired criticisms of the project that assert that local elites are increasingly reaping the benefits of the PCPR. Methodology In order to analyze the impact of the PCPR, we used an econometric methodology involving three different ways of dealing with the question of self-selection into the program. This methodology consists of simultaneous use of the pipeline, propensity score matching, and difference-in-difference methods. In order to understand these techniques better, a more detailed explanation of the problems related to self-selection is needed. Let represent the value of assets of community i in 2005 after the PCPR, if it participated in the program in 2002 and let represent the value of assets of the same community in 2005 if it did not participate in 100 the program in 2002. Thus, PCPR's effect on community assets i is and the average impact of the program on the communities that received the program in 2002 is , where indicates that community i participated in the program in 2002 and indicates that it did not participate. However, as it is impossible to observe the same community in both situations, we can use a control group to estimate the difference between the average value of assets of communities that participated in the PCPR in 2002 (the treatment group) and that of communities that form part of this control group, such that: The last term on the right-hand side of equation (1) is a measure of selection bias, representing the fact that the value of assets of communities that did not participate in the PCPR in 2002 does not necessarily correspond exactly to the value of the assets of those that did, had they not participated. In other words, in order to identify by way of comparison with a group of communities that did not participate in 2002, it would be necessary to ensure that . This last condition would be met if the participation of communities in the PCPR in 2002 had occurred randomly. However, as the selection process for the program was not random, we will use some econometric methods and identification hypotheses to estimate in a consistent fashion. The first method, known as pipeline sampling, uses for the control group only communities that enrolled on the PCPR and had their projects approved in 2005. The advantage of using this control group is that the communities enrolled that had their projects approved in 2005 tend to be similar to those that had their projects accepted in 2002 (the treatment group), in terms both of observable and non-observable characteristics. This occurs because the main differences between communities are be- tween those that have sufficient organization and management capacity to apply and obtain the project and those that do not. The identification hypothesis adopted is that the "timing" of the process of approval and selection is random (Ravallion, 2005). As this identification hypothesis may be strong in some situations, we will also use propensity score matching. In other words, we will suppose that the remaining differences between the communities that received the program in either 2002 and in 2005 relate only to observable characteris- tics so that the communities with identical observable characteristics have the same probability of having projects accepted in 2002 as in 2005--of falling into the treatment or the control group. As mentioned earlier, 101 we use propensity score matching in two separate stages. The first stage selected into the sample the most similar pairs of treatment and control communities from the population considered for this study. The second stage used more detailed data collected for this study on households and communities. In both cases, the assumption used means that and , where ^ indicates independence and X the vector of observable characteristics. This means there is no difference in terms of non-observable characteristics between the communities that par- ticipated in the program in 2002 and those that participated in 2005, once the study controlled for observable characteristics of these communities. We thereby obtain: where is the average impact of the PCPR on the value of assets of the communities with characteristics X. That is, for communities with the same observable characteristics, the difference in the value of assets between those who participated in the program in 2002 and those who participated in 2005 provides us with the average impact of the treatment on those treated. This allows us to identify the average impact of the treatment on those treated as: In case of X i being discrete we could rewrite (2) as where is the probability that a treatment community has characteristics X. Thus, one arrives at the average impact of the treatment group by estimating the differences in the average value of assets between the com- munities that participated in the PCPR in 2002 and those that participated in 2005, for each of the groups of characteristics X and, finally, obtain the weighted average of these differences, using a weighting system determined 102 by the distribution of the communities in 2002 among the various groups of identical characteristics. Angrist & Krueger (1999) estimate this. As other authors point out, the difference between this estimate and a regression of minimum squares lies in the weighting system. While the weighting system here is determined by the distribution of beneficiaries among the groups with the same char- acteristics, the weighting implicit in the estimate of minimum squares takes into account the variance of D, conditional on X. Thus, a group with a variance greater than D receives a heavier weighting.108 One problem with applying (3) occurs when X is continuous or very numerous. In these cases, an alternative is to fall back on the propensity matching score methods. Such methods use the propensity score estimate (the probability of falling within the treatment group, conditional on X) as a way of "summarizing" the information contained in X. Rosenbaum & Rubin (1983) show that if , then , where . Thus, (2) can be reformulated as: In order to estimate (4), the method adopted was that of propensity score matching, which consists of choosing, for each community (or household) from the treatment group, a community (or household) from the control group to serve as a comparison. The community (or house- hold) chosen from the control group is the one with a propensity score closest to that of each community (household) from the treatment group. The communities (households) not chosen were excluded from the esti- mation process. The final estimate (the effect of the treatment on those treated) consists of a simple comparison of the mean value of assets of the treatment group with that of the control group, chosen in this manner. The third method applied, after the pipeline and propensity score matching methods, was difference of differences (DD). This method com- pares the value of assets of the treatment and control groups before and after participation in the program. In our specific case, we compare the value of assets and other characteristics in 2002 and in 2005, both in the communities that participated in the PCPR in 2002 and in those that par- ticipated in 2005. The identification hypothesis of this method is that the non-observable differences between the treatment and the control groups 108 Note that as D is a variable that assumes values of 0 or 1, the groups gain in weight as approaches 0.5. Obviously, these two estimates tend to converge when the impact of treatment does not vary with X. Angrist and Krueger (1999) present a comparison of these estimates for more general cases. 103 are additive and fixed in time, in such a way they can be eliminated by comparing the same communities at two points in time. This identification hypothesis may fail if the rate of growth of assets depends on the initial conditions of the communities, which, in turn, may be different for the treatment and control groups (Ravallion, 2005). It is for this reason that we complemented the difference of differences method with the pipeline and propensity score matching methods. The estimation method is the same as that for the propensity score matching, except that we used the growth of assets between 2002 and 2005 as a variable outcome, instead of the level of assets in 2005. Results on infrastructure, health and physical capital In order to better understand the results presented below it is important to know the type of project that was carried out in each State. Table 4.6 below shows these figures for the sample used in this study. So far as infrastructure projects are concerned, there is a greater concentration of projects involving water supply in the States of Ceará and Rio Grande do Norte and of supply of electricity in Piauí. Productive projects are only present in the States of Piauí and Rio Grande Norte; and in these States the number of productive projects is higher than of infrastructure projects. Table 4.6 Types of projects funded by the PCPR ­ Nº of Communities Piauí Ceará Rio Grande do Norte Type of Project Treatment Control Treatment Control Treatment Control Water supply 0 2 18 11 4 4 Electricity supply 5 3 0 7 0 0 Productive 13 13 0 0 14 14 Total 18 18 18 18 18 18 We have divided this section into two subsections. The first presents the results of the analysis of household data, and the second the results for communities. Impact analysis for household variables As stated above, the idea of the pipeline procedure is to homogenize the treatment and control groups, its main concern being to remove selection bias arising from differences in non-observable variables. The working hypothesis adopted is that the procedure was successful in this regard. 104 However, there could still be differences in observable variables. The pro- pensity score matching seeks to eliminate precisely these differences. The matching procedure carried out uses the "nearest neighbor" and the kernel approaches. As the estimation is carried out for all States to- gether, the propensity score used dummies for each of the States (the point of reference being Rio Grande do Norte). The other explanatory variables are outlined in Tables 4.7 to 4.9. The logistic equation that served as the basis for estimating the second stage propensity score, that is, the likeli- hood of belonging to the treatment group, is shown in Table 4.10 and a balancing test performed on the results of this estimation indicates that it passes the test at 1%. Table 4.7 Variables representing the social capital of the community before the community association was formed Year association The year the community association was set up set up dummy that is equal to 1 if the household responded that, before the association was set up, if a problem, for example, with water Coop_antes supply in the community, suddenly arose, most residents would meet to resolve the probleme unity, suddenly arose, most of with water supply in the e association was set up,g to the treatment group, can be found in T Participated in meetings of municipal councils, assemblies, public part_reun hearings before the community association was set up (yes = 1; don't remember = 1.5; no = 2) Met or established regular contact with politicians and authori- Cont_pol ties to demand action for the community before the community association was set up (yes = 1; don't remember = 1.5; no = 2) Met with people from neighboring communities to resolve shared Cont_viz problems before the community association was set up (yes = 1; don't remember = 1.5; no = 2) Worked on a voluntary basis on charity or community work before trab_vol the community association was set up (yes = 1; don't remember = 1.5; no = 2) 105 Table 4.8 Variables representing the infrastructure conditions in the community in 2002 esc_prim_0 dummy for the existence of a primary school in the community in 2002 esc_sec_0 dummy for the existence of a secondary school in the community in 2002 posto_0 dummy for the existence of a health clinic in the community in 2002 Dist Distance of community from the main town of the municipality nº hab Number of inhabitants hab/casas Ratio between the number of inhabitants and the number of houses in the community rec_proj dummy for community having received a previous project from the PCPR; Table 4.9 Characteristics of head of household and household infrastructure in 2002 educa_0 Level of education of head of household in 2002 id_0 Age of head of household in 2002 Sexo_0 dummy for head of household being a man in 2002 at_anim_0_pc Value of asset `animals' per capita in 2002 at_agric_0_pc Value of asset `agricultural equipment' per capita in 2002 at_auto_0_pc value of asset `automobiles' per capita in 2002 at_eletr_0_pc value of asset `electrical appliances' per capita in 2002 at_fin_0_pc value of `financial' assets per capita in 2002 ac_agua_0 dummy for household having access to piped water in 2002 ac_esg_0 dummy for household having access to sewerage system or septic tank in 2002; ac_lixo_0 dummy for household having access to rubbish collection or rub- bish burning in 2002 ac_luz_0 dummy for household having access to electricity in 2002 Prop_0 dummy for home ownership in 2002 Banh_0 dummy for household having indoor bathroom in 2002 Após_0 Value of pension in 2002 Fonte_ap_0 dummy for pension being main source of income in 2002 Fonte_agr_0 dummy for farming being main source of income in 2002 Mor_0 Number of residents of household in 2002 bad_0 dummy for 2002 having been a good year for agriculture (climate) 106 Table 4.10 Propensity Score ­ Households Logit estimates Number of obs = 860 Log pseudo likelihood = -481.33273 LR chi2(15) = 229.54 Prob>chi2 = 0,0000 (standard errors adjusted for clustering on communities) Pseudo R2 = 0.1925 Dependent Robust Variable = Coef. Std. Err. Z P>|z| [95% Conf. Interval] `Treatment' Ceará 0.577*** 0.200 2.88 0.004 0. 185 0. 969 Variables about social capital of community before community association was set up part_reun 0.611** 0.273 2.24 0.025 0.076 1.299 cont_pol 0.716*** 0.297 2.41 0.016 0.134 1.299 cont_viz -1.089*** 0.358 -3.04 0.002 -1.791 -0.3086 Variables on infrastructure conditions of community posto_0 1.012*** 0.214 4.71 0.000 0.590 1.433 hab/casas -0.367*** 0.066 -5.52 0.000 -0.497 -0.236 Variables on head of household and household infrastructure educa_0 0.171** 0.072 2.37 0.018 0.030 0.313 at_eletr_0_pc 0.001*** 0.000 2.95 0.003 0.000 0.002 ac_esg_0 0.435** 0.225 1.93 0.053 -0.006 0.876 ac_luz_0 0.928*** 0.200 4.63 0.000 0.535 1.321 prop_0 0.821*** 0.222 3.70 0.000 0.387 1.255 107 banh_0 -0.581*** 0.200 -2.95 0.003 -0.981 -.199 fonte_ap_0 0.525** 0.246 2.13 0.033 0.043 1.008 fonte_agr_0 0.288 0. 178 1.62 0.105 -0.060 0.636 bad_0 0.360** 0.183 1.97 0.049 0.002 0. 719 Constant -1.894*** 0.720 -2.63 0.009 -3.305 -0.483 Standard errors in parentheses; corrected for heteroskedasticity. *,**,*** indicate statistic significance at 10%,5% and 1%,respectively Note that the variables control specifically for the initial conditions of households and communities they are located in, that is, they are prede- termined variables exogenous to the treatment. As can be seen from the results laid out in the logit above, the pipeline procedure was successful in homogenizing the groups in terms of observable variables, in addition to non-observable variables, given that there are few statistically significant differences between the groups. Matching of groups was carried out based on the propensity score estimates derived from the model above. In the nearest neighbor ap- proach, each member of the treatment group is paired with a member of the control group that had the most similar propensity score. It should be mentioned that the matching procedure was carried out with substitution and common support.109 For the nearest neighbor matching, of the 432 observations in the treatment group, 431 had information for all variables and, of these, 409 (95%) remained in the common support. Of the control group, 164 units were used, meaning that some units are paired with more than one treatment unit. Table 4.11 shows the weighting of control units and Graphs 3.1 and 3.2 represent the distribution of the propensity score before and after the nearest neighbor matching procedure (the distribution also improves with the kernel matching procedure). 109 Members of the treatment group whose propensity score is less than or greater than the minimum or maximum value respectively of the propensity scores estimated for the con- trol group are considered to lie "outside the common baseline". 108 Table 4.11 Weighting of control units resulting from the nearest neighbor matching Weight Frequency Weight*Frequency 1 80 80 2 36 72 3 19 57 4 9 36 5 5 25 6 4 24 7 2 14 8 2 16 9 2 18 11 1 11 13 2 26 14 1 14 16 1 16 Total 164 409 Figure 4.1 Distribution of propensity-score before matching 0 1 3 2 Density 1 0 0 .5 1 0 .5 Estimated propensity score Graphs by treat As the graphs show, the distributions of propensity scores for the control group (group 0) and the treatment group (group 1) are much more similar after matching, suggesting that the procedure was successful. Similar graphs for the kernel matching procedure also show that the distribution of the two groups becomes more similar after matching. The variables of interest in measuring the impact of the PCPR were: ac- cess to water, access to electricity and the total per capita household assets. 109 Figure 4.2 Distribution of propensity-score after nearest neighbor matching 0 1 3 2 Density 1 0 0 .5 1 0 .5 Estimated propensity score Graphs by treat It should be remembered that the analysis focuses on the variation in these variables (difference of differences) between 2002 and 2005. Three models were estimated, all robust for heteroscedasticity (cluster correction was not used in the matching estimates but was used in the logit propensity-score regression) : · Model 1 included only a dummy for the treatment; · Model 2 included dummies for interaction between the type of project and the treatment, in such a way as to enable the impact of the PCPR to be identified according to type of project; · Model 3 included dummies for interaction between treatment, States and type of project, for the purpose of analyzing the impact of the projects in each of the States. Impact on access to a water Table 4.12 shows the outcome of the impact on access to water. The vari- able `D_water' varies from -1 to 1; `-1' signifies that between 2002 and 2005 the household ceased to have access to water and, therefore, 1 signi- fies that it started to have access. The first model shows the impact on all the units treated, while the second allows us to associate the impact with the type of project carried out and, as expected, the water supply projects explain the rise in access to this service. There were water supply projects in the States of Ceará and Rio Grande do Norte, with 100% of projects in Ceará being water-supply projects). In Rio Grande do Norte, of the 18 communities sampled, only 4 had this kind of project. These figures corroborate the results of the third model, which shows a greater impact of the PCPR in terms of access to 110 water in Ceará compared to Rio Grande do Norte. If a household is in one of the beneficiary communities in Ceará, its chances of having access to water are 0.29 points higher than a household in a non-beneficiary com- munity. In Rio Grande do Norte the impact is small and not significantly different from zero. Results of the kernel matching are also presented for model 3; note that they are similar. Table 4.12 Impact of PCPR on access to water Sample resulting from matching D water Model 1 Model 2 Model 3 Matching method NN1 NN1 NN1 kernel Treatment 0.093*** (0.026) Treat*Abast 0.243*** (0.038) Treat*Eletrif -0.031* (0.018) Treat*Prod -0.011 (0.027) Treat*Ce*Abast 0.291*** 0.295*** (0.043) (0.044) Treat*Rn*Abast 0.031 0.035 (0.046) (0. 048) Treat*Rn*Prod 0.007 0.110 (0.044) (0.045) Treat*Pi*Eletr -0.031* -0.026 (0.018) (0.021) Treat*Pi*Prod -0.031* -0.026 (0.018) (0.021) Constant 0.031* 0.031* 0.031* 0.026 (0.018) (0.018) (0.018) (0.021) nº of observations 595 595 595 768 F (k, n-1) 13.02 . . Prob > F 0.0003 . . R-squared 0.0217 0.1024 0.1242 0.110 Standard errors in parentheses, corrected for heteroskedasticity *,**,*** Statistically significant at 10%, 5%, and 1%, respectively 111 From these data we can see that there was no variation in access to water in households that were in the treatment communities in Piauí. This is why the coefficients associated with the variables `tr_pi_eletr' and `tr_pi_prod' are exactly the same as the constant for the model, but of the opposite sign. This is also why the F figure was not generated for Models 2 and 3. It should be made clear that no sampled community from the treatment group in Piauí had projects involving water supply funded by the PCPR. These results allow us to conclude that, in the communities that had access to the PCPR; a greater proportion of families had access to water compared with communities in the control group. Impact on access to electricity Table 4.13 shows the impact on access to electricity. Model 1 shows that households in beneficiary communities had a greater increase in access to electricity compared with those in non-beneficiary communities. Model 2 tells us that, as expected, this larger increase was in communities that had electricity supply projects. Such projects are concentrated in Piauí. No community sampled in the other States had this kind of project. There are in fact some communities with electricity supply projects in Ceará, but these are in the control group, which has still not benefited from the impact of the treatment. 112 Table 4.13 Impact of PCPR on access to electricity Sample resulting from matching D electricity Model 1 Model 2 Model 3 Matching method NN1 NN1 NN1 kernel Treatment 0.074*** (0.028) Treat*Abast 0.030 (0.031) Treat*Eletrif 0.378*** (0.081) Treat*Prod 0.052* (0.030) Treat*Ce*Abast 0.041 0.042 (0.033) (0.027) Treat*Rn*Abast -0.022 -0.020 (0.050) (-0.046) Treat*Rn*Prod 0.016 0.017 (0.030) (0.023) Treat*Pi*Eletr 0.378*** 0.379*** (0.081) (0.079) Treat*Pi*Prod 0.092** 0.093*** (0.040) (0.035) Constant 0.022 0.022 0.022 0.020 (0.023) (0.023) (0.023) (0.014) nº of observations 595 595 595 768 F (k, n-1) 7.05 7.52 5.48 Prob > F 0.008 0.0001 0.0001 R-squared 0.017 0.080 0.086 0.090 Standard errors in parentheses, corrected for heteroskedasticity *,**,*** Statistically significant at 10%, 5%, and 1%, respectively In Piauí, the outcome is positive both for the communities that had electricity supply projects and for those that had productive projects financed by the PCPR, although obviously larger in the former cases. If the household is in one of the beneficiary communities in Piauí with electricity supply projects, its chances of having access to electricity is 0.4 points greater than in a household located in a non-beneficiary community. For communities with productive projects the figure stands at around 0.1. In Ceará, it can also be seen that the coefficient associated with the treatment dummy is positive and statistically different from zero, although to a much lesser extent (around 0.05). In Rio Grande do Norte the coefficients are not statistically 113 different from zero, which means that there was no difference between treatment and control. In the sample as a whole, in Ceará, more than 90% of households in both groups stated that there had been no variation in their conditions of access; and, in Rio Grande do Norte, 96% in the case of households in the treatment group and 87% in the case of households in the control group. To sum up, it can be concluded that, in communities with access to the PCPR, a larger proportion of families had access to electricity compared to those in communi- ties in the control group. Impact on total per capita household assets Table 4.14 below shows the results for the impact on total household assets per capita, excluding the value of land, between 2002 and 2005. Assets included in this measure are electrical appliances, such as television, radio and cooker; auto- mobiles, such as private cars, motorcycles, trucks; agricultural equipment, such as ploughs, digging implements, hoes, irrigation equipment; animals; and financial assets, including debits and credits. Although none of the coefficients associated with the treatment were statisti- cally significant, they were positive in all the models calculated. The study also estimated PCPR's effect on each of the five asset types--electrical appliances, auto- mobiles, animals, and financial assets. These results are presented in Appendix A. In this case as well almost none of the coefficients were statistically different from zero, but, again, they were generally positive. The lack of statistical significance of estimates of the impact on physical capital may be because an increase in the in- come of families living in poverty is mostly consumed rather than invested. It may also be because measurement errors when using recall data tend to be great, and the sample for this study was relatively small. For electrical appliances and agricultural equipment, a statistically significant impact was found in the State of Ceará. Although most projects in this State involved water supply, there was a larger variation in these assets for households benefiting from the PCPR. For agricultural assets, a statistically significant and positive outcome was found in Rio Grande do Norte, where most projects are productive. In the case of financial assets, in fact, all the coefficients are negative. This asset is the result of the difference be- tween credits and debts over the period and it can be seen that debts increased and that, according to our results, the change was greater in the treatment group. 114 Table 4.14 Impact of PCPR on variation in total per capita assets (not including land)- 2002-2005 D total per capita household Sample resulting from matching assets Model 1 Model 2 Model 3 Matching method NN1 NN1 NN1 Kernel Treatment 357 (408) Treat*abast 324 (413) Treat*eletrif 282 (413) Treat*prod 402 (425) Treat*Ce*abast 413 195 (414) (227) Treat*Rn*abast -77 -60 (503) (263) Treat*Rn*prod 640 455 (473) (320) Treat*Pi*eletr 282 39 (414) (221) Treat*Pi*prod 141 15 (418) (217) -326 -326 -326 22 Constant (400) (400) (400) (203) nº of observations 595 595 595 768 F (k, n-1) 0.77 0.37 1.39 Prob > F 0.38 0.78 0.23 R-squared 0.007 0.008 0.013 0.006 Standard errors in parentheses, corrected for heteroskedasticity. *,**,*** Statistically significant at 10%, 5%, and 1%, respectively To sum up, all the coefficients estimating PCPR's effect on household assets taken as a whole were positive and the effect on each type house- hold asset was, in the vast majority of cases, also positive, though only a few coefficients were statistically significant. Therefore, we are unable to conclude that the PCPR had a positive impact on physical capital. However, the results strongly suggest that the PCPR has a positive ef- fect on the value of the assets of the beneficiary families. This is because: i) all the coefficients associated with the treatment group (around thirty 115 in total) were positive and it would be unusual for all these coefficients to be positive if the PCPR had no impact on assets; ii) the result of Model 1, for the total value, indicates that there is a greater likelihood of the impact being positive than being zero (more than 50%); iii) the values estimated for the coefficients are high, around R$524 per capita, for a population that has an average of R$1,255 in assets; and iv) some estimates indicate specific impacts that are statistically significant, such as the impact of pro- ductive projects on agricultural equipment, for example, and, in Ceará, the positive impact of the project on electrical appliances assets. As has already been pointed out in various parts of this report, our econometric methodology for data analysis consists of associating propen- sity score matching and difference of differences'. It is interesting to com- pare the estimates obtained using these techniques with those produced in cross-section estimation (instead of looking at differences of differences) and when the whole sample is used (rather than using propensity score matching). This is done in table 4.15 below, for the three variables of in- terest. First, let us look at the variables "access to electricity" and "per capita household assets", for which the results differ substantially among the var- ious techniques used for analysis. The cross-section estimate for the whole sample (Cross-section OLS) indicates that, in 2005, the percentage of house- holds from the treatment group that had access to electricity was on average 20 percentage points higher than that of the control group. However, in terms of variation (Diffs in Diffs OLS), the likelihood of a household from the treatment group acquiring access to electricity is only 0.01 point greater than that of the control group. In the case of household assets, the "Cross- section OLS" shows that, on average, the value of per capita household assets of the treatment group in 2005 was R$ 216 greater than the value of such assets for the control group, but, in terms of variation (Diffs in Diffs OLS), the value of the assets of the treatment group grew on average less than those of the control group. The percentage of households with access to electricity, as well as assets, in the treatment group was already higher in 2002 and this result, in fact, shows a tendency to converge towards the mean. In other words, even using the pipeline mechanism to determine the control sample, which, as the logit above shows, was successful (since few variables show statistically significant differences between the two groups), it is important to combine the difference of difference procedure with propen- sity score matching (or a similar technique) when the division of the groups into treatment and control was not random (Ravallion, 2005). In the case of the variable "access to water", the results are similar for the two samples (initial and result of the matching). This happens 116 because the percentage of households that had access to a water in 2002 was already quite similar for the treatment and control groups in the initial sample (See Table H2) resulting from the pipeline. As the pipeline was especially successful for this variable, the matching did not make such a difference. Table 4.15 Comparison between Different Estimates of Impact Cross-Section Cross-Section Diffs in Diffs Diffs in Diffs OLS Matching OLS Matching Access to water Treatment 0.126 0.176 0.104 0.093 (0.033) (0.054) (0.024) (0.026) Constant 0.303 0.244 0.016 0.031 (0.022) (0.049) (0.015) (0.018) Nº of observ. 863 573 863 595 F (k, n-1) 14.980 10.470 19.740 13.02 Prob > F 0.000 0.001 0.000 0.0003 R-squared 0.017 0.035 0.022 0.0217 Access to electricity Treatment 0.208 0.027 0.009 0.074 (0.027) (0.037) (0.021) (0.028) Constant 0.681 0.856 0.083 0.022 (0.022) (0.033) (0.014) (0.023) Nº of observ. 863 573 863 595 F (k, n-1) 58.94 0.54 0.21 7.05 Prob > F 0 0.4644 0.6467 0.008 R-squared 0.064 0.0016 0.0002 0.017 Per capita household assets Treatment 216.613 -9.618 -117.956 357 (317.90) (321.233) (132.403) (408) Constant 1039.467 1151.457 118.325 -326 (289.424) (300.180) (103.273) (400) nº of observ. 864 573 864 595 F (k, n-1) 0.460 0.000 0.790 0.77 Prob > F 0.496 0.976 0.373 0.381 R-squared 0.001 0.000 0.001 0.007 Standard errors in parentheses, corrected for heteroskedasticity. 117 Impact on community variables Since the community sample size is much lower (54 communities treated and 54 in the control), the number of regressors in the logit for estimating the propensity score was smaller. Apart from the dummies for the States (Rio Grande do Norte being the point of reference), we used the variables that appear in Table 4.16. Table 4.16 Variables that represent the infrastructure conditions of the community Esc_ dummy for the existence of a primary school in the prim_0 community in 2002 Esc_sec_0 dummy for the existence of a secondary school in the community in 2002 posto_0 dummy for the existence of a health clinic in the community in 2002 n_casas_água_0 number of houses that have access to water in 2002 n_casas_luz_0 number of houses that have access to an electricity in 2002 n_casas_esg_0 number of houses that have access a sewerage or septic tank in 2002 Dist Distance of the community from the main town of the municipality n hab Number of inhabitants of community hab/casas Ratio between number of inhabitants and number of houses in the community at_com_2002 Total value of community assets in 2002 The following Table 4.17 shows the results of the propensity score for the communities. It can be seen that the "chi-squared" shows that, with a 10% confidence level, it is not possible to reject the null hypothesis that all the coefficients are equal to zero, which constitutes evidence that the treatment and control communities were initially very similar. This result provides evidence of the success of the pipeline procedure in determining the communities to be included in the sample. In order to carry out the analysis of communities in the same way as done for households, we also carried out the matching procedure. As be- fore, the matching uses 1 neighbor, and is carried out with substitution and common support. Of 54 units in the treatment group, 7 lie outside of the common support. And of the 54 units in the control group, 25 were used, which means that some communities from the control group represent more than one member of the control group. Table 4.18 below shows the 118 weighting of units from the control group and figures 4.3 and 4.4 the dis- tribution of the propensity score before and after matching. As can be seen in the graphs, the matching helps to bring the distributions even closer together. The non-smooth trajectory of the distribution function is a result of the small number of observations. Table 4.17 Propensity Score for the Community Logit estimates Number of obs = 108 Wald chi2(34) = 16.44 Log pseudo likelihood = -63.57163 Prob>chi2 = 0.2260 Pseudo R2 = 0.1508 Dependent Robust Variable = Coef. z P>|z| [95% Conf. Interval] Std. Err. `Treatment' Piauí 0.403 0.560 0.720 0.471 -0.694 Ceará 0.433 0.670 0.650 0.518 -0.879 Variables that represent the social capital of the community before the community association was set up Year of foundation of association -0.010 0.059 -0.170 0.862 -0.127 0.106 Variables that represent the infrastructure conditions of the community esc_prim_0 0.571 0.469 1.220 0.223 -0.348 1.489 esc_sec_0 1.254 0.764 1.640 0.101 -0.243 2.751 posto_0 1.141 0.677 1.690 0.092 -0.185 2.468 N_casas_agua_0 0.002 0.009 0.230 0.820 -0.015 0.019 N_casas_luz_0 0.007 0.006 1.190 0.232 -0.005 0.019 N_casas_esg_0 -0.005* 0.002 -2.090 0.036 -0.009 0.000 Dist 0.001 0.020 0.030 0.979 -0.038 0.039 Nº hab -0.003 0.002 -1.210 0.224 -0.007 0.002 hab/casas -0.245 0.207 -1.190 0.235 -0.650 0.160 at_com_2002 0.000 0.000 1.300 0.193 0.000 0.000 Constant 20.980 118.813 0.180 0.860 -211.889 253.849 *Standard errors in parentheses, corrected for heteroskedasticity. 119 Table 4.18 Weighting of control group units resulting from community matching Weight Frequency Weight*Frequency 1 14 14 2 7 14 3 1 3 4 1 4 5 1 5 7 1 7 Total 25 47 Figure 4.3 Propensity Score for communities before matching 0 1 4 3 Density 2 1 0 0 .5 1 0 .5 Probability of positive outcome Graphs by treat 120 Figure 4.4 Propensity score for communities after matching 0 1 4 3 Density 2 1 0 .2 .4 .6 .8 1 .2 .4 .6 .8 Probability of positive outcome Graphs by treat The variables of interest to the analysis of communities were: `total assets', which corresponds to the sum of community and household assets and the variables for infant mortality and incidence of diseases. Again, the econo- metric analysis used the difference of difference method, and, subsequently, analyzed the change in these variables between 2002 and 2005. However, as the number of observations is small in this case, only models including a dummy for the treatment were estimated. Variation in total assets and community assets Table 4.19 shows the impact of the program on the variation in total assets and community assets between 2002 and 2005. It is important to note that community assets used in this analysis do not include assets obtained/ constructed by the PCPR. 121 Table 4.19 Impact of PCPR on variation of community assets Sample resulting from matching Total assets = comunity+household Community Assets Treatment -11,829.66 -12,249.74 (9196.26) (9288.01) Constant 215.51 532.66 (115.16) (346.67) nº of observ. 72 72 F (k, n-1) 1.65 1.74 Prob > F 0.2026 0.1915 R-squared 0.0178 0.0187 *Standard errors in parentheses, corrected for heteroskedasticity. The coefficients relating to the treatment group, although negative, are not statistically different from zero for both variables of interest to the study. The fact that community assets account for, on average, 75% of the total assets explains the similarity between the coefficients. The negative result obtained may, in fact, indicate a "crowding-out" effect, that is, when the communities receive PCPR funding for their infrastructure projects, they transfer their own assets to ends other than the accumulation of commu- nity assets. It is worth remarking that the observation of the data shows that there is little variation in community assets. In Piauí, of the 15 communities that remained in the common support of the treatment group, two reported losses in various kinds of assets and one a gain in the case of only one asset; and, of the 10 selected from the control group, only two reported a gain in one kind of asset each. In Ceará, of the 17 communities from the treatment group, three reported a gain in the case of one kind of asset; and, of the 29 communities selected from the control group, three reported a gain in one kind of asset. Finally, in Rio Grande do Norte, none of the 15 communities from the treatment group reported a change in the value of assets; and, one of the eight from the control group reported a gain in one kind of asset. Variation in health conditions One of the possible positive external effects of the PCPR projects may be improving the general health of the population. For example, a household 122 having access to piped water can reduce the incidence of diseases associated with the quality of water, such as worms, as well as to decrease the infant mortality rate. The next set of results evaluates the variation in the incidence of cases of infant mortality and of diseases between 2002 and 2005, com- paring the treatment and control groups. It is worth pointing out that the information on health indicators was obtained from group interviews with various community members and leaders, including always the local health "agent". Thus, the data obtained about the variation of incidence of diseases and infant mortality represents the general perception of these groups. The dependent variable in Table 4.20, changes in number of cases of infant mortality, varies between 1 and 3, 1 signifying a reduction in the number of cases; two no change; and 3 a rise in the number of cases be- tween 2002 and 2005. Thus, if the coefficient for treatment is negative, it means that a larger number of communities in the treatment group, com- pared to the control group, perceived a reduction in the number of cases of infant mortality. We can therefore conclude that, according to the results, there was a greater reduction in cases of infant mortality in communities belonging to the treatment group than in those belonging to the control group. Although the coefficient is small, with a confidence level of 10%, it is statistically different from zero. Table 4.20 Impact of the PCPR on infant mortality ­ 2002-2005 D infant mortality Sample resulting from matching Treatment -0.190 (0.111) Constant 1.841 (0.077) Nº of observ. 65 F(k,n-1) 2.9 Prob > F 0.094 R-squared 0.043 *Standard errors in parentheses, corrected for heteroskedasticity. Another interesting aspect to analyze is the total incidence of disease, which cor- responds to the sum of the incidence of six types of illnesses: diarrhea, worms, dengue fever, hepatitis, asthma, and Chagas' disease. Table 4.21 shows the results for each of these illnesses and for the index of all six combined. In this case, we use an ordered logit regression. This time, the dependent variable also varies between 1 and 3, but is interpreted in the opposite manner; 1 signifies an increase in the number of cases, 2 signifies that the number remained constant, and 3 means that it went down. The procedure followed involves building up such an indicator for 123 each of the diseases and then calculating the sum of these individual indicators. If the treatment indicator is positive, it means that there was a reduction in the cases of diseases in the treatment group compared with the control. Table 4.21 Impact of the PCPR on the incidence of diseases ­ 2002 - 2005 Dengue Chagas' All D diseases Diarrhea Worms Fever Hepatitis Asthma Disease Diseases Treatment 0.350 0.467 0.022 1.040* 1.229* 1.219* 0.704 (0.592) (0.548) (0.550) (0.534) (0.538) (0.643) (0.512) _cut1 -2.215 -1.222 -2.010 -2.113 -1.572 -2.194 -3.462 (0.506) (0.480) (0.506) (0.506) (0.420) (0.494) (1.080) _cut2 -0.591 0.408 0.443 1.432 1.722 2.194 -2.503 (0.506) (0.502) (0.498) (0.473) (0.434) (0.494) (0.706) _cut3 -1.638 (0.499) _cut4 -0.331 (0.488) _cut5 0.249 (0.488) _cut6 1.000 (0.533) _cut7 1.574 (0.384) _cut8 2.319 (0.443) _cut9 3.202 (0.498) Nº of obs. 72 72 72 72 71 66 66 Chi2(k) 0.350 0.730 0.000 3.800 5.210 3.600 1.890 P > chi2 0.554 0.394 0.969 0.051 0.023 0.058 0.169 Pseudo R2 0.004 0.007 0.000 0.037 0.048 0.045 0.009 The coefficients estimating the various cut-off points are not shown in this table * Standard errors in parentheses, corrected for heteroskedasticity * indicates that the coefficient estimated is statistically significant at 5% For all diseases and, consequently for the total indicator, the coefficient is as expected, i.e. there is evidence that there was a greater decrease in cases of the illnesses in the treatment group than in the control. In the case of hepatitis, asthma and Chagas' disease, the coefficients were statistically dif- ferent from zero. 124 Summary of Main Results This study carried out an evaluation of the PCPR in the States of Piauí, Ceará and Rio Grande do Norte, analyzing the impact of the program on access to infrastructure services and the accumulation of assets at house- hold level, as well as the accumulation of assets and health conditions at the community level. According to the results obtained, the Program had a positive impact on access to electricity in Piauí and a positive impact on access to water in the State of Ceará. These results are, in principle, in accordance with what was to be expected, given the emphasis of the projects on electricity supply in Piauí and water in Ceará. As for the accumulation of per capita household assets, none of the coefficients associated with the treatment were statistically significant, but the effect was positive according to all the models estimated. The failure to obtain statistically significant results may be due to a number of factors. First, a rise in the income of families living in poverty is probably consumed rather than invested, so measuring only the impact on assets will not reflect the real effect on income. Secondly, the sample size is relatively small for measuring small increases in capital; and, third, the errors occurring when measuring recall data are very large and generate a wide variation. Therefore, more in-depth analysis of the effects on income and physical capital needs to be carried out by future studies. One possibility that could be easily implemented is to return to a sample for which data already exists on a baseline study of 3,000 households, col- lected in 2004 by Buainain and Fonseca (2005b) for treatment and control groups, which includes detailed information on income. This should be when sufficient time will have passed for the effects on income and capital to have occurred since the treatment communities received projects in 2004. The data reveal little variation in community assets between 2002 and 2005. In the case of health indicators, the estimates for all the indicators are as expected, with better health conditions in the treatment group com- munities than in the control group. It is important to note that the results are statistically significant for infant mortality (with a 10% confidence level), hepatitis (5% confidence level), asthma (5%), and Chagas' Disease (10%). 125 The Social Impact of the PCPR: Building Institutions, New Patterns of Social Relations and Social Capital Social capital is a key dimension in the PCPR's implementation strategy, which was designed to contribute to the processes of building up, strength- ening and employing social capital as an instrument for reducing poverty and improving the quality of life in the rural areas of Northeast Brazil. Considering this objective, early literature on the topic has empha- sized both the unfavorable historical, social and political context in which PCPR's interventions take place and PCPR's lack of efficient tools to overcome these traditional barriers.110 Hence, from the beginning PCPR was faced by a strong skepticism. More recent studies have changed this perspective and shown PCPR's success on building legitimate and represen- tative community associations, in increasing community participation and social control of public interventions, and in avoiding or, at least, reducing elite's capture. They bring evidence that community empowerment and effective action on the part of poor rural communities regarding their col- lective needs have been effectively increased as a major PCPR's contribu- tion.111 All these studies carried out up to 2005 have had methodological limi- tations that affect the reliability of their results. A major methodological shortcoming is that they do not take due account of the problem of sample selection bias or of the influence of other factors, observable or not, on social capital. Our analysis reduces these deficiencies and its results emphati- cally confirm the positive effect of the PCPR on social capital. They indicate that there has been a significant growth in the levels of participation in as- sociations and community activities and that communities have increased social control over the State and taken on new responsibilities as citizens. In the rest of this section we first make a brief definition of social capital and a brief review of the results of previous studies of social capital in the PCPR. Then we describe the methodology of analysis. The results obtained at the household, community and municipal levels are then pre- sented and discussed. 110 The Northeast rural poor communities would be an unfavorable context because of (a) long- standing tradition of dependence on local elites, (b) low levels of social capital and commu- nity organization, (c) lack of a tradition of citizenship, (d) distrust in relation to the State, and, (e) the ability of local and regional elites to reap the benefits of government investment and control institutional civil representation. A seminal and representative work on this literature is Kottak, Costa and Prado (1994). 111 Representatives of this new trend are Van Zyl, Sonn and Costa (2000) and Buainain and Fonseca (2005a). The initial skepticism has lost its cutting edge, but it did not entirely die away. On the contrary, it was recently refueled by research carried out by the World Bank's Operations Evaluation Department (Kumar: 2006). 126 A Definition of Social Capital We start out from a limited definition of social capital. In essence, here we use the concept to refer to the capacity of a social group or community to act collectively when collective action is needed. In order for this con- certed action to be possible there must be the motivation and the available institutional arrangements to do so. These two aspects are the inputs into social capital formation. The motivations that groups of people have for acting collectively, such as relations of mutual solidarity and coopera- tion are the most elementary inputs of social capital and are described in the literature as cognitive social capital (Uphoff: 2000; Grootaert & van Bastelaer: 2001). Collective action requires institutional arrangements that permit access to information, decision-making and to take action. This aspect is known as structural social capital. The capacity to act collectively has many effects on the participation level, here we emphasize two of them: community and civil participa- tion. The first is understood as the capacity of a group to act collectively whenever a concerted collective action is necessary to solve a community problem or to supply a community need. It reflects the forms of collective action and decision-making at community level. Civil participation refers to the action of individuals on issues involving the relation with govern- ment, especially those that involve exercising social control over the State. Community and civil participation are described as the main outputs of the social capital formation. Is it possible to build up social capital in rural poor communities in Northeast Brazil? In its analysis of the impact of the PCPR on social capital formation, the Quasi-Experimental study focused on four basic questions. Is it possible to build up social capital in poor rural communities in Northeast Brazil? If so, what is the impact of the PCPR on this process? What are the present trends in this respect? And what are the consequences of this social capital formation in poor rural communities in terms of municipal governance and management? In order to answer these questions, interviews were conducted at mu- nicipal, community and household level: at municipal level with the local authorities, representatives of civil society organizations and rural commu- nity associations with a seat and a vote on the PCPR Municipal Councils; at community level with the directors and members of the local commu- nity association benefiting from PCPR and other relevant stakeholders; 127 and at household level with the nuclear family resident in each household. These interviews and meetings covered a wide range of issues. Original Skepticism, its Revision and Resistance The emergence of an analytical concern with the issues of popular partici- pation, decentralization of decision-making, and empowerment of rural communities (which came later to be known as the issue of social capital) coincides with the adoption of the community-driven development (CDD) approach by the Rural Producers' Support Program in 1993. The search for an answer to the question of whether it is possible to generate sus- tainable and long-lasting social capital in this context was raised amidst profound skepticism regarding the appropriateness and efficiency of CDD local development strategies. This skepticism guided studies of the impact of the PCPR and the programs that preceded and followed it up to the end of the millennium. In subsequent years, studies questioned the legitimacy of community associations, municipal councils and subprojects implemented locally by the PCPR. They also questioned their capacity to facilitate genuine participation by the poorest sectors of the population, to represent their best interests, to make their decisions hold weight, and to generate new institutional arrangements and new forms of municipal governance that would be free of the bad habits of the past and would be sustainable in the medium- to long-term. However, since the end of the 1990s and as a result of more in-depth field research and the wider use of decentralized PCPR implementation strategies, a new analytical paradigm has begun to emerge. More evidences which was collected indicates that popular participation, community empowerment, legitimate representation and effective action on the part poor rural communities regarding their collective needs--in a word, social capital--has been effectively generated, strengthened and employed in the context of the implementation of PCPR subprojects. This formation of social capital has occurred because the Project rules (and essentially the distribution of its resources through the municipal councils with an over- whelming attendance on the part of representatives of community organiza- tions) have taken the material resources out of the hands of political actors with a potential interest in perpetuating practices involving political depen- dence. They have not only encouraged practices involving political equality and participation, but also led to the empowerment of the representatives of community associations and a weakening of the ties of personal dependence on political leaders, whose base of legitimacy depends on having a mo- nopoly in terms of access to public resources. Besides they have established an obligation to be transparent in decision-making and to make information 128 on the Program widely available to the public. Finally they have promoted a climate of mutual checks and balances, in so far as it is in the collective interest of each association to be ready to receive new subprojects (Buainain and Fonseca 2005a). Thus, the more recent literature clearly indicates that the initial skepti- cism in relation to the appropriateness and efficiency of the CDD-based local development strategies used by the PCPR has over time become less pertinent. The initial skepticism lost its cutting edge, but it did not entirely die away.112 All the studies of social capital carried out up to 2005 have method- ological shortcomings and this calls their conclusions into question. They do not take due account of the problem of sample selection bias and the influence of other factors, observable or not, on social capital. The 2005 Quasi-Experimental Study overcame these deficiencies and its results em- phatically confirm the positive effect of the PCPR on social capital, be it in terms of inputs (motives for acting collectively and institutional arrange- ments that facilitate collective action) or outputs (participation in com- munity activities, civil participation, and the reconfiguration of relations between the community and the government, or relations of governance). The results of this study indicate that there has been a significant growth in the levels of trust, solidarity and cooperation among residents of poor rural communities. Their networks of collaborative relationships with residents of other communities, government authorities and outsiders have also been strengthened. Participation in associations and community activ- ities has increased and a new way of individuals dealing--as individuals or through representatives of their organizations--with government authori- ties has emerged and developed, whereby social control is exercised over the State and responsibilities and duties are taken on as citizens. Overall view of social capital in the PCPR In the view of the stakeholders brought together in almost all the mu- nicipalities, the community-driven development approach achieves better results than local development projects implemented using a supply-driven approach, and, in most cases do so at lower cost. These stakeholders point out the sustainability of the majority of PCPR community subprojects and their strong social impact in terms of strengthening community as- sociations, improving quality of life, and increasing family income in rural areas. Although consistent with the results of the review of the literature, some of these estimates (chiefly those relating to the impact on income) 112 The critical view of the impact of the RPAP on community participation and social capital formation was refueled by research carried out by the World Bank's Operations Evaluation Department in 2004/2005 (Kumar 2005). 129 appear to be excessively optimistic when compared with the findings of the Quasi-Experimental Study (See Table 4.22). Table 4.22 Results Obtained at Municipal Level Questionnaire Item Results The CDD achieved better results than "supply-driven development" 98% The CDD approach achieved better results at lower cost| 85% The PCPR's community subprojects are sustainable 83% The PCPR contributed positively to strengthening community 85% associations The PCPR contributed positively to improving the quality of life 87% The PCPR contributed positively to increasing poor families' income 42% The institutional capacity of community associations has grown since 94% 2002 The PCPR contributed greatly to increasing institutional capacity 85% The evaluation of the community associations and the municipal councils linked to the PCPR is equally positive. In most municipalities it is believed that the capacity of community associations to represent the interests of the community, to identify local priorities, to mobilize local resources and raise resources from outside the community has increased since 2002. This process is closely associated with the PCPR and its implementation mecha- nisms (See Table 4.23). Table 4.23 Results Obtained at Community Level Results Com. Com. Questionnaire Items Overall 2002 2005 Decisions are taken democratically at PCPR councils 46% 44% 49% The number of members of the community associa- 46% 49% 43% tion has grown over the last three years The proportion of members who participate actively 36% 29% 43% in the association has grown over the last three years The proportion of members who contribute finan- 27% 22% 32% cially to the association has also grown Frequency at community association meetings has 31% 25% 38% been up The majority of residents take part in local decision- 64% 58% 70% making processes 130 The number of residents involved in local decision- 43% 36% 49% making processes has grown since 2002 This number has grown since the association was 61% 45% 77% set up The number of residents who participate in collec- 25% 16% 34% tive activities has grown since 2002 This number has grown since the association was 33% 25% 42% set up Associations encourage participation in decision- 61% 45% 77% making processes Community associations are capable of helping resi- 72% 75% 70% dents who need help Community associations represent local interests ap- 87% 80% 94% propriately Community associations effectively resolve local 71% 67% 75% problems Associations mobilize resources in the community 44% 38% 49% for collective activities Associations cooperate with other social groups and 54% 42% 66% communities Community association leaders have great influence 54% 44% 64% on community life The influence of leaders has risen over the last three 40% 25% 51% years The influence of leaders in conflict resolution has 23% 18% 28% grown over the last three years They have great trust in working in partnership with 44% 40% 47% the municipality and the State Their trust in working with the municipality and the 33% 22% 45% State has grown over the last three years The number demands made by the community to 54% 50% 58% the municipality has grown since the association was set up At community level there is also a positive view of institutional arrange- ments that the PCPR's implementation strategy promotes. The levels of participation of the residents of rural communities in associations, col- lective decision-making and collective activities have grown. Community associations and PCPR municipal councils are perceived as being inclusive, participatory and democratic institutions, where decisions are made by the 131 majority.113 The institutional capacity of the associations--and, in partic- ular, their capacity to represent the community and solve its problems--is deemed to be high114 and to have grown significantly over the last three years (an increase of 33%). Most consider that community associations are very influential in the life of the community, that they are chosen democratically (a view shared by 91% of the communities) and that their influence has grown, both in terms of resolving internal conflicts, and with regard to matters of general interest to residents. Above all, the growing strength of associations and their broad participation in municipal councils has brought about a great change in relations between rural communities and the State authorities.115 This process is revealed by the perception of a rise in the municipal public authorities responding to demands or requests on the part of the community and a resulting rise in levels of confidence in government and in their willingness to forge partnerships with government (See Table 4.24). Table 4.24 Results Obtained at Household Level Results Com. Com. Questionnaire Items Overall 2002 2005 Families interviewed that participated in the 86% 87% 85% meeting to choose the PCPR subproject Families who said they approved of the 93% 92% 94% community demand selected 113 In 26% of the communities with representatives on the FUMAC Municipal Councils, the mayor of the municipality or his/her representatives was stated as being the stakeholder with the greatest influence over the decisions of municipal councils. In 31% of them, this status was attributed to a president of a community association. In communities benefiting from the RPAP in 2002 with representatives on these councils, the status of the most influential council-member was attributed to the mayor of the municipality or his/ her representatives in 26% of cases and to the president of the community association in 35%. The same proportion of communities benefiting in 2005 (26%) attributed this role to the representatives of the municipal authorities and to the presidents of community associations. These figures allow us to refute the serious criticisms regarding the lack of legitimacy and the control of these institutions by local and regional elites or by those holding political office, which were the main pillars of the skepticism that originally prevailed in the literature on the RPAP and its influence on social capital. They also allay the fears that the RPAP municipal councils were being "taken over" by the Mayor's office because of the direct influence of the mayor on the choice of council-members and on the decision-making process. Although this still occurs, it is far from being the norm. 114 These opinions were voiced by 87% and 71% respectively of community meetings 115 In 58% of communities the growth in the capacity of the community to make complaints and receive positive answers from the municipal authorities is associated with the setting up of the local community association and, in more than 18% this is considered to be a phenomenon that arose after 2002. In the communities served by the RPAP in 2002, these figures stand at 55% and 17%, respectively, and in the communities served in 2005, at 62% and 18%. 132 Results Com. Com. Questionnaire Items Overall 2002 2005 Families satisfied or very satisfied with the opera- 93% 92% 96% tion of the subproject18 Decisions are taken democratically at the commu- 71% 71% 70% nity associations Associations strengthen ties of friendship between 97% 96% 98% residents Associations help people in need 86% 86% 87% Associations promote collaboration between com- 86% 85% 87% munities Community associations contribute to improving 78% 79% 77% family income Community associations are capable of bringing 93% 92% 95% subprojects that benefit the community Community associations are capable of 94% 93% 96% representing the community's interests Community associations are capable of solving local 80% 78% 83% problems There has been a variation in their capacity to 14% 9% 19% solve local problems over the last three years Families participate in collective activities in the 68% 66% 69% community Family participation in collective activities has 32% 34% 29% increased over the last three years Family participation in collective activities has 40% 41% 39% increased since the association was set up Solidarity among community residents has in- 20% 17% 22% creased over the last three years Solidarity among community residents has in- 29% 26% 31% creased since the association was set up Trust in residents of other communities has in- 15% 15% 14% creased in the last three years Trust in residents of other communities has in- 17% 17% 18% creased since the association was set up Trust in government authorities has increased over 22% 21% 22% the last three years Trust in government authorities has increased since 21% 19% 22% the association was set up 133 Results Com. Com. Questionnaire Items Overall 2002 2005 The interviewees have great trust in the leaders of 61% 63% 58% associations The interviewees trust that their decisions will 81% 82% 78% benefit most or all residents The leaders of associations are highly respected by 81% 81% 80% their peers Leaders have more financial resources than most 28% 27% 30% residents The influence of leaders in conflict resolution has 24% 21% 26% risen over the last three years Leaders help to resolve most conflicts between 39% 43% 35% residents At household level it is revealed that the levels of formal and genuine participation of beneficiaries in identifying local priorities and selecting the community action to be implemented with PCPR resources are high.116 Equally high are the numbers of those satisfied with the demands made by the communities and the investment carried out.117 Moreover, community associations and their leaders have assumed a central role in organizing community action and representing their home communities. They have won the respect and trust of residents and are highly regarded among them. The setting up community associations is also strongly linked to the perception of a growth in levels of trust, solidarity, cooperation and par- ticipation in collective activities at local level and beyond. Moreover, this view prevails among the greater part of those interviewed. 116 This result differs radically from the findings made by a recent World Bank OED study (Kumar et al., 2005, p. 138 and Figure N.1) available at http://www.worldbank.org/oed/ cbdcdd), carried out in the State of Rio Grande do Norte, which involved interviews with households that benefited from RPAP subprojects and those that did not. This study iden- tified that only 37% of those interviewed remembered having participated in community meetings to choose local subprojects and 22% having made an active contribution to these meetings. This led the authors to conclude that the RPAP and the CDD approach of promoting the genuine participation of rural populations, creating and increasing com- munity social capital are incapable of producing results. However, as the interviewees included a high percentage of people not benefiting from the Program, it was only to be expected that not many would participate in the meetings. 117 In communities benefiting from the PCPR in 2002, 63% of householders interviewed claimed that they were very satisfied, 29% satisfied, and only 8% said they were dissatis- fied with the functioning of the locally implemented PCPR subproject. In the communities benefiting in 2005, while 42% of householders interviewed were unable to give an opinion on why the local subprojects were still at the implementation phase and were not yet operating, 56% (that is, 97% of those who expressed an opinion on the functioning of the subprojects) claimed that they were satisfied or very satisfied. 134 Most residents describe community leaders as equals.118 Seventy-six percent of households interviewed stated that they chose their community leaders democratically. With reference to the criticism that members of the local elite take over community associations, the study reveals that most households viewed their leaders as more knowledgeable and learned, having more contact with influential people, and more time and willing- ness to dedicate themselves to the community than the average resident. However, only a little over one-quarter of those interviewed considered them to have more in the way of financial resources than the majority of residents of the community (Table 4.25) Table 4.25 Distinctive Characteristics of Community Leaders Communities Communities benefiting in benefiting in Characteristics General 2002 2005 More knowledge and Study 66% 69% 64% More time and willingness 73% 73% 74% to dedicate themselves to the community More contacts with influential 76% 77% 76% people More financial resources 28% 27% 30% The PCPR is, therefore, associated with a new institutional arrange- ment to which positive impacts on public administration, the sense of citizenship, and thereby social capital are attributed. To what extent are these opinions valid and justified? The answer should be found by trying to measure the stock of social capital and its variation over time and by analyzing the measurable impact of the PCPR on this process. Measuring Changes in Social Capital: Analytical Procedures In this study, the four dimensions of collective action described above: two inputs and two outputs of social capital, are measured by use of aggre- gated indices (see below and Table 4.26). These are used to measure the evolution of social capital for beneficiaries of the PCPR over time. Given 118 Information collected on household assets are inconclusive regarding the similarities or differences in terms of the living conditions of those who are involved to varying degrees in community association activities. 135 that the control group used in the other analyses in this paper (the pipeline group) does not serve as a control for social capital because the influence of the PCPR on social capital starts at the moment these groups enter the "waiting list", the analysis uses another strategy to control for the effect of factors other than the PCPR on social capital. We estimate a tendency over time, or the time trend for social capital that occurs exogenously to the PCPR. The estimation of this time trend uses data from the same communi- ties investigated, but from a time prior to the effects of the PCPR. It uses information on the stock of social capital existing before community as- sociations were set up­ which occurs always before the PCPR subprojects can be obtained. In our sample this covers a period of 26 years (1979 to 2005). Since information on social capital in the period before the forma- tion of associations was obtained by recall and it is likely that memories of specific periods in the remote past are very unreliable, this study used only observations in communities whose associations had been set up after 1993 (when the PCPR adopted the CDD approach). If the time trend is positive, the results obtained from the compari- sons over time among beneficiaries would overestimate the real impact of PCPR; if the tendency is negative, the results would be an underestimate; and if there is no tendency, the results of the comparisons are the real mea- sure of the impact of the PCPR on social capital. The analysis of the evolution of social capital for beneficiaries looks at changes between (a) the period prior to the setting up of the community association, (b) the year 2002, when the subprojects were implemented in the communities that first had access to the PCPR (in this section referred to as the Communities 2002), and (c) the date field work was carried out, September and October 2005, when the subprojects were being imple- mented in the most recently benefited communities (in this section referred to as Communities 2005). The analysis is based on intervalar indices [0, 1] that are additive and reflect each of the dimensions of social capital described above, as well as a general index. Table 4.26 gives an overview of the household level vari- ables used to draw up the indices. 136 Table 4.26 Indicators of stock of the social capital and its four basic components Indicators Variables measured at each of the three points in time participation of families in community or voluntary Cognitive social capital activities, serving as an indicator of intra-group solidarity and cooperation level of access to relevant information regarding the structural social capital public sector, and participation in inter-community cooperation activities involvement of the majority of community residents community participation in collective responses to crisis situations that re- quire concerted group action participation in public fora for deliberation and decision-making and the intensity of regular contacts civil participation between family members and the authorities to demand action to benefit the community in which they live index of social capital simple mean of the four basic components Based on the indices at each point in time, three indices for changes over time were created for each of the household units: (1) the change between the year the community association was set up and 2002 (DT1); (2) the change between the year the association was set up and 2005 (DT2); and (3) the change between 2002 and 2005 (DT3). Based on these, we identi- fied five types of impact: · Total Short-Term Impact (STI), which can be measured in Communities 2002 and Communities 2005 and corresponds to the cumulative impact of setting up the community association and the implementation of the PCPR project; · Total Long-Term Impact (LRI), only measured for the Communities 2002; corresponds to the cumulative impact of setting up the com- munity association, implementation of the PCPR project, and the additional variation that occurred until 2005; · Additional Variation (AG), only measured for the Communities 2002, corresponds to the evolution in the period between implementation of the PCPR project and 2005; · Direct Impact of the Implementation of the project (PI), which can only be measured for the 2005 group in which associations were set up before 2002; and, 137 · Impact due to the Setting up of the Community Association (AFI), can only be measured for the Communities 2005 which set up its associa- tions before 2002. For each one of the two types of communities (Communities 2002 and Communities 2005), each variation over time relates to a distinct type of impact. Table 4.27 outlines the types of impact and the source of data used for each type of community. Table 4.27 Measurement of Impact of PCPR on Social Capital Time interval used Communities Communities Measurable Impacts 2002 2005 Total Short-Term Impact (STI) DT1 DT2 Total Long-Term Impact (LRI) DT2 Additional Variation (AG) DT3 Impact of Implementation of PCPR Subproject DT3 (PI) Impact of Setting Up of Community DT1 Association (AFI) Estimating the Time Trend for Social Capital Table 4.28 below presents the time trend results for the mean index of so- cial capital. The estimation used information on the stock of social capital before the setting up of community associations; the year these organiza- tions were set up ­ over a 12-year period (1993 to 2005); and a propensity score index estimated specifically for the analysis of social capital119. The results clearly indicate that, over the last 12 years, social capital has not changed in rural communities of Northeast Brazil independently from the PCPR, as the estimate of the variable "trend" is not significantly different from zero.120 119 This estimation is analogous to table 10. The only difference is that all variables related to the process of community organization and participation were excluded since we are measuring the impact on social capital. 120 The same result was obtained when this exercise was applied using all the observations covering the period from 1979 to 2005. 138 Table 4.28 Estimates of the time trend of social capital Number of obs = 780 Regression with robust standard errors F( 2,777) = 2.60 Prob> F = 0.0747 R-squared = 0.0068 Root MSE = 0.25111 Dependent Variable = `index of social capital Robust - isc' Coef. Std. Error t P>|t| [95% Conf. Interval] trend 0,00246 0,028 0,9 0,929 -0,00521 0,00571 ptreat 1 0,886 0,389 2,28 0,023 0,0122 0,1651 constant 0,845 0,581 4,90 0,000 0,1705 0,3986 1.The variable ptreat is an estimate of the probability of belonging to the 2002 group of communities, a result of the propensity-score analysis specific to social capital. This estimation was repeated separately for each of the dimensions of so- cial capital and obtained identical results. Therefore, the measured varia- tions in social capital over time among beneficiaries can be attributed to the setting up of community associations and the implementation of PCPR community subprojects. These are presented below. Social Capital Impacts at Household Level The indices measuring the stock of social capital at each point in time are presented in Table 4.29. Table 4.29 Stocks of Social capital according to time period and type of community1 Group of 2002 Group of 2005 Point Overall communities communities in T of T of T of Indicators Time Mean T-test Mean T-test Mean T-test Social capital T1 0.37 -13,33 0.40 -7,75 0.35 -11,12 index T2 0.45 -4,64 0.47 -2,21 0.44 -4,30 T3 0.52 1,72 0.52 1,54 0.51 0,89 1. T-tests of hypothesis: mean=0 T1 ­ Time of formation of community association T2 ­ Time of 2002 World Cup T3 ­ Time of research (2005) 139 These results suggest that social capital has a general tendency to grow. Also, the stock existing at the time of the formation of the community association was greater in the Communities 2002, suggesting that com- munities holding more `social capital' were able to organize and access PCPR's resources faster and earlier. Still, in 2005 both groups of com- munities have similar social capital suggesting a "catch-up" process by the Communities 2005 group. Based on the existing stocks of social capital at each of the three points in time, it is possible to build up indicators of changes over time. Since there has not been, over the last 12 years, any underlying independent ten- dency to build up social capital, as concluded by the time trend analysis, these are measures of the impact of the PCPR on social capital. Table 4.30 presents the results of this analysis using the definition of impacts described in Table 4.27 above. Table 4.30 Measurement of Impact of PCPR on Social Capital 2002 Group of Communities 2005 Group of Communities Impact Mean T of T-test Mean T of T-test STI 26.4% 8.05 45.7% 12.67 LRI 42.8% 10.82 AG 19.3% 6.50 AFI 19.9% 6.54 PI 33.2% 8.97 In view of the scale of the measured changes in social capital, the results signal that a process of intensive growth in the stock of social capital is un- derway and that the PCPR is having an enormous influence on this. They show that the formation of social capital reaches its peak rate of growth during the implementation of community PCPR projects (as expressed by the high value for PI in the 2005 Communities), when there is an intensi- fication of community participation and relations of co-responsibility with structures of government in the execution of public policy. This contrasts with the traditional relations of dependence of communities on a patron- izing provider State. After this phase, social capital growth continues but at a slower pace (as suggested by the value of AG for the 2002 group), which indicates that the social capital created is used less than it could or should be, but that it does not revert to earlier levels either. Likely the lack of further and broader stimuli to collective action and joint responsibility 140 in the management of public policy is a reason for a later slower rate of growth. Comparisons across time were also obtained for each of the four com- ponents of social capital and the social capital inputs and outputs (see tables 4.31 and 4.32). In relation to cognitive social capital, the results reveal a tendency for growth that is higher between the time the community associa- tion was set up and 2005, than between 2002 and 2005, and higher in the 2005 communities than in those from 2002. The results relating to struc- tural social capital reveal that it too has grown and has continued to grow after the implementation of the projects took place, though more rapidly under the direct stimulus of the projects' implementation. Table 4.31 Measurement of the Impact of the PCPR on Social Capital Inputs 2002 Group of Communities 2005 Group of Communities Impact Mean T of T-test Mean T of T-test STI 4.2% 2.72 14.6% 5.43 LRI 7.5% 3.92 AG 3.7% 2.72 These results on social capital inputs refute the key hypothesis of the early literature and critics of CDD programs, which indicate that after imple- mentation of projects there would be a reduction in the levels of involve- ment and participation of communities. Although a euphoria effect has been identified at the time of implementation of PCPR projects, at which time social capital growth reaches its peak, there is no subsequent reduc- tion in the level of motivation or in the number of institutional arrange- ments. On the contrary, after the implementation of projects, social capital inputs continue to grow. This suggests that, once the process has been set in motion by the PCPR, its effects are long-lasting. This suggestion is reinforced by the findings regarding social capital outputs. Community participation has grown significantly in communities benefiting from the PCPR, at a faster pace in communities that are imple- menting the PCPR at present than in those that implemented it three years ago, but it is still growing in both groups. The indicators for change over time in levels of civil participation reveal rates of growth that are always positive. 141 Table 4.32 Measurement of Impact of PCPR on Social Capital Outputs 2002 Group of Communities 2005 Group of Communities Impact Mean T of T-test Mean T of T-test STI 16.4% 4.46 22.1% 5.39 LRI 31.4% 6.11 AG 12.8% 3.73 These results reveal also that the relative impact of the PCPR is greater on social capital outputs than on inputs, as can be seen in the comparison of tables 4.31 and 4.32 where the mean impact for each type of impact in both community groups is larger for outputs. It suggests that the PCPR uses the elementary forms of feelings of reciprocity existing in the commu- nities and invests these to increase both community and civil participation. Generally, the stock of social capital outputs, particularly civil participa- tion, in rural poor communities in the Northeast is lower than the stock of inputs, reflecting the traditions that for so long posed obstacles to any attempts to introduce new governance structures and the development of citizenship. Thus the importance of interventions such as those sponsored by the PCPR which aim to overcome these obstacles. The greater impact of the PCPR lies in the use of new institutional arrangements to help commu- nities become more active, to engage them in networks of inter-community collaboration, as well as to transform their relations with the State from one of dependency into a partnership, from one of mutual risk to one of mutual trust. Be this as it may, our findings are a clear rebuttal to the skepticism with which part of the literature considers the ability of PCPR and CDD method- ology to generate social capital. As we saw in a previous chapter, it has been argued that PCPR uses rather than creates social capital; in consequence, it performs better just on the social scenarios in which a certain amount of social capital has been already developed. This conception is partially true and partially false. Essentially it is mislead by a superficial analysis of social capital formation. What our refined conception of social capital ­ which distinguishes social capital inputs from social capital outputs and, thus, con- siders the different dimensions or phenomena collapsed under the concept of social capital ­ and our data suggest is that, indeed, PCPR uses some inputs of social capital ­ internal amounts of trust, solidarity and coopera- tion that generate a sense of community without which no social group can exist as a community ­ and performs better when they are higher; neverthe- less, PCPR renders them stronger and converts them in meaningful outputs that previously were too weak or absent in the beneficiary group ­ namely, 142 community participation in issues of public interest ordinarily conceived of as the realm of state intervention and civilian participation.121 Results for the Community The research at community level involved a meeting with at least ten participants, including community association leaders, teachers and head teachers from the local school, community health officers, residents and beneficiaries of the PCPR subprojects. The absence of data on the stock of social capital before the setting up of the community association made it impossible to attempt to establish the endogenous tendency over time to build up social capital in the case of data collected at community level. The analysis is based on comparisons over time and comparison of the differences in the variations over time between the 2002 and 2005 Communities. It employs two types of indicators: one of the stock of com- munity participation existing at each of the three points in time under con- sideration and those of the perception of changes in the three components of social capital. The indicator for the stock of community participation was drawn up using the same type of variable employed at household level -- the typical reaction of the community to a hypothetical crisis situation which requires a response involving collective action (See Table 4.33). 121 In the poor rural Brazilian Northeast society, the sentiment of community and the soci- etal arrangements it generates constitute a kind of last capital owned by families bereft of any other sort of resources. Poor rural families from the same community frequently organize collaborative activities to prepare land for planting crops and help each other to confront crises such as the regular droughts and their devastating economic impact. Therefore, in this social context, it is unquestionable that there is to some extent and as the key condition of existence for these poor rural families, a certain quantity of elementary social capital accumulated in their motivation to act collectively. It is also un- questionable that the ability to mobilize this stock of social values is fundamental for the success of community demand driven development projects. Nevertheless, it is also true that relations with public authorities and state institutions have historically been marked by a high and justifiable level of fear and distrust. They have always been conceived of as risky and such feelings were justified by the traditional use of public agencies and resources by the local elites to maintain and reproduce the conditions of dependence, sub- ordination and poverty of the poorest stratum of society, to get their hands on the larger share of benefits deriving from government investment and to protect their social, eco- nomic and political advantages at the top of a rigidly hierarchical society. It is this lack of trust between poor community and state agencies that development projects driven by community demand, such as the PCPR, needed to defy, have been attempting to oblit- erate, and now seem to have been able to transform by (i) establishing networks of col- laboration between the poor rural communities and wider ranging social and government institutions, (ii) breaking with the tradition of political clientilism and (iii) redefining the patterns of vertical social relations between poor rural communities and the state in the Northeast region. 143 Table 4.33 Measurement of the Levels of Community Participation: Communities Communities Overall 2002 2005 T of T of T of Point in Time Mean T-test Mean T-test Mean T-test Setting up of 34.3% -3.43 27.3% -3.75 41.5% -1.24 Association 2002 World Cup 51.8% 0.38 43.6% -0.94 60.4% 1.53 Final Field Research 57.4% 1.55 45.5% -0.67 69.8% 3.11 (2005) The results for the community show identical tendencies and similar fig- ures to those obtained at household level. They reveal a general tendency for growth in community participation, but also show that the 2005 com- munities started out with a greater stock and have now increased their advantage over the 2002 communities, as their rate of growth for the stock of community participation was also much greater over the periods of time under study. These results should be compared with those for households, as, in relation to the figures for the existing stock of social capital, it can be observed that the members of the 2005 communities brought together for the community interview are much more optimistic than the family groups interviewed at household level regarding the levels of community participation existing in 2002 and 2005, as at household level rates of 51.9% and 55.8%, respectively, were recorded. There is no significant difference in the evaluation of community participation at the time the as- sociation was set up, as the result obtained at household level was 40.6%. In the 2002 communities, the opposite trend is observed. The members brought together for the community interview made a much more critical assessment than those interviewed at household level regarding the existing stock of community participation at each of the three points in time. At the household level, these figures were 38.8% for the time the association was set up, 48.0% for the time of the 2002 World Cup Final and 51.8% for the present day. Such a difference in attitude between those interviewed at community level in the 2005 and 2002 communities must be related to the euphoria effect presently being experienced by the 2005 communities as a result of the introduction of the PCPR subprojects. This relationship is expressed in the results arising from an additional question posed at com- munity level immediately following the battery of questions on the typical 144 response of the community to crises. This question dealt with situations that required collective action similar to those mentioned in the previous questions that the community had in fact lived through in the last twelve months and on the attitude of residents to this. While 77% of the 2005 communities identified such an occurrence, only 45% of the 2002 com- munities did so. Implementation of the PCPR subproject is responsible for such a big difference. Significantly, however, in the proportion of communities in which it was stated that such an event had occurred, the percentage of responses that affirmed that the majority of residents had collaborated in solving the problem was 74%, with a slight difference between the 2002 communities (72% of the communities in which a similar fact was claimed and 33% of all the communities) and the 2005 communities (76% of the communities in which a similar fact was claimed and 58% of all the com- munities). Equally significant results were obtained when the methodology ad- opted to analyze the impact of the PCPR on these indicators was applied. These results are presented in Table 4.34 below. Table 4.34 Measurement of the Impact of the PCPR on Community Participation at Community Level 2002 Group of Communities 2005 Group of Communities Impact Mean T of T-test Mean T of T-test STI 16.4% 3.25 28.3% 4.53 LRI 18.2% 2.84 AG 1.8% 0.44 AFI 18.2% 3.09 PI 6.8% 1.77 It can be verified, therefore, though with less confidence (given the reduced size of the sample) that community participation continued to grow in the 2002 Communities group after the implementation of the subproject, but at a much slower pace. It can also be seen, again corroborating the analyses carried out at household level, that the euphoria effect caused by the imminent implementation of the PCPR subproject in the 2005 com- munities, contributed positively to increasing their advantage in terms of community participation compared with the 2002 communities. Without further stimulus, the community participation of these communities did not decline, but grew much more slowly. 145 What, then, do the broader community indicators of the perception of changes in terms of social capital and its components reveal? They were drawn up using the same procedures adopted at household level. The in- dicator of perception of changes in cognitive social capital used a variable that gauges the opinion of those interviewed regarding the evolution of feelings of solidarity in relation to those in need. The indicator referring to structural social capital uses a variable that gauges opinion regarding the evolution of the level of participation of local residents in decision-making processes regarding issues of interest to the community.122 The indicator for the perception of changes in community participation used a variable that reflects opinions regarding the evolution of genuine contributions on the part of local families to activities that aim to improve the quality of life of the community. Finally, the indicator referring to civil participation reflects opinions regarding the relations between the community and the municipal public authorities in terms of the evolution of the capacity of the municipality to meet the demands of the community. The results of this analysis are presented in Table 4.35 below. Table 4.35 Indicators of Perception of Change Communities Communities Overall 2002 2005 T of T of T of Indicators Period Mean T-test Mean T-test Mean T-test Cognitive social DT1 0.315 5.86 0.346 4.16 0.283 4.16 capital DT2 0.389 6.81 0.327 4.19 0.453 5.43 Structural social DT1 0.278 4.08 0.164 1.64 0.396 4.37 capital DT2 0.509 7.82 0.273 2.67 0.755 11.53 Community DT1 0.204 2.89 0.055 0.65 0.358 3.22 participation DT2 0.556 6.82 0.564 4.77 0.547 4.85 Civil participation DT1 0.083 1.32 -0.018 -0.22 0.189 2.02 DT2 0.462 7.38 0.423 4.79 0.500 5.62 Social capital DT1 0.220 5.41 0.136 2.48 0.307 5.27 index DT2 0.486 11.98 0.404 7.13 0.567 10.09 122 Numerous questions were also made regarding the current levels of involvement of residents in the community association and on the evolution of this since the time of the 2002 World Cup Final. It made no sense to use the time of the setting up of the associa- tion as a point of reference for collecting this information, and so these could not be in- cluded in the indicator for the perception of changes in structural social capital and could not be used in this analysis, except in a subsidiary fashion. 146 The first fact that stands out from these results is that, at the community level, there is a consensus of opinion that social capital and its principal components have evolved positively in the rural communities studied. This agrees with the perception of change observed in those interviewed at household level, but it is significant that the community indicators tend to be higher than those measured at household level. There is also a consensus that changes during the DT2 period (since the setting up of the association) were greater than those during the DT1 period and this result is the opposite of that obtained at household level. How is this to be explained? First, the collective memory regarding facts about the community may be keener than the memory of individuals, principally because groups of people who are more active in community affairs probably took part in the meetings. The memory of one individual sparks off that of another and the group as a whole fills in the gaps in the memory of each individual. Secondly, it is probable that individuals emphasize the institutional arrangements for collective action which they participate in more when they are together with others. Whatever the reason, the conclusion is that when residents of these rural communities in Northeast Brazil meet together, they tend to confer on the process of setting up community associations an enormous responsibility for trans- forming the lives of rural communities, the horizontal relations between their members and the vertical relations between their members and other social agents, and much more emphatically than they do when consulted in private. This process can, therefore, be seen as an essential step in building up social capital in the rural parts of Northeast Brazil. Equally fundamental is the fact that the perception of changes is much greater in the 2005 than in the 2002 communities. There is also a differ- ence in this respect in relation to the indicators for community and house- hold level, but this acts to corroborate and strengthen the results arrived at through analysis of the indicators for change in the stock of social capital at household level. The results are also reinforced by the fact that there is a much greater difference between the 2002 and the 2005 communities in terms of perceptions of changes in social capital outputs than is the case with inputs. At the community level, we find, for the first time, signs that there is a decline in the indicators for social capital among the communi- ties studied some time after the implementation of the subproject. The observation that there is a tendency--albeit tiny--for a more widespread perception of a decline since 2002 in civil participation in the 2002 com- munities is consistent with the slowing of the pace of growth in the case of this component, which was noted using the measurable indicators of change at household level. As a whole, then, these results are consistent 147 with the thesis we have put forward here regarding the euphoria effect arising from the implementation of PCPR subprojects. Two further aspects of this analysis of the results for communities should be pointed out. One relates to the strengthening of community as- sociations as institutions, the other to the perception of the consequences of this. The results for the communities do not only show that people see a very strong association between the setting up of a community association and the strengthening of social capital in these communities. They also suggest that these organizations achieve a wider reach within their home communities and greater participation in their activities on the part of the local population. These factors can be measured by the variation in formal and genuine participation. As shown in Table 4.35 above, the results for the community show that the number of members, the intensity of their participation, and their contribution to the activities promoted by commu- nity associations grew significantly in the last three years. Furthermore, the results for the community (as well as those for house- holds) show that these subjects understand the strengthening of community associations as a process that strengthens their institutional capacity and legitimacy as tools for representation and their efficiency as instruments for solving local problems. As this view is identical to that seen in the results of the research at household level, it can be concluded that local people see the community associations as having more than a crucial role in building up social capital at community level. In their view, they are becoming increas- ingly capable of representing the interests of the community in government circles, of exercising control over government officials, and of obtaining ef- fective responses to the legitimate collective demands of the community.123 From this point of view, which is widely shared by local people, the new institutional arrangements set up or strengthened by them are as- suming growing legitimacy and efficiency in representing their communi- ties and populations. As collective agents they are succeeding in doing what individual families do not do very much--i.e. participating in civic action, putting pressure on government officials, making them respond 123 As seen above, in Tables 4.23 and 4.24, for 87% of the communities (80% of the 2002 Communities and 94% of the 2005 Communities) and for 94% of those interviewed in the household research (93% of the 2002 Communities and 96% of the 2005 Communities), the community associations are capable of adequately representing the interests of the communities. Furthermore, for 71% of the communities (67% of the 2002 Communities and 75% of the 2005 Communities) and for 80% of those inter- viewed in the household research (78% of the 2002 Communities and 83% of the 2005 Communities), they are capable of effectively and efficiently resolving local problems. The differences between the 2002 and 2005 Communities are once again due the euphoria effect associated with implementation of the subprojects, with the lack of new stimuli causing the positive evaluation of community associations by the their own members to wane with the passage of time. 148 to collective demands and changing the traditional pattern of vertical relations between the communities and the State. If it is accurate, this per- ception would mean that, although the civil participation of family units remains (as we have seen) underdeveloped, the community associations ­ as legitimate intermediaries in vertical relations between poor communities and the State ­ have become capable of providing their communities with the sufficient and adequate means to participate in civic life. This would mean that, in this context, where traditionally each family is too poor or weak to stand up to the local elites and the impersonal rules of the state, without having recourse to relations of dependence or a scheme involving trading favors for subordination, civil participation and the exercise of citizenship emerge through and result from the feeling of community or- ganization and community participation. In this context, what continues to be weak at the level of individuals and families becomes firmer when action is taken by a collective subject, as the association can represent the community before the State and obtain responses in a way that individuals do not have the necessary strength to do. In a word, it is the whole that operates in civic life and not its fragile parts, and it acts effectively in so far as it is capable of transforming prevailing patterns of vertical relations between poor rural populations and communities and local elites and the state authorities. The adequacy of this point of view is the subject of the next section, in which attention is turned to the results of the research at municipal level. First, however, it should be pointed out that this optimistic view of community associations' capacity for civil representation and solving local problems does not cancel out the awareness of the difficulties that typi- cally surround relations between poor rural communities and the state. Questioned regarding the frequency with which community demands are met by the municipal government, only 15% of the communities stated that their demands are very often or frequently met. While more than 60% admitted that their demands are occasionally met, 19% stated that they are never met and around 6% responded that they do not normally de- mand anything of the municipal government. It is likely that the euphoria effect led to a more positive perception of the capacity and interest of the municipal government in the 2005 communities than in the 2002 commu- nities, but the point worth making is that, although the communities are better organized, feel more capable of exercising citizenship and confident in their capacity to obtain a response from the municipal authorities and forge partnerships with them to solve local problems, the capacity of the latter to respond to these demands is still very low. This relative incapacity is the result both of the severely restricted budgets of these small-scale 149 municipalities, and, chiefly, of the traditional models of municipal adminis- tration and governance, which are not open to popular representation and participation. This perpetuates the challenges which poor rural communi- ties face in attempting to make the most of their existing stock of social capital in the absence of any external stimulus and underlines the impor- tance of public policy guided by the principles of popular co-participation, such as the PCPR. Results for the Municipalities At the municipal level, information was collected by way of meetings with representatives of a variety of institutions and segments of society at the PCPR Municipal Councils in order to analyze collective perceptions of changes in the levels of community and civil participation and of the impact they have had on municipal administration. Perceptions of the social processes associated with vertical relations between rural communi- ties and municipal governments forged by the social players brought to- gether at municipal level are consistent with the perceptions expressed by households and by the community level interviews about how community associations operate as an instrument of representation and civil participa- tion. Information collected about the operational and political representa- tion of poor rural communities by legally-established government entities confirms the limitations of these entities. It also stresses the importance of new--perhaps parallel--institutional arrangements to override the tradi- tional forms of political domination in the rural Northeast and to make it possible for the interests of the poorest strata of the population to be expressed in civil life in a proactive manner. On the one hand the evaluation carried out by the social players brought together at municipal level was positive with regard to the institu- tional capacity of community associations and their impact on municipal governance. In most municipalities it is believed that: · most rural communities are now capable of identifying their needs and establishing priorities; · most of the leaders of community associations are capable of impar- tially representing the interests of their peers and keeping the latter well-informed with regard to issues of public interest; and · the capacity of community associations to represent the interests of the community, to mobilize resources locally and obtain resources from outside has grown since 2002. In most municipalities the view also prevails that there have been more meetings between community representatives and the municipal authori- 150 ties since the time when the PCPR Municipal Council was set up. These meetings, the strengthening of community associations and the work of the municipal councils within which they operate have given rise to profound changes in municipal administration. They are believed to have led to the introduction of more transparent procedures, greater social control over decisions made by municipal officials and a greater interest on the part of the latter in meeting the demands expressed by communities. Thus, in 79% of municipalities it is believed that the organization of the communities has had a positive effect on municipal administration and in 81% it is believed that the setting up of the municipal councils has had the same effect. In only 16% of municipalities was it concluded that neither community organiza- tion nor the operation of the municipal councils in which the community associations take part has affected the administration of the municipality. Backing up this perception that great changes in municipal administration have resulted from the process of organization of the community and the operation of the municipal councils, data collected on the ways in which municipal action and investment is disseminated indicate that there is a ten- dency to greater openness and transparency in this. Between 2002 and 2005, there was a significant reduction in the proportion of municipalities that had not made this information public in any way.124 On the other hand, although there has been a rise in the use of the media to communicate public policy, the use of municipal councils as channels for providing information on the actions of the municipal ad- ministration (and, thereby, as fora for the control of these actions) has remained stable and occurs in only 12% of municipalities. It should be stressed that, although a significant proportion of PCPR municipal coun- cils have already become fora for the discussion of other issues of interest to the municipality and of public policy, it is still argued that most of these continue to have a strictly deliberative function restricted to the PCPR and PRONAF interventions. In 35%, 36%, and 45% of municipalities, these councils already operate as fora for discussion of other federal, state, and municipal programs, respectively. However, in only 28%, 20% and 28% of municipalities do they serve as a forum for deliberation regarding other federal (including PRONAF), state, and municipal projects and resources, respectively. After almost a decade of the existence of these councils, these two results are not very encouraging and suggest that these municipal councils, which broadly represent organized civil society are still not (as 122 While 21% did not publicize action and investment in 2002, only 5% still do not do so. The medium most frequently used is the radio, but there has also been a growth in the use of posters in public places and the publishing of information through the Council Chamber. 151 they should be) fully incorporated as instruments for the implementation of public policy and public accounting by the public authorities. Likewise, we should mention the predominant features of the political and electoral participation of the poor sectors of the population. Views and data collected on the last two municipal elections--at municipal and community level--suggest that the impact of community organization and its representative bodies on the process of governance does not happen through the legally established channels of political representation. These data reveal that community associations and their leaders are increasingly participating in politics and elections, but they are still in a minority. In 46% of municipalities was the opinion shared that these engaged ef- fectively as political players in the previous round of municipal elections and in 57% of these municipalities was it considered that they had great influence and that they had had a decisive influence on the result of the elections. In other words, in only 26% of the municipalities studied are rural community associations considered to have any political or electoral importance. Between the two last elections, there was in fact a reduction in the number of community leaders elected. In 2000, the average number of community leaders elected in all the municipalities was 0.400, while in 2004 it had dropped to 0.377.125 This reduction in the average number of community leaders elected contrasts with the slight rise in the proportion of municipalities that had at least one community leader elected to political office ­ this having occurred in 32% of the municipalities in 2000, and 38% in 2004. Data collected as part of the research at community level also con- firm that community leaders have little influence on the political and elec- toral process, with only 35% of communities putting up candidates in 2000, and only 38% in 2004. In the two elections together, only 17% of them suc- ceeded in getting one of their candidates elected. Regression analyses further show that putting forward candidates and, in particular, their success at the ballot box is determined strongly by the size of the community.126 125 Taking into consideration only the municipalities in which community leaders were elected, the average number of candidates elected at the last two elections was 1.79 in 2000, and 1.33 in 2005. 126 In the sample of communities studied, the size of the community varied from 8 to 343 households. Logistic binary regression (using "logit" as the "link function") of the likeli- hood of a community getting one of its candidates elected a councilor in 2000 and 2004 by size of community produced the following results: Predictor: Size of Community Coef SE Coef Z P Odds Ratio Responses : Likelihood of electing a councilor in 2000 0.015859 0.006322 2.51 0.012 1.02 Likelihood of electing a councilor in 2004 0.025779 0.007851 3.28 0.001 1.03 152 Given this, political and electoral legal processes for political represen- tation have not been accessible to the leaders of poor rural communities such as those that benefit from the PCPR. It can, therefore, be suggested that these communities, being isolated units, control little in the way of electoral capital and that political action and representation continue to be under the control of the traditional political agents. The fact that poor rural populations and the representatives and leaders selected from among them are excluded from political action and representation therefore increases the importance of community associations as organizations for representation and civil participation and of the municipal councils as fora for expression of the needs and interests of these poor rural populations. These populations have been excluded from legally established institu- tional channels for political representation, or at least excluded from the possibility of playing a proactive role at these levels of government. They, therefore, have found new fora for inclusion, representation, and partici- pation in decision-making processes on public policies that affect their interests. These fora include the associations and municipal councils whose numbers have grown over the last decade, as it is essentially through these that civil participation can emerge and be consolidated. The fact that it is widely perceived by a broad range of different social players that the rural Northeast is going through significant changes in terms of social capital, citizenship, and governance backs up the results reported by the most recent literature on the outcomes and impacts of the PCPR. Furthermore, it proves the importance of the existence of commu- nity associations and municipal councils as social and political representa- tives of poor rural communities, and, thus the importance of the positive effect the PCPR has had in providing the stimulus for setting them up in the rural Northeast. Up to now, 37,600 community associations have received PCPR and its predecessor programs and 1,505 municipal coun- cils, with an unprecedented level of representation of rural community associations, have been set up, covering 89% of rural municipalities in the Northeast. The results of the quasi-experimental study demonstrate that 32.4% of the associations that make up the sample were created under the influence of the PCPR. Extrapolating this figure for all the organizations benefiting from the project, this would mean that 12,000 rural communi- ties have organized because of the PCPR. If this is correct, the PCPR is responsible for establishing a new pattern of institutions, which is associ- ated with an extremely positive impact on social capital, on the sense of citizenship, on civil control of public administration, and, thereby, on overcoming the structural conditions that bring about social vulnerability and poverty. 153 Conclusions regarding social capital The PCPR has had a positive and significant impact both on the inputs and the outputs of social capital, with a sustainable and long lasting effect. The rate of growth of the outputs is higher than that of the inputs, suggesting that the PCPR uses the social capital already existing in rural communities, the inputs, transforming it into social capital outputs and, thus, generating further social capital. This transformation occurs insofar as the stock of motivation for acting collectively is transferred from their traditional ap- plication, usually mutual aid networks and organizing religious events, to new social and political spheres, such as local development, and relations between communities and with the local government. The study leads to the conclusion that the PCPR has been effective in generating social capital in rural communities in Northeast Brazil. All the results indicate, with statistical significance and with a high degree of confidence, that the PCPR has a strong impact on social capital forma- tion. This impact derives both from the creation of associations and from the execution of projects. However, the project implementation produces what can be called the euphoria (or exuberance) effect, characterized by a sharp rise in collective activities at local level, in relations of cooperation between communities, and between associations and local authorities. This effect is revealed by the fact that short-term growth in social capital is greater in the 2005 than in the 2002 communities. During this period of euphoria, the rate of growth of social capital is high; subsequently the rate of change decreases, but continues to be positive. That is, social capital generated under the PCPR's stimulus does not decline. The PCPR generates or strengthens institutional arrangements that enjoy a high level of confidence and credibility in rural communities since they effectively contribute to solving local problems. The strengthening of community associations and their participation in municipal councils is widely associated with a rise in the capacity of poor rural communities to ensure that their interests are represented, to exercise social control over the activities of the municipal administration and to put pressure on them to provide effective responses to the demands of communities. According to some, the PCPR is responsible for this reputation because a community subproject chosen and implemented by members of a community associa- tion is an indispensable factor in building the credibility of community as- sociations and generating a feeling of confidence among poor people. All of this suggests that there is a great unused stock of social capital in these communities. Nevertheless, and as a result of the PCPR's restricted budget and absence of other public policies that adopt a community 154 demand driven approach, the rural Northeast has seen, since 1993, the establishment of a wide range of social institutions that are not being fully used. Few communities served by the PCPR succeed in obtaining a second subproject and there are few other examples of public policy that has genuinely endorsed the demand-driven approach and community partici- pation in decision-making and implementation. This restricts the impact that the strong increase of social capital may have on local government and municipal administration. Participatory community associations and the municipal councils can represent the interests of the poorest sectors of society. They can ef- ficiently take action at local level and strengthen horizontal trust relations, solidarity, and cooperation among equals. Community associations have pushed for more democratic vertical relations between poor rural com- munities and the state--which have been traditionally dominated by local elites, rigid hierarchical ties, and elite capture of benefits. These associa- tions can build high levels of social capital; however, they have not yet reached their full capacity in public administration and local government. It comes, therefore, as no surprise that the "type of social capital" most negatively affected by the lower rate of growth of social capital that takes place once the euphoria effect ends, is civil participation and citizenship. Finally, the quasi-experimental study shows that an extremely positive evaluation prevails among those interviewed at local and municipal level regarding the influence of community and civil participation on municipal administration, partnerships between the public authorities and communi- ties, making public information on the actions of municipal administrators and social control of these. Such views, however, do not stop them from agreeing that the capacity and interest to respond to the demands of com- munities on the part of the municipal authorities are still weak. They also do not stop them from seeing that the political and electoral participation of community associations and their leaders remain limited, or underes- timated. Neither do they stop them from stating that, in fact, the legal channels established for political representation continue to be largely inaccessible to legitimate representatives of the poorest rural communities and populations. In combination, these facts and points of view support the importance of the new institutional arrangements encouraged by the PCPR (and consequently to the stimulus brought to bear by the PCPR and by similarly oriented public policy initiatives) in order to promote the legitimate representation of the interests and the genuine participation of poor rural populations in decision-making processes at local and munic- ipal level. Through such arrangements, poor populations in the Northeast have found the best way of developing their social capital. They have more 155 confidence in dealing with the state and more capacity to be proactive in local development processes. They receive a fairer share of the benefits of public policy, and they have broadened their influence over government and municipal administration through more effective community and civil participation. In the face of a centuries-long tradition of clientilistic practices rooted in a society marred by profound and mutually reinforcing social, economic, and political differences, these processes are evidently not complete. Yet, even the most skeptical studies--which have in the past analyzed the impact of PCPR on popular participation, social capital for- mation, and recovering a sense of citizenship as an end in itself and as an indispensable means to reducing poverty--have not been blind to the fact that this is the most promising, if not the only, way forward. Conclusions This report has confirmed the main conclusions reached by the study car- ried out by van Zyl et al (2000), which was the most wide-ranging study of the PCPR before this one. The present study demonstrates, using rig- orous methodologies of analysis, that the Program has a significant impact on access to water and electricity in the Northeast, and that it has helped reduce infant mortality and the incidence of various diseases. It also con- cludes that the Program has had a great and sustainable impact on social capital, which could not have been created without the PCPR and which has not diminished subsequent to implementation of the subprojects. However, some results are still inconclusive. The results suggest that the Program has a positive impact on the household accumulation of physical capital, but the effects are not statistically significant. As the baseline data used in the analysis were collected by recall, that is, those interviewed in 2005 were asked about conditions in 2002, the impact of the PCPR on income could not be measured. Instead of this, the study analyzed the impact on accumulated physical capital. However, income increases in poor families may for the most part be consumed rather than invested, and thus a large part of the effect on income cannot be measured through changes in physical capital. Likewise, the impact of the Program on health should be analyzed afresh using direct observation. The results of this study, which suggest positive effects in terms of infant mortality and the incidence of some diseases, were based on the responses of qualified interviewees and not on direct observation. Therefore, it is still necessary to measure the impact on income and confirm the results of this study on health. The authors suggest that the 156 same sample used in the 2004 FECAMP baseline study (Buainain and Fonseca, 2005b) be used for an impact study, preferably returning to the same households. The FECAMP baseline study used a sample of benefi- ciaries and a control and collected detailed data on households, including on income, from the period prior to implementation of the subprojects. This suggested impact study should incorporate the range of methodolo- gies available for adjusting for selection bias, including propensity-score matching and the difference of differences method. 157 Appendix A (Chapter 4) Results for household assets Table A1: Impact of PCPR on assets (electrical appliances) - 2002-2005 D assets (electrical Sample resulting from matching appliances) per capita Model 1 Model 2 Model 3 Trat 28.645 (22.780) Trat*abast 45.450 (25.610) Trat*eletrif 15.981 (32.428) Trat*prod 16.394 (26.036) tr*ce*abast 60.486* (27.242) tr*rn*abast -20.805 (32.554) tr*rn*prod 22.469 (29.225) tr*pi*eletr 15.981 (32.485) tr*pi*prod 9.872 (32.598) Constant 67.800 67.800 67.800 (20.139) (20.175) (20.210) nº of observ. 573 573 573 F (k. n-1) 1.580 1.190 1.730 Prob > F 0.209 0.314 0.126 R-squared 0.005 0.007 0.013 Standard errors in parentheses, corrected for heteroskedasticicty * indicates that the coefficient estimated is statistically significant at 5% 157 158 Table A2: Impact of PCPR on assets (automobiles) - 2002-2005 Sample resulting from matching D assets (automobiles) per capita Model 1 Model 2 Model 3 Trat 131.350 (149.670) Trat*abast 149.694 (144.132) Trat*eletrif 65.317 (141.613) Trat*prod 128.314 (178.865) tr*ce*abast 172.788 (146.772) tr*rn*abast 47.937 (143.365) tr*rn*prod 199.031 (258.01) tr*pi*eletr 65.317 (141.862) tr*pi*prod 52.386 (140.522) Constant -25.165 -25.165 -25.165 (138.453) (138.696) (138.940) nº de observ. 573 573 573 F (k. n-1) 0.77 1.19 1.28 Prob > F 0.3805 0.3122 0.2729 R-squared 0.003 0.0032 0.0045 Standard errors in parentheses, corrected for heteroskedasticicty 159 Table A3: Impact of PCPR on assets (agricultural equipment): 2002-2005 D assets (agricultural Sample resulting from matching equipment) per capita Model 1 Model 2 Model 3 Trat 8.236 (5.466) Trat*abast -0.487 (4.805) Trat*eletrif -2.320 (4.067) Trat*prod 17.986* (9.553) tr*ce*abast 0.810 (5.558) tr*rn*abast -6.202 (3.209) tr*rn*prod 37.616* (17.668) tr*pi*eletr -2.320 (4.074) tr*pi*prod -3.091 (2.964) Constant 4.612 4.612 4.612 (2.723) (2.728) (2.733) nº of observ. 573 573 573 F (k. n-1) 2.27 1.48 2.16 Prob > F 0.1325 0.218 0.0573 R-squared 0.0035 0.0127 0.0336 Standard errors in parentheses, corrected for heteroskedasticicty * indicates that the coefficient estimated is statistically significant at 5% 160 Table A4: Impact of PCPR assets (animals) ­ per capita figures: 2002-2005 D assets (animals) Sample resulting from matching per capita Model 1 Model 2 Model 3 Trat 434.695 (446.422) Trat*abast 389.157 (449.925) Trat*eletrif 362.516 (454.325) Trat*prod 488.974 (448.628) tr*ce*abast 404.073 (451.616) tr*rn*abast 323.436 (470.084) tr*rn*prod 550.272 (455.791) tr*pi*eletr 362.516 (455.125) tr*pi*prod 423.158 (448.272) Constant -429.60 -429.60 -429.60 (444.88) (445.67) (446.45) nº of observ. 573 573 573 F (k. n-1) 0.950 1.090 0.760 Prob > F 0.331 0.354 0.577 R-squared 0.016 0.017 0.017 * Standard errors in parentheses, corrected for heteroskedasticicty 161 Table A5: Impact of the PCPR on financial assets: 2002-2005 D financial assets per Sample resulting from matching capita Model 1 Model 2 Model 3 Trat -79.136 (61.999) Trat*abast -78.434 (66.956) Trat*eletrif -11.143 (70.426) Trat*prod -93.214 (74.397) tr*ce*abast -35.408 (61.253) tr*rn*abast -268.020 (175.946) tr*rn*prod -60.286 (62.870) tr*pi*eletr -11.143 (70.551) tr*pi*prod -128.568 (113.554) Constant -97.973 -97.973 -97.973 (54.437) (54.533) (54.629) nº of observ. 573 573 573 F (k. n-1) 1.63 0.96 0.81 Prob > F 0.2023 0.4136 0.5426 R-squared 0.0053 0.0063 0.0131 Standard errors in parentheses, corrected for heteroskedasticicty 162 163 Appendix B (Chapter 4) Results of propensity score used for analysis of social capital Table B1:Propensity Score at household level for Social capital exercise Logistic regression Number of obs = 860 Waldchi2(28) = 83.46 Prob> chi2 = 0.0000 Log pseudo likelihood = -484.82234 Pseudo R2 = 0.1867 Dependent Variable Robust = `Treatment' Coef. Std. Error Z P>|z| [95% Conf. Interval] Piaui 0.609 0.610 1.000 0.318 -0.586 1.804 Ceará 0.578 0.630 0.920 0.359 -0.658 1.813 Variables that represent community infrastructure conditions esc_prim_0 0.393 0.467 0.840 0.401 -0.523 1.309 esc_sec_0 1.112 0.726 1.530 0.126 -0.312 2.535 posto_0 0.930 0.561 1.660 0.098 -0.170 2.030 Dist 0.007 0.018 0.380 0.703 -0.028 0.042 nº hab -0.002 0.001 -1.270 0.202 -0.004 0.001 hab/casas -0.300 0.215 -1.390 0.163 -0.721 0.122 rec_proj -0.523 0.689 -0.760 0.448 -1.874 0.827 Variables for head of household and household infrastructure educa_0 0.139 0.091 1.530 0.126 -0.039 0.317 id_0 -0.007 0.010 -0.710 0.480 -0.026 0.012 sexo_0 0.023 0.303 0.080 0.939 -0.571 0.618 at_anim_0_pc 0.000 0.000 0.930 0.355 0.000 0.000 at_agric_0_pc 0.001 0.001 0.920 0.357 -0.001 0.002 at_auto_0_pc 0.000 0.000 -0.410 0.679 0.000 0.000 at_eletr_0_pc 0.001* 0.000 2.890 0.004 0.000 0.002 at_fin_0_pc 0.000 0.001 0.550 0.585 -0.001 0.001 ac_agua_0 -0.206 0.425 -0.490 0.627 -1.039 0.626 ac_esg_0 0.413 0.378 1.090 0.275 -0.329 1.154 163 164 ac_lixo_0 0.006 0.246 0.020 0.982 -0.477 0.489 ac_luz_0 1.145* 0.434 2.640 0.008 0.295 1.995 prop_0 0.731* 0.344 2.130 0.034 0.057 1.406 banh_0 -0.509 0.351 -1.450 0.147 -1.198 0.180 após_0 0.000 0.001 0.220 0.829 -0.001 0.001 fonte_ap_0 0.349 0.341 1.020 0.306 -0.319 1.017 fonte_agr_0 0.279 0.264 1.060 0.291 -0.239 0.797 mor_0 0.041 0.046 0.890 0.371 -0.049 0.131 bad_0 0.428 0.276 1.550 0.121 -0.112 0.968 Constant -1.821 1.348 -1.350 0.177 -4.464 0.821 *Heterocedasticity robust standard errors in parentheses.. * Statistically significant at 5% Variables explaining the propensity score equation for the analysis of Social Capital · dummy for the State of Ceará (CE); · dummy for the State of Piauí (PI); Variables that represent the infrastructure conditions of the community · dummy for the existence of a primary school in the community in 2002 (escp_0); · dummy for the existence of a secondary school in the community in 2002 (escs_0); · dummy for the existence of a health clinic in the community in 2002 (posto_0); · distance of community from main town in the municipality (dist); · number of inhabitants in the community (pop) ; · ratio between number of inhabitants and number of houses in the community (hab/ casas) · dummy for already having received a previous project (rec_proj); Variables for characteristics of head of household and household infrastructure · level of education of head of household in 2002 (educa_0); · age of head of household in 2002 (id_0); · dummy head of household being a man in 2002 (sexo_0); · value of asset `animals' per capita in 2002 (at_anim_0_pc); · value of asset `agricultural equipment' per capita in 2002 (at_agric_0_pc); · value of asset `automobiles' per capita in 2002 (at_auto_0_pc); · value of asset `electrical appliances' per capita in 2002 (at_eletr_0_pc); · value of financial assets per capita in 2002 (at_fin_0_pc); · dummy for household having access to water network in 2002 (agua_0); · dummy for household having access to sewerage network or septic tank in 2002 (esg_0); · dummy for household having access to rubbish collection network or burning rubbish in 2002 (lixo_0); · dummy for household having access to electricity network in 2002 (luz_0); 165 · dummy for home ownership in 2002 (prop_0); · dummy for indoor bathroom in 2002 (banh_0); · value of income from pension in 2002 (apos_0); · dummy for pension being main source of income in 2002 (fonte_ap); · dummy for agricultural activity being main source of income in 2002 (fonte_agr); · number of residents in household in 2002 (mor_0); and · dummy for 2002 having been a god winter (bad_0) 166 167 References (The classification of the methodology used in each PCPR study that was reviewed is indicated at the end of the reference. See Chapter 1 for a de- scription of the methodologies) ADESE (Agência de Desenvolvimento do Seridó). "Abastecimento de Água em Pequenas Comunidades Rurais Dispersas no Semi-Árido Brasileiro: Parte B ­ Estudo de Caso da Região do Seridó (Estado do Rio Grande do Norte)". Bank-Netherlands Water Partnership Program e Banco Mundial, 2002. General classification: C Amazonas, Fatima Relatório Feira de Bologna. Banco Mundial, Recife. 2002. Angrist, J. e Krueger, A. "Empirical Strategies in Labor Economics", in Ashenfelter e Card (eds), Handbook of Labor Economics, vol. 3A, Elsevier. 1.999 Argôlo de Souza, Renato. Projeto de Combate à Pobreza Rural do Estado do Maranhão ­ PCPR ­ MA (Comunidade Viva) ­ Relatório Final, São Luis, Maranhão, 2004. General classification: S ASPEC - Associação de Pesquisa e Estudos Científicos, Avaliação Progressiva do Projeto São José no Estado de Sergipe, PRONESE, Aracaju, 1999. General classification: BAWO BADEN, S. (2000) Gender, Governance and `The Feminization of Poverty', in. Women and Political Participation: 21st Century Challenges. New York: UNDP. Banco Mundial, MDA, CONTAG - Apresentação: A Economia Globalizada X A Produção da Agricultura Familiar no Brasil ­ Uma Estratégia de Acesso ao Mercado ­ Terceiro Fórum Temático América Latina e Caribe Sociedade Civil e Banco Mundial ­ Angra dos Reis ­ Brasil 21­23 Outubro 2003. Barbosa, Túlio. "Brasil - Relatório de Revisão de Meio Termo Projetos de Combate à Pobreza Rural: Empréstimo 3917- BR ­ Rural Poverty Alleviation Project ­ Bahia, Empréstimo 3918- BR ­ Rural Poverty Alleviation Project ­ Ceará; Empréstimo 3919- BR ­ Rural Poverty Alleviation Project ­ Sergipe ". Versão original: outubro 1999, Revisada em março de 2000. Banco Mundial, Brasília, 2000. General classification: PSR 167 168 Braceras. Fidel et al. Gestão do Desenvolvimento Local: Experiências Exitosas no Rio Grande do Norte. Governo do Estado do Rio Grande do Norte, SETHAS, IICA, 2004. General classification: C Buainain, Antônio Márcio e Rinaldo Barcia Fonseca (coord.). Projeto de Combate à Pobreza Rural (Renascer-Pernambuco): Funcionamento e Resultados Imediatos. Relatório preliminar. Instituto de Economia, Universidade de Campinas, Campinas, 2004a. General classification: BAWO Buainain, Antônio Márcio e Rinaldo Barcia Fonseca (coord.). Projeto de Combate à Pobreza Rural (São José- Ceará)- Parte I de II: Resultados Imediatos e Desempenho. Relatório preliminar. Instituto de Economia, Universidade de Campinas, Campinas, 2004b. General classification: BAWO Buainain, Antônio Márcio e Rinaldo Barcia Fonseca (coord.). Projeto de Combate à Pobreza Rural (Produzir - Bahia) ­ Parte I de II: Resultados Imediatos e Desempenho. Relatório preliminar. Instituto de Economia, Universidade de Campinas, Campinas, 2004c. General classification: BAWO Buainain, A . M e R. Fonseca (coord.). Projeto de Combate à Pobreza Rural Fase II-Projetos Renascer (PE), Produzir (BA) e São José (CE): Evolução do Perfil Socio-econômico dos Beneficiários e Efeitos Imediatos. Relatório de Consultoria Técnica, Unicamp, Campinas, 2005a. General classification: BAWO Buainain, A . M e R. Fonseca (coord.). Projeto de Combate à Pobreza Rural Fase II-Projetos Renascer (PE), Produzir (BA) e São José (CE): Perfil Socioeconômico da População Beneficiária Fase II. Relatório de Consultoria Técnica, Unicamp, Campinas, 2005b. General classification: geral: WO, e em algumas análises WWO SM. Buanain, Antônio Márcio e Rinaldo Barcia Fonseca (coord). Estudo de Avaliação de Impactos do Programa Cédula da Terra, 2003. Relatório Final, novembro 2003. General classification Geral: BAWO, alguns resultados: BAWWOST Buainain, Antônio Márcio (coord), Perfil dos Beneficiários, Projeto Cédula da Terra e INCRA. Relatório Preliminar, junho 2002. UNICAMP/NEA- MDA/NEAD General classification: Geral: WWO SS; Resultados sobre focalização: WWOSM Buarque, Cristina. 2004. Visão de gênero no mundo rural brasileiro contemporâneo. Caderno Técnico n 31. Instituto Interamericano de Cooperação para a Agricultura Caminha, Raimundo Nonato. "Projeto Desenvolvimento Integrado do Maranhão: Uma experiência de trabalho em grupo". Banco Mundial, Recife, 2004. General classification: S CEPLAN - Consultoria Econômica e Planejamento. Estudo de Desempenho Físico do PCPR Pernambuco, Versão Final, Recife, 2004. General classification: WO Coirolo, L. and Lammert, J. Community-Driven Development in Northeast Brazil: Achieving Results through Community Driven Development. Commissioned by PADECO, Co. Ltd. for the World Bank­Japan Trust Fund Project "Reducing Poverty: What Works, What Doesn't." 2008. 169 Costa, Alberto Coelho Gomes. "Cidadania e Participação Comunitária no Programa de Alívio à Probreza do Nordeste: Esperanças e Riscos para Associações Comunitárias do Nordeste". 1998. General classification:WO Costa, Alberto C. G. "O Capital Social Acumulado nas Comunidades Rurais do Estado da Bahia, Banco Mundial, Recife". 1999ª. General classification: C Costa, Alberto C. G. "O Capital Social Acumulado nas Comunidades Rurais do Estado do Ceará. Banco Mundial". Recife, 1999b. General classification: C Costa, Alberto C. G. "Capital Social Acumulado nas Comunidades Rurais do Estado de Sergipe. Banco Mundial". Recife, 1999c. General classification: C Costa, Alberto C. G. "PCPR, Organização Comunitária, Participação Popular e Capital Social: Estudo dos Casos de Pernambuco e Piauí". Banco Mundial, Recife, 1999d. General classification:WO Costa, Alberto C. G. "Relatório da Missão a Sergipe para Aprimorar Dados do Estudo de Impacto Comunitário Realizado pela ASPEC". Banco Mundial, Recife, 1999e. General classification: PSR Costa, Alberto C. G. "Organização Comunitária, Capital Social e Gestão de Subprojetos ­ A Experiência dos PCPRs". Brasília, 2002ª. General classification:S Costa, Alberto C. G. "Organização Comunitária, Capital Social e Gestão de Serviços de Saneamento". Trabalho apresentado na "Oficina Temática: Gestão de Serviços de Saneamento em Pequenos Municípios e suas Localidades: Ações Inovadoras Buscando a Sustentabilidade". Brasília, 2002b. General classification:S Costa, Alberto C. G. Relatório de Missão Maranhão ­ PCPR. Banco Mundial, Recife, 2003. General classification: PSR Costa, Alberto C.G. "Estudo de Avaliação dos Subprojetos de Mecanização Agrícola -Projeto de Revitalização da Produção Agrícola na Região Nordeste do Estado da Bahia". Não publicado, 2004. General classification: C Couto Soares, Maria Clara et al. "Fundos Sociais e Ambientais Financiados ou Administrados pelo Banco Mundial no Brasil". Rio de Janeiro, março de 2000 FATRES Fundação de Apoio ao Trabalhador Rural da Região do Sisal. Programa Produzir: Uma Experiência de Controle Social em Municípios do Semi- Árido Baiano Salvador, Bahia, 2003. General classification: WO FETARN (Federação dos Trabalhadores na Agricultura do Rio Grande do Norte) e IICA (Instituto Interamericano de Cooperação para a Agricultura), "Perfil de Entrada - Características Sócio-Econômicas No Espaço de Atuação do PCPR No Rio Grande Do Norte". Natal, 1999. General classification: RB FLACSO ­ Faculdade Latino Americana de Ciências Sociais. "Avaliação de Impacto e Estudo de Desempenho Físico do Projeto São José no Estado do Ceará - Programa de Combate à Pobreza Rural". Relatório Complementar - "Avaliação dos Pesquisadores de Campo" Fortaleza, 1998ª. General classification: PSR 170 FLACSO (Faculdade Latino - Americana de Ciências Sociais). "Estudo de Desempenho Físico e Avaliação dos Impactos Sócio - econômicos e Culturais do Projeto São José no Estado do Ceará". Fortaleza, 1998b. General classification: WO Fonsêca, G. G e Raimundo J. C. de Melo. "Estudo de Perfil de Entrada". Relatório Final. Teresina, 1999a. General classification: RB Fonsêca, G. G e Raimundo. J. C. de Melo. "Estudo de Desempenho Físico dos Subprojetos Financiados pelo PCPR-PI". Teresina, 1999b. General classification: WO Fonsêca, Graziani G. e Raimundo J. C. de Melo. "Relatório de Revisão de Meio Termo do Projeto de Combate à Pobreza Rural do Estado do Maranhão". São Luis, 2002. General classification: WWOSS Garrison, J et al. "Participatory Portfolio Review of Small-Grant Funds within Bank Supported Loans in Brazil ­ Methodology and Summary of Main Findings". World Bank, Brasilia, 2000. General classification: PSR Garrison, J. "Do Confronto à Colaboração: Relação entre a Sociedade Civil, o Governo e o Banco Mundial no Brasil". Banco Mundial, Brasília, 2000. General classification: PSR Germano de Figueiredo, Sonia M. "Colaboración y movilización de grupos comunitarios y participación de la mujer". In: Acción Local, Mejores Vidas: Implementación de Proyectos Participativos y Descentralizados ­ Taller Regional para América Central, IDE del Banco Mundial y IICA Guatemala, 1998. General classification: C GOETZ, A. M. and Jenkins, R. (2005) Reinventing Accountability: Making Democracy Work for Human Development. Baskingstoke: Palgrave Macmillan. Gois, Magaly Nunes, Conselhos Gestores: Instrumentos de Construção de Nova Esfera Pública? Um estudo com Conselhos de Desenvolvimento Municipal de Sergipe. Dissertação (Mestrado em Educação), Universidade Federal de Sergipe, São Cristóvão (SE), 2004. Grootaert, Christiaan and van Bastelaer, Thierry. "Understanding and Measuring Social Capital: A Synthesis of Findings and Recommendations from the Social Capital Initiative." In Social Capital Initiative Working Paper No. 24. Washington, D.C.: World Bank, 2001 Guzzo, J. R. Crescimento, Solução Para a Miséria, Revista Exame, 1 de fevereiro de 2006 Hydros ­ Engenharia e Planejamento Ltda, Estudo de Desempenho Físico ­ PCPR Sergipe, Aracaju, 2004. General classification: WO IADB (1998) The use of Social Investment Funds as an Instrument for Combating Poverty: Strategy Paper. Washington: InterAmerican Development Bank. IICA (Instituto Interamericano de Cooperação para a Agricultura); FETARN (Federação dos Trabalhadores na Agricultura do Rio Grande do Norte) e AACC (Associação de Apoio às Comunidades do Campo): Projeto de Combate à Pobreza Rural (RN) - Estudo de Desempenho Físico (EDF), Natal 1999. General classification: WO Instituto Civitas ­ Cidadania e Políticas Públicas. "Estudo de Desempenho Físico (EDF) ­ PCPR Piauí". Teresina, 2004a. General classification: WO 171 Instituto Civitas ­ Cidadania e Políticas Públicas. "Relatório para Revisão de Meio Termo do Projeto de Combate à Pobreza Rural ­ PCPR II". Teresina, Piauí, 2004b. General classification: WO Instituto Civitas ­ Cidadania e Políticas Públicas. "Relatório para Revisão de Meio Termo do Projeto de Combate à Pobreza Rural ­ PCPR II do Estado do Piauí. (Documento 3)-- Estudos De Casos". Teresina, 2004c. General classification: C INTERCOOP (Cooperativa Interdisciplinar de Serviços Profissionais Ltda). "Estudos de Desempenho Físico e Avaliação de Impacto do Programa de Apoio às Comunidades Rurais ­ PRODUZIR". Salvador, 1998. General classification: WO Jara, Carlos J. A Sustentabilidade do Desenvolvimento Local ­ Desafios de um Processo em Construção. PRORURAL/IICA, 1998 Jatobá, J.(coord) & Leornado Guimarães Neto. "Nordeste do Brasil: Avanços, Mudanças e Desafios". Recife, dezembro 2005 Jucá, Walmar Isacksson. "Vínculos Burocráticos, Normativos e Institucionales entre los Proyectos Basados en la Demanda y el Gobierno Estatal". In: Acción Local, Mejores Vidas: Implementación de Proyectos Participativos y Descentralizados ­ Taller Regional para América Central, IDE del Banco Mundial y IICA, Guatemala, 1998. Kahn, Ahmad Saeed & Silva, Lúcia Maria Ramos. "Avaliação do Projeto São José no Estado do Ceará: Estudo de Caso". UFCE, 2002. General classification: BAWO; income analysis: BAWWO ST Kottak C., Costa, A. and Prado, R. "A Study of Popular Participation in Brazil: Northeast Rural Development Program (NRDP-PAPP)". Paper presented at Workshop on Participatory Development. May 17-20 (Washington DC: World Bank), 1994. General classification: RB Kottak, C. and Costa, A. "A Follow-Up Study of Popular Participation in Brazil: Northeast Rural Development Program". Mimeo (Ann Arbor, MI: University of Michigan), 1994. General classification: BAWWOSS Kumar, Nalini. "The Effectiveness of World Bank Support for Community Driven Development (CDD): An OED Evaluation". The World Bank, Operations Evaluation Department, Washington, 2005. General classification: geral WWOSS; alguns resultados envolvendo análises usando "recall" são BA WWOSS Lisboa, Marcos de Barros (org). A agenda perdida: diagnóstico e propostas para a retomada do crescimento com maior justiça social. Rio de Janeiro, setembro de 2002. Mansuri, Ghazala and Vijayendra Rao. Community-Based and ­Driven Development: a Critical Review. World Bank Policy Research Working Paper 3209, February 2004 Matos Filho, João. A Descentralização das Políticas de Desenvolvimento Rural ­ Uma Análise da Experiência do Rio Grande do Norte. Tese de Doutoramento apresentada ao Instituto de Economia da UNICAMP. Campinas, 2002. General classification: C Matos Filho, J.et al.. "Estudo de Desempenho Físico ­ EDF ­ PCPR, RN". Natal, 2005a. General classification: WO 172 Matos Filho, João et al. "Projetos de Combate à Pobreza Rural do Rio Grande do Norte I e II (PCPR I e PCPR II) Avaliação de Resultados". Natal, 2005b . General classification: WO Merlet, Michel. "CDD Experience in Northeast Brazil, based on examples from the States of Pernambuco and Rio Grande do Norte". 2003. General classification: S Moreira da Silva, Amenair. Participação da Mulher no Projeto Cédula da Terra (Limitações e Perspectivas). Recife, Banco Mundial, 1998. Nankani, Helen B. Targeting Afro Brazilians--Lessons from the Maranhao PCPR. (Back-to-Office Report: Learning Mission in the Context of the Rural Poverty Alleviation Project--Maranhão), The World Bank, Brasília, July, 2002. General classification: PSR NCA Engenharia, Arquitetura e Meio Ambiente S/C Ltda. In: Revisão Temática ­ Projetos de Infra-estrutura Rural no Brasil, Banco Mundial, Brasília, junho 2002. General classification: QAR Nichter, Simeon, Productive Investments in Pernambuco. Projeto Renascer, Recife, Pernambuco, 2003. General classification: C. OED, World Bank, Community-Driven Development in Rural Mexico and Brazil. Précis, Summer 2000, Number 200. Parker, A. Decentralized Rural Development and Local Government Performance in Northeast Brazil. PhD thesis, Department of Agricultural Economics, University of Pretoria (Pretoria, South Africa), 1998. General classification: RB. Pozzoni, Barbara e Nalini Kumar. A Review of the Literature on Participatory Approaches to Local Development For an Evaluation of Community- Driven Development Approaches Supported by the World Bank. The World Bank. Operations Evaluation Department, 2004. General classification: S. Projeto Renascer. "Avaliação dos Conselhos do FUMAC-P". Pernambuco, Recife 2002. General classification: C. Projeto Renascer. "Projeto Piloto Integrado de Desenvolvimento Local, Regiões Moxotó e Agreste Meridional". 2003. General classification: PSR Projeto Renascer. "Relatório de Meio Termo PCPR do Estado de Pernambuco". Recife, 2004. General classification: PSR PRONESE ­ Empresa de Desenvolvimento Sustentável do Estado de Sergipe. "Relatório para Fins de Revisão de Meio-Termo (RMT) do Projeto de Combate à Pobreza Rural do Estado de Sergipe -PCPR II - Empréstimo 4649-BR ­ Rural Poverty Reduction Project ­Sergipe". Aracaju, Sergipe, 2004. General classification: WO Ramalho, André. "Relatório de Revisão Independente de Licitações (PCPR Paraíba)". Banco Mundial, Brasilia, 2004. General classification: QAR Ravallion, M. "Evaluating Anti-Poverty Programs". Prepared for: Evenson, R.E. and Schultz, T. Paul (eds.) Handbook of Agricultural Economics Volume 4, North-Holland, 2005. Rede Nós/BNB II Poverty Measurement and Information System Seminar - BNB Passaré, Fortaleza-CE, Brazil 3-4 February, 2003 Rêgo, Manuel. "Estudo Sobre Sistemas Localizados de Abastecimento de Água Para Pequenas Comunidades Rurais Dispersas na Região Semi-Árida 173 do Estado do Ceará: Relatório de Avaliação". Banco Mundial, Brasília, 2002. General classification: WO Rizvi, Andrea Ryan and Alberto Costa. "Measuring Social Capital. Findings from the Rural Northeast Brazil". Power Point Presentation, World Bank. Washington, 2003a. General classification: BAWWOSS Rizvi, Andrea Ryan and Alberto Costa. "O Programa de Combate à Pobreza Rural e o Capital Social no Espaço Rural do Nordeste Brasileiro". Power Point Presentation. 2003b. RIZVI, Andrea Ryan and Alberto Costa. Rizvi, Andrea Ryan and Alberto Costa. "The Relevance of Social Capital Measurement to World Bank Operations (draft)". The World Bank, Washington, 2003c . General classification: BAWWOSS Rizvi, Andrea Ryan and Alberto Costa. "Can community driven infrastructure programs contribute to social capital? Findings from the Rural North East of Brazil". Power Point Presentation. World Bank, Washington, 2003d . General classification: BAWWOSS Romano, Claudia. "Case studies of PCPR in Ceará". Banco Mundial, Recife, 2003. General classification: C Roumani, Anna. Shanghai Conference ­ Brazil ­ CDD. The World Bank, Washington, 2004 Sampaio, Yony (Coordenador) et al. "Projeto de Combate à Pobreza Rural ­ PCPR Pernambuco - Perfil de Entrada e Estudo de Desempenho Físico". Recife, 1999. General classification: RB Serrano, R. Who Knows What's Best for the Poor? Demand-driven Policies and Rural Poverty in Northeast Brazil. Unpublished Masters thesis Department of Urban Studies and Planning, Massachusetts Institute of Technology (Boston), 1996. General classification: C Sonn, Loretta e Alberto Costa. "Estado da PARAÍBA - Avaliação do PAPP Reformulado". Não publicado, Banco Mundial, Recife, 1996ª . General classification: C Sonn, Loretta e Alberto Costa. "Estado de ALAGOAS - Avaliação do PAPP Reformulado". Não publicado, Banco Mundial, Recife, 1996b . General classification: C Sonn, Loretta e Alberto Costa. "Estado de MARANHÃO - Avaliação do PAPP Reformulado". Não publicado, Banco Mundial, Recife, 1996c . General classification: C Sparovek, Gerd (coord.) Diagnóstico dos Projetos do Crédito Fundiário e Combate à Pobreza Rural. Final draft, 2004. General classification: WO Tendler, J. New Lessons from Old Projects: The Workings of Rural Development in Northeast Brazil. Operations Evaluation Department (Washington DC: World Bank, 1993 Teruel, Rodolfo et al. "Brasil: Descentralización y Desarrollo Municipal Sustentable ­ La Experiencia de Seridó". 2001. Uphoff, Norman "Understanding Social Capital: Learning from the Analysis and Experience of Participation." In Partha Dasgupta e Ismail Seragldin (eds.), Social Capital: A Mulifaceted Perspective. Washington, D.C.: World Bank, 2000. 174 Van Zyl, Sonn and Costa. Decentralized Rural Development and Enhanced Community Participation: A Case Study From Northeast Brazil World Bank. Policy Research Working Paper 1498, August 1995. General classification: C Van Zyl, J., Sonn, L. e Costa, A. Decentralized Rural Development, Enhanced Community Participation And Local Government Performance: Evidence From Northeast Brazil. The World Bank, Recife, 2000. General classification: WO Vieira Rocha, Dom Jaime. "A Experiência do Seridó". Seminário Banco Mundial - 10 Anos dos Projetos de Combate à Pobreza Rural no Nordeste do Brasil - Novas Estratégias e Perspectivas. Recife, 2003. General classification: S Weiss, Joseph. "Lições Aprendidas da Experiência Comparada Entre Fundos no Brasil e no Exterior". In: Garrison et al. Estudo Sobre Fundos Sociais e Ambientais Financiados ou Administrados pelo Banco Mundial no Brasil ­ Relatório Geral I. 2000. General classification: S World Bank. "Maranhão Integrated Program: Rural Poverty Reduction. Annex 12: Indigenous People's Participation Plan. Project Appraisal Document". Report No. 28647-BR, Washington, 2004 World Bank. "Bahia Integrated Project: Rural Poverty. Annex 7. Safeguard Policy Issues. Indigenous People". Washington, 2005 World Bank. "Rural Poverty Reduction Project II. Annex 4: Productive Sub ­ Projects. Project Appraisal Document". Report No. 40489-BR, Washington, 2007 World Bank (2008) Gender and Agricultual Livelihoods Sourcebook. Washington: The World Bank (in press).