WPS6895 Policy Research Working Paper 6895 Does Access to Information Empower the Poor? Evidence from the Dominican Republic Emmanuel Skoufias Renata Narita Ambar Narayan The World Bank Latin America and the Caribbean Region Poverty Reduction and Economic Management Department May 2014 Policy Research Working Paper 6895 Abstract This paper summarizes the results of the impact impact is found on awareness and the use of information evaluation of the Access to Information pilot project under the specific Access to Information rules. However, on empowerment of citizens in poor municipalities in it is observed that individuals address more general the Dominican Republic. Among the dimensions of complaints to governments as a result of the Access empowerment investigated are civic knowledge, awareness to Information program regardless of whether these and use of the right to information, perceptions of are classified under the ATI law or not. Some positive and trust in public services and institutions, civic and statistically significant impacts are found on local participation, and measures of local governance. Data government responsiveness, prioritization and decisions were collected in two rounds: a baseline round at the about the municipal budget, and trust in and satisfaction end of 2010 and a follow-up round in mid-2012. No with some local government services. This paper is a product of the Poverty Reduction and Economic Management Department, Latin America and the Caribbean Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http:// econ.worldbank.org. The authors may be contacted at eskoufias@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team DOES ACCESS TO INFORMATION EMPOWER THE POOR? EVIDENCE FROM THE DOMINICAN REPUBLIC Emmanuel Skoufias (World Bank) Renata Narita (Universidade de São Paulo) Ambar Narayan (World Bank) JEL classification: H0, C14 Keywords: Access to Information law, empowerment, program evaluation Corresponding author: Emmanuel Skoufias, The World Bank, 1818 H Street NW, (MSN: I 4- 405) Washington DC 20433-USA. tel: (202)458-7539. fax: (202) 522-3134. e-mail: eskoufias@worldbank.org Acknowledgements: The authors are grateful to Sarah Keener, Helen Grandvoinette, and Anupama Dokeniya for valuable suggestions. 1. Introduction The monitoring of public institutions by citizens is widely believed to be a necessary condition for the successful implementation of institutional reforms aiming towards more effective and accountable governments. In the last decade, governance and institutional reforms have been promoted in variety of middle income and developing economies. The implementation of these reforms recognizes that the quality of institutions in these countries plays an important role for economic development and poverty reduction. [Keefer and Knack (1995, 1997), Mauro (1995), Hall and Jones (1999), La Porta et al (1999), Acemoglu, Johnson and Robinson (2001)]. One example of such reforms is the right-to-information (RTI) laws – laws that establish the rights of citizens to have access to public information or exercise such a right already written in the constitution. 1 Since the 1980s, RTI laws have been implemented in more than 90 countries, starting with developed democracies. However, most of the new adopters are countries in Eastern Europe, Asia, Africa, Middle East and Latin America facing difficult governance environments and pressures from international aid agencies and intergovernmental bodies to adopt reforms that improve transparency and create more established democracies (Dokeniya, 2013) Having a law of access to information about government decision-making and expenditures can help citizens hold governments accountable to their citizens. But simply passing a law does not necessarily or immediately translate into improved governance (Banisar 2006; Calland and Bentley 2011). Rather, a number of conditions have to be in place for freedom of information laws to increase government transparency. In addition to a functioning government system, there must be some degree of political will on the part of government, a civil society actively engaged in pushing for and monitoring use of the law, and an involved and active media. Thus programs towards improving supply and demand for information are thought to be the way to promote a more effective use of the RTI laws. 1 The acronyms RTI (right to information), ATI (access to information) and FOI (freedom of information) will be used interchangeably for laws of access to information. 2 One example of such programs is the Access to Information (ATI) pilot project in the Dominican Republic (DR). Following the adoption of the Freedom of Information laws in 2004 and 2007, this project has aimed to educate citizens in poor municipalities about their legal right to request and receive information from government institutions, with the long-term objective to increase transparency and accountability in public service provision. Project implementation begun in the third quarter of 2009 completed in December 2011. The insights gained from this pilot are expected to influence the next phase of the program as well as Bank support to initiatives aimed at increasing accountability and transparency in government. To the best of our knowledge, little is known about the effects of such interventions in the Dominican Republic or in other developing countries. The little evidence that exists provides useful qualitative insights and before-and-after comparisons. In Mexico, for example, findings suggest that while the average person rarely exercised the right to obtain information; the most uneducated groups almost never used it. In response, the Federal Freedom of Information Institute developed the Comunidades project to raise awareness of the law among the country’s most disadvantaged individuals. The project involved twenty civil society organizations that worked with people in poor communities of nine Mexican states to teach them about the potential uses of the law and procedures for requesting public information. An evaluation of this project associates the use of the law in these communities with an increase in trust, training, and monitoring of the process (Guerrero- Amparan and Toledo, 2009). Similarly, Roberts (2010) studies the Right-to-information Act in India and finds that a substantial barrier to accessing information is lack of awareness about rights under the Act, particularly in rural areas, among females and socially disadvantaged groups. In addition, the author points other correlates of low use of the law such as lack of administrative capacity, low enforcement and hostility faced by individuals when placing requests. A notable shortcoming of the studies mentioned above is the absence of a credible counterfactual that allows “causal” inferences about the effects of ATI interventions on 3 different dimensions of empowerment among citizens. 2 This paper is an effort to fill this lacuna through the blending of qualitative methods with rigorous quasi-experimental quantitative methods employed to construct a credible counterfactual group. 3 Specifically, we employ the difference-in-differences method on individual survey data collected before and after the completion of the ATI pilot project using observations from treatment municipalities and a comparison group of municipalities selected by the method of “nearest neighbor” matching. The ATI pilot relates to empowerment in many ways. First, information itself is power. In particular civic education allows individuals to be better equipped to take advantage of opportunities, access services, exercise their rights, and demand better institutions and services. In addition knowledge about the freedom of information law’s roles on institution development could turn individuals into legal activists empowered to exercise the law. 4 Second, participation and inclusion of the poor together are critical at ensuring that limited public resources represent local priorities and do not exacerbate inequalities that could be reinforced by the exclusive participation of the local elites. The World Development Report 2004 is devoted entirely to the role of citizens’ voice and participation in holding politicians accountable. Empowerment of poor citizens can increase their influence in policymaking or align their interests with those of politicians, thus making basic services reach the poor. The report also defends that “non-governmental or civil society organizations can also help amplify the voice voices of the poor, coordinate 2 Our research also contributes to the literature on information and participatory programs. Previous impact evaluations of such programs in developing countries have mostly focused on public health and education service delivery. Studies typically find positive impacts on knowledge about health issues (Duflo, Dupas, Kremer and Sinei, 2006), on awareness of parents about the school roles or education (Pandey, Goyal and Sundararaman, 2007); Keefer and Khemani, 2011), on participation in community meetings or in the monitoring of service providers (Pandey et al, 2007; Pandey, Goyal and Sundararaman, 2007; Bjorkman and Svensson, 2009), in volunteering for small group community work (Banerji, Duflo, Glennerster and Khemani, 2010), on investment in schools (Reinikka and Svensson, 2011; Bjorkman, 2008; Andrabi, Das and Khwaja, 2009), and on parental investment in education (e.g., Keefer and Khemani, 2011). 3 Goncalves (2009) is perhaps the closest study to ours. She analyzes the gradual implementation of Participatory Budgeting (PB) across Brazilian municipalities. This program is similar to the ATI pilot, but more specifically, it consists of a participatory mechanism in which citizens negotiate with government officials over budget allocation and investment priorities. Using the difference-in differences method, she finds that the adoption of PB increased the spending that matches popular preferences as expressed in the participatory fora. In particular, it almost doubled spending in health and sanitation in the PB municipalities compared to the municipalities where PB was not implemented. 4 For related discussion on legalist studies, see for example Gauri (2009), World Bank (2005) and references therein. 4 coalitions to overcome their collective action problems, mediate on their behalf through redress mechanisms, and demand greater service accountability.” Finally, trust in state institutions and civil society contributes to greater participation and builds pressure for improved local governance, whether in setting priorities for expenditures, or providing access and quality of basic public services. We explore several dimensions of empowerment by analyzing the short-run effects of the intervention on intermediate outcomes measured by indicators of individual civic knowledge, awareness and use of the right to information, perceptions of and trust in public services and institutions, civic participation, as well as measures of local governance. The next section provides some background on the Access to Information project in the Dominican Republic. Section 3 presents the evaluation design and the combination of quantitative and qualitative methods used to evaluate the impacts of the ATI program. Section 4, discusses the findings while section 5 concludes with a discussion of the implications of the evaluation findings for the expansion of the pilot to the national level. 2. Background on the ATI Program In July 2004, the Dominican Congress approved the Law of Free Access to Information (FOI), [no. 200 of 07/2004], which makes it compulsory for government agencies to make information accessible to the public by allowing every citizen to monitor government actions. This kind of information includes budget allocation, status of public works projects, program implementation status, procurement and the government’s payroll, among others. The law also specifies the creation of an Office of Access to Information (OAI) in every government agency. These offices must guarantee permanent free access to up-to-date public information to all citizens. All agencies must establish information systems that are capable of providing this information on demand, and are liable to penalties if they refuse to provide information to the public. With the support of the Japanese Social Development fund and the World Bank, the Dominican government launched an intervention pilot project with intervention components aimed at enhancing both the supply of and the demand for public information 5 in seven poor municipalities in the DR. (see Box 1) Grupo Gestion Moderna (GGM), a consultancy firm, and Centro Juan Montalvo (CJM), a local NGO, were jointly selected to implement the project. The project included a number of activities aiming to increase the supply and demand for information. On the supply side GGM worked with local government agencies to raise their awareness about the importance of complying with the law. GGM also assisted local government agencies with organizing their management information systems and with establishing their OAI. On the demand side, CJM worked with local community organizations to raise their awareness about the importance of exercising their right to access public information. The project trained community organizations on a range of issues, including what types of information can be requested, how to request it, what to do in the case of refusal, and how the information once received can be used to hold municipal governments accountable. The knowledge was transmitted through workshops organized by CJM in different community organizations in each treatment municipality. 3. Evaluation Design and Data The intervention was carried out at the municipality level. A total of seven municipalities were selected by the program implementers, Centro Juan Montalvo (CJM) and Grupo Gestion Moderna (GGM). These municipalities were selected from five provinces covering different regions in the country, urban and rural areas, based on their relatively high poverty rates (above 40 percent, province level), willingness of the local mayor and council to cooperate in developing information systems and opening offices of information, and the existence of a functioning participatory budgeting process. Given that the treatment was not randomly selected and the lack of other sources of data to evaluate the outcomes of interest, matching was a natural choice. Each intervention municipality was matched with a group of five municipalities using nearest neighbor matching to minimize the statistical distance between the units based on population size (total, young and elderly), poverty rate, quality of life index, illiteracy, unemployment and remittances rate; the proportion of children per household, mean adult education, fraction 6 of male household heads, proportion of households lacking basic services in energy, garbage collection, subject to contamination by stagnant water and trash accumulation. As a final step, only one municipality, the first or the second closest, was selected based on consultation with the program implementers as a “control” for each municipality in the treatment group. Thus, a total of seven treatment and seven control municipalities were selected into the study. It is important to bear in mind that the participatory budgeting process is also present in the control municipalities. 5 It was not possible to ascertain the willingness of the local mayors in the control municipalities in developing information systems and opening offices of information as was done in the process of selecting the seven treatment municipalities. Table A.1, in the Appendix, lists the municipalities which entered the quantitative impact evaluation and indicates those selected for the qualitative baseline study. Quantitative data were collected in two rounds for a broad range of indicators from the seven ATI municipalities, and from seven comparable municipalities that did not receive project support. 6 The baseline survey round took place between December 2010 and January 2011, prior to the implementation of the ATI pilot program. The follow-up round took place in June 2012, six months after the completion of the intervention activities (in December 2011). The study was designed to detect a 10% relative increase in the primary and related outcomes indicating knowledge of the specific rights under the ATI law, satisfaction with requested information from public offices, civic participation, local governance, trust in public and non-public institutions, and perception about main public services. This resulted in a sample size of 1,932 households (966 households for the treatment and 966 for the control group), representing seven municipalities per treatment group, 10 clusters (estratos) per municipality, and around 13 to 14 households per cluster. 5 Additional details on the potential role of participatory budgeting in the impact evaluation of the ATI pilot are provided below. 6 Gallup Dominican Republic was contracted to conduct the field work for the baseline and end-line surveys. All instruments were translated into Spanish, and pre-tested prior to use in the field. Data collection instruments were administered to respondents in Spanish by native speakers. 7 Both the treatment and control groups comprise a representative sample of the population of households in the 14 municipalities of the study in each round. 7 The baseline survey above was complemented by the collection of qualitative data that included twelve focus group discussions (separate groups were organized with men and women, with a total of 95 participants) that took place in two treatment and two control communities with representatives of civil society organizations (CSO), individual interviews with representatives from community organizations, including neighborhood associations, women’s organizations, and a range of other social, cultural and development associations; radio and print journalists, and three mayors who had been in office about a month before the survey took place (Dudwick et al, 2012). Meetings were also held with two to three municipal council members in each of the four communities. An operational or process evaluation was also carried out to verify consistency between project design and actual implementation. The process evaluation report shed light on the challenges faced during the course of the intervention and helped interpret the findings of this impact evaluation. The combination of quantitative and qualitative data provided a very good basis for understanding the dynamics of change in the pilot communities. The group and individual discussions revealed the complexity of the context and the many activities already taking place in municipalities, including donor projects and NGO activities, directed at increasing people’s awareness of public affairs. Another issue that surfaced was the pervasive fear that requesting information would lead to public officials to see them as confrontational. Overall, the main issues identified during the baseline and implementation of the ATI project may be summarized as follows. Six years after the FOI law was enacted, only about twenty-six percent of respondents knew about transparency legislation. This generally reflects the low levels of education in the poor communities in the study, where just under 10 percent of households had a member with a university degree. Members of community organizations however 7Attrition is not an issue since the two rounds of survey represent cross-sections of households in the 7 treatment and 7 control municipalities (i.e. not a panel of households). The follow-up survey contained 1,918 interviewed households. 8 were more likely to be aware of the law, even when characteristics such as education, occupation, gender, and age were taken into account. Community organizations are the principal conduits for obtaining information. This is linked to (i) existing transparency organizations such as Participacion Ciudadana who have worked with local NGOs to disseminate information about Participatory Budgeting and transparency legislation, (ii) improved capacity and empowerment of many CSOs in the past ten years, especially those representing urban and relatively better educated segments of the population, (iii) concerns that officials would consider requests for information about financial matters confrontational, and respond by firing a close family member or friend working in the public sector. Other individuals also reported fear of physical reprisals, related to a long history of authoritarian regime. Local politicians and civil servants were considered as the most serious bottlenecks when it comes to accessing information. Local mayors are very powerful as acknowledged by community representatives and municipal officials. On the other hand, local officials were aware of rising public expectations, and understood they were under greater scrutiny from CSOs and media. Yet, they stressed the financial challenges faced in setting up mechanisms for providing information to public, they also expressed their intention to disseminate information through the local mass media, government bulletins and newsletters but were vaguer about setting up procedures for responding to information requests. This is probably a reflection of their perception about what access to information should be or simply from the fact that relatively few officials knew the procedures for requesting information under the FOI law, and some were unsure about whether individual citizens could pursue requests. A recent process evaluation on the status of ATI project implementation suggested that this project fulfilled its objective to establish Social Audit Committees (SAC) in each municipality of intervention. Further, all SACs participated in a broader national network (Foro Ciudadano) through the local support from Centro Juan Montalvo. This connection would guarantee that requests for information would be placed formally and through an established national organization, thus ensuring a successful and speedy delivery of information. 9 Despite this, the SACs formed lacked basic governance rules. A presidential decree establishes that five members of any SAC should be selected during an open meeting with community organizations. Any resident in the community can become a member, except elected civil servants or those working directly in the activities executed by the public institution hosting the SAC. In addition, the creation and composition of the SACs should be notified to the Department for Prevention of Administrative Corruption (DPCA), thus preventing the establishment of long-lasting commissions which could lead to conflict of interest and capture. The implementation evaluation report reveals that the creation of SACs in some municipalities did not follow these rules. Many of the SAC members in the municipalities of the intervention are civil servants or CBO members with limited skills to monitor infrastructure projects or to analyze municipal budgets, hence showing low independence and little capacity within the SACs. While there were oversight institutions, the GGM and the CJM, supervising the actions of SACs and working together with the CBOs probably played a major role in ensuring that local governments provided information. As soon as the ATI project finished in November 2011, SACs in at least three out of seven municipalities faced severe problems to finance their operations, since participants could not afford paying for transportation. As a consequence, meetings and requests for information stopped after the completion of the ATI pilot project. These challenges encountered during the pre-program and implementation phases of the program provide useful explanations for the observed impacts (or absence of impacts) of the ATI pilot program. a. Indicators of impact Quantitative data were collected to investigate whether the ATI project had impacts on knowledge, access to information, civic participation, local governance, trust in public and non-public institutions and perception about main public services. These are believed to be important intermediate mechanisms for the information legislation to ultimately have positive results impacts on public transparency and accountability. Specifically, we investigate the ATI impacts on knowledge about the FOI law, existence of offices of access 10 to information, knowledge on existing related processes/projects such as the Participatory Budgeting and other civic knowledge for example local and central government responsibilities in terms of main services provision, local and central government activities, projects and knowledge on who is the mayor or the president. In terms of use of information, we look into whether individuals have asked for public information to a government official or department. As for civic participation, we investigate individual participation into collective action activities: participation in CBOs, in Participatory Budgeting meetings, in protests or strikes. In addition, we investigate whether the individual placed a complaint to the government, voted in municipal elections, or supported campaigns for city council candidates. While knowledge, access to information and civic participation are intended to measure the first order impact on the demand side, local governance is intended to measure the direct impact on the supply side of information. Proxies for local governance indicate whether local government consults the citizens’ opinions for project prioritization, for making decisions and whether the government is spending in key areas, always as perceived by households. This said, it is possible that an improvement in local governance is driven by an increase in awareness of individuals pushing for greater transparency rather than the supply side directly improving government information provision. While are not able to disentangle supply from demand effects, we can infer which one is likely to be more predominant given the intervention design and how the program was implemented in practice. Regarding individual perception of public institutions, public services and CSOs that could improve the collective participation of individuals and demand information, we investigate whether individuals trust government institutions (central, local, justice courts and police) as well as whether they trust non-government institutions such as CBOs, NGOs, political parties, church, besides neighbors in their community. In addition, we look at the satisfaction of individuals with each of the key public services in their community. 11 b. Balancing tests The baseline quantitative paper presents several balance tests and detailed descriptive analysis for all the variables contained in the questionnaire. In this paper we present the balancing tests only for the main variables. Tables A.1 and A.2 show comparison of means between treatment and control groups at the baseline. Given that we used matching to obtain the control group, the p-values refer to a test in which we compare means in the treatment group with ‘weighted’ means in the control group. 8 In terms of the main household characteristics, Table A.1 shows that the null hypothesis of equality of means between treatment and control groups is not rejected for most key variables. The tests where the null is rejected show that at the baseline the treatment may be slightly worse than the control group in terms of basic services of sanitation, sewer, electricity, water, garbage collection and house property. All other variables show great balance of household characteristics between treatment and control groups. Table A.2 reports the main outcome variables. All except one are found very similar across treatment and control groups. Under a 90% confidence level, trust in CBOs is lower in the ATI municipalities than in the control ones. Other than this, all other tests indicate that at the baseline, there were no major differences between outcomes in the treatment and control groups. c. Difference-in-Differences Matching (DDM) estimator Our estimation relies on quasi-experimental methods. The program implementers chose the treatment municipalities and we paired the treated municipalities with comparison or control municipalities using a matching procedure. Baseline analysis showed that most key exogenous and program variables are well balanced, which reinforces our attempt to obtain robust impact estimates. Our strategy then consists of comparing means of outcomes between municipalities assigned to treatment to mean outcomes in the control municipalities before and after the initiation of the ATI pilot. The parameter of impact that we estimate is the intent-to-treat effect (ITE), which is an estimate of the impact of making 8 These weights are proportional to the number of times a control municipality (from the universe, excluding the treatment group) is matched with another municipality in the treatment group. 12 the ATI program available in the treatment municipalities regardless of whether households actually participate (by choice or not) in the training sessions offered by the ATI program. The ITE is of particular interest to policy makers since it captures the operational efficiency of the program, such as inefficiencies in the delivery of the program, as well as any potential general equilibrium or spillover effects of the program. Below we detail our estimation method. The typical matching estimator pairs each municipality in the treatment group to an observationally similar municipality (or several) in the control group and compares the differences in their outcomes as the effect of the program (Rosenbaum and Rubin, 1983). In practice, using outcomes post- intervention, this would be estimated from: � M = 1 ∑:{ =1}∩S{1 − W(i, j)Y0j } (1) #{ =1} where 1 represents the average of outcome Y within the treatment municipality i and the average of outcome Y within the control municipality j which was paired to i. W(i, j) are weights which depend on the metric used for the distance between treatment and matched to treatment municipalities. 9 One of the maintained assumptions for the validity of this ITE estimate of the impact of ATI based on the matching estimator is the conditional independence assumption (CIA) assumption which assumes the absence of the role of any unobserved variables such as political affiliation of mayors etc. into the selection into the treatment group (i.e. selection based on observables). In fact the choice of the seven treatment municipalities was decided by GGM and CJM in charge of the program implementation and was based on the willingness of the municipality government to engage on the supply-side of the project and the existence of a functioning participatory budgeting process. 9Only one control municipality satisfying the common support region was chosen to pair with each treated municipality. Weights are proportional to the number of times a control municipality is used to form a match in the treatment group (see Abadie and Imbens, 2002). 13 The availability of the baseline survey allows us to estimate the impact of ATI based on the double-difference approach whereby the matching estimate of the ITE ex-post is netted out of any preexisting differences between the treatment and control municipalities. Since observed characteristics and outcomes were balanced across the treatment groups at baseline, inclusion of the baseline does not change the results, but leads to more precise estimates. Given that we have repeated cross-sectional data, as opposed to longitudinal data, the estimator following Smith and Todd (2005) can be implemented as: 1 I� DDM = � {1 − W(i, j, t)Y0jt } − #{ = 1} :{ =1}∩S 1 ∑:{ {1 ′ − W(i, j, ′ )Y0j ′ } =1}∩S (2) #�′ =1� ′ where t and t’ indicate periods after and before the program start date, respectively. This procedure differs from the one above in that matching is performed now twice, before and after the program starts. 10 The second term in the ITE-DDM estimator differences out from the ITT-M estimator any pre-program differences in the outcomes between treatment and control municipalities, hence producing more robust estimates of program effects. We consider time invariant covariates, the ones we use to construct our control group. Given this and that the control municipality remained control after the program started, it is plausible to assume W(i, j, t)= W(i, j, t′), i.e. the matching procedure is performed only once, before the intervention. The standard error for a DDM estimator does not have an analytical form. Therefore, we construct standard errors using bootstrapping. d. Potential threats to the evaluation design 10 The difference-in-differences matching estimator is defined in Heckman, Ichimura and Todd (1997) and Heckman, Ichimura, Smith and Todd (1998). Blundell and Costa-Dias (2008) also present a variant of the estimator by Smith and Todd (2005) applied to a similar context, with repeated cross section data. 14 The evaluation design above does not distinguish explicitly the potential effect of the Access to Information pilot from the effect of participatory budgeting. As pointed out above, one of the selection criteria for the municipalities in the pilot ATI, was the existence of a functioning participatory budgeting process. However, the implementation of participatory budgeting meetings was not uniform across the seven treatment municipalities. For example, the Monte Plata municipality had functioning participatory budgeting, whereas San José de los Llanos and Hato Mayor municipalities did not. 11 It is important to point out, however, that the participatory budgeting process is also present in the control municipalities. According to the baseline survey about 25% of the households in both treatment and control municipalities knew what a participatory budget process was, and most of them heard about it through friends and family (43% and 38% in control and treatment). 12 Approximately 53% and 60% of respondents in the control and treatment municipalities, respectively, who reported knowing what a participatory budget process was, reported that participatory budget meetings were being undertaken in their municipality (this difference is statistically significant at the 10% level but not at the 5% level), and approximately half of them had participated in at least one meeting, with no significant differences between the two groups. Among those people that had participated in the meetings, 76% in the control municipalities and 60% in the treatment municipalities had expressed their preferences during these meetings (statistically significantly different at the 5% level). To the extent that the participatory budgeting processes were already in place in both treatment and control municipalities and their effects on people's perceptions have already taken place prior to the implementation of the ATI, then one could reasonably argue that the application of the DiD estimator is likely to yield an estimate of the impact of the ATI program that is "net" of the effects of participatory budgeting on key outcome indicators. However, this statement needs to be qualified further, since participatory budgeting was "work in progress" with possibly measurable impacts on people's perceptions and attitudes during the life of the ATI project. 11 See Lima (2011). 12 See Table 1 Appendix B of Lima (2011). 15 4. Results In this section, we review the results on awareness and use of information, participation in collection action activities, local governance, trust in public institutions and CSOs, and satisfaction with public services. The reported estimator is the intent-to-treat effect obtained using difference-in-differences with matching (DDM) as we explained in the previous section. 13 Awareness of and Access to Information Table 1 shows the main results of the ATI project on awareness and use of the FOI law. The ATI pilot’s impact is positive, since it appears to raise knowledge about rights under the FOI law by 3.6 percentage points (on average). However, the impact is not statistically significant. While the average impact is not statistically significant, the project could have had positive impacts on awareness of FOI law among certain sub-groups, which is not possible to determine due to the limited sample sizes. Since initial awareness of transparency legislation associates positively with covariates such as the level of education or wealth, the ATI project could have been more effective among more educated and wealthier groups. In addition, it could have been more effective among individuals who participate in community organizations since these have been actively disseminating information in many communities. Findings from the qualitative evaluation highlight that many of the community leaders and officials interviewed during the study credited their awareness to workshops and seminars that Participacion Ciudadana had held throughout the country on transparency legislation. Table 1 here Although provision of information by public institutions is a mandate, by 2011, 60 percent of public institutions still lacked information offices (World Bank 2011). While 13In Appendix C, we also estimation results using the simple ex-post difference with matching to compare with the DDM estimates. Also, in Appendix D, we report results of a bias correction method, which uses matching with regression. 16 there was potential for the ATI intervention to improve government capacity and thus knowledge on existence of OAIs, our evaluation findings show no relevant impact of the intervention on knowledge about existence of an OAI. It appears that this knowledge has increased similarly in both treatment and control groups which would be consistent with the presence of spillover effects across treated municipalities. It also appears that knowledge is greater in the chosen ATI municipalities both in the baseline and endline surveys, thus implying no differential effect from the intervention. In terms of general civic knowledge, when households were asked to identify whether a certain public service is provided by the central or local government, the proportion of correct answers is higher in treatment than in control municipalities for key services. Specifically, the proportion of correct response on education – that it is a central government responsibility – was 4.2 percentage points higher in treatment areas than in control areas. Surprisingly, despite similarity between the ATI program and other transparency projects such as the Participatory Budgeting, knowledge on the latter wasn’t influenced by the ATI program. Neither of these findings was found to be statistically significant. Regarding the use of information, the surveys show a large increase in requests for information to public offices or officials in the ATI municipalities and a small decrease in the control municipalities. The differential effect is positive, 1.5 percentage point, but not statistically significant. That this parameter is small also reflects findings raised by the qualitative baseline study that identified several barriers to accessing information, especially in the poorer communities in the DR. First, participants in the focus groups blamed pervasive illiteracy for making people unaware of their rights as citizens. Because people do not understand their rights, they do not make demands – as a result, professionals or lawyers are the ones most able to pursue these rights on their behalf. Second, although people wanted to know more about what the public authorities were doing, they feared that demanding information about financial matters could appear confrontational. Participants of the study attributed fear of reprisal to the legacy of dictatorship, under which they lived for 32 years. Thus they seemed to believe that the 17 government would only respond to well-educated, well-spoken or well-connected people (for example, CBO members), but ignore those of lower status. Nonetheless, our quantitative evaluation shows that CBO members were just a bit more likely to access information than non-CBO members across ATI and non-ATI municipalities, and that this difference is not significant. In principle, this shows that barriers to access information seem to extend to everyone including organized individuals. Another dimension of access to information that was studied was whether the individual had complained to the local government in the year prior to the survey date. The project led to an increase of 4.5 percentage points in the proportion of households that submitted complaints to the government. Unlike placing requests under the FOI law, submitting complaints is much more general and covers a much broader range of issues than the FOI law does. In this regard, the qualitative paper for this evaluation shows that all interviewed individuals (in selected treatment and control municipalities) acknowledged the right of citizens to access information from public institutions. However, this notion was present among those who knew as well as those who did not know about the existence of the law. In addition, the rights to access information are stated in a very general and vague way. There is no clear understanding of the ways this information should be accessed or the type of information people would like to obtain. In many cases it is even confused with social demands like holding public officials accountable to their campaign promises, or getting the council to pay for medicine and funeral arrangements for members of the community (Vega, 2011) Participation in collective action activities Membership in community-based organizations (CBO) is considered high in the DR. Nearly fifty per cent of the households have someone who belongs to a religious, farmer, neighbor or a non-governmental organization. Table 2 shows that pre-program difference in CBO membership between treatment and control regions is not significant, and that membership has increased for both groups. Surprisingly, this is higher in the non-ATI than in the ATI municipalities. While there is no statistical significance, this is rather puzzling, 18 since the ATI project worked through CBOs to raise awareness about the law. One would expect that this would have created stronger incentives for group membership in treated more than in control municipalities. One possible reason why it did not could be a substitution effect. Where individual CBO members become more empowered to request information, there is likely to be lower demand for collective action. However, the fact that group membership has grown over the two rounds of the survey in both treatment and control municipalities is interesting in itself and probably a reflection of the Municipal Law 176-07 that complements the FOI law 200-04. A qualitative baseline assessment reports that the Municipal Law 176-07 promoted the development of CSOs, through greater access to public information, use of public media and financial assistance, to propel their participation in the municipal administration. 14 Engagement in other transparency projects such as the Participatory Budgeting has similarly increased in both ATI and non-ATI municipalities, which is to say, independently of the ATI project intervention. Table 2 here Local governance As Table 3 reports, the impacts of the ATI on local governance appear to be the strongest so far. They indicate a higher rate of consultation of citizens in government decision making across samples and periods. The general decline from the baseline to the follow up survey is somewhat puzzling. It may have occurred due to the local elections that took place just before the collection of the baseline survey, thus reflecting candidates’ campaign or commitment to greater accountability than previous administrations. Nonetheless, looking at the differential effect of the ATI, we observe a positive and large effect, i.e. the reported consultation rate of citizens by local governments in the treatment municipalities is higher by 7.8 percentage points compared to consultation rate 14 The financial incentives to the creation and development of non-profit organizations consist of a competitive grant in which the organizations with the project which could benefit mostly the community would win, as established by art. 227 of the Law No. 176-07. 19 in the control municipalities. Proportion of citizens who believe that local governments are spending in key areas for their community is also higher (by 8.1 percentage points) in the treatment than in the control municipalities. Table 3 here The importance and statistical significance of these results are intricately linked to how the program operates at the local level. However, we cannot disentangle which factor, supply of or demand for information, is primarily responsible for this result. The pre-program and implementation evaluation seem to point to failures of the ATI intervention in creating institutions and capacity that would improve the quantity and quality of information. On the other hand, the increasing strength and training of local CBOs are believed to be the strongest component of the program, pushing governments to address complaints and to consult citizens for the execution of public investments in key areas for the community. Qualitative findings from interviews with mayors also reveal their perception about transparency in the ATI municipalities of Bayaguana and Hato Mayor del Rey. According to these, a transparent management is perceived to increase citizens’ trust in mayors, which would favor the re-election of mayors who promote such trust. Hence, trust of households in the government seems to be an important channel through which transparency laws can translate into greater transparency of public institutions. In this context, the impacts of ATI intervention on trust in public private institutions, discussed below, acquire special meaning. Trust Table 4 reports the effects of the ATI program on trust in public and non-public institutions. The measure in use is the proportion of households who responded as having complete trust or some trust in these institutions. Consistent with the positive results we found for most local governance outcomes, we also observe a positive effect of the ATI program on the level of trust of households in the local (municipal) government. One of the components of the ATI intervention involved working with municipalities to increase capacity to respond to demand for information. 20 That reported trust in local governments increased by 9.5 percentage points in the municipalities with the ATI program – while trust in other institutions of the public sector did not show a significant increase – is consistent with a direct effect of the program on the supply (and maybe even quality) of public information at the local level, at least in the way individuals perceive it. Trust in individuals and CSOs Though positive, the effects of the ATI program on trust in CSOs (such as church, political parties, NGOs and CBOs) are not statistically significant. However, it is worth noting that the trust in CBOs is large and higher by 8.6 percentage points due to the program (though statistically insignificant). The second component of the ATI program consisted of working intensively with the CBOs to increase awareness in the communities. The increase in trust in CBOs, present only in the treatment group, is indicative of the effectiveness of this channel. Findings from the qualitative study also indicate that one reason that people were more likely to trust civil society organizations is their history of resistance to the authoritarian regime. Social movements or activism were historically supported by peasant movements, Christian communities, cultural and sports clubs, and NGOs. Table 4 here Satisfaction with public services As Table 5 shows, there is little or no evidence of project impacts on satisfaction with public services, with the exception of maintenance of public parks. Services for which there is no significant impact on satisfaction include those provided by the central government (such as health, education, electricity, sewer and water) or the municipality (such as garbage collection, roads, and transport). For most of these services, there seems to be a common time trend of increased satisfaction in both treatment and control municipalities. The resulting net treatment impact is null in the case of health, education, transportation and roads. The only significant and positive impact observed is on satisfaction with maintenance of public parks (greater by 9.3 percentage points in the ATI municipalities). That any positive impact on satisfaction in ATI municipalities was limited to a service 21 under the purview of the local government makes sense – the limited size of the pilot meant that any increase in demand for accountability among citizens would have affected the incentives of only the municipal government. Table 5 here What remains less clear, however, is why the positive impact on satisfaction occurred for one municipal government service (maintenance of public parks) only and not the others. The qualitative study appears to provide some clues. First, most people care about municipal budgets and information on payrolls and expenditures. Second, spending on maintenance or construction of parks (and to a lesser extent) is among the most visible to citizens. During focus group meetings and interviews, people seemed concerned with taxes and budgets, and particularly with what is being done with their taxes, focusing mostly on visible signs of improvements (or lack thereof) in their community. As seen in Chapter 1, a respondent mentioned a children’s park that was to be paid for from the local budget – “from our pockets” - but has remained unfinished for years. Others talked about using the right to information to get the council to give money to their organizations or to find ways to have the local government build things for the community, such as parks and bridges. For most people, having more information was a way to oversee the expenditure allocation of local, and sometimes central governments (Vega, 2011). Knowing about these concerns, and responding to increased demand from citizens for accountability in the ATI treatment areas, local governments in ATI municipalities might have focused on spending to improve the quality of visible services like public parks. Heterogeneous impacts For younger households (40 years or less), effects are twice as much effects for the average, for all outcomes except access to rights. The p-values reduce in all cases, suggesting that effects are much higher so detectable with the sample we have. In the limit, this also suggests a positive and higher effect of the ATI pilot on awareness of younger households, if we had a larger sample. Table 6 here 22 5. Conclusions and Considerations for Project Expansion This paper summarizes the results of the impact evaluation of the ATI pilot project on empowerment of citizens in poor municipalities in the Dominican Republic. Among the dimensions of empowerment investigated are civic knowledge, awareness and use of the right to information; perceptions of and trust in public services and institutions; civic participation; and measures of local governance. The quantitative data collected in two rounds; baseline at the end of 2010 and follow up in mid-2012, is based on the premise that the project could have some measurable impacts one and a half years after its initiation. This relatively short time horizon may be sufficient for some indicators to change but certainly not all. Indeed, as it is apparent in the paper, many indicators do not change relative to the comparison group, or even if they do the changes (or the impacts) are too small or statistically insignificant. Nevertheless it is important to point out that some positive (and statistically significant) impacts are observed on local government responsiveness, on prioritization and decisions about municipal budget, and on trust in and satisfaction with some local government services. Bearing these caveats in mind, the results of this paper shed some light on the type of outcomes that can be influenced by interventions enhancing the supply of and demand for public information in poor areas, and potential areas of improvement for such interventions. Specifically, we observe no impact on awareness and access of information under the specific ATI rules. However, we do observe that individuals manage to address more general complaints to governments as a result of the ATI program regardless of whether these are classified under the FOI law or not. We also do not find any impact on participation in collective action activities. But the ATI pilot appears to have been more effective in other dimensions that are detailed below: The ATI intervention increased the consultation of citizens by government about where to invest at the local level, as perceived by citizens. It also resulted in an increase in expenditures that match people’s preferences in the communities. These results seem to be linked to how the program operates. There seems to be a greater role played by the demand side of the intervention as pointed out during the pre-program and interim stages 23 of the evaluation. The increasing strength and training of local CBOs are believed to be the strongest component of the program, pushing governments to address complaints and to consult citizens for the execution of public investments in key areas for the community. The ATI intervention also improved trust in local governments. This is in line with the previous result indicating significant improvement in local government transparency, as perceived by citizens. Greater pressure for improved local governance and more response from mayors probably contributed toward the program’s success in improving trust in municipal governments. The ATI intervention increased satisfaction with maintenance of public parks, but had no significant impact on satisfaction with other services provided by the municipality or the central government. Citizens seem to associate freedom of access to public information with the right to know about municipal budgets – how the taxes they are contributing to the municipal governments are being spent by the government. It seems intuitive that people tend to be more focused on investments that are more visible to them, such as public parks, which could explain why municipal government might have focused on improving the quality of this type of spending. Even though the ATI intervention demonstrated some success in providing incentives for better governance at the municipality level, there are a few potential areas of improvement in the design of these types of interventions that deserve serious consideration if they are to lead to greater empowerment and improved governance. Most of the interventions supported by the ATI pilot – raising awareness and educating citizens about how to demand information – occurred through community organizations. As mentioned earlier, CBOs are also generally seen as the principal conduits for request for information, and the ATI pilot leverages (and reinforces) this perception. In the past ten years, CBOs have been supported as part of initiatives to promote democratization and government transparency. However, this has occurred primarily among urban and relatively better educated segments of the population. In poor municipalities such as the ones included in this pilot, CBOs are likely to lack the resources and the training required to demand information effectively. While the ATI pilot attempted to correct for the lack of capacity through initiatives to train CBO members, these may have 24 been inadequate and short-lived. Future programs of a similar type can become more effective by focusing even more on – and improving the content and length of – such training. Intervening through community organizations is probably the most effective way to increase demand for information, as members and leaders of such organizations are relatively better off, more educated, more engaged in entrepreneurial activities, and thus more knowledgeable and successful in requesting information about government actions. On the other hand, channeling information requests through community leaders may run the risk of capture by local elites, which could in turn imply that the concerns of the poorest or most disadvantaged citizens are inadequately represented. While the risk of elite capture is something this study has not analyzed, it is a risk that merits consideration in the design of future ATI-type interventions. Whether elite capture of CBOs has occurred in these areas will also be a subject of upcoming research using the evaluation survey data. Since CBOs are likely to have limitations in terms of outreach among citizens, should future programs use media sources, such as television, more intensively? Recent literature shows the potential for ICT to enhance government outcomes (Zhen-Wei Qiang et al, 2006). The process evaluation report reveals that information campaigns through TV and radio were carried out only for one quarter of the calendar year, towards the end of the program implementation period. Despite this, the quantitative baseline survey reveals that when individuals are asked to respond spontaneously about how they heard about the FOI law, 60 percent answered TV, 27 percent radio, 18 percent family and friends, 11 percent newspapers, 5 percent community meetings, 5 percent CBOs, and less than 1 percent from other activities such as workshops. These numbers were similar in the follow up survey, but with the difference that the percentage of those who heard about the law from community meetings or CBOs increased to 9 and 7 percent respectively. Given this, future projects of this type should consider using media channels more intensively to disseminate information, to complement efforts through CBOs. The results from the surveys also suggest that efforts to raise awareness about right to information laws are likely to be more successful if the type of information that is disseminated is customized or tailored to fit more closely with the immediate concerns and 25 needs of citizens. A program to attract the interest of people would be one that is localized, and has clear relevance to issues close to the immediate concerns of citizens. In poor municipalities where the ATI pilot was implemented, people seem to be most interested to demand information on services that create jobs or generate income, or visibly affect the quality of their day-to-day lives (such as poor maintenance of public parks) – and related to that, what their municipality is doing with the taxes they pay. Given this, the focus of the ATI pilot on information about services provided by (and expenditures by) the municipal government seems to be appropriate. That said, there may be scope to further strengthen the “localization” component of future programs with similar objectives. The process evaluation suggests that the governance of social audit committees (SAC) was a major challenge. Most SAC members were civil servants or CBO members with limited skills to monitor infrastructure projects or to analyze municipal budget, resulting in low independence and capacity within these committees. We cannot disentangle the effects attributable to supply or demand side interventions. However, based on what we know about the implementation of the project, it appears that the impacts we find were mainly a result of demand side interventions, namely ones that seek to increase demand for information and accountability (of municipal government) among citizens. The ATI pilot seems to have an important implication for future programs: demand- side interventions by themselves, even with uneven implementation, can have some impact on citizen empowerment and local governance. Beyond this, what the results imply for future projects – on the efficacy of supply versus demand-side interventions – is unclear. Given that the impacts appear to have resulted mainly from demand-side interventions, one could argue that it would be more cost-effective for future programs to focus even more on the demand side at the expense of the supply-side. On the other hand, one could also argue that the impacts of the program could have been higher with a more effective implementation of the supply side of the intervention, by improving the quantity and quality of information provided to citizens. Finally, it is interesting to note that the impacts seem to have occurred in the absence of any significant impact on citizen awareness or knowledge of the ATI law. This seems to suggest that improving knowledge about the law among the average citizen is not 26 a necessary condition for citizen empowerment to occur through demand-side interventions. In the case of the ATI intervention, empowerment seems to have occurred mainly through increased engagement of CBOs in the process, even though these organizations are low on capacity and have limited outreach among citizens. Whether this is the most cost-effective way to use ATI law to improve citizen empowerment is however a question that this project cannot shed light on. To address this question, it would be useful to design interventions that combine different potential ways to raise awareness on ATI law, and evaluate these against each other to find the most cost-effective ways of increasing awareness and citizen empowerment. This could include, for example, outreach through CBOs, media channels and combinations thereof – and each method could be evaluated against the others for impact and cost-effectiveness. We hope that the findings of this study will be a catalyst for such experimentation and research, since the use of ATI law is a powerful potential tool for citizen empowerment, which can raise the quality of governance and services that affects the lives of poor people everywhere. 27 References Abadie, Alberto and Guido W. Imbens (2002). “Simple and Bias-Corrected Matching Estimators for Average Treatment Effects”, NBER Technical Working Paper 283. Abadie, Alberto and Guido W. Imbens (2006). “Large Sample Properties of Matching Estimators for Average Treatment Effects”, Econometrica, 74(1), pp. 235-267. Acemoglu, Daron, Simon Johnson and James A. Robinson (2001). “The Colonial Origins of Comparative Development: An Empirical Investigation”, American Economic Review, vol. 91(5), pp. 1369-1401. Andrabi, Tahir, Jishnu Das and Asim Ijaz Khwaja (2009). “Report Cards: The Impact of Providing School and Child Test-scores on Educational Markets”, Duke University, mimeo. Banerjee, Abhijit, Rukmini Banerji, Esther Duflo, Rachel Glennerster and Stuti Khemani (2008). “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India”, NBER Working paper no. 14311. Banisar, D. (2006). “Freedom of Information Around the World 2006: A Global Survey of Access to Government Information Laws.” London: Privacy International. Blundell, Richard and Monica Costa-Dias (2008). “Alternative Approaches to Evaluation in Empirical Microeconomics”, Cemmap Working Paper CWP 26/08, Department of Economics, UCL. Bjorkman, Martina (2008). “Public Funding in the Educational Sector and Its Effect on Test Scores”, IIES Monograph Series no. 56, Stockholm University. Bjorkman, Martina and Jakob Svensson (2006). “Power to the People: Evidence from a Randomized Experiment of a Citizen Report Card Project in Uganda”, mimeo, IIES, Stockholm University. Bjorkman, Martina and Jakob Svensson (2009). “When is Community-Based Monitoring Effective? Evidence from a Randomized Experiment in Primary Health in Uganda”, mimeo, IIES, Stockholm University. Calland, R. and K. Bentley (2011). The impact and effectiveness of accountability and transparency initiatives; Freedom of Information. Mimeo. Dokeniya, Anupama (2013). “Implementing Right to Information: Lessons from Experience”, The World Bank, Poverty Reduction and Economic Management, Washington, DC. Dudwick, Nora, Lycia Lima, Valery Vega, and Renata Narita (2012). “Access to Information in Dominican Republic: What a Mixed-Methods Survey Tells Us About Attitudes and Expectations.” Background Paper for the ATI Project Report, The World Bank, PREM Poverty Reduction and Equity Group, Washington, DC. Duflo, Esther, Pascaline Dupas, Michael Kremer and Samuel Sinei (2006). “Education and HIV/AIDS Prevention: Evidence from a Randomized Evaluation in Western Kenya”, WPS4024, World Bank. Dupas, Pascaline (2011). “Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya”, American Economic Journal: Applied Economics, vol. 3, pp. 1–34. 28 Gauri, Varun. (2009). “How Do Local-Level Legal Institutions Promote Development? An Exploratory Essay”, Justice and Development Working Paper Series no. 6, The World Bank. Goncalves, Sonia (2009). “Power to the People: The Effects of Participatory Budgeting on Municipal Expenditures and Infant Mortality in Brazil”, mimeo, London School of Economics. Guerrero-Amparan, J. P and M. S. Toledo (2009). “The Right to Information for Marginalized Groups: The Experience of Proyecto Comunidades in Mexico: 2005-2007.” Mexico City: William and Flora Hewlett Packard Foundation. Hall, Robert E. and Charles I. Jones (1999). “Why Do Some Countries Produce So Much More Output Per Worker than Others?”, Quarterly Journal of Economics, vol. 114 (1), pp. 83-116. Heckman, James J., Hidehiko Ichimura and Petra Todd (1997). “Matching as an Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme”, Review of Economic Studies, 64(4), pp. 605-654. Heckman, James J., Hidehiko Ichimura, Jeffrey A. Smith and Petra Todd (1998).”Characterizing Selection Bias Using Experimental Data”, Econometrica, 66(5), pp. 1017-1098. Imbens, Guido W. (2004) “Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review”, Review of Economics and Statistics, 86(1), pp. 4-29. Imbens, Guido W. and Jeffrey M. Wooldridge (2009) “Recent Developments in the Econometrics of Program Evaluation”, Journal of Economic Literature, 47(1), pp. 5- 86. Keefer , Phillip and Stuti Khemani (2011). “ Mass Media and Public Services: The Effects of Radio Access on Public Education in Benin”, Policy Research Working Paper no. 5559, The World Bank. Keefer, Philip and Stephen Knack (1995). "Institutions and Economic Performance: Cross- Country Tests Using Alternative Institutional Measures", Economics and Politics, vol. 7(3), pp. 207-227. Keefer, Philip and Stephen Knack (1997). "Why Don’t Poor Countries Catch Up? A Cross- National Test of an Institutional Explanation", Economic Inquiry, vol. 35, pp. 590- 602. La Porta, Rafael, Florencio Lopez-de-Silanes, Andrei Shleifer and Robert Vishny (1999). "The Quality of Government", Journal of Law, Economics, and Organization, vol. 15(1), pp. 222-279. Lima, Lycia (2011). “Expanding Access to Information to the Poor and Vulnerable Communities in the Dominican Republic: Findings from the Impact Evaluation Baseline Quantitative Survey”. Background Paper for the ATI Project Report, PREM Poverty Reduction and Equity Group, Washington, DC. Mauro, Paulo (1995). "Corruption and Growth", Quarterly Journal of Economics, vol. 110, pp. 681-712. Olken, Benjamin A. (2007). “Monitoring Corruption: Evidence from a Field Experiment in Indonesia”, Journal of Political Economy, vol. 115(2), pp. 200-249. Olson, Mancur (1965). “The Logic of Collective Action: Public Goods and the Theory of Groups”, Cambridge: Harvard University Press. 29 Pandey, Priyanka, Sangeeta Goyal, and Venkatesh Sundararaman (2007). “Community Participation in Public Schools: Impact of Information Campaigns in three Indian States”, Unpublished manuscript, World Bank, Washington, D.C. Pandey, Priyanka, Ashwini R. Sehgal, Michelle Riboud, David Levine, and Madhav Goyal (2007). “Informing Resource-Poor Populations and the Delivery of Entitled Health and Social Services in Rural India: A Cluster Randomized Controlled Trial”, Journal of the American Medical Association, vol.298, pp. 1867-1875. Reinikka, Ritva and Jakob Svensson (2011). “The Power of Information in Public Services: Evidence from Education in Uganda”, Journal of Public Economics, vol. 95, pp. 956- 966. Roberts, A. (2010). “A Great and Revolutionary Law? The First Four Years of India’s Right to Information Act.” Public Administration Review, 70 (6): 914-924. Rosenbaum, Paul R. and Donald B. Rubin (1983). “The Central Role of the Propensity Score in Observational Studies for Causal Effects”, Biometrika, 70(1), pp. 41-55. Rosenbaum, Paul R. and Donald B. Rubin (1985). "The Bias Due to Incomplete Matching" Biometrics, 41, 103-116. Smith, Jeffrey A. and Petra E. Todd (2005). “Does Matching Overcome LaLonde’s Critique of Nonexperimental Estimators?”, Journal of Econometrics, vol. 125, pp. 305-353. Vega (2011). “Qualitative Research Report”. Background Paper for the ATI Project Report, The World Bank, PREM Poverty Reduction and Equity Group, Washington, DC. World Bank. (2004). World Development Report: Making Services Work for Poor people, The World Bank and Oxford University Press. World Bank (2005). Law and Development Movement. Law and Justice Institutions. Brief. http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTLAWJUSTINST/0,,con tentMDK:23103515~menuPK:1989584~pagePK:210058~piPK:210062~theSitePK :1974062~isCURL:Y,00.html World Bank (2011). Country Partnership Strategy Progress Report for the Dominican Republic. Zhen-Wei Qiang, Christine; Lanvin, Bruno; Minges, Michael; Swanson, Eric; Wellenius, Bjorn; clarke, George R.; Halewood, Naomi; Adamali, Aref; Coffey, John Oliver; Safdar, Zaid (2006) “Global trends and policies - 2006 : Information and Communications for Development.” Washington DC, The World Bank. http://documents.worldbank.org/curated/en/2006/01/6739395/global-trends- policies-2006-information-communications-development 30 Table 1 - Awareness of and Access to Information Baseline Follow up p- Subject Indicators DID* Control Treatment Control Treatment value Awareness (i) Knowledge of Rights under the FOI law: Yes 0.253 0.270 0.301 0.339 0.036 0.313 Existence of ATI office: Yes 0.078 0.111 0.086 0.130 0.009 0.621 (ii) Electricity Responsibilities of (central each level of government) 0.728 0.632 0.674 0.622 0.024 0.611 government in Garbage collection terms of service (local provision (key government) 0.953 0.965 0.948 0.949 0.002 0.901 services) Roads, streets, pathways (local government) 0.574 0.583 0.524 0.549 0.034 0.421 Health (central government) 0.823 0.815 0.762 0.756 0.029 0.558 Education (central government) 0.849 0.838 0.760 0.790 0.042 0.425 (iii) Participatory Heard about PB budgeting process meetings: Yes 0.254 0.265 0.287 0.295 0.004 0.845 Use of the FOI law (i) utilization of Asked public the Access to information to Information rights any official or government: Yes 0.044 0.032 0.041 0.041 0.015 0.214 Ever complained to government: Yes 0.109 0.097 0.069 0.085 0.045 0.017 * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 31 Table 2- Participation in collective action Baseline Follow up p- Subject Indicators DID* Control Treatment Control Treatment value (i) Group Participation in membership/part CBOs: Yes 0.478 0.462 0.651 0.603 -0.027 0.595 icipation Participation in PB meetings: Yes 0.286 0.308 0.325 0.345 0.008 0.683 Participation in protests/strikes: Yes 0.182 0.161 0.128 0.108 0.003 0.908 Voted municipal election: Yes 0.803 0.758 0.788 0.763 0.025 0.589 Supported campaign for city council candidate: Yes 0.582 0.552 0.566 0.534 -0.002 0.978 * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 32 Table 3 - Local governance Baseline Follow up p- DID* Subject Indicators value Control Treatment Control Treatment (i) Government Governance/Tran consults citizens sparency for project prioritization: Often 0.222 0.260 0.167 0.213 0.010 0.699 Government consults citizens for making decisions: Always or sometimes 0.568 0.598 0.501 0.586 0.078 0.078 Government spends in key areas in the community: Yes 0.449 0.507 0.354 0.469 0.081 0.066 * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 33 Table 4 – Trust in public and non-public institutions Baseline Follow up Subject Indicators DID* p-value Control Treatment Control Treatment (i) in public institutions Central (complete or some government 0.696 0.664 0.778 0.758 0.007 0.840 trust) Municipal government 0.652 0.633 0.599 0.663 0.095 0.057 Justice courts 0.539 0.533 0.583 0.605 0.024 0.592 Police 0.525 0.478 0.591 0.566 0.010 0.797 (ii) in individuals or neighbors 0.822 0.801 0.850 0.823 -0.007 0.979 CSOs (complete or some trust) family 0.963 0.978 0.978 0.967 -0.022 0.785 church 0.929 0.914 0.935 0.930 0.014 0.760 political parties 0.408 0.359 0.478 0.475 0.048 0.241 NGOs 0.708 0.652 0.726 0.693 0.019 0.689 CBOs 0.797 0.732 0.798 0.811 0.086 0.119 * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 34 Table 5- Satisfaction with public services Baseline Follow up p- DID* Subject Indicators value Control Treatment Control Treatment (i) with public Sewer system 0.759 0.690 0.648 0.735 -0.002 0.974 services (Yes) Electricity 0.444 0.565 0.518 0.608 -0.059 0.227 Water 0.749 0.725 0.725 0.729 0.028 0.517 Garbage collection 0.852 0.925 0.859 0.841 -0.078 0.134 Roads, streets, pathways 0.321 0.321 0.349 0.366 -0.004 0.924 Transport 0.526 0.539 0.586 0.615 0.015 0.725 Maintenance of public parks 0.491 0.358 0.521 0.477 0.093 0.027 Health 0.514 0.486 0.543 0.488 -0.020 0.730 Education 0.732 0.717 0.787 0.768 0.001 0.924 * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 35 Table 6- Heterogeneity of impacts Whole sample Only younger heads Subject Indicators DID p-value DID p-value Awareness of FOI law: YES 0.036 0.313 0.067 0.265 Even complained to Access to rights government: YES 0.045 0.017 0.059 0.077 Trust in the municipal government 0.095 0.057 0.192 0.022 Satisfaction Maintainance of public parks 0.093 0.027 0.169 0.016 Local Governance Government expenditure in key areas: YES 0.078 0.078 0.143 0.039 36 Appendix A A.1- Treatment and Control Municipalities: Population, Poverty and Location TREATMENT Population Poverty CONTROL Population Poverty MUNICIPALITIES (%) MUNICIPALITIES (%) CRISTOBAL 2,546 84.3 UBILLA 2,989 73.4 SAN JOSE DE LOS LLANOS 18,282 74.0 EL VALLE 7,966 69.7 BAYAGUANA 33,122 68.4 DON JUAN 8,267 75.4 SABANA GRANDE MONTE PLATA 24,863 64.8 DE BOYA 24,357 73.8 EL PENON 4,040 66.5 EL CACHON 2,101 70.5 SABANA DE LA MAR 14,676 61.6 MICHES 10,545 56.0 HATO MAYOR DEL REY 43,544 54.0 QUISQUEYA 18,528 62.0 * The names in italic indicate those municipalities selected for the qualitative baseline study. 37 Appendix B B.1 – Baseline comparison of means: Main household characteristics Control Treatment Subject/Indicator p-value # Mean SD # Mean SD Demographics Gender of HH head: male 982 0.659 0.474 987 0.678 0.468 0.627 Age of HH head 982 48.7 16.575 987 50.099 16.9 0.673 Age of spouse 548 40.8 14.641 560 41.736 15.2 0.399 Education of HH head: Primary 892 0.674 0.469 871 0.674 0.469 0.559 Secondary (and has not started tertiary) 892 0.215 0.411 871 0.191 0.393 0.280 Incomplete tertiary 892 0.044 0.205 871 0.052 0.221 0.299 Complete tertiary 892 0.045 0.207 871 0.048 0.214 0.991 Postgraduate 892 0.003 0.058 871 0.009 0.095 0.290 Other education 892 0.019 0.137 871 0.026 0.160 0.253 Education of HH spouse: Primary 487 0.581 0.494 494 0.601 0.490 0.494 Secondary (and has not started tertiary) 487 0.253 0.435 494 0.247 0.432 0.927 Incomplete tertiary 487 0.074 0.262 494 0.061 0.239 0.731 Complete tertiary 487 0.068 0.252 494 0.061 0.239 0.643 Postgraduate 487 0.006 0.078 494 0.008 0.090 0.982 Other education 487 0.018 0.135 494 0.022 0.148 0.465 Occupation of HH head: Wage earner 982 0.281 0.450 987 0.282 0.450 0.566 Self employed 982 0.359 0.480 987 0.373 0.484 0.952 Student 982 0.003 0.055 987 0.009 0.095 0.052 Other 982 0.356 0.479 987 0.336 0.473 0.513 Occupation of HH spouse: Wage earner 548 0.175 0.380 555 0.207 0.406 0.232 Self employed 548 0.151 0.359 555 0.144 0.352 0.586 Student 548 0.040 0.196 555 0.043 0.204 0.523 Other 548 0.633 0.482 555 0.605 0.489 0.872 HH characteristics/assets HH size 982 3.749 1.911 987 3.721 2.041 0.705 HH rooms 978 3.767 1.512 986 3.773 1.544 0.940 HH bedrooms 980 2.185 0.893 987 2.172 0.891 0.855 Sanitation: private toilet 982 0.311 0.463 987 0.345 0.476 0.259 shared toiled 982 0.026 0.161 987 0.035 0.185 0.084 38 private latrine 982 0.411 0.492 987 0.350 0.477 0.013 shared latrine 982 0.160 0.367 987 0.124 0.329 0.042 none 982 0.090 0.286 987 0.146 0.353 0.000 Sewer: residual waters system 982 0.080 0.272 987 0.186 0.390 0.000 cesspool 982 0.334 0.472 987 0.309 0.462 0.350 none 982 0.576 0.494 987 0.497 0.500 0.045 don't know 982 0.006 0.078 987 0.007 0.084 0.823 Electricity provider: companies 982 0.979 0.145 987 0.946 0.226 0.547 none 982 0.017 0.130 987 0.044 0.204 0.006 other (power plants, solar) 982 0.004 0.064 987 0.010 0.100 0.259 Water source: Free water truck 982 0.001 0.032 987 0.013 0.114 0.001 Paid water truck 982 0.021 0.145 987 0.010 0.100 0.036 Private well 982 0.030 0.169 987 0.125 0.330 0.000 Public water companies 982 0.736 0.441 987 0.621 0.485 0.009 Public well 982 0.129 0.336 987 0.143 0.350 0.911 Spring/ river/ pond/ rain 982 0.050 0.218 987 0.042 0.200 0.051 Other 982 0.032 0.175 987 0.047 0.211 0.029 Garbage collection: Yes 982 0.791 0.407 987 0.690 0.463 0.035 House property: Loan (favor from family/friend) 982 0.107 0.309 987 0.085 0.279 0.091 Loan (exchange for labor) 982 0.030 0.169 987 0.064 0.245 0.010 Owned (donation/gift, inherited) 982 0.115 0.319 987 0.085 0.279 0.139 Owned (paying mortgage) 982 0.015 0.123 987 0.016 0.126 0.845 Owned (paid) 982 0.557 0.497 987 0.599 0.490 0.214 Rented 982 0.176 0.381 987 0.151 0.358 0.188 If owned, has document? Yes 675 0.570 0.495 691 0.637 0.481 0.159 Working cell phone? Yes 978 0.816 0.388 986 0.808 0.394 0.996 Working computer? Yes 978 0.097 0.296 986 0.100 0.301 0.920 Working TV? Yes 978 0.755 0.431 986 0.776 0.417 0.399 Working radio? Yes 978 0.454 0.498 986 0.489 0.500 0.479 Working internet? Yes 978 0.052 0.222 986 0.051 0.220 0.799 HH behavior listen to radio 982 0.601 0.490 987 0.623 0.485 0.513 read local newspapers 982 0.078 0.269 987 0.104 0.306 0.049 watch TV 982 0.862 0.346 987 0.882 0.322 0.479 access the internet 982 0.074 0.262 987 0.091 0.288 0.292 read paid national newspapers 982 0.258 0.438 987 0.265 0.442 0.839 read magazines 982 0.241 0.428 987 0.235 0.424 0.983 39 read free national newspapers 982 0.161 0.368 987 0.168 0.374 0.905 Social programs, self-rated poverty Received benefits from any social welfare program? Yes 982 0.821 0.384 987 0.820 0.385 0.878 Poverty level today: 1(poorest)- 10(richest) 981 3.128 1.617 987 2.978 1.652 0.301 Poverty level 3 years ago: 1(poorest)-10(richest) 981 3.325 1.993 987 3.196 1.991 0.363 Poverty level 3 years ahead:1(poorest)-10(richest) 981 4.732 2.451 987 4.548 2.565 0.405 40 B.2 - Baseline comparison of means: main outcomes Control Treatment Subject/Indicator p-value # Mean SD # Mean SD Awereness Knowledge of rights under the ATI law: Yes 982 0.253 0.435 987 0.270 0.444 0.562 Knowledge of existence of an OAI: Yes 982 0.078 0.269 987 0.111 0.315 0.015 Use of the FOI law Have asked information to any government office or official? Yes 982 0.044 0.205 987 0.032 0.177 0.198 Participation in collective action Participation in CBOs: Yes 982 0.478 0.500 987 0.462 0.499 0.211 Have complained to government last year? Yes 982 0.109 0.312 987 0.097 0.296 0.185 Have voted in last municipal election? Yes 982 0.803 0.398 987 0.758 0.429 0.298 Have supported campaign for city council candidate? Yes 982 0.582 0.493 987 0.552 0.498 0.512 Local governance How much the government consults citizens for project prioritization? Often 982 0.222 0.416 987 0.260 0.439 0.113 How much the government takes into account the citizens when making decisions? Always/Sometimes 982 0.568 0.496 987 0.598 0.491 0.503 Does the government spend resources in key areas? Yes 982 0.449 0.498 987 0.507 0.500 0.174 Trust Trust in central government? Complete or some trust 982 0.696 0.460 987 0.664 0.473 0.431 Trust in local government? Complete or some trust 982 0.652 0.477 987 0.633 0.482 0.352 Trust in CBOs? Complete or some trust 982 0.797 0.402 987 0.732 0.443 0.058 41 Appendix C Comparing Double-difference with Ex-post difference results In this appendix, we show results calculated using only the ex-post difference. Given that the outcomes at the baseline were in general statistically similar, in principle the ex-post difference is unbiased and generates more precise estimates. We find that using DID or ex- post difference (both with matching) leads to similar conclusions in 28 out of 37 cases. Among the effects found significant using DID, only the effect on “satisfaction with maintenance of public parks” has an opposite sign and is not significant using the ex-post difference. In five cases (knowledge of rights under the FOI law, of existence of an ATI office, satisfaction with sewer, with garbage collection, and whether government often consults citizens for project prioritization), the DID results fail to reject the null while the ex-post difference is significant at 1 or 5% level. Especially regarding the main outcomes of knowledge of rights under the FOI law and of existence of an ATI office, municipalities in the intervention group seem ex-ante more knowledgeable than the control group. The balancing tests (in Table B.2) show however that a simple ex-post difference would overestimate the impact of the program on knowledge of existence of an ATI office. Therefore, our results using DID are conservative in that we allow for pre-program differences between intervened and not intervened municipalities, even though for most outcomes the balancing tests reject these two groups are statistically different. Using only ex-post difference shows that knowledge on the ATI law improved by about 5% and this is statistically significant. Satisfaction with sewer system also improved however satisfaction decreased with garbage collection. Results in terms of satisfaction could go either way. In the short run, as people are more knowledgeable, they are able to make a better judgment of the services they consume. One striking result is that sewer and garbage collection – along with maintenance of parks – are provided by the municipality, so it seems that individuals are more aware of local services than with those provided by the national government. Together with other findings pointing to higher trust in the municipal government and better local governance, these results seem to suggest that the ATI was 42 effective in improving awareness at the local level in particular, thus people are now able to judge or to tell their perceptions about the local government and locally provided services. C.1 – Double-difference vs. ex-post difference Ex-post p- Subject Indicators DID* p-value Differenc value e Awareness (i) Knowledge of Rights under the FOI 0.036 0.313 0.049 0.050 law: Yes Existence of ATI 0.009 0.621 0.045 0.004 office: Yes Participation in collective action activities Ever complained to 0.045 0.017 0.024 0.037 government: Yes Trust (i) in public institutions (complete or some trust) Municipal government 0.095 0.057 0.063 0.077 Satisfaction (i) with public services Sewer system -0.002 0.974 0.054 0.000 (Yes) Garbage collection -0.078 0.134 -0.099 0.007 Maintenance of 0.093 0.027 -0.043 0.212 public parks Local governance 43 (i) Government consults Governance/Transparenc citizens for project 0.010 0.699 0.049 0.019 y prioritization: Often Government consults citizens for making 0.078 0.078 0.096 0.001 decisions: Always or sometimes Government spends in key areas in the 0.081 0.066 0.130 0.000 community: Yes * Difference-in-difference matching estimator. N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 44 Appendix D Comparing DID using matching with DID using both matching and regression methods Unlike the previous estimator, the DID using both matching and regression is a semi- parametric method combining regression with the same matching procedure as explained in the body text. The regression is an additional attempt to correct for potential biases arising from differences in observed characteristics between treatment and control units, though they are close because of the matching process (see Imbens, 2004; Imbens and Wooldridge, 2009; Abadie and Imbens, 2002, 2006). It is ambiguous whether adding regression to matching methods will improve precision, as it combines both parametric and non-parametric methods. However the resulting estimates are showed to be consistent and asymptotically normal. Taking on similar notation (as that in section II.5), the estimator is implemented as: 1 I� DDMR = � {1 − W(i, j, t)[Y0jt + 0 ′(1 − 0 )]} #{ = 1} :{ =1}∩S 1 − � {1 ′ − W(i, j, ′ )[Y0j ′ + 0 ′ ′(1′ − 0′ )]} #{ ′ = 1} :�′ =1�∩S with 0 and 0′ estimated from regressions using individuals in the control group ′ 0 = 0 + 0 �1 − 0 � + 0 ′ 0′ = 0′ + 0′ �1′ − 0′ � + 0′ using weighted least squares with weights varying by municipality corresponding to the number of times a control municipality is used as a match (Abadie and Imbens, 2002). The results using this additional bias correction confirm the main predictions of tables 2.1- 2.5. In similar magnitude, they show that the ATI project increased complaints to the government, improved trust on the municipal government, increased satisfaction with maintenance of public parks, and the perceptions of people as to whether the government spend resources in key areas for their community. One estimate is not robust to this bias correction and rejects that the ATI had an impact on whether governments make public 45 consultations about where to invest. The matching with regression procedure was also applied as a bias correction for the ex-post difference estimator reported in Appendix C. More strongly, it supports the previous findings that the ATI had an impact on the knowledge about the existence of an ATI office, but then rejects that there exists an impact on the knowledge about the ATI law itself. D.1 - Double-difference: matching vs. matching with regression Matching Matching with Regression Ex- Subject Indicators p- p- post p- DID DID value value Differe value nce Awareness (i) Knowledge of Rights under the 0.036 0.313 0.018 0.637 0.034 0.200 FOI law: Yes Existence of ATI 0.009 0.621 0.008 0.785 0.042 0.028 office: Yes Participation in collective action activities Ever complained to government: 0.045 0.017 0.048 0.041 0.024 0.119 Yes Trust (i) in public institutions (complete or some trust) Municipal government 0.095 0.057 0.096 0.050 0.063 0.072 Satisfaction (i) with public services Sewer system -0.002 0.974 0.041 0.657 0.086 0.241 (Yes) Garbage -0.078 0.134 -0.076 0.104 -0.100 0.002 collection 46 Maintenance of 0.093 0.027 0.102 0.021 -0.036 0.254 public parks Local governance (i) Government Governance/Transparen consults citizens cy for project 0.010 0.699 0.004 0.910 0.042 0.085 prioritization: Often Government consults citizens for making 0.078 0.078 0.066 0.162 0.089 0.009 decisions: Always or sometimes Government spends in key 0.081 0.066 0.076 0.067 0.131 0.000 areas in the community: Yes * N=1,969 and 1,918 households (baseline and follow up, resp.). P-values are obtained using 500 bootstrap samples. 47