WPS4268 Power to the People: Evidence from a Randomized Field Experiment of a Community-Based Monitoring Project in Uganda Martina Björkman* and Jakob Svensson# Abstract: Strengthening the relationship of accountability between health service providers and citizens is by many people viewed as critical for improving access to and quality of health care. How this is to be achieved, and whether it works, however, remain open questions. The paper presents a randomized field experiment on increasing community-based monitoring. As communities began to more extensively monitor the provider, both the quality and quantity of health service provision improved. One year into the program, there are large increases in utilization, significant weight-for-age z-score gains of infants, and markedly lower deaths among children. The findings on staff behavior suggest that the improvements in quality and quantity of health service delivery resulted from an increased effort by the staff to serve the community. Overall, the results suggest that community monitoring can play an important role in improving service delivery when traditional top-down supervision is ineffective. World Bank Policy Research Working Paper 4268, June 2007 The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the countries they represent. Policy Research Working Papers are available online at http://econ.worldbank.org. This project is a collaborative exercise involving many people. Foremost, we are deeply indebted to Frances Nsonzi and Ritva Reinikka for their contributions at all stages of the project. We would also like to acknowledge the important contributions of Gibwa Kajubi, Abel Ojoo, Anthony Wasswa, James Kanyesigye, Carolyn Winter, Ivo Njosa, Omiat Omongin, Mary Bitekerezo, and the field and data staff with whom we have worked over the years. We thank the Uganda Ministry of Health, Planning Division, the World Bank's Country Office in Uganda, and the Social Development Department, World Bank, for their cooperation. We are grateful for comments and suggestions by Paul Gertler, Esther Du.o, Abhijit Banerjee, and seminar and conference participants at LSE, Oxford, IGIER, MIT,World Bank, NTNU, Namur, CEPR/EUDN conference in Paris, and BREAD & CESifo conference in Venice. Finally, we wish to thank the Bank-Netherlands Partnership Program (BNPP), the World Bank Research Committee, the World Bank Africa Region division, and the Swedish International Development Agency, Department for Research Cooperation, for funding this research. Björkman also thanks Jan Wallander.s and Tom Hedelius.Research Foundation for funding. *IGIER, University of Bocconi, and CEPR. Email: mbjorkman@unibocconi.it. #IIES, Stockholm University, NHH, and CEPR. Email: jakob.svensson@iies.su.se. 1 Introduction Approximately 11 million children under ...ve die each year. Almost half of these deaths occur in Sub-Saharan Africa where roughly one in ...ve children dies before reaching the age of ...ve. More than half of these children ­nearly 6 million ­will die of diseases that could easily have been prevented or treated if the children had had access to a small set of proven, inexpensive services.1 Why are these services not provided? While there is no simple answer, a wealth of anecdotal, and recently more systematic, evidence shows that the provision of public services to poor people in developing countries is constrained by weak incentives of service providers ­schools and health clinics are not open when supposed to; teachers and health workers are frequently absent from schools and clinics and, when present, spend a signi...cant amount of time not serving the intended bene...ciaries; equipment, even when fully functioning, is not used; drugs and vaccines are misused; and public funds are expropriated.2 The traditional approach to accountability in the public sector relies on external control. This is a top-down approach where someone in the institutional hierarchy is assigned to monitor, control and reward/punish agents further down in the hierarchy. The tacit assumption is that more and better enforcement of rules and regulations will strengthen providers'incentives to increase both the quantity and quality of service provision. But, in many poor countries, the institutions assigned to monitor the providers are typically weak and malfunctioning, and may themselves act under an incentive system providing little incentives to e¤ectively monitor the providers. As a result, the relationship of accountability of provider-to-state is ine¤ective in many developing countries.3 1See Lancet (2003) and UNICEF (2003). It is estimated that 2 million children under ...ve die from diarrhea, which in most cases can be treated with simple oral rehydration therapy. Another 2 million children die from pneumonia, where once more there is su¢ cient evidence of e¤ective treatment (antibiotics). Malaria kills one million children under ...ve, most of whom could have been protected by preventive measures and treatment with anti-malarias. Globally, neonatal disorders account for the highest proportion of deaths of children ­many of them could have been saved if mothers had had access to basic antenatal and delivery care. Approximately half a million children under ...ve die from measles, for which these is a cheap and e¤ective vaccine (Black et al., 2003; Jones et al., 2003). 2For anecdotal and case study evidence, see World Bank (2003). Chaudhury et al. (2006) provide systematic evidence on the rates of absenteeism based on surveys where enumerators made unan- nounced visits to primary schools and health clinics in seven developing countries. Averaging across countries, 35 percent of the health workers were absent. Banerjee et al. (2004) and Duo and Hanna (2005) con...rm these ...ndings. On misappropriation of public funds and drugs, see Reinikka and Svensson (2004) and McPake et al. (1999). 3In addition, while well-functioning legal and ...nancial systems can curtail obvious cases of mis- management, they only partially constrain the discretionary powers of public sector managers and employees. The complexity of the tasks performed by a typical public sector unit and its informa- tional advantage relative to the monitor (which typically relies on accounting data) make it nearly impossible to design legal and accounting measures to address all types of misuse and thus, to curtail less obvious cases of mismanagement (such as shirking, budget prioritization in favor of sta¤ and po- litical considerations). Finally, audit reports and legal procedures are often di¢ cult for nonspecialists to interpret and will therefore go unnoticed unless the commissioning agency acts on them. 2 As a complementary strategy, it has therefore been argued that more e¤ort must be placed on strengthening bene...ciary control, i.e. strengthening providers'accountabil- ity to citizen-clients (see e.g., World Bank, 2003). However, despite the enthusiasm for such an approach, there is little credible evidence on the impact of policy interventions aimed at achieving it (Banerjee and He, 2003; Banerjee and Duo, 2005). This paper attempts to provide some. To examine whether bene...ciary control works, we designed and conducted a ran- domized ...eld experiment in 50 "communities" from nine districts in Uganda.4 In the experiment, or intervention, communities were provided with baseline information on the status of service delivery, both in absolute terms and relative to other providers and the government standard for health service delivery. Community members were also encouraged to develop a plan that identi...ed areas of concerns in health provision, and how to address them within the existing resource envelope. The intervention sought to relax two constraints typically faced by communities in monitoring providers: lack of access to reliable and structured information on the community's entitlements and the status of service delivery, and inadequate local orga- nizational capacity. Access to reliable and structured information about current status of service delivery and entitlements is critical for citizens'ability to monitor service providers. Although people know whether their own child died or not, and whether the health workers did anything to help them, they typically do not have any information on aggregate outcomes, such as how many children in their community did not survive beyond the age of 5 or where citizens, on average, seek care. Provision of informa- tion on outcomes and performance improves citizens'ability to challenge abuses of the system, since reliable quantitative information is more di¢ cult for service providers to brush aside as anecdotal, partial, or simply irrelevant. But information provision may not have any considerable impact unless there are members of the community who are willing to make use of the new information. Exerting accountability (mon- itoring providers) is subject to potentially large free-rider problems. Elite capture further complicates the process of holding providers accountable. By enhancing local organization capacity and encouraging the community to develop its own monitoring strategy, these constraints are sought to be relaxed. The community-based monitoring project increased the quality and quantity of pri- mary health care provision. One year into the program, we ...nd a signi...cant di¤erence in the weight of infants (0.17 z-scores increase) and a markedly lower number of deaths among children under ...ve (a 33 percent reduction in child deaths) in the treatment communities. Utilization (for general outpatient services) was 16 percent higher in the treatment compared to the control facilities. We also ...nd signi...cant di¤erences in the number of deliveries at treatment facilities and the use of antenatal care and family planning. Treatment practices, as expressed both in perception-based responses by households and in more quantitative indicators (immunization of children, waiting 4A "community" is operationalized as the households (and villages) residing in the ...ve-kilometer radius around the facility (see section 5 for details). Approximately 110,000 households (600,000 individuals) reside in these communities, of which half reside in the treatment communities. 3 time, examination procedures, absenteeism), improved signi...cantly in the treatment communities, thus suggesting that the changes in quality and quantity of health care provision are due to behavioral changes of the sta¤. We ...nd evidence that the treat- ment communities became more engaged and began to monitor the health unit more extensively. No e¤ect is found on investments, or the level of ...nancial or in-kind support (from the government). Furthermore, supervision of providers by upper-level government authorities remained low in both the treatment and the control group. This reinforces our con...dence that the ...ndings on the quality and quantity of health care provision resulted from increased e¤orts by the health unit sta¤ to serve the community in the light of better community monitoring. The paper is organized as follows. The next section reviews the literature. Section 3 discusses the concept of community monitoring. Section 4 briey describes the institutional environment in Uganda and in the project areas. The community-based monitoring intervention is described in section 5. Section 6 lays out the evaluation design and the results are presented in section 7. Section 8 concludes. 2 Literature Review Improving governance and public service delivery through community participation is an approach that has gained prominence in recent years. For example, the World Development Report 2004 is entirely devoted to the concept of increasing poor citizens' voice and participation in service delivery in order to help them monitor and discipline providers. But despite the enthusiasm for such approaches, there is little credible evidence on the impact of policy interventions aimed at achieving them. On the one hand, most (all) comprehensive community based monitoring initiatives have not been rigorously evaluated. On the other hand, the few studies relying on rigorous impact evaluation strategies have not evaluated more comprehensive attempts to inform and involve the community in monitoring public o¢ cials. On the latter issue, Olken (2005) evaluates di¤erent ways of monitoring corruption in a road construction project in Indonesia. In one of the experiments, invitations were sent out to village-level meetings where project o¢ cials documented how they spent project funds for local road construction. However, although the invitations increased the number of people participating in the meetings, the meetings were still dominated by members of the village elite. Moreover, corruption is not easily observable and project o¢ cials may very well be able to hide it when reporting on how funds were used. The data also reveal that corruption problems were seldom discussed in these meetings.5 Thus, it is unclear to what extent non-elite community members were 5The information problem is illustrated in the novel but burdensome way in which Olken (2005) estimates the extent of corruption. Speci...cally, Olken (2005) assembled a team of engineers and surveyors who dug samples in roads to estimate the quantity of materials used and then, using price information from local supplies, estimated the extent of "missing" expenditures. The corruption estimates were not reported in the village meetings. 4 really more informed about corruption in the project, or if they had any means of inuencing outcomes, in response to the intervention. Given these constraints, it is not surprising that Olken (2005) only ...nds minor e¤ects of the intervention. Using a randomized design, Banerjee, Deaton and Duo (2004) evaluate a project in Rajasthan in India where a member of the community was paid to check once a week, on unannounced days, whether the auxiliary nurse-midwife assigned to the health center was present at the center. Unlike Olken's study, getting reliable information is not a concern here. In fact, external monitors con...rmed the absence rates documented by the community members assigned to the project. The issue is rather how the informed community member could use his or her information on absenteeism to invoke community participation. The intervention had no impact on attendance. Thus, informing one person, even if this is done is a structured and regular way, may not have much impact. Jiminez and Sawada (1999) examine how decentralizing educational responsibil- ity to communities and schools a¤ects student outcomes. They study El Salvador's Community-Managed Schools Program, EDUCO, and its e¤ect on students'achieve- ment on standardized tests and attendance. The evidence suggests that enhanced community and parental involvement in EDUCO schools improved students'language skills and diminished student absences. A key estimation issue in this paper is en- dogenous program participation and although the authors instrument for program participation by using the proportion of EDUCO schools in a municipality, it is not obvious that they succeed in obtaining the causal treatment e¤ect. There is a growing empirical literature on the relationship between information dissemination (through the media) and accountability. With few exceptions, this lit- erature studies the relationships of accountability of politicians to citizens and deal with one (periodic elections), out of several, mechanism through which citizens can make politicians and policymakers accountable.6 For example, Strömberg (2003, 2004) considers how the press inuences redistributive programs in a model of electoral poli- cies, where the role of the media is to raise voter awareness, thereby increasing the sensitivity of turnout to favors granted. Besley and Burgess (2002) focus on the media's role in increasing political accountability also in a model of electoral policies. Ferraz and Finan (2005) study the e¤ects of making information about corruption in the lo- cal governments public on the probability of the incumbent winning the election. Our work di¤ers in several important dimensions. First, we focus on mechanisms through which citizens can make providers, rather than politicians, accountable. Thus, we do not study the design or allocation of public resources across communities or programs, but rather on how these resources are utilized. Second, we use micro data from house- holds and health stations rather than disaggregated national accounts data. Finally, we identify impact using an experimental design. The source of identi...cation will thus come directly from a randomized experiment. Reinikka and Svensson (2005a) also study the relationship between information, accountability, and outcomes at the provider level. They exploit a newspaper cam- 6For a review, see Khemani (2006). 5 paign aimed at reducing capture of public funds by providing schools (parents) with information to monitor local o¢ cials'handling of a large education grant program. Head teachers in schools closer to a newspaper outlet are found to be more knowl- edgeable of the rules governing the grant program and the timing of releases of funds by the central government. These schools also managed to claim a signi...cantly larger part of their entitlement after the newspaper campaign had been initiated. Reinikka and Svensson (2005b) and Björkman (2006) take these results as a starting point to explore the e¤ects of increased "client power" on school outcomes. They show that the reduction in capture had a positive e¤ect on both enrollment and student learning. The newspaper campaign in Uganda, however, may not be easy to scale up in other sectors or for more complex government programs. Speci...cally, the capitation grant is a very simple entitlement program and a small item in a vast government budget. They also identify impact using a non-experimental approach. 3 Community-based Monitoring Community-based monitoring, or social accountability, is an approach towards build- ing accountability that relies on civic engagement where citizens and civil society organizations directly or indirectly participate in extracting accountability (Malena et al., 2004). It can take a variety of forms, although most interventions have in com- mon that they inform citizens about their rights and status of service delivery and encourage participation.7 Citizens/communities typically face several constraints in initiating local collec- tive action to improve service delivery outcomes. First, citizens may not be able to challenge abuses of the system since they lack reliable information on outcomes. Community members'own experience of service provision, or private information, is typically an imprecise signal of overall (or average) quality (Khemani, 2006). Second, like most local collective actions, community-based monitoring is subject to possibly large free-riding problems: The community would like to ensure that the provider per- forms, but everyone would rather have someone else monitor performance. Third, the community may lack the ability to sanction, either directly or indirectly, the provider in case of poor performance, or reward good performance. Community-based monitoring, however, is considered to have several potential ad- vantages. For example, it is likely to be cheaper for the bene...ciaries to monitor the providers since they (at least as a group) are better informed about the status of service delivery than the external agent assigned to supervise the provider. They may also have means of punishing the provider that are not available to others, such as verbal 7Examples of this approach include participatory budgeting in Porto Allegre, Brazil; citizen report cards in Bangalore, India; right to information on public works and public hearings or jan sunwais in Rajasthan, India; public information campaign to reduce capture of school funds in Uganda; and community scorecards in Malawi (see Reinikka and Svensson, 2004; World Bank, 2003; Paul, 2002; and Singh and Shah, 2002). 6 complaints or social opprobrium (Banerjee and Duo, 2005). Similarly, they may be able to induce higher e¤ort of health workers by providing non-pecuniary rewards (so- cial rewards) for good performance. To the extent that the service is valuable to them, they should also have strong incentives to monitor the provider ­incentives which the external agent assigned to supervise the provider may lack. Naturally, there is no guarantee that community monitoring will work even if the community is informed, can coordinate actions and there is demand for the service. In many developing coun- tries, the bene...ciaries of health services in rural areas are socially inferior to health care workers. Bene...ciary groups may also be captured by the service provider or other authorities through their social or political connections (Banerjee and Duo, 2005). Thus, in the end, if and to what extent community monitoring works is an empirical question. 4 Institutional Setting Uganda, like many newly independent countries in Africa, had a functioning health care system in the early 1960s. Accessibility and a¤ordability were relatively extensive. The 1970s and 1980s saw the collapse of government services as the country underwent political upheaval. Health indicators fell dramatically during this period until peace was restored in the late 1980s. Since then, the government has been implementing major infrastructure rehabilitation programs in the public health sector. Some health indicators have improved, while others have not. This is despite a GDP growth rate exceeding 64 percent and a 40-percent reduction in consumption poverty in the 1990s (Appleton 2001) As of 2001, public health services are free of charge. Anecdotal and survey evidence (see below), however, suggests that users still encounter varying costs when visiting public health facilities. The health sector in Uganda is composed of four types of facilities: hospitals, health centers, dispensaries (health center III), and aid posts or sub-dispensaries. These fa- cilities can be government, private for-pro...t, or private not-for-pro...t operated and owned. The impact evaluation focuses on dispensaries. Dispensaries are in the lowest tier of the health system where a professional interaction between users and providers takes place. Most dispensaries are rural (89 percent). According to the government health sector strategic plan, the standard for dispensaries includes preventive, promo- tional, outpatient care, maternity, general ward, and laboratory services (Republic of Uganda 2000). In our sample of facilities, on average, a dispensary was sta¤ed by an in-charge or clinical o¢ cer (a trained medical worker/doctor), three nurses (including midwives), and three nursing aids or other assistants. The health sector in Uganda is decentralized and supervision and control of the dispensaries are governed at the district level. A number of actors are responsible for the functioning of the dispensaries. The Health Unit Management Committee (HUMC) is supposed to be the main link between the community and the health 7 facility. Each dispensary has an HUMC which consists of members from both the health facility sta¤ and non-political representatives from the community (elected by the sub-county local council). The HUMC should monitor drugs and ...nances disbursed to the health facility, as well as the day-to-day running of the health facility (Republic of Uganda 2000). The HUMC can warn the health facility sta¤on issues of indiscipline, rudeness to patients and misappropriations of funds by recommending that the sta¤ is transferred from the health facility. However, the HUMC has no authority to dismiss a worker. In cases of problems at the health facility, the working practice is that the chair person of the HUMC raises the issue with the in-charge. If there is no improvement, the issue should be referred to the Health Sub-district. The Health Sub-district monitors funds, drugs and service delivery at the dispen- sary. Supervision meetings by the Health Sub-district are supposed to appear quarterly but, in practise, monitoring is infrequent. The Health Sub-district has the authority to reprimand, but not dismiss, health facility sta¤ for indiscipline. In severe cases of indiscipline, therefore, the errand will be referred to the Chief Administrative O¢ cer of the District and the District Service Commission, which is the appointing authority for the district and has the authority to suspend or dismiss sta¤. Another actor in the health sector is Community-based organizations (CBOs). Their main focus is on health education in antenatal care, family planning, and HIV/AIDS prevention. 5 The Project: Citizen Report Card In response to perceived continued weak health care delivery at the primary level, a pilot project (Citizen report cards) aimed at enhancing community involvement and monitoring in the delivery of primary health care was initiated in 2004. The project was designed by sta¤from Stockholm University and the World Bank, and implemented in cooperation with a number of Ugandan practitioners and 18 community-based orga- nizations. The 50 project facilities (all in rural areas) were drawn from nine districts in Uganda (see the appendix for details). With the catchment area (or the community) of each dispensary de...ned as the households and villages residing within a ...ve-kilometer radius from the clinic, about 110,000 households reside in the communities supposedly served.8 The facilities were ...rst strati...ed by location (districts) and then by size (the number of households re- siding in the catchment areas). From each group, half the units, with corresponding catchment areas, were randomly assigned to the treatment group and the remaining 25 units were assigned to the control group. Each district thus had both treatment and control groups. The main objective of the Citizen report card project was to strengthen providers' accountability to citizen-clients by enhancing communities'ability to monitor providers. 8Dispensaries are designed to serve households in a catchment area roughly corresponding to the ...ve-kilometer radius around the facility (Republic of Uganda, 2000). 8 Speci...cally, the project aimed at: (i) providing communities with baseline information on the status of service delivery, both in absolute terms and relative to other providers, and the government standard for health service delivery at the dispensary level; and (ii) encouraging people to develop a plan that identi...ed areas of concerns in health provision, and how to address them within the existing resource envelope. These com- ponents are discussed next. A time-line and a schematic view of the intervention and expected outcomes are depicted in ...gures 1 and 2. 5.1 Data Collection and Report Cards Data collection was governed by two objectives. First, data were required to assemble report cards on how the community at large views the quality and e¢ cacy of service delivery. We also wanted to contrast the citizens'view with that of the health unit sta¤. Second, data were required to rigorously evaluate impact. To meet these objectives, two surveys were implemented: a survey of health care providers and a survey of health care users. Both surveys were implemented prior to the intervention (data from these pre-intervention surveys formed the basis for the report cards) and one year after the project had been initiated. A quantitative service delivery survey (QSDS) was used to collect data from the health service providers. Since agents in the service delivery system may have a strong incentive to misreport (or not report) key data, the data were obtained directly from the records kept by facilities for their own need (i.e. daily patient registers, stock cards, etc.) rather than from administrative records submitted to the district-level govern- ment. The former, often available in a highly disaggregate format, were considered to su¤er the least from any incentive problems in record-keeping. The household survey collected data on both households' health outcomes and health facility performance, including performance parameters such as usage, avail- ability, access, reliability, quality and satisfaction. To the extent that it was possible, household responses were supported by patient records, i.e., patient exercise books and immunization cards. These records helped the household recall details about its visits to the health facility and also minimized problems of misreporting. The post- intervention household survey also included a shorter module on health outcomes. Speci...cally, data on under-...ve mortality were collected and we measured the weight of all infants in the surveyed households. A strati...ed random sample of households within the catchment area of the facility were surveyed. In total, roughly 5,000 households have been surveyed in each round. The design and implementation of the surveys are explained in more detail in the appendix. The data from the two pre-intervention surveys were analyzed and a smaller subset of the ...ndings were assembled in report cards for the treatment localities.9 The data 9Thus, the design and size of the surveys were largely driven by the second objective ­to evaluate impact. 9 included in the report cards were identi...ed as key areas subject to improvement and include utilization, quality of services, informal user charges and comparisons vis-à-vis other health facilities in the district and the country at large. Each treatment facility and its community had a unique report card summarizing, in a format easily accessible to the communities, the ...ndings from the surveys conducted in their area. The report cards were translated into the main language spoken in the commu- nity.10 To support the non-literate community members, posters were designed by a local artist so that otherwise complex information and concepts were easily under- stood. Because the information in the report cards was largely statistical, the posters visually conveyed the main messages, such as where people go to seek medical care and why they do so.11 5.2 Dissemination and Participation Getting people to retain and use information to achieve a speci...c objective is a complex problem.12 Extensive piloting concluded that simply reporting the facts would be likely to have little impact. Thus, to maximize the likelihood that the information in the report cards would be used when people decide what actions to take, a participatory approach was chosen where community members themselves actively interpreted and analyzed the information.13 To this end, the process of providing information and encouraging participation and monitoring was initiated through a series of meetings: a community meeting; a sta¤ meeting; and an interface meeting. Sta¤ from various Community-based organizations (CBO) acted as facilitators in these meetings.14 The community meeting was a two-day (afternoons) event with approximately 100 invited participants drawn from the surveyed villages in the catchment area of the health facility. To avoid elite capture, the invited participants consisted of a selection of representatives from di¤erent spectra of society (i.e. young, old, disabled, women, 10 In the end, the report cards were translated into six di¤erent languages: Ateso (Soroti), Lusoga (Iganga), Lango (Apac), Luganda (Masaka, Wakiso, Mukono and Mpigi), Runyankore (Mbarara) and Lugbara (Arua). 11 See the appendix for a prototype poster. 12 See, for example, Lupia (2004) who systematizes and draws conclusions from clinical, psycholog- ical, and economic research on information transmission and processing. 13 The approach we used is closed linked to the so-called "participatory rural appraisal approach". Participatory rural appraisal (PRA) is a label given to a growing family of participatory approaches and methods with the common aim of enabling people to make their own appraisal, analyses, and plans. PRA evolved from a set of informal techniques used by development practitioners in rural areas to collect and analyze data (World Bank, 1996). 14 Since the CBOs were in regular interaction with the communities and had a mandate drawn from a long-term presence on the ground working with the community, these facilitators were perceived to be a good conduit through which the project could be delivered. The CBO facilitators were trained for seven days in data interpretation and dissemination, utilisation of the participatory methodology, and conict resolution and management. It should be noted that various CBOs (including some participating in the project) also operate in the control districts. Thus, the presence (and numbers) of CBOs in the project communities is similar across treatment and control groups. 10 mothers, leaders). The facilitators mobilized the village members by cooperating with village council representatives in the catchment area. Invited participants were asked to spread the word about the meeting and, in the end, a large number of uninvited participants from other villages who had found out about the event also attended the meeting. A typical village meeting was attended by more than 150 participants per day. In the community meeting, the facilitators used a variety of methods, including maps, venn diagrams, role-play, and focus group discussions, to disseminate the infor- mation in the report cards in a participatory, or interactive, way.15 Information on patients'rights and entitlements was also discussed.16 As the objective was not only to inform but to encourage people to participate in developing a shared view on how to improve service delivery and monitor the provider, the facilitators structured the discussions through a series of questions on the various elements of accountability in the primary health sector (who is accountable to whom?; what is a particular actor accountable for?; how can these actors account for their actions?; and how are these elements reected in the report card ...ndings?). At the end of the meeting, the com- munity's suggestions for improvements (and how to reach them without additional resources) were summarized in an action plan. The action plan contained information on health issues/services that had been identi...ed as the most important to address; how these issues could be addressed and how the community could monitor improve- ments (or lack thereof). An abbreviated version of one such action plan is depicted in the appendix. While the issues raised in the action plans di¤ered across communities, a common set of concerns included high rates of absenteeism, long waiting-time, weak attention of health sta¤, and di¤erential treatment. After the meeting, participants were given posters and copies of the report card to bring back to their villages and share with their village members. The health facility sta¤ meeting was a one-day (afternoon) meeting held at the health facility with all sta¤ present. In this meeting, the facilitators contrasted the information on service provision as reported by the provider with the ...ndings from the household survey. The meeting enabled the providers to review and analyze their performance, and compare their performance with other health clinics in the district and across the country. An interface meeting with participants (chosen by people that attended the commu- nity meeting) from villages in the catchment area and the health facility sta¤ followed the community and health facility meetings. Based on the action plan developed in the community meeting and the discussions from the health facility meeting, the interface 15See the appendix for a more detailed description of the various methods used during the meetings. 16Information on patients' rights and entitlements was based on the Yellow Star program. In 2000, the MoH developed a quality of care strategy called the Yellow Star Program with the aim of improving and maintaining basic standards of care at government and NGO health facilities. The rationale behind this strategy was the general consensus that the quality of health services had been a major deterrent to service utilization. The Yellow Star Program lists a set of basic standards of quality. The standards fall into six categories: Infrastructure and Equipment; Management systems; Infection prevention: Information: Education and Communication; Clinical skills; and Client services. 11 meeting devised a strategy for improved health care provision. During the interface meeting, the community representatives and the health facility sta¤ presented and discussed their suggestions for improvements. A role-playing exercise was used to dis- seminate the results from the survey, with community participants and sta¤ reversing roles. The participants discussed their rights and responsibilities as patients or medical sta¤. The outcome was a shared action plan, or a contract, outlining the community's and the service provider's agreement on what needs to be done, how, when and by whom. The "community contract" also identi...ed how the community could monitor the agreements and a time plan. Because the problems raised in the community meet- ings constituted the core issues discussed during the interface meetings, the community contract was in many respects similar to the community's action plan. Copies of the community contract were kept with the community and the health facility to support the following monitoring process. 5.3 Ongoing process of monitoring The three separate meetings aimed at kick-starting the process of community monitor- ing. Thus, after the initial meetings, and based on the agreements in the community contract, the communities were themselves in-charge of establishing ways of moni- toring the provider. The facilitators supported the communities in this process with follow-up meetings. This was done as an integrated part of the CBO's ordinary work in the villages. Each community had approximately two follow-up meetings in the six- month period that followed. In these meetings, facilitators raised the issues identi...ed in the community contract with citizens and community leaders. After a period of six months, the communities and health facilities were revisited to conduct a mid-term review ­a repeat engagement on a smaller scale. Including a one-day community meeting and a one-day interface meeting, the review tracked the implementation of the community contract. The earlier community contracts were printed on posters to spark discussions. Health facility sta¤ and community members jointly discussed suggestions on actions for sustaining or improving progress, or in the case of no improvements, why so. Where improvements had been made, suggestions for sustainability were recorded. The community and the health facility kept the updated community contract to assist in further monitoring. 6 Evaluation Design and Expected Outcomes 6.1 A stylized framework The following sub-section presents a stylized framework that illustrates the channel(s) through which community-based monitoring could a¤ect health outcomes. 12 The key behavioral change induced by more extensive community-based monitoring is expected to be increased e¤ort by health unit sta¤ to serve the community. Health workers have little pecuniary incentives to exert high e¤ort. Typically, public money does not follow patients and hiring, salaries and promotions are largely determined by seniority and educational quali...cations ­not by how well the sta¤ performs. While formal sanctions, such as suspensions and dismissal, are possible, they are in practice uncommon and only applied in cases of severe neglect and mismanagement. An in- dividual worker may still put in high e¤ort if shirking deviates from her ideal choice, given the behavior of other sta¤ and the situation (Akerlof and Kranton, 2005). The e¤ort choice may also be inuenced by the social rewards from community members or social sanctions against shirking health workers. Social rewards and sanctions are key instruments available to the community to boost health worker's e¤ort. The e¤ort of a health worker could theoretically complement or substitute the ef- fort of other health workers. Complementarities could directly arise in health service production. There could also be externalities working through the informal instru- ments at the community's disposal (social rewards and sanctions). For example, to the extent that social rewards and sanctions are facility speci...c, a high e¤ort worker may bene...t little from social rewards and su¤er from social sanctions if other workers are shirking. This, in turn, could generate multiplier e¤ects in average individual e¤ort and thus open up the possibility of multiple health facility equilibria: some with high e¤ort by all sta¤ and others with poor overall service quality. As the community receives more accurate information about service quality and can coordinate on expected reforms, i.e. the intervention, the community is likely to be in a better position to monitor e¤ort but may also choose to more regularly exploit the instruments at their disposal, i.e., praise workers when service provision improves and complain when it does not. Workers may then ...nd coming to work, or more generally exerting e¤ort, more attractive. Complementarities in workers'e¤ort combined with a more engaged and supportive community can therefore result in a virtuous circle, where higher e¤ort by some sta¤ makes it more attractive for others to also come to work as the social prestige of working in a well functioning health clinic rises. As service quality improves, community members in turn shift from self-treatment to the facility in question, which further boosts the return to e¤ort as more households will commend the sta¤ for its e¤ort. The switch from self-treatment to professional care and the increase in quality could both have a positive e¤ect on health outcomes. 6.2 Outcomes The main outcome of interest is whether the intervention increased the quantity and quality of health care and, thus, improved health outcomes in the treatment commu- nities. However, we are also interested in evaluating changes (if any) in all steps in the accountability chain depicted in ...gure 2: Did the intervention increase treatment communities'ability to exercise accountability? Did it result in behavioral changes of the sta¤ (i.e., did sta¤ exert higher e¤ort to serve the community)? 13 As a robustness test, we also assess alternative explanations. Some of these al- ternative mechanisms are illustrated in ...gure 3. One concern is spillovers. Spillovers could a¤ect the estimates in two ways. If information about the intervention spread to control areas and, as a result, control communities became more involved in mon- itoring the providers, the estimated treatment e¤ect would be biased downward. If, on the other hand, households in control communities shifted from seeking care at the control facility to the nearest treatment clinic, it is possible that the estimated treatment e¤ect would be biased upward. This is a potentially serious concern but also a mechanism which we can test. It is also possible that the intervention did not only (or primarily) increase the extent of community monitoring, but had an impact on other agents in the service delivery chain. For example, the various upper-level authorities in the health sector (e.g. the Health Sub-district) may have become more involved in monitoring the providers, or the district government may have increased its administrative or ...nancial support, following the intervention. While this would not invalidate the causal e¤ect of the intervention it would, of course, a¤ect interpretation. Therefore, this alternative hypothesis is also subject to a battery of tests. Given the wealth of information, we report the main results and tables in the text and refer the reader to appendix for additional ...ndings. 6.3 Statistical framework Given the randomized assignment of the Citizen Report Card project, we expect the 2004 pre-data in the treatment areas to be similar those in the control areas. We have both facility-speci...c data (on utilization, for example) and household-speci...c data (on waiting time, for example). Denoting yijdt the outcome variable of household i (when applicable), health facility j in district d and period t, we start by checking that there is no di¤erence between treatment and control facilities/communities prior to the intervention: yijdPRE = PRE + (1) PRE Tjd + "ijdPRE ; where t = PRE denotes the pre-intervention period, Tjd is a dummy indicating whether health facility j is in the treatment group and "ijdPRE is the error term. In regressions using household data, the disturbance term is adjusted to allow for correlations within catchment areas (communities). To estimate the causal e¤ect of the program, we then run the same regression in the post-period (t = POST): yijdPOST = POST + (2) POST Tjd + "ijdPOST : We also estimate an extended version of equation (2): yijdPOST = 0POST+ 0POST Tjd + XijdPOST + d+ "ijdPOST : (3) 14 Speci...cation (3) includes district ...xed e¤ects ( ) and facility and household vari- d ables (X) controlling for pre-treatment di¤erences across health facilities and com- munities that were present despite randomization. This increases the precision of the coe¢ cient estimates. For a subset of variables, we can also stack the pre and post data and explore the di¤erence-in-di¤erences in outcomes, i.e., we estimate:17 yijt = POSTt + (4) DD (Tj POSTt) + j + "ijt; where POST is a post period dummy, is a facility speci...c ...xed e¤ect, and is j DD the di¤erence-in-di¤erences estimate (program impact).18 7 Results 7.1 Pre-intervention di¤erences Prior to the intervention, the treatment and the control group were similar on most characteristics. We report the test of di¤erence in means across control and treatment groups in table 1. At the baseline, we do not ...nd any statistically signi...cant di¤erences in utilization (number of outpatient treated and deliveries per month), households' use of di¤erent service providers (including drug shops) in case of illness, waiting time, equipment usage, government funding of clinics, citizens'perceptions of sta¤ behavior, catchment area characteristics (such as the number of villages and households in catchment area), distances from the health facility to the nearest local council and government facility, or health facility characteristics (such as type of water source, availability of drinking water at the facility, whether a separate maternity unit is available, electricity shortages). In one out of ...ve measures of monthly supply of drugs (i.e., Quinine), the treatment group, on average, has a marginally higher supply in the year prior to treatment. In one out of four user-charge measures, there is some evidence (the estimate is signi...cant at the 10 percent level) that patients served by the treatment facilities are more likely to pay for service delivery. Overall, though, the randomization appears to have been successful. 17It is a subset of variables since the post intervention surveys collected information on more variables and outcomes. 18A slightly more restricted di¤erence-in-di¤erence (DD) speci...cation substitutes the facility ...xed e¤ects for Tjd. In that case, time invariant factors will be captured by Tjd. Both DD speci...cations yield identical point estimates of . DD 15 7.2 Processes The initial phase of the project, i.e., the three separate meetings, followed a pre-design structure. A parallel system (visit by a member of the survey team) also con...rmed that this initial phase of the intervention was properly implemented. After these initial meetings, it was up to the community to sustain and lead the process that the intervention intended to initiate. In this section, we present some evidence on this ...rst component in the accountability chain depicted in ...gure 2; namely if the treatment communities become more involved in monitoring the providers. To avoid inuencing local initiatives, the parallel system was only in place during the ...rst round of meetings. Therefore, we are not able to document all actions taken by the communities in response to the intervention. Still, we have two sources of infor- mation on how processes in the community have changed. First, the CBOs submitted reports on what type of changes they observed. This evidence is complemented by facility and household survey data as well as data assembled through a local council survey. According to the CBO reports, the community-based monitoring process that fol- lowed the ...rst set of meetings was a joint e¤ort mainly managed by the local councils, HUMC (Health Unit Management Committee) and community members. In the com- munities, the performance of the health facility was discussed during village meetings. The local council survey also con...rms this. A typical village in the treatment group had, on average, six local council meetings in 2005. In those meetings, 89 percent of the villages discussed issues concerning the project health facility. The main subject of discussion in the villages concerned the community contract or parts of it, such as behavior of the sta¤. The CBOs report that concerns raised by the village members were carried forward by the local council to the health facility or the HUMC. However, although the HUMC is an entity that should play an important role in monitoring the provider, it was in many cases viewed as being ine¤ective. As a result, mismanaged HUMCs were dissolved and new members elected, while others felt the pressure from the community to act and follow up on the issues covered in the community contract. These claims are also con...rmed in the survey data: More than one third of the HUMCs in the treatment communities were dissolved and new elected or received new members following the intervention. In the control communities, we observe no dissolved HUMC. Further, the CBOs report that the community also monitored the health facility sta¤ during health visits to the clinic, when they rewarded and questioned issues in the community contract which had or had not been addressed, suggesting a more systematic use of non-pecuniary rewards. Tools such as suggestion boxes (where community members could anonymously leave suggestions for change or comment on the lack of change that was supposed to have taken place), numbered waiting cards (to ensure a ...rst-come- ...rst serve basis), and duty roasters, were also reported to be put in place in several treatment facilities. In table 2, we formally look at the program impact on these processes. We use data collected through visual checks by enumerators during the post-intervention survey. 16 As reported in table 2 (regressions 1-2), one year into the project, treatment facilities are signi...cantly more likely to have suggestion boxes (no control facility had these, while 36 % of the treatment facilities did) and numbered waiting cards (only one control facility had these, while 25 % of the treatment facilities did). A higher share of treatment facilities also post information on free-services and patient's rights and obligations (regressions 3-4). The enumerators could visually con...rm that 70 percent (17 out of 25) of the treatment facilities had at least one of these "monitoring tools" (suggestion boxes, numbered waiting cards, and/or posters on free-services), while only 4 out of 25 control units had at least one of them. The di¤erence is statistically highly signi...cant (column 5). The results based on household data mirror the ...ndings reported in columns 1-5. For example, the performance of the sta¤is more often discussed in local council meet- ings in the treatment communities (regression 6), suggesting that the treatment com- munities became more engaged. Three out of four households surveyed have attended at least one village meeting in 2005. Of those attending, 40 percent (13 percentage points) more households in the treatment community report that the functioning of the health facility was discussed. Combining the evidence from the CBO reports and the household survey data thus suggests that both the "quantity" of discussions about the project facility and the subject (from general to speci...c discussions about the community contract) changed in response to the intervention.19 7.3 Treatment practices The qualitative evidence from the CBOs and, to the extent that we can measure it, the ...ndings reported in table 2, con...rm that the treatment communities became more involved in monitoring the provider. Did community monitoring a¤ect the health worker's behavior and performance? We turn to this next. We report the results on treatment practices and sta¤ behavior, both as expressed in perception responses by households (in appendix A.3) and in quantitative indicators such as the immunization of children, waiting time, sta¤ absenteeism, examination procedures, management of the clinic, and extent of preventive care. We start by looking at examination procedures.20 Regression 1, table 3, presents the result of estimating (4) with the dependent variable being an indicator of whether any equipment (for instance thermometer or blood pressure equipment) was used dur- ing the examination. 49 percent of the patients in the treatment community reported that equipment was used the last time the respondent (or the respondent's child) 19Additional evidence on community engagement and monitoring is reported in appendix A.3. 20Naturally, the relevant treatment is conditional on illness and the condition of the patient. How- ever, since the project was randomly allocated across communities, there is no reason to believe that the type of illness and the condition of the patients should di¤er systematically across groups. In fact, we have information on reported symptoms for which the patient seeks care (from the household survey). There are, on average, no systematic di¤erences in reported symptoms across treatment and control communities. 17 visited the project clinic, as opposed to only 41 percent in the control group. The di¤erence-in-di¤erences estimate, 8 percentage points or a 19% increase, is highly sig- ni...cant. In regression 2, table 3, we look at an alternative measure of sta¤ performance ­ the waiting time ­de...ned as the di¤erence between the time the user left the facility and the time the user arrived at the facility, subtracting the examination time. On average, the waiting time was 133 minutes in the control facilities and 119 in the treatment facilities. The di¤erence is highly signi...cant.21 Table 4, column 1, reports the results on absenteeism.22 The point estimates suggest a substantial treatment e¤ect. On average, the absence rate, de...ned as the ratio of workers not physically present at the time of the post-intervention survey to the number of workers employed, is 19 percent (10 percentage points) lower in the treatment facilities. Column 2 presents the result when only using the nominator as the dependent variable. In the treatment facilities, 3.1 workers were present on average as compared to 2.3 in the control clinic. Thus, in response to more extensive community monitoring, health workers are more likely to be at work. Enumerators also visually checked the condition of the health center, i.e. whether oors and walls were clean, the condition of the furniture and the smell of the facility. Each condition was ranked on a score from 1 (dirty) to 3 (clean). Through principal components analysis, we transform these four variables into a summary score (the ...rst component): "condition of the clinic". There is a large and signi...cant improvement in the treatment clinic. The point estimate implies that treatment clinics, on average, score 0.56 standard deviations (in the sample of control facilities) higher than the control facilities. Thus, treatment clinics appear to have put more e¤ort into keeping the clinic in decent condition in response to the intervention.23 The ...ndings on immunization of children under ...ve are reported in tables 5a- 5d.24 We have information on how many times (doses) in total each child has been 21 The point estimates for the treatment e¤ect in table 3 are similar, but somewhat less precisely estimated, when only using data from the post-intervention survey, i.e. when estimating (2) instead of (4). 22 The post-intervention survey was not announced in advance. At the start of the survey, enu- merators physically veri...ed the provider's presence. A worker was counted as absent if, at the time of the visit (during facility hours), he or she was not in the clinic. Sta¤ reported to be on outreach were omitted from the absence calculation. In the full sample, 47 percent of the health workers were absent. Chaudhury et al. (2006), based on a larger sample of both rural and urban health centers in Uganda, report that 37 percent of the workers, on average, are absent. 23 Improvements in treatment practices are also substantiated by the qualitative data assembled. As reported in the appendix, table A.8., for all three subjective measures (overall change in the quality of services provided over the last year, change in sta¤ politeness, change in availability of medical sta¤), there are positive and signi...cant di¤erences between the treatment and control communities' responses. 24 We report results of estimating (2) rather than the di¤erence-in-di¤erences equation (4), since the pre-treatment vaccination outcomes were strongly inuenced by a mass immunization campaign implemented prior to the survey period. Due to reported irregularities in the top management of the unit in charge of the immunization campaigns, we have not been able to assemble accurate information on the actual timing of the campaign prior to the intervention. 18 immunized with polio, DPT, BCG, and measles. To the extent that this is possible, these data were collected from households'immunization cards. According to the Uganda National Expanded Program on Immunization (UNEPI), each child in Uganda is suppose to be immunized against measles (one dose at 9 months and two doses in case of an epidemic); DPT (three doses at 6 weeks, 10 weeks and 14 weeks); BCG (one dose at birth or during the ...rst contact with a health facility); and polio (three doses, or four if delivery takes place at the facility, at 6 weeks, 10 weeks and 14 weeks). To account for these immunization requirements, we create dummy variables taking the value of one if child i of cohort (age) j had received the required dose(s) of measles, DPT, BCG, and polio, respectively, and zero otherwise. We then estimate (2), using these binary indicators (for measles, DPT, BCG, and polio) as dependent variables for each age group (0-12 months, 13-24 months, 25-36 months, 37-48 months, and 49-60 months). The results are reported in tables 5a-5d. There are signi...cant positive di¤erences between households in the treatment and control community for all four vaccines, although not for all cohorts. The program impact on measles vaccination is presented in table 5a. Approximately 34 percent of the children under one year have received at least one dose against measles. There is no signi...cant di¤erence between treatment and control groups (regression 1). For one- year old children (13-24 months), however, we ...nd a signi...cant di¤erence (regression 2). In the control group, 79 percent of the children have been immunized, while the corresponding number in the treatment group is 5.6 percentage points higher. A smaller, but signi...cant, di¤erence also shows up in the cohort of three-year old children (37-48 months) (regression 4). Table 5b reports the results on immunization against polio. There are positive and signi...cant di¤erences in all but the oldest age group (regressions 6-9). The di¤erence is largest for the youngest cohort (9.5 percentage points). For DPT, in table 5c, we ...nd a signi...cant positive di¤erence in two out of ...ve cohorts and for BCG, in table 5d, we ...nd a positive and signi...cant di¤erence (7.8 percentage points) for the youngest cohort (regression 1). According to the government health sector strategic plan, preventive care is one of the core tasks for health providers at the primary level. Although we did not collect data on households'knowledge about health and various preventive measures, we have data on to what extent households have been informed about the potential dangers of self-treatment and if they have received information about family planning. Table 6 shows that a signi...cantly larger share of households in the treatment communities have received information about the dangers of self-treatment (regression 1), and the importance of family planning (regression 2). The di¤erence is 9 and 7 percentage points, respectively.25 25As a reference point, the share of households that have received information about the dangers of self-treatment and the importance of family planning are 32 percent and 30 percent, respectively, in the control communities, implying a 28% and 23% increase in health knowledge. 19 7.4 Utilization The evidence presented so far shows that treatment communities began to monitor the health unit more extensively in response to the intervention and that in light of better community monitoring, the health unit sta¤ responded by improving the provision of health services. We now turn to the question of whether increased community monitoring also resulted in improved quantity and quality of care. Tables 7 and 8 report estimates of the treatment e¤ect on quantity. We collected detailed data from the health facilities on the number of out-patients, the number of deliveries, the number of antenatal care patients, and the number of people seeking family planning services.26 Table 7 presents the results, for the four di¤erent utilization variables, from the estimations of equations (2) and (4). There are positive and signi...cant di¤erences between treatment and control facilities across all four services. One year into the program, utilization (for general outpatient services) is 16 percent higher in the treat- ment facilities. When controlling for district ...xed e¤ects, the point estimate is slightly larger and more precisely estimated (signi...cant at the 1 percent level). The di¤erence in the number of deliveries at the facility (albeit starting from a low level) is even larger (68 percent, regression 4) and fairly precisely estimated. There are also positive and signi...cant di¤erences in the number of patients seeking antenatal care (22 percent, regression 8) and family planning (60 percent, regression 10). As a complement to the di¤erence approach, columns 3 and 6 present the results from the estimation of a value added speci...cation.27 Di¤erence-in-di¤erences esti- mates, i.e., equation (4), are reported in table A.10 in the appendix.28 The point estimates from both speci...cations are positive and highly signi...cant. The di¤erence- in-di¤erences estimates also suggest a larger treatment e¤ect (28%) for outpatient services. Table 8 reports changes in utilization patterns based on household data. We col- lected data on where each household member sought care during the last year in case of illness that required treatment. Apart from recording visits to the project facility (treatment or control facility), we recorded visits to private providers (for-pro...t and NGOs), traditional healers, self-treatment (i.e., purchases of medicine in drug shops), or other government facilities (i.e., not a project facility). Consistent with the ...nd- ings reported in table 7, we ...nd a positive and signi...cant di¤erence in the use of the project facility between treatment and control facilities (regression 1). The increase, 26As discussed in section 5, these data were assembled by counting the number of patients from daily patient records, maternity unit records, the antenatal care register, and the family planning register. 27Data on the number of antenatal care patients and the number of people seeking family planning services were not collected from medical records in the pre-treatment survey. 28The value added speci...cation is yjdPOST = V A + V ATjd + yjdPRE + "jdPOST : 20 14 percent higher in the treatment group as compared to the control group, is similar to that reported in table 7 (using facility records). Table 8 also shows that households in the treatment community reduced the num- ber of visits to traditional healers and the extent of self-treatment (regressions 4 and 5), while there are no statistically signi...cant di¤erences (regressions 2, 3, 6 and 7) across the two groups in the use of other providers (NGO, for-pro...t, or other government facilities). Thus, households in the treatment communities switched from traditional healers and self-treatment to the project facility in response to the intervention. 7.5 Health outcomes The main objective of the community-based monitoring project was to improve health outcomes in rural areas of Uganda where health indicators have been stagnating. To achieve this objective, the project intended to enhance communities'abilities to monitor the public health care provider, thereby strengthening providers'incentives to increase both the quality and quantity of primary health care provision. As reported above, the project was successful in raising both utilization and, to the extent that this can be measured, service quality. Next, we turn to health outcomes. Data on two health outcomes were collected. First, we collected information on whether the household had su¤ered from the death of a child (under ...ve years) in 2005, i.e., the ...rst year of the community monitoring project. Second, we measured the weight of all infants (i.e., under 18 months of age) and children (between 18 and 36 months of age) in the surveyed households.29 Health outcomes (under-...ve mortality and weight of infants) could have improved for several reasons. As noted in the Introduction, access to a small set of proven, inexpensive services could, worldwide, have prevented more than half of all under-...ve deaths. In the community monitoring project speci...cally, having patients switching from self-treatment or traditional healers to seeking care at the treatment facility could have an e¤ect. Holding utilization constant, better service quality and increased im- munization of children (particular measles) could also result in a reduction in mortality and improved health status. The increased use of preventive care (health education) may also have an e¤ect. Table 9 presents the results on child mortality. 3.2 percent of the surveyed house- holds in the treatment community had su¤ered from the death of a child in 2005. The corresponding number in the control community is 4.9 percent. The di¤erence ­a 33 percent reduction in child deaths in the treatment communities ­is signi...cant and 29The weighing scale was a regular hanging baby scale with trousers (Salter type). Two trained enumerators assisted in the task. During the weighing process, the enumerators took help from family members, mostly mothers. When the infant/child was hanging calmly on the scale, the enumerators recorded the weight. 21 fairly precisely estimated when controlling for district ...xed e¤ects (regression 2).30,31 With a total of approximately 55,000 households residing in the treatment communi- ties, the treatment e¤ect (0.017) corresponds to 546 averted under-...ve deaths in the treatment group in 2005.32 The dependent variable in regression 3, table 9, is under-...ve mortality rate in the community.33 Consistent with the ...ndings in columns 1-2, the point estimate suggest a substantial treatment e¤ect. The average under-...ve mortality rate in the control group is 145, close to the o¢ cial ...gure of 133 for 2005 (UNICEF, 2006). In the treatment group, the under-...ve mortality rate is 97 and the di¤erence is signi...cant at the 5-percent level.34 The program impact on the weight of infants is reported in table 10. Growth charts for boys and girls are depicted in ...gure 4. As in Cortinovis et al's (1997) study of over 4,000 children from 31 villages in Mbarara (a district in south-western Uganda), we ...nd that Ugandan infants have values of weight far lower than the NCHS/CDC international reference. The gap increases for older infants. The median weight of six-month old boys in the sample is close to the 25th percentile of the NCHS/CDC reference chart. For the 18 months old, the median weight for boys lies close to the 10the percentile of the NCHS/CDC chart. Figure 5 plots the distribution of weight-for-age (z score).35 A population similar to the reference population (NCHS) will have a mean z score of zero, with approximately 2.5 percent of the population below a z score of -2 (the threshold for moderately underweight). In the sample of measured infants, 17.4 percent fall below this threshold. 30The numbers on child deaths are comparable to other survey based measures on child mortality in Uganda. In a sample of 1178 children under the age of ...ve from north-western Uganda (from both urban and rural villages), Vella et al (1992) ...nd a mortality rate (percent of children who died during the last year) of 3.9 percent. Mortality rates were around 10% during the ...rst year of life, 3.1% in the second year, 4.0% in the third year, and about 0.5% thereafter. 31The treatment e¤ect reported in table 9 is quantitatively important, even as compared to medical ...eld trials where infant mortality is a measured outcome. For example, of the 23 measures (i.e. biological agent or action intended to reduce child mortality) for which Jones et al. (2003) conclude that there is su¢ cient or limited evidence of e¤ect on child mortality, the mean e¤ect was a 37% reduction in infant mortality. 32We get an almost identical estimate (540 averted deaths) when we weight with distance to the health facility. Since villages closer to the facility were oversampled, the sample of treatment villages is not fully representative of the total population in the treatment communities. 33The under-...ve mortality rate is estimated as the number deaths of children under ...ve in the community as a fraction of number of live births in 2005 (i.e. number of infants younger than one year at the end of 2005 plus the number of infants under one year that died in 2005) expressed per 1,000 live births. 34To put this into perspective, an under-...ve mortality rate of 97 implies that child mortality in the treatment group is in parity with that of Ghana - a country with a 50 percent higher GDP per capita (PPP US$) in 2001 (UNDP, 2002). 35The z-score is a normally distributed measure of growth de...ned as the di¤erence between the weight of an individual and the median value of weight for the reference population (2000 CDC Growth Reference in the U.S.) for the same age, divided by the standard deviation of the reference population. We exclude z scores > j4:5j as implausible. Four observations (out of 1142) with z scores < 4:5 were consequently dropped. 22 8.5 percent of the infants (up to 18 months) are severely underweight (< 3 z scores). Almost a quarter of the infants falls below the mildly underweight threshold (< 1 z score). The di¤erence in means of z scores of infants between the treatment and the control group is reported in regression 1, table 10. The estimated e¤ect (di¤erence) is 0.164 z score in weight-for-age. Regression 2 applies a more stringent restriction on the data to avoid problems of misreporting.36 The di¤erence in mean is 0.17 z score and is precisely estimated. Figure 6 plots the distribution of z scores for treatment and control groups. The di¤erence in measured weight is most apparent for underweight children. Underweight status causes a decrease in immune and non-immune host defenses. Thus, since underweight children are at a higher risk of su¤ering from infectious diseases (and more severe complications of infectious diseases), and therefore in higher demand for/need of health care, the data in ...gure 6 are consistent with a positive treatment e¤ect arising from improved access and quality of health care, rather than a general increase in nutritional status. Regression 3 adds district ...xed e¤ects and controls for age and gender. The results remain qualitatively unchanged. The incidence of underweight increases with age. We cannot reject the hypothesis that the treatment e¤ect is the same for girls and boys. The treatment e¤ect is quantitatively important. For this purpose, the baseline proportion of infants in each risk category (severe, < 3 z scores; moderately, 3 z scores < 2; mild, 2 z scores < 1) in the control group was calculated. Applying the shift in the weight-for-age distribution (adding 0.17 z score) with the odds ratio for each category ­children who are mildly [moderately] {severely} underweight have about a two-fold [...ve-fold] {eight-fold} higher risk of deaths from infectious disease (Jones et al, 2003) ­ the reduction in average risk of mortality is estimated to be approximately 8 percent (...gure 6).37 Columns 4-5 in table 10 report the program impact on child weight for children between 18-36 months of age. The treatment e¤ect is small and insigni...cant.38 36Speci...cally, we drop observations with a recorded weight above the 90th percentile in the growth chart reported in Cortinovis et al (1997). Since weight is measured by trained enumerators, the reporting error is likely due to misreported age of the child. 37To put this into perspective, a review of controlled trials designed to improve the intake of complementary food for children aged six months to ...ve years showed a mean increase of 0.35 z score (Jones et al, 2003). If the present coverage level were increased to universal coverage (99%), Jones et al estimate that complementary feeding alone would prevent 6% of the under-...ve deaths in the 42 countries with the 90% of worldwide child deaths in 2000. According to Jones et al, this is one of the most e¤ective (in the sense of preventing under-...ve deaths) preventive interventions feasible for delivery at high coverage in a low-income setting. 38Measurement errors due to misreported age of the child are likely to be a more serious concern for children above 18 months than for infants. 23 7.6 Robustness Given that within each district there are both treatment and control units, one concern with the evaluation design is the possibility of spillovers from one catchment area to another. For example, if a treatment facility improved the quality of health provision due to the intervention, households in villages in the catchment area of a control community might choose to seek service in the treatment facility. If this is the case, we would overestimate the e¤ects (on utilization) of the intervention. Naturally, it is also possible that community members in the control facilities copied the monitoring approach of the treatment facilities, in which case the bias would go in the opposite direction. In practice, there are reasons to believe that this is not a serious concern. First, the average (and median) distance between the treatment and control facility is 30 kilometers. Second, in a rural setting, it is unclear to what extent information about improvements in treatment facilities has spread to control communities. Still, the possibility of spillovers is a concern. One way of testing for spillover e¤ects is to estimate an augmented version of (2) for the sample of control facilities.39 That is, we estimate yidPOST = + DISTid + "idPOST; (5) where DISTi is the distance (in kilometers) between the control facility i and the closest treatment facility. The results of estimating (5) for the various utilizations measures are reported in table 11. In all speci...cations, the estimate of di¤ers insigni...cantly from zero. Table 12 reports a di¤erence-in-di¤erences version of (5). Again, the point esti- mates are insigni...cantly di¤erent from zero.. Another concern, which does not inuence the casual e¤ect of the project but the interpretation, is if the district or sub-district management changed their behavior or support in response to the intervention. For example, the Health Sub-district or local government may have provided additional funding or other support to the treatment facilities. The results in tables 13-15 do not provide any evidence of this being the case. Di¤erence-in-di¤erences estimates of the monthly supply of drugs indicate that the treatment and control facilities are similar. If anything, drug supplies are smaller in the treatment clinics (table 13). The treatment facilities did not receive more funding from the sub-district or district (table 12, regression 6) as compared to the control facilities. The di¤erence-in-di¤erences estimate is negative, but insigni...cant. There are no di¤erences in constructions or infrastructure during the ...rst project year (table 14), and there are no di¤erences in the availability of equipment at the health facility (table 15). A similar interpretational concern arises if the upper-level authorities increased their supervision and control of treatment facilities in response to the intervention. 39Pooling the sample of control and treatment facilities and adding a dummy for treatment facilities yields identical results. 24 However, this does not seem to be the case either. The supervision of providers by upper-level government authorities remained low in both the treatment and the control group (table 16, regressions 1-2). The incidence of supervision and control visits may be an imprecise measure of the e¤ectiveness of monitoring by the upper-level authorities. A complementary measure is implemented sanctions. We have data on the extent to which sta¤ was dismissed or transferred during the ...rst year of the project. As noted in section 4, only the District Service Commission has the authority to dismiss and transfer sta¤. There is only a handful of sta¤ that has been dismissed or transferred in 2005 and there is no systematic pattern that distinguishes treatment from control facilities (table 16, regressions 3-4). Likewise, there is no di¤erence between treatment and control facilities in the number of sta¤ that voluntarily left the facility during 2005 (regression 5). Taken together, these ...ndings reinforce our con...dence that the improved quality and quantity of health care provision resulted from increased e¤orts by the health unit sta¤ to serve the community in light of better community monitoring. 8 Conclusion In this paper, we have studied the e¤ects of enhancing rural communities'ability to hold primary health care providers accountable. We ...nd that both the quality and quantity of the health service provision improved in the treatment communities: One year into the program, average utilization was 16 percent higher in the treatment communities; the weight of infants higher, and the number of deaths among children under ...ve markedly lower. Treatment communities became more extensively involved in monitoring the providers following the intervention and the results suggest that the health unit sta¤ responded by exerting a higher e¤ort in serving the community. By strengthening the providers'incentives to serve the poor, health provision and in the end health outcomes can be signi...cantly improved. The starting point of this work is the mounting evidence showing that the provi- sion of public services to poor people in developing countries is constrained by weak incentives of service providers. As argued in Chaudhury et al. (2006), this evidence is symptomatic of failures in "street-level" institutions and governance. However, al- though these failures constitute a direct obstacle to economic and social development, they have, until recently, received much less attention in the literature than weaknesses in macro institutions. This paper is an attempt at partly closing this gap. Although the Citizen report card project appears to be successful, it is too early to use these ...ndings as a basis for continued or increased support and funding for various activities with the aim of strengthening bene...ciary control. There are still a number of outstanding issues. One important concern is to what extent the processes initiated by the Citizen report card project are permanent. Since the project is ongoing and scaled up to involve an additional 25 project units, this question can be answered at a 25 later stage. At the same time, it is possible that the treatment communities'ability to coordinate citizen actions has also been applied to other areas of concern (education, local road construction, etc.), in which case the aggregate return is even larger than what the above results suggest. It is also possible that even better results can be achieved by combining bottom-up monitoring with a top-down approach (supervision and possibly sanctions/rewards from someone in the institutional hierarchy assigned to monitor and control the primary health care providers). Before scaling up, it is also important to subject the project to a cost-bene...t analysis and relate the cost-bene...t outcomes to other possible interventions. This would require putting a value on the improvements we have documented. To provide a avor of such a cost-bene...t analysis, consider the ...ndings on averting the death of a child under ...ve. The intervention resulted in 1.7 percentage points fewer child deaths in the treatment communities during the ...rst project year. To the extent that this number is representative of the total treatment population, this would imply that approximately 550 under-...ve deaths were averted as a result of the intervention. A back-of-the-envelope calculation then suggests that the intervention, only judged on the cost per death averted, must be considered to be fairly cost-e¤ective. The estimated cost of averting the death of a child under ...ve is around $300 in the Citizen report card project. This can be compared to the numbers reported by Filmer and Pritchett (1999). They contrast the cost of averting the death of a child derived from increasing public expenditures on health (regression estimates range from $47,112 to $100,927), to more conventional health interventions based on cost-e¤ectiveness estimates of the minimum required cost to avert a death (ranges from $1,000 to $10,000 for diarrheal diseases, from $379 to $1,610 for acute respiratory infection, $78 to $990 for malaria, and $836-$3,967 for complications of pregnancy).40 The Citizen report card project was implemented in nine di¤erent districts of Uganda and reached approximately 55,000 households. Thus, in this dimension, the project has already shown that it can be brought to scale. Still, this project is a controlled experiment in some dimension. Speci...cally, data collection and data analy- ses were supervised by the evaluators. To the extent that these tasks were delegated to local actors in the various communities, they could have been subject to capture. This is an issue on which our ...ndings do not shed any light. What our ...ndings strongly suggest, though, is that experimentation and evaluation of new tools to en- hance accountability should be an integral part of the research agenda on improving the outcomes of social services. This is an area where at present, research on what works and what does not is lagging behind policy. 40These numbers should be viewed with caution. Naturally, the 95 percent con...dence interval would also include a much smaller estimate of program impact than the 1.7 percentage points used here. Moreover, since the largest cost item was the collection of data and these data were partly used in the intervention and partly to evaluate impact, the cost is a rough estimate. Filmer and Pritchett's (1999) estimates of the cost of averting a child death derived from increasing public expenditures on health are subject to a variety of estimation problems and the health interventions based cost-e¤ectiveness estimates of the minimum required cost to avert a death are, as noted by Filmer and Pritchett, at best suggestive. 26 References Appleton, Simon (2001), "The Rich Are Just Like Us, Only Richer': Poverty Functions or Consumption Functions?", Journal of African Economies 10(4): 433-469. Banerjee, Abhijit and Esther Duo (2005), "Addressing Absence", Journal of Eco- nomic Perspectives 20 (1): 117-132. Banerjee, Abhijit, Angus Deaton and Esther Duo (2004), "Wealth, Health, and Health Service Delivery in Rural Rajasthan", American Economic Review Papers and Proceedings 94(2): 326-330. Banerjee, Abhijit, and Ruimin He (2003), "The World Bank of the Future", American Economic Review 93(2): 39-44. Besley, Timothy, and Robin Burgess (2002), "The Political Economy of Government Responsiveness: Theory and Evidence From India", The Quarterly Journal of Eco- nomics 177(4):1415-51. Besley, Timothy, and Andrea Prat (2005), "Handcu¤s for the Grabbing Hand? Media Capture and Government Accountability", American Economic Review, forthcoming. Björkman, Martina (2006), "Does Money Matter for Student Performance? Evidence from a Grant Program in Uganda", Working Paper, IIES, Stockholm University. Chaudhury, Nazmul, Je¤rey Hammer, Michael Kremer, Karthik Muralidharan, and F. Halsey Rogers (2006), "Missing in Action: Teacher and Health Worker Absence in Developing Countries", Journal of Economic Perspectives 20(1): 91-116. Esther, Duo and Rema Hanna (2005), "Monitoring Works: Getting Teachers to Come to School", Working Paper, Department of Economics and Poverty Action Lab, MIT. Filmer, Dean and Lant Pritchett (1999), "The Impact of Public Spending on Health: Does Money Matter?", Social Science and Medicine 49(10). Jimenez, Emmanuel and Yasuyuki Sawada (1999), "Do Community-Managed Schools Work? An Evaluation of El Salvador's EDUCO Program", The World Bank Economic Review 13(3): 415-441. Khemani, Stuti (2006), "Can Information Campaigns Overcome Political Obstacles to Serving the Poor," mimeo, The World Bank. Malena, Carmen, Reiner Forster and Janmejay Singh (2004), "Social Accountability: An Introduction to the Concept and Emerging Practice", Social Development Papers 76, Participation and Civic Engagement Group, The World Bank. McPake, Barbara, Delius Asiimwe, Francis Mwesigye, Mathias Ofumbi, Lisbeth Orten- blad, Pieter Streeand and Asaph Turinde (1999), "The Economic Behavior of Health 27 Workers in Uganda: Implications for Quality and Accessibility of Public Health Ser- vices,"Social Science and Medicine 49(7): 849-865. Moeller, Lars Christian (2002), "Uganda and the Millennium Development Goals", Human Development Network, World Bank, Washington D.C, Processed. Olken, Ben (2005), " Monitoring Corruption: Evidence from a Field Experiment in Indonesia", NBER Working Paper No.11753. Reinikka, Ritva and Jakob Svensson (2004), "Local Capture: Evidence from a Central Government Transfer Program in Uganda", The Quarterly Journal of Economics 119 (2): 679-705. Reinikka, Ritva and Jakob Svensson (2005a), "The Power of Information: Evidence from a Newspaper Campaign to Reduce Capture", Working paper, IIES, Stockholm University. Reinikka, Ritva and Jakob Svensson (2005b), "Fighting Corruption to Improve School- ing: Evidence from a Newspaper Campaign in Uganda", Journal of the European Economic Association 3 (2-3): 259-267. Reinikka, Ritva and Jakob Svensson (2005c), "Working for God", Working paper, IIES, Stockholm University. Republic of Uganda (2000), "National Health Policy and Health Sector Strategic Plan 2000/01-2004/05", Ministry of Health, Kampala Republic of Uganda (2002), "Infant Mortality in Uganda 1995-2000: Why the Non- Improvement?", Discussion Paper No. 6, Planning and Economic Development, Kam- pala. Singh, Janmejay and Meera Shah (2002), "Community Score Cards in rural Malawi", World Bank, Washington, D.C. Processed Strömberg, David (2003), "Mass Media and Public Policy", European Economic Re- view 45(4-6): 652-63. Strömberg, David (2004), "Radio's Impact on Public Spending", The Quarterly Jour- nal of Economics 119(1): 189-221. Samuel Paul (2002), Holding the State to Account: Citizen Monitoring in Action, Books for Change, Bangalore. UNICEF (2006), The State of the World's Children 2007: Women and Children - The double dividend of gender equality, UNICEF. UNDP (2002), Millennium Development Goals: A compact among nations to end human poverty, Human Development Report 2003, UNDP. 28 World Bank (2003), Making Service Work for the Poor People, World Development Report 2004, World Bank and Oxford University Press. World Bank (1996), "World Bank Participation Sourcebook", Environmental Depart- ment papers No. 19, Washington. World Health Organization (2006), World Health Statistics 2006, WHO Press, Switzer- land. 29 A Appendix A.1 Sampling Strategy The starting point for the sample frame is the QSDS data set for 2000 and the second round of QSDS data for 2004 (Reinikka and Svensson, 2005c). The QSDS data set consists of a total of 155 health facilities. The sample design for the QSDS was governed by three principles. First, the attention was restricted to dispensaries (i.e., health centre III) to ensure a degree of homogeneity across sampled facilities. Second, subject to security constraints, the sample was meant to capture regional di¤erences. Finally, the sample had to include facilities from the main ownership categories: government, private non-pro...t, and private for-pro...t providers. These three considerations led to the choice of a strati...ed random sample, which was based on the Ministry of Health facility register for 1999. The register includes government, private non-pro...t, and private for-pro...t health facilities, but is known to be inaccurate with respect to the latter two. A total of 155 health facilities were sampled. On the basis of existing information on private-for pro...t and non-pro...t, it was decided that the sample would include 81 government facilities, 44 private non-for-pro...t facilities, and 30 private for- pro...t facilities. As a ...rst step in the sampling process, 8 districts (out of 45) had to be dropped from the sample frame due to security concerns.41 From the remaining districts, 10 districts, strati...ed according to geographical location, were randomly sampled in proportion to district population size. Thus, three districts were chosen from the Eastern and Central regions, and two from the Western and North regions. A.1.1 Part 1: Sampling of Villages Our initial sample frame for the household survey thus consists of 81 government facil- ities and their "catchment"areas. The catchment area of a facility is operationalized as the ...ve-kilometer radius around the facility. For di¤erent reasons, all these facil- ities/catchment areas could not be included in the sample. First, three government facilities in Soroti could not be surveyed in the second round of the QSDS due to security concerns. Second, detailed maps (covering at least the ...ve-kilometer radius around the facility) and the corresponding census data could not be collected for three units.42 Third, for some facilities, a signi...cant part of the catchment area lies outside the facilities'administrative boundaries. These facilities/catchment areas were there- 41 The eight districts were Bundibugyo, Gulu, Kabarole, Kasese, Kibaale, Kitgum, Kotido, and Moroto. 42 Uganda Administrative Maps from the Cartography Department at the Uganda Bureau of Sta- tistics. These maps are drawn with the sub-county level as the highest administrative unit and the village as the smallest unit. The maps were drawn in September 2001 (some earlier) as a preparation for the 2001/2002 Census. 30 fore dropped from the sample.43 Finally, ...ve districts had been split since the initial survey; Kaberamaido previously part of Soroti, Kayunga previously part of Mukono, Mayuge previously part of Iganga, Sironko previously part of Mbale, and Wakiso pre- viously part of Mpigi. As a result, for some districts, we end up with too few facilities. The districts with too few (less than four) facilities were therefore dropped. Altogether, we end up with a sample of 50 government facilities/catchment areas (CA). Combining information on geographical location (from the detailed maps provided by Uganda Bureau of Statistics (UBOS)) and census data, we could list all villages and enumeration areas and their size (number of households) for each catchment area (CA). Summary data on the number of villages in CA are provided in Tables A.1- A.3. Altogether, there are 804 enumeration areas, covering 1,194 villages and 109,296 households in the 50 CAs. On average, a CA consists of 20 enumeration areas and 29 villages, half of which are outside the 3 km radius. The average (median) village has 92 (84) households. Three general principles governed our choice of sample. First, we wanted our sample of households to be representative of the potential users of the facility in the CA. This, in turn, is a function of both the size of the population in the CA and the distance to the facility. Second, for the intervention to be feasible (and within our budget constraint), we wanted to restrict the number of villages to be surveyed within a given CA. For the same reason, we wanted to ensure that the villages surveyed are clustered together in a smaller set of clusters within each CA. Finally, we wanted to include the village where the facility was located (typically the village where the sta¤ resides). To ensure this, we chose a four-stage sampling design. First, we determined how many villages should be selected from each CA. Balancing the need of being represen- tative of the potential users of the facility in the CA and designing a ...nancially and logistically feasible survey strategy, the "village rule"was set to no: villages = 3:3 + 0:1 (no: villages in CA): (6) Second, we determined the share of these villages that should be sampled from the one, three, and ...ve kilometer radius (strata 1, 3, and 5), i.e., the "strata rule".44 For each CA, these shares were set so as to replicate the shares of villages in the di¤erent strata in the CA, with one exception. Since households in villages closer to the facility, everything else equal, are more likely to visit the facility, we oversampled the villages from the one-kilometer radius by a factor of 2 and undersampled the share of facilities within the ...ve-kilometer radius (excluding the facilities within the three-kilometer radius) by a factor of 0.7. Third, to ensure that the villages surveyed are clustered together and that the 43Speci...cally, we dropped facilities/catchment areas where more than 25, [33] or {50} percent of the catchment area were outside the 1 [3] {5} km radius. 44Strata 1 is de...ned as the area within the one-kilometer area; strata 3 is de...ned as the area within the three-kilometer area excluding the area within the one-kilometer area; strata 5 is de...ned as the area within the ...ve-kilometer area excluding the area within the three-kilometer area. 31 village where the facility is located is included in the sample, we ...rst identi...ed the enumeration areas (EA) of the village where the facility is located and second, we selected an additional 2-4 EAs within each CA, with a probability proportional to population size. The number of EAs selected was determined by (6).45 Finally, within the sampled EAs, we randomly selected the stipulated number of villages in the 1, 3, and 5 kilometer strata in the CA. The total and the average number of villages sampled according to the sampling strategy and the actual number of villages surveyed are depicted in Table A.4.46 Summary statistics of the sample of villages surveyed are depicted in Table A.5 and Table A.6. Overall, 293 villages were surveyed (from 242 EAs) with a total population of 29,405. The average village in the sample has 102 households, slightly larger than the average village in the sample frame. A.1.2 Part 2: Sampling of Households in Selected Villages Using the most updated census data, we enumerated all 293 villages included in the ...nal sample and coded them. Two codes were created; one unique code for each household in each village (HHSVC), and one unique code for each household in the whole sample of households in the 293 villages (HHSSC). Then, we determined the number of households that should be surveyed in each village (SHHS). The rule was set as follows: SHHS Condition 10 if total no. of households in village 2 [20, 50] 0.2*(no. hhs in village) if total no. of households in village 2 [50, 100] 20 if total no. of households in village 2 [100, 200] 25 if total no. of households in village > 200 This resulted in a total sample of 4,978 households to be surveyed in the ...nal sample. The sample design to select the households to be surveyed from the set of eligible households (i.e., the enumerated households) is as follows. First, a random number between 1-10 (between 1-5 in villages with less than 100 households) was drawn. This number is denoted "START"and is the ...rst household selected. Let the last number in the village list of households (HHSVC) be denoted by "LNO". Then, the remaining (SHHS-1) sampled households are determined by selecting every xth (denoted "EVERY") household, starting from START up to the point where the total number of sampled households is equal to SHHS. The variable EVERY is de...ned as the maximum integer such that 45That is, enough EAs were chosen so that the stipulated number of villages in the 1, 3, and 5 kilometer radius could be surveyed. 46Four villages were dropped due to too few households residing in the village (less than 20 house- holds). We also had to replace a handful of villages where enumeration was not possible. This accounts for the di¤erence between the sample rule and the actual sample. 32 EV ERY = (max[integer LNO] START)=(SHHS 1) (7) Intuitively, we determined EVERY such that the sequence of households to be sampled is evenly distributed over the list of households in the village, i.e. evenly distributed over HHSVC.47 A replacement strategy was also designed. The replacements are selected as follows. If a selected household with HHSVC code x could not be surveyed, the household with HHSVC code x+1 should be selected. If that is not feasible (because there is no x+1 household or because that household could not be interviewed, or because that household has already been interviewed), the household with HHSVC code x-1 should be selected. If that is not feasible, the household with HHSVC x+2 should be selected, and thereafter x-2, etc. A.1.3 Ex-post Survey The same sample of health facilities, villages and households that were sampled and surveyed in 2004 were re-surveyed in the ex-post survey at the beginning of 2006, and there was likely to be cases where the previously surveyed household could not be in- terviewed for some reason (i.e. the household had moved or died etc.), a replacement strategy was designed. The replacements were selected as follows. If a selected house- hold with HHSVC code x could not be surveyed, pick the household residing to the right of household x. If that is not feasible (because there is no household to the right or because that household could not be interviewed either, or because that household has already been interviewed), pick the household residing to the left of household x. If that is not feasible, pick the household residing two houses to the right of household x, and then two houses to the left of household x, etc. In total, 4,996 households were surveyed in the ex-post survey, 4,373 of which were resurveyed. 47Denote LAST as the last household in the list to be surveyed (i.e. the sampled household with the highest HHSVC). Then LAST = START + EVERY*((SHHS -1). 33 Table A.1. Total number of households, villages and enumeration areas in sample frame (50 units). Total Within 1 km Within 3 km radius Within 5 km radius radius excl. those within excl. those within the 1 km radius the 3 km radius Households 109,296 11,572 41,665 56,059 Villages 1,194 113 458 623 Enumeration areas 804 Source: UBOS maps and census data Table A.2. Number of households, villages and enumeration areas in sample frame (50 units) Mean Median Min Max Households in catchment area 2,483 2,728 490 3,938 Households within 1 km radius in CA 344 240 60 1014 Households within 3 km radius excl. 1096 991 127 2,357 those within the 1 km radius in CA Households within 5 km radius excl. 1,303 1,231 173 2,428 those within the 1 and 3 km radius in CA Villages in catchment area 29 26 7 58 Villages within 1 km radius 3 3 1 8 Villages within 3 km radius excl. 13 11 2 30 those within the 1 km radius in CA Villages within 5 km radius excl. 15 15 2 31 those within the 1 and 3 km radius in CA Enumeration areas in catchment area 20 19 4 35 Villages in enumeration area 1.9 2 0 6 Source: UBOS maps and census data. Table A.3. Village characteristics in sample frame (50 units). Mean Median Min Max Number of households in village 92 84 0 273 Distance to facility 3.9 5 1 5 Source: UBOS maps and census data 34 Table A.4. Sampled villages according to village and strata rules and actual sample (50 units). According to Sample village/strata rule Villages (total) 295 293 Villages, average in CA 6 6 Villages in strata 1, total 64 70 Villages in strata 1, average in CA 1 2 Villages in strata 3, total 117 121 Villages in strata 3, average in CA 2 3 Villages in strata 5, total 114 102 Villages in strata 5, average in CA 2 2 Source: UBOS maps and census data. Table A.5. Total number of households, villages and enumeration areas in actual sample Total Within 1 km Within 3 km radius Within 5 km radius radius excl. those within excl. those within the 1 km radius the 3 km radius Households 29,405 7,696 11,653 10,056 Villages 293 70 121 102 Enumeration areas 242 Table A.6. Village characteristics of sample. Mean Median Min Max Number of households in village 102 92 22 232 Distance to facility 3.2 3 1 5 35 A.2 Participatory Methods The report card was disseminated to the community using a Participatory Rural Ap- praisal (PRA) methodology. In the early 1990s, the participatory rural appraisal methodology was mainly used by non-government organizations in East-Africa and South-Asia but are today widely used in many di¤erent organizations all over the world.48 Participatory rural appraisal evolved from a set of informal techniques used by development practitioners in rural areas to collect and analyze data. It empha- sizes local knowledge and the importance of having bene...ciaries making their own appraisal, analysis, and plans for monitoring and evaluation of service providers. It is a participatory process intended to mitigate the collective action problem by facilitat- ing the analysis of people's environment and identi...cation and discussion of problems. The method employs a wide range of tools and techniques such as maps, diagrams, role-plays and action planning. Next, we briey describe the speci...c tools used in the Citizen Report Card project in Uganda. Venn diagrams were used to discuss power issues in service delivery. Participants were asked to list the di¤erent stakeholders in health service delivery (i.e. health fa- cility sta¤, citizens, health management committee, district o¢ cials etc). Thereafter, the participants discussed the di¤erent roles and responsibilities of these players in en- suring the quality of the service, i.e. who is accountable to whom; what is a particular stakeholder accountable for, and how can these actors account for their actions. The outcome was used in the interface meeting to identify the stakeholders who have the power to ensure that quality service is delivered. The outcome also contributed to the process of developing a shared vision of how to monitor the provider. Focus group discussions were used to generate discussions among and across sub- groups. Participants were divided into key social groups such as women, men, youths, disabled, local leaders and elderly in order to get their perspectives over issues con- cerning service delivery and determine their preferences for change. Each group indi- vidually discussed the issues covered in the report card and recorded suggestions for improvements. Thereafter, each group presented the results to the other participants by using ip charts. In this way, the voice and priorities of all social groups were taken into considerations. "Now, Soon, Later" approach is a technique aimed at helping the community identify issues they would like to address in the short term and those they would address in the longer term, considering the resource envelope at hand. Thereafter, the participants were asked to prioritize the needs according to their resource envelope and discuss which factors are important and necessary for making a change. This tool was intended to help the community analyze the resources available, the time frame for implementing the desired change and the importance of the issue. Role play was used to illustrate community and health facility interactions as per- ceived by the respective parties. This tool facilitated the discussion and dialogue in the interface meeting between health sta¤ and community members. The story of the play illustrated the participants'interpretation of an ordinary day at the health facil- 48See World Bank (1996). 36 ity. In the play, community members were asked to act the roles of health facility sta¤ (In-charge; Mid-wife; Records Assistant; Watch Man; Laboratory Assistant; Senior Nurse etc) and health facility sta¤ acted the roles of users of the facility (pregnant women; patients; poor patients; community leader; Chairman). Role plays are viewed as an e¤ective tool for di¤using sensitive issues (such as absenteeism or weak attention of sta¤). It is also a tool that can be used to illustrate constraints and opportunities, enabling users and providers to forge a way forward. Not only did the role play fo- cus on the current situation at the health facility but in a second role play, the plot exempli...ed what the participants would like the situation to be like in six months. Roles and Responsibility Analysis is used to provide clarity as to who is responsi- ble for what activity. In this analysis, the participants review all planned activities in the action plan and ensure that each activity becomes someone's responsibility. This tool de...ne roles and responsibilities and helps strengthening the relationship of ac- countability between health service providers and citizens with regard to the activities determined in the action plan. The facilitator guides the participants to discuss the activities recorded in the action plan and help them agree on the criteria for taking up a responsibility for a particular activity. Thereafter, the participants identify who among the community or health facility sta¤ would suit the criteria and discuss this responsibility with the person or group identi...ed. The groups or individuals assigned to be responsible for a certain activity are then recorded in the action plan. Action planning was a tool used in the ...nal stage to summarize and record the community's suggestions for improvements (and how to reach them without additional resources). The action plan states the health issues/services that had been identi...ed by the community and the sta¤ as the most important to address; how these issues could be addressed; when they are supposed to be achieved; by whom this will be done; and how the community could monitor the improvements (or the lack thereof). The action plan is a contract between the community and the health facility. It forms the basis for local monitoring and makes it easier for the community to keep track of the implementation of agreed recommendations. A.3 Additional results A.3.1 Processes Table A.7. reports additional ...ndings on changes in processes at the community level following the intervention. As reported in column 1, community members in the treat- ment group are better informed about patients'rights and obligations according to the government set standard for health service delivery at the primary level.49 The treat- ment communities are also more likely (although most households do not know this) to 49These data are based on simple knowledge tests administered to households. Speci...cally, respon- dents were asked to list the main "rights" (right to con...dential treatment, right to polite treatment according to ...rst come-...rst serve basis, right to receive information on ailment and drugs, free health care, attended with one hour) according to the Yellow Star Program (see section 5.2). The dependent 37 know when the project facility receives drug deliveries (regression 2) and signi...cantly more likely to have been informed about the HUMC's role and responsibilities. Table A.7. Program impact on processes: performance of sta¤ discussed in village meeting and information about patients'rights Dependent Informed Informed about Informed about variable about patients' drug deliveries HUMC's role & rights responsibilities Speci...cation (1) (2) (3) Program impact 0.03* 0.03** 0.05*** (0.02) (0.01) (0.01) Mean in control group 0.34 0.11 0.08 District ...xed e¤ects Yes Yes Yes Observations 4996 4996 4996 R2 0.02 0.06 0.05 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable in speci...cations: (1) Dummy variable indicating if the household could list at least one of the rights according to the Yellow Start program, (2) Dummy variable indicating if the household knows when the health facility receives drugs, (3) Dummy variable indicating if the household has been informed about HUMC's role and responsibilities. c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. A.3.2 Treatment practices Table A.8. reports treatment e¤ects based on households'perception of the quality of service delivery at project facilities. Although these estimates constitute causal e¤ects of the community monitoring project, there are several reasons why they should be interpreted with care. For all three subjective measures (overall change in the quality of services provided over the last year, change in sta¤ politeness, change in availability of medical sta¤), there are positive and signi...cant di¤erences between the treatment and control communities'responses. Most households in the control communities (53 %) perceive that the quality of services provided at the project facility has become worse or not improved during the last year. In the treatment communities, on the other hand, a majority (564 %) [ngt fel på denna si¤ra!!!!] of the households surveyed report that the quality of services provided at the project facility has improved. The di¤erence is signi...cant and precisely estimated once controlling for district ...xed e¤ects (regression 1). We ...nd similar patterns in households'perceptions of the politeness of sta¤ and the availability of medical sta¤ when visiting the clinic (regressions 2 and 3 in table A.8).50 variable (table 3, speci...cation 2) takes the value of 1 if the respondent could list at least one of these rights and zero otherwise. We ...nd a positive and signi...cant e¤ect (treatment e¤ect) on both the extensive and intensive margin (not reported), i.e.; more informed respondents and conditional on being informed, better knowledge about patients'rights following the intervention. 50We ...nd similar e¤ects and of the same magnitude (positive and signi...cant) using ratings on the attention given to the patient by the sta¤ when visiting the project facility and whether the patient 38 Table A.8. Citizens'perception of changes in the quality of health care over the last year Dependent Overall quality Sta¤ politeness Availability of variable medical sta¤ Speci...cation (1) (2) (3) Program impact 0.09** 0.08** 0.09*** (0.04) (0.03) (0.03) Mean in control group 0.47 0.53 0.51 Controls Yes Yes Yes District ...xed e¤ects Yes Yes Yes Observations 3343 3343 3343 R2 0.09 0.05 0.06 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable in speci...cations: (1) Dummy variable indicating improvement in overall qual- ity; (2) Dummy variable indicating improvement in sta¤ politeness; (3) Dummy variable indicating improvement in availability of medical sta¤. c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. d. Control variables include: Distance to nearest local council from the health facility, distance to other government health facilities in the area and electricity at the health facility. As of 2001, public health services are free of charge. However, the survey evidence indicates that patients still encounter varying costs, although a large majority of pa- tients do not pay (informal) user fees. In the pre-treatment data, 7 percent of the households surveyed reported having to pay user charges for outpatient services; ap- proximately 15 percent had to pay for injections (when needed); and 67 percent paid for delivery.51 In table A.9, we report the program impact on these informal charges. The in- tervention had no signi...cant e¤ect on the share of households that needed to pay for drugs (regression 1) or delivery (regression 4). However, it had an impact on general outpatient services (regression 2) as well as on injections (regression 3). felt he/she was free to express herself when being examined. 51 Average payment (for those that had to pay) was UGX 1,435 (USD 0.80) for out-patient service, UGX 370 (USD 0.21) for injections, and UGX 4,955 (USD 2.75) for delivery. 39 Table A.9. Di¤erence-in-di¤erence estimates of the program impact on user charges at the health facility. Dependent variable Drugs General treatment Injections Delivery Speci...cation (1) (2) (3) (4) Program impact -0.01 -0.06* -0.14** -0.07 (Treatment*2005) (0.01) (0.029) (0.07) (0.11) 2005 0.002 -0.018** 0.11** -0.13* (0.005) (0.007) (0.04) (0.07) Mean control group 2005 0.01 0.02 0.37 0.50 Facility ...xed e¤ects Yes Yes Yes Yes Observations 5660 5734 2511 507 R2 0.003 0.18 0.27 0.42 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Speci...cation: (1)-(4) Dummy variables indicating whether the health facility charged for service provided during the last visit. c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. A.3.3 Utilization The di¤erence-in-di¤erences estimates on the number of outpatients and deliveries are reported in table A.10. For the number of outpatients, we present the results from estimations of di¤erence-in-di¤erences speci...cations in both levels and logarithms. The treatment e¤ect is positive and signi...cantly di¤erent from zero for both out- patients served and the number of deliveries. The point estimates in the out-patient speci...cations suggest a substantial treatment e¤ect. Table A.10. Di¤erence-in-di¤erences estimates of the program impact on health fa- cility utilization. Dependent variable Out-Patient Log of Out-Patient Delivery Speci...cation (1) (2) (3) Program impact 215.5** 0.28** 3.48* (Treatment*2005) (93.4) (0.11) (1.98) 2005 -247.3 -0.25*** 1.73 (70.1) (0.07) (0.89) Mean control group in 2005 661 9.2 Facility ...xed e¤ects Yes Yes Yes Observations 100 100 100 R2 0.77 0.82 0.90 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Robust standard errors in parenthesis. 40 Table 1. Average health facility and citizen characteristics, pre-treatment. Treatment Control Di¤erence group group Utilization: Out-patient care 587 908 -51 (141) Delivery 10.32 7.48 2.84 (2.61) Utilization pattern: Project facility 0.31 0.34 -0.03 (0.03) NGO health facility 0.02 0.02 -0.002 (0.003) Private-for-Pro...t health facility 0.24 0.26 -0.02 (0.01) Traditional healer 0.034 0.03 0.004 (0.007) Self treatment (drug shop) 0.36 0.32 0.04 (0.03) Other government health facility 0.18 0.17 0.01 (0.05) Other provider 0.014 0.007 0.007 (0.005) Quality measures: Waiting time 148 144 4.3 (4.2) Equipment usage 0.47 0.48 -0.01 (0.02) Funding at the facility: 1000 shillings 4766 3429 1337 (905) The results are catchment area (health facility) averages. Robust standard errors in parentheses. Signi...cantly di¤erent from zero at 99 (***), 95 (**), and 90 (*) percent con...dence. Description of variables: Utilization variables are the average number of patients visiting the health facility per month; Utilization pattern is the citizens'use of di¤erent service providers in case of illness (reported in percentages); Waiting time is calculated as the di¤erence between the time the citizen left the facility and the time the citizen arrived at the facility minus the examination time; Equipment usage is a dummy variable indicating whether the sta¤ used any equipment during examination; Funding at the health facility is the average funds received at the health facility per month from the district and the Health Sub-district (measured in 1000 shillings). 41 Table 1 continued. Average health facility and citizen characteristics, pre-treatment. Treatment Control Di¤erence group group Catchment area statistics: Number of villages per health facility 23.2 24.6 -1.3 (3.14) Number of villages per health facility 2.6 1.8 0.80* in strata 1 (0.45) Number of villages per health facility 8.9 9.5 -0.64 in strata 3 (1.7) Number of villages per health facility 11.7 13.2 -1.5 in strata 5 (1.69) Number of households per health facility 2140 2224 -84 (275) Number of households per village 93.9 95.4 -1.42 (8.2) Health facility characteristics: Piped water 0.04 0.04 0 (0.00) Rain tank/Open well 0.52 0.36 0.16 (0.14) Borehole 0.44 0.60 -0.16 (0.14) Drinking water 1.76 1.48 0.28 (0.20) Separate maternity unit 0.16 0.16 0 (0.00) Distance to nearest Local Council I 0.72 0.85 -0.13 (0.26) Distance to nearest public health provider 8.68 7.76 0.92 (1.90) Number of days without electricity 18.3 20.4 -2.12 in the last month (4.14) The results are catchment area (health facility) averages. Robust standard errors in parentheses. Signi...cantly di¤erent from zero at 99 (***), 95 (**), and 90 (*) percent con...dence. Description of variables: Catchment area statistics are determined from UBOS maps and census data; Piped water, Rain tank and Borehole are dummy variables indicating the health facility's watersource; Drinking water is a dummy variable indicating whether the health facility has drinking water available; Separate maternity unit is a dummy variable indicating whether the health facility has a separate maternity unit; Distance to nearest Local Council I and distance to nearest public health provider is measured in kilometers; Number of days without electricity in the last month is measured out of 31 days. 42 Table 1 continued. Average health facility and citizen characteristics, pre-treatment. Treatment Control Di¤erence group group Citizen perceptions: Polite behavior 3.06 3.02 0.04 (0.04) Attention 3.17 3.16 0.01 (0.03) Free to express 3.8 3.77 0.03 (0.02) Citizens'informations about 0.14 0.16 -0.02 drug deliveries (0.05) Supply of drug deliveries at the health facility: Erythromycin 420 346 74 (131) Chloroquine 3410 2915 495 (567) Septrine 2690 2430 260 (623) Quinine 573 335 238* (130) Mebendazole 1597 1500 97 (230) User charges: Drugs 0.024 0.011 0.013 (0.012) General treatment 0.10 0.03 0.07* (0.04) Delivery 0.50 0.58 0.08 (0.10) Injection 0.24 0.20 0.04 (0.06) The results are catchment area (health facility) averages. Robust standard errors in parentheses. Signi...cantly di¤erent from zero at 99 (***), 95 (**), and 90 (*) percent con...dence. Description of variables: Citizen's perceptions describes his/her experience during the last visit at the health facility and are measured on a scale from 1 to 4 where a higher value represents higher satisfaction; Citizen's information about drug deliveries is a dummy variable indicating if the citizen knows when the health facility receives drugs from the district/Health Sub-district; Supply of drug deliveries per month is measured as the average number of tablets received at the health facility per month from the district/Health Sub-district; User charges are a dummy variable indicating if the household had to pay for the service provided at the health facility. 43 y if has (4) iable y s; ni arv it liticaf ice y tingaci y the liti acilf 2) serv ind ss fac 6)( 33 se h ealthh meetings 13*** (0.0 0. Y 1193 0.11 Dumm th 0. health ealth the Discu if al LC (1) riableav the he :s fi free y lo ator gn out tingaci to er ab ind Dumm one * um dicati leb 6)( 6. es 70 (5) Y en in least oring ming 0.16 50 0. ,) t A itno 0.56* 0.11)( or ariav the iable inf y ssione m yb arv vicesr sk y ts tersop regr Dumm e-seerf ni a gh on ri tions hecc mmu (5) D al ash on 10) es s;n areas ts' (4) .120 .120 Y 50 2) y tn oster obliga (0. 0.26 isuv ien s;n it P . atp acilf gatio sretsop and datio th obli s,d hmec atc .ytilicaf through al car he and st gn meetings vicesr ) ctede commener the cil within aiti htlaeh formingni se ) 9** 12 se (3 0. Y 05 if ighr w 0.47 coll coun ts' deret 0.1 (0.09 a and terso free cal atd of tsn edreb clus Lo e eht P el. tienap no indicating mun at ar lev plai on ta sd t d moc yt iable sloot msret eredb car * base arv xes,ob facili es 30 (2) ercenp orf osterp y a ce on th ting 0.20* 0.10)( 0.04 Y 50 0. are xob sti al gnirotino Num ai (10) 5)( mmu has he w y 5][ )- D geg (1 the 1 on 3)(; liticaf isturban (su D of h g .si m sti no xob 5 the uggestions ts ols" 10) (1) 0 es a to 38*** Y 50 at cations onin esht 0. (0. 0.3 ealth patien gn ti tcap Sugge ci...e ash ancec y the ori paren sp st liti its if uncf e ni in mi fac for itno margor act groupl ec¤e signi... h m" th rs ote iables rdsac tingaci es ss erro the tn ion mpi tro den arv ealth gn ind fo discu dard P el atc dex... am onc aiti .2 ndee ict ationsvr t eht leb (*) en if w one ab eci... an trs nde de ariav ants ehold el Dep ariv 2 **][ y Sp Progr Me Di Obse R astel ust ba *** Dep tingaci erb a hous Rob T a. b. ind umn Dumm has the c. 44 Table 3. Di¤erence-in-di¤erence estimates of the program impact on treatment prac- tices at the health facility. Dependent variable Equipment usage Waiting time Speci...cation (1) (2) Program impact 0.08** -14.0* (Treatment*2005) (0.03) (7.7) 2005 -0.07*** -9.6* (0.02) (5.3) Mean control group 2005 0.41 133 Facility ...xed e¤ects Yes Yes Observations 5280 5148 R2 0.15 0.12 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Speci...cation: (1) Dummy variable indicated whether the sta¤ used any equipment during exam- ination when the citizen visited the health facility; (2) Waiting time is calculated as the di¤erence between the time the citizen left the facility and the time the citizen arrived at the facility minus the examination time. c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. 45 Table 4. Program impact on management Dependent variable Absence rate Sta¤ present Condition of clinic Speci...cation (1) (2) (3) Program impact -0.10* 0.78* 1.13*** (0.058) (0.46) (0.31) Mean control group 0.53 2.3 -0.52 District ...xed e¤ects Yes Yes Yes Observations 50 50 50 R2 0.35 0.30 0.47 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Speci...cation: (1) Absence rate is the ratio of workers not physically present at the time of the post- intervention survey to the number of workers employed; (2) Sta¤ present is the number of workers (veri...ed by the enumerators) to be present at the time of the (surprise) post-intervention survey; (3) Condition of clinic is the ...rst component from a principal components analysis of the variables "condition of the oors of the health center", "condition of the walls", "condition of furniture", and "smell of the facility". Each condition is ranked from 1 (dirty) to 3 (clean) by the enumerators. c. Robust standard errors in parenthesis. 46 s.. 4 )01( 400.0- )610.0( th se se Y Y 365 20.0 mon 3 )9( ***330.0 49-60 )110.0( se se Y Y 9221 10.0 een w etb selsae si 2 )8( M 400.0 )020.0( 4 se se Y Y 5301 10.0 ths; mon 1 )7( *040.0 )320.0( se se Y Y 2201 30.0 37-48 n ee w 0 )6( 730.0- )40.0( reas.a se se Y Y 855 30.0 t etb en is hm 3 4 )5( .nerdlihc 600.0 )910.0( ***49.0 )10.0( catc o o N N 365 100.0 s;ht mon within 25-36 3 fo )4( **230.0 )410.0( ***19.0 )110.0( o o N N 9221 400.0 deret een clus e noitazinu el. wteb ar selsae si 2 )3( 800.0 M )320.0( ***88.0 )20.0( lev o o N N 5301 200.0 t msret 2; ths ercenp m mon mi selsae 1 )2( *650.0 )920.0( ***97.0 )20.0( cenabr o o N N 2201 500.0 (10) 24- 5][ istu .8. 13 D A 1 .si ene m the w able T no thes in tcap 0 )1( 720.0- )50.0( ***53.0 )40.0( o o N N 855 100.0 at etb is 1 ancec paren (d) ni mi ths; margor tcap stce¤e s teon e signi... rorre mon se ote 12 w s:el P .a5 el ba tnednepe mi dex... den dardn ab elob D elbairav eg A noitac...icepS margor tnatsno slortno tcirtsi (*) ationsvr tas ariv is ol **][ 0 ust tr 2 P C C D Obse R *** Rob Age: Con T a. b. c. d. 47 s.. 4 )01( 710.0 )920.0( th se se Y Y 075 20.0 mon 3 )9( **730.0 )910.0( 49-60 se se Y Y 3121 20.0 een w etb oilo 2 )8( P *030.0 )710.0( si se se Y Y 1201 4 40.0 ths; mon 1 )7( *740.0 )620.0( se se Y Y 0101 30.0 37-48 n ee 0 )6( **590.0 )930.0( reas.a w se se Y Y 065 70.0 t etb en is hm 3 4 )5( 420.0 .nerdlihc )820.0( ***98.0 )020.0( catc s;ht o o N N 075 200.0 mon within 25-36 3 )4( fo *240.0 )320.0( ***78.0 )810.0( o o N N 3121 400.0 deret een clus e el. wteb ar si noitazinu oilo 2 )3( P 430.0 )620.0( ***48.0 lev )20.0( o o N N 1201 200.0 t msret 2; ths ercenp m cenabr mon mi 1 oilop )2( *350.0 )130.0( ***97.0 )20.0( o o N N 0101 500.0 (10) 24- 5][ istu .8. 13 D A 1 .si ene the w able T no tcap 0 )1( 280.0 )150.0( ***95.0 )40.0( o o N N 065 700.0 at thes etb in is 1 ancec paren (d) ni mi s ths; teon margor tcap stce¤e e signi... rorre mon se ote 12 w s:el den ab P . b5 el ba tnednepe mi dex... dardn (*) tas elob ariv D elbairav eg A noitac...icepS margor tnatsno slortno tcirtsi is ationsvr ol **][ 0 ust tr 2 *** Rob Age: Con P C C D Obse R T a. b. c. d. 48 4 )01( 100.0- )130.0( s.. se se Y Y 665 30.0 th mon 3 )9( ***070.0 )320.0( se se Y Y 0321 49-60 80.0 een w etb T P 2 D )8( 720.0 )220.0( si se se Y Y 7301 80.0 4 ths; 1 )7( **150.0 )520.0( mon se se Y Y 6101 01.0 37-48 n ee w 0 )6( 330.0 )240.0( reas.a se se Y Y 165 50.0 t etb en is hm 3 4 )5( 110.0 )730.0( ***78.0 )30.0( catc o o N N 665 100.0 s;ht mon within .nerdlihc 25-36 3 )4( **180.0 )930.0( ***97.0 )30.0( o o N N 0321 110.0 deret een clus e fo el. wteb ar si noitazinu T t 2; P 2 )3( D 320.0 )140.0( ***87.0 )30.0( lev o o N N 1037 100.0 msret ths ercenp mon m mi 1 )2( T 650.0 )440.0( ***37.0 )40.0( cenabr o o N N 6101 400.0 (10) 24- 5][ istu .8. 13 D A 1 P .si ene the w able D T no thes in tcap 0 )1( 710.0 )050.0( ***65.0 )40.0( o o N N 165 100.0 at etb is 1 ancec paren (d) ni mi ths; margor tcap stce¤e s teon e signi... rorre mon se ote 12 w s:el P .c5 el ba tnednepe mi dex... den dardn ab elob D elbairav eg A noitac...icepS margor tnatsno slortno tcirtsi (*) ationsvr tas ariv is ol **][ 0 ust tr 2 P C C D Obse R *** Rob Age: Con T a. b. c. d. 49 s.. 4 )01( 600.0 )610.0( th se se Y Y 655 20.0 mon 3 )9( 510.0 )010.0( 49-60 se se Y Y 2121 20.0 een w etb G C 2 )8( B 800.0- )110.0( si se se Y Y 9201 10.0 4 ths; 1 )7( 700.0 )510.0( mon se se Y Y 1101 10.0 37-48 n ee 0 )6( **660.0 )130.0( reas.a w se se Y Y 079 40.0 t etb en is hm 3 4 )5( 310.0 )910.0( ***59.0 )20.0( catc o o N N 655 100.0 s;ht mon within .nerdlihc 25-36 3 )4( 120.0 )310.0( fo ***39.0 )10.0( o o N N 2121 200.0 deret een clus e el. wteb ar si noitazinu G t 2; C 2 )3( B 700.0- )410.0( ***49.0 )10.0( lev o o N N 9201 000.0 msret ths ercenp mon m mi 1 )2( G 600.0 )810.0( C ***29.0 )10.0( cenabr o o N N 1101 000.0 (10) 24- 5][ istu .8. 13 D A 1 .si ene B the w able T no thes in tcap 0 )1( **870.0 )530.0( ***67.0 )10.0( o o N N 079 10.0 at etb is 1 ancec paren (d) ni mi ths; margor tcap stce¤e s teon e signi... rorre mon se ote 12 w s:el P . d5 el ba tnednepe mi dex... den dardn ab elob D elbairav eg A noitac...icepS margor tnatsno slortno tcirtsi (*) ationsvr tas ariv is ol **][ 0 ust tr 2 P C C D Obse R *** Rob Age: Con T a. b. c. d. 50 Table 6. Program impact on citizens'information. Dependent Health Importance of variable information family planning Speci...cation (1) (2) Program impact 0.09*** 0.07*** (0.02) (0.02) District ...xed e¤ects Yes Yes Observations 4996 4996 R2 0.16 0.10 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable in speci...cations: (1) Dummy variable indicating if the household receives information about the importance of visiting the health facility and the danger of self-treatment, (2) Dummy variable indicating if the household receives information about family planning. c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. 51 in lages vil of gni erb * .1) es es 44 (10) 9.1* (4 Y Y 50 0. umn ,yt Plann 5 *** facili (9) 5. amilyF 4.9)( 3.5)( No No 50 .030 h 15.2 ealth 4* se se the (8) 17. 9.6)( Y Y 50 0.65 at natal te ) 1 ater *** w An (7 16. 16.1)( 78.9 11.9)( No No 50 0.02 * ** inkingrd 09) (6) of 3.45 2.03)( No No 50 1.01* (0. .660 y yre liti ab . ** it es es (5) 7.3 2.5)( Y Y 50 .610 ailva un Deliv ,yt ytinr 3* *** (4) noitazilitu 6. 3.3)( 9.2 1.6)( No No 50 .070 facili tea el. m th lev ate *** ** t ealh ar 05) (3) 47.8)( No No 50 49 (0. 0. sep ytilicaf 123.3 0.31* ercenp the a at tne has * (10) y ati .4) es es 40 5][ it htlaeh (2) Y Y 50 137.0* (54 0. 1 .si acilf atersource Out-P the w at thes of no 0) 0) 5 health tcap (1) No No 50 107.4* (65. 60.8***6 (35. 0.0 ancec paren epyt the ni s mi st de:ul erht signi... margor act ec¤e rorre inc whe ote and den P tn ion mpi dardn ae .7 el atc (*) riablesav lization tn dex... tas ar t el ndee am ta ict ol ab uti rolst ationsvr **][ ust eci... trs tr ba Dep ariv 2 hmen Sp Progr Pre- Cons Con Di Obse R *** Rob Con T a. b. c. catc 52 r ) the 02) (7 5*** 06 se 00 08 O -0.01 (0. 0.0 0.01)( 0. Y 92 0. tn y .nrettap ernmev liticaf 05 3** 2) es go h (6) -0.0 0.05)( (0.0 0.12 Y 9200 0.02 -0.04 gnikees erh ealth .) Ot t tages htlaeh'snezitic elf-S 5)( 02) 37 se 00 03** encrep -0.03* (0. 0. 0.01)( 0. Y 92 .090 reas.a in treatmen t en ted or hm ep tionali ) 008) 028 se 0 03 (r no (4 .005) Y catc 0. tcap adr ealerh .013*0- (0. .0020- (0 0. 920 ess T lnil within fo mi r-o margorp -fe t 4) deret asec es at (3) pro... 0.03 0.02)( in -0.02 0.25 Y 9200 0.04 (0.01 clus e Priv el. ar ersd lev vi eht O 6) 8 t 3 pro 004 fo (2) 005) es 09 msret NG 0.- 0.0 Y 9200 0. ice (0.00 0.00 seta (0. ercenp y cenabr ervs (10) mitse liti tne ) 5][ istu fac D 1 secnere¤id-ni-ecnere¤i t 8*** (1) 013) es 0.26 Y 0.08 .si er¤id jecor 0.034* 9200 (0.019 -0.0 (0. the of at thes P seu' ancec paren ns ni ize 2005 s cit signi... is ) iable ctse rorre arv act groupl ote e¤ iable den D tn arv ion mpi t*2005n tro dardn t .8 atc am onc xed... (*) tas en y el ndee eci... eatmer an liti ationsvr **][ ust end ba Dep Sp Progr (T 2005 Me acF 2 Obse R *** Rob Dep T a. b. c. 53 Table 9. Program impact on health outcomes: Under-...ve child deaths. Dependent variable Child death (children < 5 year) Under-5 mortality rate Speci...cation (1) (2) (3) (4) Program impact -0.016* -0.017** -48.0** -46.0* (0.01) (0.009) (24.2) (25.8) Constant 0.049*** 144.9*** (0.006) (16.9) Controls No Yes No Yes District ...xed e¤ects No Yes No Yes Observations 2922 2922 50 50 R2 0.002 0.01 0.08 0.18 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable (columns 1-2) is a dummy variable indicating whether any children under ...ve in the household have died during the last year and estimated under-5 mortality rate in the community expressed per 1,000 live births (columns 3-4). c. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. d. Control variables: see note (d) in Table A.8. 54 Table 10. Program impact on health outcomes: Child weight of infants (weight-for- age z-scores). Dependent variable weight-for-age z-scores 1-18 months 19-36 months Speci...cation (1) (2) (3) (4) (5) Program impact 0.16* 0.17** 0.15** 0.012 0.04 (0.09) (0.08) (0.07) (0.09) (0.06) Child age (log) -1.28*** 0.07 (0.07) (0.17) Female 0.26*** 0.08 (0.09) (0.06) Constant -0.64*** -0.71*** -0.95*** (0.07) (0.06) (0.08) Controls No No Yes No Yes District ...xed e¤ects No No Yes No Yes Observations 1167 1135 1135 1300 1300 R2 0.002 0.003 0.22 0.00 0.04 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable is weight-for-age z-scores. c. Speci...cation: (1) Includes all children under 18 months, (2) Includes all children under 18 months with observations with recorded weight above the 90th percentile in the growth chart reported in Cortinovis et al (1997) dropped. (3) Includes all children under 18 months plus controls. (4) Includes all children between 18 and 36 months plus controls. (5) Includes all children between 18 and 36 months plus controls. d. Robust standard errors in parenthesis. Disturbance terms are clustered within catchment areas. e. Control variables: see note (d) in Table A.8. 55 Table 11. Robustness test: The e¤ect on utilization at the control facilities when controlling for proximity to project facility. Dependent variable Out-Patient Delivery Family Antenatal Services planning care Speci...cation (1) (2) (3) (4) Distance to nearest project facility -1.13 -0.10 0.07 -0.56 (2.11) (0.07) (0.22) (0.52) Constant 696*** 12.4*** 13* 96*** (66) (2.75) (7) (18.1) District ...xed e¤ects No No No No Observations 25 25 25 25 R2 0.02 0.06 0.01 0.03 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Robust standard errors in parenthesis. 56 Table 12. Robustness test: Di¤erence-in-di¤erence estimates on the e¤ect on utiliza- tion at the control facilities when controlling for proximity to project facility. Dependent variable Out-Patient Delivery Services Speci...cation (1) (2) Distance to closest project -3.41 0.04 facility in 2005 (4.72) (0.04) 2005 -142.1 0.54 (154.0) (1.3) Facility ...xed e¤ects Yes Yes Observations 50 50 R2 0.75 0.91 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Robust standard errors in parenthesis. 57 g in th urd ale H ricttsi and tc b-d Su distri th gni ale eht 8 H es 72 undF (6) 24- 65) 94 from (1224) 1261 (9 4471 Y 0. dna th ict e mon distr rep azol 6 nde es the y (5) 114 533)( 984** (412) 2483 Y 100 0.6 omrf liticaf deviecer Meb th ltha eni mon he ) gnidnuf es erp eht (4) -237 -30 99 Quin (154 101)( 305 Y .570 y it at ed acilf h eiv dna ecr ine tr ylppus 3)( 82) 73 se ealth -2.9 (6 -457 (415) 19 Y 001 0.60 Sep eht nds fu ta care sgurd eniu el. lings).il ed sh lev eiv ) 8) 4 se 0 t da rec health fo (2 -176 (68 -531 (415) 238 Y 10 0.55 c gan seta Chlorq ercenp bli U tablets pu (10) mitse of of 1000 ycin 5][ 1 erb tn ni ecnere¤id-ni-ecnere¤i es (1) 51.81 127)( (65) 257 Y 96 0.73 -145** the mun amou uredsa .si ythromr at esht E ancec agereva age (me er Av e arey paren ni 2005 th rs signi... is 1-5); astl ) ctse el s the erro riableav act groupl ote e¤ ab gn den D .31 tn ion mpi t*2005n ariv dard tro t (column urid atc am onc xed... (*) ants y den ict el ndee eary eci... eatmer an liti ationsvr **][ en istr ust ba Dep Sp Progr (T 2005 Me acF 2 last Obse R *** Dep b-d Rob T a. b. the Su c. 58 Table 14. Program impact on infrastructure at the health facility. Dependent variable New units Toilets Water Electricity source Speci...cation (1) (2) (3) (4) Program impact -0.09 0.09 0.05 0.05 (0.12) (0.11) (0.10) (0.10) Controls Yes Yes Yes Yes District ...xed e¤ects Yes Yes Yes Yes Observations 50 50 50 50 R2 0.50 0.34 0.29 0.42 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable is a dummy variable indicating whether any constructions or renovations of infrastructure have been done at the health facility during the last year. c. Robust standard errors in parenthesis. d. Control variables: see note (c) in Table 6. 59 Table 15. Di¤erence-in-di¤erence estimates on equipment at the health facility. Dependent variable Bicycles Examination Blood pressure Weighing Microscope beds equipment scale Speci...cation (1) (2) (3) (4) (5) Program impact 0.04 0.20 -0.08 0.08 0.22 (Treatment*2005) (0.19) (0.25) (0.19) (0.11) (0.15) 2005 0.40*** 0.20 0.36** 0.12* -0.001 (0.14) (0.11) (0.14) (0.06) (0.001) Mean control group 2005 2.92 2 2.04 2.6 0.44 Facility ...xed e¤ects Yes Yes Yes Yes Yes Observations 100 100 100 100 100 R2 0.97 0.84 0.92 0.99 0.99 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable is the number of each equipment available at the health facility. c. Robust standard errors in parenthesis. . 60 Table 16. Program impact on monitoring of upper-level authorities and dismissals and transfers of sta¤ at the health facility Dependent variable Sub-county Parish Dismissals Transferred Left o¢ cial o¢ cials Speci...cation (1) (2) (3) (4) (5) Program impact 0.12 0.06 -0.054 -0.083 0.079 (0.12) (0.11) (0.07) (0.23) (0.15) District ...xed e¤ects Yes Yes Yes Yes Yes Observations 50 50 50 50 50 R2 0.43 0.45 0.22 0.16 0.17 a. *** [**] (*) denote signi...cance at the 1 [5] (10) percent level. b. Dependent variable in speci...cation (1) dummy indicating if the facility has received a moni- toring/support visit from any Sub-county o¢ cials in 2005; (2) dummy indicating if the facility has received a monitoring/support visit from any Parish o¢ cials in 2005; (3) number of sta¤ that has been dismissed in 2005; (4) number of sta¤ that has been transferred from the facility in 2005; (5) number of sta¤ that voluntarily left the facility in 2005. c. Robust standard errors in parenthesis. 61 Figure 1: Timing of project End of 2004 Beginning of 2005 Beginning of 2006 Time line Collection of Report card intervention Community monitoring Collection of household (5 days) based on community household and facility · benchmark information on contract and benchmark and facility level data current status of service information. level data delivery relative other providers and the government standard; and · community contract Figure 2: Schematic view of intervention and expected outcome Intervention: Ability to exercise Service provider Quality and Improved - report card dissemination accountability exerts higher quantity of health - facilitate the agreement of (monitor provider) effort to serve health care outcomes a "community contract" increases the community provision increase Figure 3: Alternative hypotheses Increased Intervention: government Service provider Quality and Improved - report card dissemination funding exerts higher quantity of health - facilitate the agreement of effort to serve health care outcomes a "community contract" the community provision Increased increase supervision by upper level authorities Spillovers Staff responds directly to intervention added oup 5 4 re gr - orf- scores,z2- scoz control scores.z1- ght months) 2 wei 18- and3- 0.17 to 2,- 3,- of (1 re es effect es 0 ution infants orcsz denote sco-z denote 0 ent orcsz lines for mean lines Distrib solid Treatm solid 5: ores -2 re scz denotes 7: Figu age Vertical: line re oltr Vertical: -4 Figu onC oup gr Notes dashed -5 Notes 52. 2. 51. 1. 50. 0 52. 2. 51. 1. 50. 0 ytisned ytisned 15 4 group nt group. ysoB 10 thno eatmenttr ores M in ts 5 scz Treatme 2 char groups control of mean in 0 owth 01 9 8 7 6 5 4 ons elit necr ep ht 05 htg ieW ontrolc es mean 0 Gr orcsz denotes 4: and line re 15 Distributi denotes 6: ment solid -2 line Figu lsri 10 G thno re pu treat M rogl 5 Figu for ront Vertical: dashed -4 0 Co Notes group, 01 9 8 7 6 5 4 e litn ecre p ht0 5 ht gi eW 3. 2. 1. 0 ytisned 64