102967 world development report BACKGROUND PAPER Digital Dividends When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness? Tiago Peixoto World Bank Group Jonathan Fox American University 2016 World Development Report Digital Dividends www.worldbank.org/wdr2016 BACKGROUND PAPER When Does ICT-Enabled Citizen Voice Lead to Government Responsiveness?* Tiago Peixoto, World Bank Jonathan Fox, American University JANUARY 2016 IN COLLABORATION WITH Evaluation Team | DEET We are thankful to Sanna Ojanpera who provided research assistance on case description. Thanks also to Brandon Brockmyer, Susan Crawford, Duncan Edwards, Archon Fung, Brendan Halloran, Zahid Hasnain, Renee Ho, Anna Levy, Rosemary McGee, and Dennis Whittle, for valuable comments and insights, and to Claudio Mendonca for support with graphic design. An abridged version of this paper is also published in the January 2016 issue (Vol. 47, number 1) of the IDS Bulletin. 2016 World Development Report Background Paper Abstract This paper reviews evidence on the use of 23 information and communication technology (ICT) platforms to project citizen voice to improve public service delivery. This meta-analysis focuses on empirical studies of initiatives in the global South, highlighting both citizen uptake (‘yelp’) and the degree to which public service providers respond to expressions of citizen voice (‘teeth’). The conceptual framework further distin- guishes between two trajectories for ICT-enabled citizen voice: Upwards accountability occurs when users provide feedback directly to decision-makers in real time, allowing policy-makers and program managers to identify and address service delivery problems – but at their discretion. Downwards accountability, in contrast, occurs either through real time user feedback or less immediate forms of collective civic action that publicly call on service providers to become more accountable and depends less exclusively on deci- sion-makers’ discretion about whether or not to act on the information provided. This distinction between the ways in which ICT platforms mediate the relationship between citizens and service providers allows for a precise analytical focus on how different dimensions of such platforms contribute to public sector respon- siveness. These cases suggest that while ICT platforms have been relevant in increasing policymakers’ and senior managers’ capacity to respond, most of them have yet to influence their willingness to do so. Introduction Around the world, civil society organisations (CSOs) and governments are experimenting with information communication technology (ICT) platforms that try to encourage and project citizen voice, with the goal of improving public service delivery. This meta-analysis focuses on empirical studies of initiatives in the global South, highlighting both citizen uptake (‘yelp’) and the degree to which public service providers respond to expressions of citizen voice (‘teeth’). The conceptual framework is informed by the key distinction between two distinct genres of ICT-enabled citizen voice – aggregated individual assessments of service provision and collective civic action. The first approach constitutes user feedback, providing precise information in real time to decisionmakers. This allows policymakers and programme managers to identify and ad- dress service delivery problems – but at their discretion. Collective civic action, in contrast, can encourage service providers to become more publicly accountable – an approach that depends less exclusively on decisionmakers’ discretion about whether or not to act on the information embodied in feedback. This con- ceptual distinction between two different ways in which ICT platforms mediate the citizen–service provider relationship allows for a more precise analytical focus on how different dimensions of these ICT platforms contribute to public sector responsiveness. This study begins with a conceptual framework intended to clarify the different links in the causal chain in between ICT-enabled opportunities to express voice (platforms) and institutional responses. In other words, how and why are these platforms supposed to leverage responses from service providers? The answers turn out not to be so obvious. Our approach was informed by a close review of the available evidence, primarily quantitative, about experiences with 23 ICT platforms in 17 countries1. This focus on unpacking causal chains is informed by two factors. First, the broader literature on the drivers of accountability increasingly emphasis- 1 This also included an international platform, Change.Org. The data analysis in that case referred to a total of 132 countries (World Bank 2014b). EVALUATION TEAM | DEET 2 2016 World Development Report Background Paper es using causal chains to address the analytical puzzle of how to distinguish how and why citizen action may or may not lead to public sector response (Fox 2014; Grandvoinnet et al. 2015; Joshi 2014; Peixoto 2013). Second, analysis revealed that we do not see a generic type of platform leading to a generic type of response. Instead, we see key differences in the institutional (not technological) design of the interface that may be relevant for voice, citizen action and institutional response. The evidence so far indicates that most of the ICT platforms that manage to leverage responsiveness somehow directly involve government. While ICT-enabled voice platforms vary widely across many dimensions, this analysis emphasises several differences that are hypothesised to influence both citizen uptake and institutional response. These include the degree of public access to information about the expression of voice – does the public see what the pub- lic says? Does the ICT platform document and disclose how the public sector responds? They also include institutional mechanisms for public sector response – do the agencies or organisations take specific offline actions to prompt service providers’ response? As a first step towards homing in on these variables, this paper maps the 23 platforms studied in terms of various empirical indicators of these distinct dynamics. This exercise is followed by a discussion of propositions that may or may not link voice to institutional response. Note that this study does not focus on two ways in which service delivery agencies use ICT that are very relevant for understanding their full array of relationships with users. First, many public agencies are us- ing mobile phones and social media to disseminate information efficiently. However, if those interfaces are one-way (‘inside-out’, or ‘top-down’), then they do not ‘count’ as ICT-enabled citizen voice for the purposes of this study. Second, agencies can use ICT for internal administrative reforms that can bolster their capacity to respond to citizen concerns –by reducing the discretionary power of front-line providers through increasing the capacity of managers to monitor service provider performance, as well as by helping consistently track whether and how problems are being addressed. This study covers evidence of institutional response to ICT-enabled systems for users to exercise voice, rather than the broader set of cases of relevant e-government initiatives. Conceptual map: Unpacking digital engagement The broader analytical context for this paper involves three simultaneous trends in the literature on the role of information in leveraging public accountability. First, the number and diversity of practitioner-led digital engagement for service delivery initiatives continue to grow, involving both effervescent experimen- tation and efforts to scale up. Experimentation with social accountability tools has been growing within the portfolios of both large public and private aid donors for the past decade, and some involve ICT. For instance, many World Bank projects with ‘identifiable beneficiaries’ now include some kind of feedback mechanism, and citizen engagement has become a policy framework which includes the use of ICT (World Bank 2014a). Major private donors, such as the Omidyar Network and Google, are also making significant investments to encourage ‘civic technology’– in both the global North and South. New donor partnerships are also encouraging experimentation with civic technology in very low-income countries, led most notably by Making All Voices Count.2 2 Making All Voices Count is supported by DFID, USAID, Sida and Omidyar Network EVALUATION TEAM | DEET 3 2016 World Development Report Background Paper Second, while growing media coverage of ICT-enabled voice platforms is often enthusiastic, social science research on the dynamics and impacts of these initiatives lags far behind, and the limited existing evidence does not yet support unqualified optimism.3 This study is distinctive in that it draws on a recent round of unusually comprehensive empirical studies that involve both large-scale surveys and access to govern- ment agency data. This new research suggests that the key dynamics that drive both voice and institutional response may be different from some of the widely-held impressions projected by the media, donors and platform developers. Take for example the case of Kenyan urban water agency’s MajiVoice (see also Welle and Williams, this IDS Bulletin), a large-scale user feedback system widely presented as an ICT-enabled voice platform. Recent surveys find significant evidence of institutional response, grounded in an effective complaint tracking system – yet three-quarters of the complaints are filed in person, 21% by phone and less than 3% by SMS or online (Belcher and Lopes 2016). Third, the focus on the potential for citizen voice to improve public service delivery involves at least four dis- tinct yet overlapping arenas of practice – the open data movement, open government reforms, anti-corrup- tion efforts and social accountability initiatives. In spite of the apparent new policy consensus that all these good things go together, in practice, the limited synergy between these distinct approaches suggests that the whole is still not greater than the sum of the parts (Carothers and Brechenmacher 2014). Most of these governance reform approaches rely heavily on the potential power of information to stimulate voice, yet they assign information different roles. There are several conceptual challenges involved in specifying the causal mechanisms that may link voice and institutional response – aside from the empirical questions involved (documenting uptake is more straightforward than institutional response). The first analytical chal- lenge is to disentangle voice from responsiveness. Much of the first wave of research on ICT-enabled voice platforms focuses primarily on citizen uptake (e.g. Gigler and Bailur 2014), without clear evidence that the feedback loop actually closes. In practice, the concept of feedback loop often used to imply that uptake (e.g. citizen usage of crowd-sourced platforms to report feedback) necessarily leads to positive institutional responses. In other words, there is a high degree of optimism embedded in the way the concept tends to be used. In contrast, the framework proposed here avoids this assumption by treating the degree of insti- tutional response as an open question. The second conceptual challenge is to specify the relationship between the role of ICT-enabled voice plat- forms and the broader question of the relationship between transparency and accountability. In spite of the widely-held view that ‘sunshine is the best disinfectant’, the empirical literature on the relationship between transparency and accountability is far from clear (Fox 2007; Gaventa and McGee 2013; Peixoto 2013). The assumed causal mechanism is that transparency will inform and stimulate collective action, which in turn will provoke an appropriate institutional response (Brockmyer and Fox 2015, Fox 2014).4 In this model, both analysts and practitioners have only just begun to spell out the process behind that collective action (Fung, Graham and Weil 2007; Joshi 2014; Lieberman, Posner and Tsai 2014). In light of widely held unrealistic expectations about the ‘power of sunshine’, convincing propositions about causal mechanisms involved 3 The current enthusiasm – among development stakeholders and the media – over the potential of technology in citizen participation in the developing world is reminiscent of the wave of optimism surrounding such initiatives in Europe over the past decade, despite the significantly less favorable conditions of developing countries. Even in Europe, with generous funding and a more favorable institutional and technological context, most experiences present limited results at best (see, for instance, Prieto-Martin et al 2011; Susha and Gronlund 2014; Diecker and Galan 2014). 4 Note that this widely assumed causal mechanism does not distinguish explicitly between two different kinds of accountability – preventative (reforms that make future transgressions more transparent) and reactive (answerability and the possibility of sanctions). EVALUATION TEAM | DEET 4 2016 World Development Report Background Paper need to specify how and why the availability of an ICT platform (a) would motivate citizen action and (b) why the resulting user feedback would motivate improvements in service provision. After all, decisionmakers’ lack of information about problems is not the only cause of low-quality service provision. Third, the relationship between ICT-enabled voice platforms and the transparency/accountability question is complicated by the fact that, in practice, a significant subset of those platforms does not publicly dis- close the user feedback. Yet if citizen voice is not made visible to other citizens, where does its leverage come from? Such feedback systems aggregate data – by asking citizens to share their assessments of service provision – but if the resulting information is not made public, then it cannot inform citizen action. In these systems, if users’ input is going to influence service provision, voice must activate ‘teeth’ through a process other than public transparency – such as the use of data dashboards that inform senior managers’ discretionary application of administrative discipline. These conceptual propositions suggest that it is relevant to distinguish explicitly between two different accountability pathways that link voice and ‘teeth’ – shorthand for institutional willingness and capacity to respond (Fox 2014). In downwards accountability relationships, service providers are held accountable by citizen voice and action. The arrow of answerability points downwards, insofar as it is driven by the poten- tial political cost to policymakers of not responding to a publicly visible concern. In contrast, in upwards accountability relationships, frontline and middle level service providers are held accountable to senior policymakers and programme managers, who use the user information to take administrative action. The arrow of answerability points upwards. In this approach, the incentives for policymakers to act on user information are less clear. Clearly, both mechanisms can operate together, but they are empirically and analytically distinct (see Table 1). Table 1. How does voice trigger teeth? Upwards and downwards accountability PRIMARY CAUSAL MECHANISM Voice pathway Upwards accountability Downwards accountability Individual user feedback From frontline service providers to managers and policymakers by identifying problems and triggering administrative action Collective civic action From public sector to society, by bring- ing external pressure to bear and raising the political cost of non-responsiveness Based on these conceptual propositions, this review of 23 ICT-enabled voice platforms distinguishes be- tween two different types of citizen voice, ‘user feedback’ and ‘civic action’. While these two approaches can overlap in practice, they are analytically distinct. Their common denominator is the use of dedicated ICT platforms to solicit and collect feedback on public service delivery. The differences between them in- volve three dimensions: i) whether the feedback provided is disclosed; ii) through which pathway individual EVALUATION TEAM | DEET 5 2016 World Development Report Background Paper or collective citizens’ preferences and views are expressed; and iii) whether these mechanisms tend to promote downwards or upwards accountability. Note that this analytical approach differs from the World Bank’s current policy framework, which considers user feedback to be a variant of ‘citizen engagement’ (World Bank 2014a). The approach proposed here, in contrast, does not treat the adjectives ‘citizen’ and ‘civic’ as pure synonyms (though they overlap). We use citizen (as in ‘citizen voice’) to refer to individual, non-public actions, while civic refers to public, collective actions. The two approaches are potentially mu- tually reinforcing and in practice, some voice platforms combine them (see Figure 1, below). With regard to the first dimension, we will assess cases in terms of the extent to which the feedback pro- vided by individuals is publicly disclosed or not, thus enabling citizens to potentially act to hold govern- ments accountable. Citizens’ capacity to hold governments accountable depends, among other things, on the accessibility of publicly available relevant and actionable information (Fung, Graham and Weil 2007). In this respect, whether the feedback provided by citizens on service delivery is publicised or not is directly related to the extent to which citizens can hold governments accountable for their performance and actions. Thus, a first distinction between user feedback and civic engagement is that, while a grow- ing number of ICT platforms collect input from individuals, only user feedback that is made public counts here as civic engagement (in Figure 1, this is the area of overlap between the two circles, involving both individual feedback and public disclosure). For instance, in the case of Punjab Proactive Governance model, the government solicits feedback via mo- bile phones on the quality of services provided on a large scale, on an ongoing basis (Bhatti, Zall Kusek and Verheijen 2014). However, the feedback provided is not disclosed to the public, only to senior policymakers, as it is intended to inform internal administrative monitoring processes. This process does not contribute to citizens’ ability to act based on the feedback. In contrast, Uruguay’s PorMiBarrio is a mobile and web-based platform that enables Montevideo’s citizens to report problems like vandalism and breakdowns of public in- frastructure. The problems reported, and the actions taken in response by government (e.g. repaired, or not), are displayed on a map on the public website. Not only is the government able to act on citizen reports, the publication of the feedback makes it possible for citizens to hold governments accountable. The second dimension that we use to categorise platforms assesses the mechanisms by which citizens’ views and preferences are expressed – either individually or collectively. Individualised mechanisms refer to those that do not involve collective action, yet the feedback provided by a single individual is expected to trigger a response, possibly through aggregation in order to identify problem areas in public service delivery. This is the case, for instance, of web-based citizen reporting initiatives such as PorMiBarrio, FixMyStreet in Georgia and I Paid a Bribe in India. In these cases, each individual report of very specific service issues needing attention is assumed to be enough to lead to a governmental response. In contrast, collective mechanisms refer to those in which it is the magnitude, nature and intensity of the aggregation of citizen concerns that is expected to trigger governmental action. Examples of platforms for collective voice include online petitions such as Change.org and mobile and web-voting in Brazil’s state-wide Rio Grande do Sul Participatory Budgeting (PB) process. In both initiatives, it is the collective mobilisation around a cause or preference that is intended to trigger government responsiveness. The core of the technological platforms that support these mechanisms lies in the reduction of transaction costs for collective action that can ad- dress policy agenda-setting, in contrast to reacting to policy outputs. This collective dimension, we argue, EVALUATION TEAM | DEET 6 2016 World Development Report Background Paper is what gives the character of ‘civic-ness’ to ICT-enabled voice platforms, insofar as they enable individuals to engage in collective action – or at least to address public concerns. In contrast to feedback systems that receive individual reactions to specific service delivery problems, ICT platforms that enable the public aggre- gation of citizens’ views have more potential to constitute input into the setting of broader policy priorities. This potential civic agenda-setting contribution goes beyond the conventional understanding of feedback, in which the agendas that citizens are supposed to respond to are set from above (See Box 1 below). Thus, our conceptual distinction can be summarised as follows: citizen feedback initiatives provide feed- back from individual clients of services. Where such feedback is not publicly disclosed, the causal pathway to governmental response is via upwards accountability, from frontline and mid-level public servants to se- nior managers and policymakers. Conversely, civic engagement refers to mechanisms where the feedback is publicly disclosed, which allows for collective action and downwards accountability to also take place. Figure 1 below illustrates our conceptual model. Figure 1. Unpacking user feedback and civic action: Difference and overlap On the left side of Figure 1 (blue) feedback is individual and undisclosed, which we can describe as a typical case of governmental user feedback platforms. On the right side (yellow), citizen voice is simul- taneously collective and disclosed, meeting the two criteria for our definition of civic engagement. At the intersection point, however, we find platforms that both collect individually specific feedback and make those inputs public (sometimes also reporting whether and how the government responds). This overlap involves the fact that, while individualised feedback mechanisms are not designed to spur online collective action within the platform itself, the fact that the feedback is publicised may inform and facilitate collective action – offline as well as online. This may be the case, for instance, when the sum of individual feedback EVALUATION TEAM | DEET 7 2016 World Development Report Background Paper in a certain platform, such as FixMyStreet, reveals to the public the patterns of failure in a certain service, or in certain locations. In this case, even though the platform is not specifically designed to support collective action, the disclosure of evidence of patterns of failure in a given service may support well-targeted collec- tive action to address service delivery problems. Figure 2 below presents the diagram populated with the cases we analyse in this study. The platforms that generated a high degree of tangible response from the service delivery agencies are indicated in green (7 of 23). High responsiveness to citizen voice is measured here as tangible service delivery agency action, registered in more than half of cases. In eight cases, user uptake was high – though only three of these were also among the eight cases of high responsiveness. Figure 2. Mapping citizen voice platforms and degrees of institutional responsiveness Maji Voice Digital State Participatory Budgeting Rio 1746 Pressure Pan Punjab Proactive Change.org My Voice U-Report Karnataka BVS Degree of Institutional Proactive Listening Electricity Check my School Responsiveness Lungisa LAPOR High I Change my City Huduma Medium Por Mi Barrio IMCO Low Maji Matone e-Chautari I Paid a Bribe Barrios Digital Chemi Kucha Sauti Za Wananchi As shown in Figure 2, approximately a quarter of the cases are found in the user feedback category, an- other quarter in the civic action category, and 14 of 23 at the intersection between those two, called citizen engagement here. The cases in the user feedback category are mostly web- and mobile-based systems for collecting citizen views on the provision of services in a specific sector, such as electricity, water and health. Here the service provider plays either a passive or an active role in the collection of feedback. In the first role, the citizen voluntarily initiates the contact to report an issue with public services via mobile- or web-based systems – sometimes in combination with offline, face-to-face citizen attention windows (as in the case of MajiVoice in Kenya). One large-scale example in this category is LAPOR, Indonesia’s complaint handling system, which allows citizens to submit their reports on issues ranging from teacher absenteeism to damaged roads through a number of channels which include SMS, mobile apps and social media. EVALUATION TEAM | DEET 8 2016 World Development Report Background Paper The user feedback category also includes a second mechanism by which data is collected, which we call ‘proactive listening’ – also called ‘proactive feedback’ by its practitioners (Bhatti, Zall Kusek and Verheijen 2015; Masud 2015). Here, government service providers proactively reach out to citizens in order to gather feedback from them on the quality of services received. This mechanism is best illustrated by Punjab’s Citi- zen Feedback Model, where a system generates SMS and calls to public service users in order to ask them about satisfaction with the services received and potential corruption incidents. The Punjab government has deployed this approach on an unprecedentedly massive scale, with more than 6 million outreach calls so far. Recent large-scale surveys of service users have found that these outreach efforts actually reached and received responses from 15% of citizens called (Bayern 2015; World Bank 2015). The citizen engagement platforms (those found at the intersection between user feedback and civic action) predominately utilize web and mobile-based mechanisms for reporting public service issues, similar to many of the user feedback platforms. However, what distinguishes these platforms is that the user feedback pro- vided to service providers is also disclosed publicly. For example, the Lungisa website allows Cape Town residents to report service delivery problems (e.g. sanitation, electricity) using an online form, which is then routed to the relevant government agency and further investigated by Lungisa staff. Unlike many user feed- back systems, however, Lungisa allows residents to view all other reports that have been submitted, as well as the status of each issue (i.e., in progress, closed). Indeed, if ICT platforms ultimately seek to facilitate dis- closure about whether and how governments respond to citizen voice, then the capacity to track both citizen feedback and government response are necessary, but not sufficient, design features. Citizen engagement platforms also seem to differ from user feedback platforms in terms of their ownership. While user feedback platforms tend to be built by service providers, citizen engagement platforms have been launched primarily by CSOs or donor organizations (see Table 3). Generally, platforms built by service pro- viders tend to generate far more user uptake than those launched by CSOs or donors, with a few exceptions. Finally, Figure 2 shows that several cases do not involve individualized user feedback and fall entirely with- in the civic action category. In these cases, the ICT platform’s primary goal is to support collective action through the aggregation of individual citizen inputs. In other words, the role of individual inputs is not sim- ply to identify specific service delivery problems, but to demonstrate the extent of citizen concern, through the process of aggregation. The civic action cases considered here are significantly less numerous and more heterogeneous than either the user feedback cases or the citizen engagement cases. They include projects as diverse as web-based participatory budgeting in Rio Grande do Sul and the international online petitioning platform Change.org. However, if the scope of this research was broadened to include e-par- ticipation, crowdsourced political deliberation, or the role of social media in enabling political protest, the number of relevant ICT platforms would increase. However, the focus here is on citizen voice platforms that specifically address public service provision. EVALUATION TEAM | DEET 9 2016 World Development Report Background Paper Digital engagement initiatives: Categorizing platforms in terms of variables of interest In this section, we categorize our 23 ICT platform cases by considering a number of factors (i.e., indepen- dent variables) that may contribute to our outcome of interest: institutional response. We define “institu- tional response” as a clearly identifiable action taken by government/service providers, following individual or collective input by citizens. For example, institutional response is evidence in the case of the Proactive Listening initiative of EDE Este, an electricity distribution company in the Dominican Republic.. The initiative combines a traditional complaint handling mechanism with proactive outreach to users. This online/mobile phone platform allows citizens to report problems with electricity services, ranging from malfunctioning connections to bribe requests by maintenance crews. Following the handling of the complaint (e.g. re-con- nection of electricity), the company proactively re-contacts a random sample of users to gather feedback on the quality of services provided. The feedback received is systematically used to inform sanctions (e.g. administrative procedures) and rewards (e.g. performance-related wage bonuses for company workers). Since its implementation in 2011, the initiative has recorded growing resolution rates of reported issues, with close to 100 percent of the feedback provided indicating good or excellent levels of satisfaction. The average of instances of disrespectful treatment of clients registered at the beginning of the project was drastically reduced, and reported cases of corruption fell by 70 percent. Turning next to our independent variables, we have identified eleven factors that may have a relation- ship with institutional responsiveness—Disclosure of feedback, Disclosure of response, Proactive listen- ing, Voicing modality, Accountability directionality, Uptake, Combined offline action, Driver, Partnerships between public service provider and civil society organization(s), Level of government, and Institutional responsiveness. Of these, uptake—the degree to which citizens actually use digital platforms—deserves particular attention here: Uptake is often used as a key outcome for evaluating ICT platforms. Yet, while uptake may be necessary, it is far from sufficient for triggering institutional response (as the data below show). As described above, our main outcome of interest here is governmental response. Rather than treating citizen voice as an end in and of itself, our analysis treats uptake as an intermediate output that is relevant to the extent that it in- forms governmental decisions about whether and how to respond (see Table 2). Making this distinction is not intended to diminish the intrinsic value of expressing citizen voice. To the contrary, citizen voice is a so- cially valuable practice with the clear potential to encourage learning. Nonetheless, differentiating between uptake as an output, and institutional response as an outcome, provides crucial conceptual clarity that allows us to disentangle a number of different hypotheses about how a number of factors might influence institutional responsiveness. Table 2 details this approach further, distinguishing between inputs, outputs, outcomes and impacts. EVALUATION TEAM | DEET 10 2016 World Development Report Background Paper Table 2: ICT-enabled voice platforms: inputs, outputs and impacts INPUT > OUTPUT 1 > OUTPUT 2> OUTCOME > IMPACT Platform: Expression of citizen Aggregation of Institutional response Tangible change Channel for voice voice (uptake) voices (e.g., breaking in service delivery bottlenecks, repairs, access resource allocation) Publicity: Disclosed or not? Disclosed or not? Disclosed or not? Disclosed or not? Considering uptake as an output helps us to better understand the role that it may play in generating the outcome of interest, institutional responsiveness. Hypothetically, it should be relatively straightforward to find evidence supporting a causal relationship between uptake and responsiveness. All other things being equal, governments are more likely to respond when more citizens are engaged. Indeed, the odds of suc- cessful collective action increase as the number of participants grow (Lohmann 2000). In a cross-national study by the World Bank (2015) of online petitioning, the higher the number of signatories of a petition, the more likely governments are to respond. In fact, a number of both traditional and digital citizen participation platforms are explicitly designed to trigger governmental response only when citizen participation reach- es a pre-set benchmark. This is the case with citizen initiatives, referenda, and the official e-petitioning systems in the United Kingdom and the United States. However, some development practitioners argue that sustained uptake itself can be used as a proxy for government responsiveness. Otherwise, the argu- ment goes, citizens would not “keep coming back.” While this assertion is partially supported by empirical evidence (e.g., Sjoberg, et al, 2015), there are a number of instances where one finds sustained uptake despite low levels of institutional responsiveness, perhaps best exemplified in Downs’ (1957) work on the “paradox of voting.” Thus, treating citizen uptake as an indicator of government responsiveness remains problematic (as we shall demonstrate later). Below, we provide a description of each variable of interest. While this list is by no means exhaustive, the selection of these variables is informed by the literature on digital engagement and institutional respon- siveness, and reflects the availability of data across all cases. Further analysis would be necessary to as- sess the relative weights of each variable. The main focus of the subsequent discussion will be on broad patterns that emerge across all 23 cases. For brevity’s sake, discussion of specific cases and the explicit rationale used to code them will be limited. Description of Variables Disclosure of feedback – Refers to the extent to which the feedback provided by the citizen is made public or not. Disclosure of response – Refers to whether the official response to citizen feedback (individual and collective) is publicly disclosed or not. This would reveal the extent to which citizen input has led to institutional responsiveness. Proactive listening – Indicates whether at some point the service provider proactively contacts the citizen in order to collect feedback on the quality of services provided. EVALUATION TEAM | DEET 11 2016 World Development Report Background Paper Voicing modality – Whether the feedback provided through the ICT platform is individual or collective. This indicates whether ICT-enabled collective action is involved in triggering a response. Accountability directionality – Determines if the causal pathway is more likely to promote accountability between service providers and higher authorities (upwards accountability) or between citizens and service providers (downwards accountability). Uptake – An essentially quantitative measure of the number of individuals who provide feedback or who join a collective action. Uptake was coded in absolute terms of input provided (e.g. number of votes, reports) in a discon- tinuous range of low (between 1 and 10,000), medium (between 10,001 and 100,000) and high (above 100,000). Combined offline action – Identifies whether additional actions are taken offline in order to encourage government responsiveness. This could refer to a structured process of citizen follow-up on participatory budgeting, or to dedicated DE platform staff that follow up with the relevant authorities (e.g. Lungisa). Driver – Identifies the main institution driving the initiative, such as government, donors and CSOs. Partnerships between public service provider and civil society organization(s) – This refers to the exis- tence of formal and/or informal relationships between government and civil society, where there is some degree of coordination towards a common outcome. For example, this is the case for the Por Mi Barrio project, a part- nership between the organization DATA and the municipal government of Montevideo. This relationship allows for direct communication between the digital platform (developed by DATA) and the governments’ existing com- plaint response mechanism. Another example is the formal partnership between the IPaidaBribe.com project and Indian governmental authorities, which facilitates communication and allows for coordinated follow-up of bribes reported to the government. Coding options include government-led, CSO-led and donor-led. Level of government: Describes the level at which services are provided and feedback is provided, sub-divided as national, sub-national and local. Institutional responsiveness: This indicator reflects the degree to which there is clearly documented ev- idence of government response to feedback provided through ICT platforms (including combined online/ offline action). Whenever possible, coding categories for institutional responsiveness reflect the share of citizens’ inputs addressed, ranging from low (less than 20%) to medium (between 20 and 50% of citizen issues addressed) and high (50% and above). When that was not possible, researchers compared the current and prior status quo with regard to the explicit and implicit goals of the project. Level of respon- siveness ratings were based on existing data (e.g. I Change My City), original data analysis (e.g. Change. org) and, in some cases, interviews with DE platform staff, who were asked to provide clear evidence of responsiveness to feedback provided through the platforms. This approach is limited by dependence on self-reported administrative data in cases where verifiable system data and or user surveys are not available. Cases that lacked sufficient evidence with which to assess degree of institutional responsiveness were not included. Table 3 below presents the final coding of cases, followed by Table 4 with the specific evidence for the coding of institutional responsiveness outcomes. EVALUATION TEAM | DEET 12 2016 World Development Report Background Paper Table 4 – Mapping uptake and institutional response to ICT-enabled voice platforms High Medium Low CASE COUNTRY VARIABLES OUTPUT OUTCOME ACCOUNTABILITY DIRECTION INDIVIDUAL / COLLECTIVE PROACTIVE LISTENING RESPONSE DISCLOSE FEEDBACK DISCLOSE GOVT LEVEL RESPONSE PARTNER OFFLINE UPTAKE DRIVER PROACTIVE LISTENING ELECTRICITY DO IND GOV SUB MAJI VOICE KE IND GOV LOCAL LUNGISA ZA IND CSO LOCAL RIO 1746 BR IND GOV LOCAL DIGITAL STATE PB BR COL GOV SUB I CHANGE MY CITY IN IND CSO LOCAL POR MI BARRIO UY IND CSO LOCAL MAJI MATONE TZ IND CSO LOCAL PRESSURE PAN BR COL CSO ALL CHANGE .ORG INT COL CSO ALL PUNJAB PROACTIVE PK IND GOV SUB EVALUATION TEAM | DEET 13 2016 World Development Report Background Paper CASE COUNTRY VARIABLES OUTPUT OUTCOME ACCOUNTABILITY DIRECTION INDIVIDUAL / COLLECTIVE PROACTIVE LISTENING RESPONSE DISCLOSE FEEDBACK DISCLOSE GOVT LEVEL RESPONSE PARTNER OFFLINE UPTAKE DRIVER I PAID A BRIBE IN IND CSO ALL U-REPORT UG COL DON NAT CHEMI KUCHA GE IND CSO LOCAL CHECK MY SCHOOL PH IND CSO SUB LAPOR ID IND GOV ALL MYVOICE NG IND DON SUB HUDUMA KE IND CSO ALL IMCO MX IND CSO NAT KARNATAKA BVS IN IND DON SUB E-CHAUTARI NP IND DON SUB BARRIOS DIGITAL BO IND DON SUB SAUTI ZA WANANCHI TZ IND CSO NAT EVALUATION TEAM | DEET 14 2016 World Development Report Background Paper Table 5: Evidence for assessing institutional responsiveness CASE RESPONSE CRITERIA FOR CODING DATA SOURCE Reduction of corruption reports by 70%, increased Interview with distribution company, system reports Proactive Listening Electricity High levels of service-user satisfaction. provided by company. Increase in percentage of solved complaints, System data analysis (n=57,809), customer survey with time of response reduced by half since (n=1,064).” (Belcher & Lopes 2015). Maji Voice High implementation. Survey reveals 60% of satisfied customers. Lungisa High 98% of complaints reported as solved. Website (http://www.lungisa.org/, March 28th 2015). 99% of complaints reported as solved, user 1746 statistical report (http://www.1746.rio.gov.br/, Rio 1746 High satisfaction at 74% March 28th 2015) Digital State Participatory Institutional response based on 100 % of prioritized World Bank report, “Impact of online voting in High Budgeting projects submitted to official budget. participatory budgeting in Brazil” (Haikin, 2015) 51% of complaints resolved. Website (http://www.ichangemycity.com/, I Change My City HIgh March 28th 2015) Por Mi Barrio High 50% of complaints resolved. Montevideo’s data report, interview with project staff. Service provider actions were taken in 40% of Website (http://blog.daraja.org/2012/02/so-what- Maji Matone Medium reports received. have-we-learnt-summarising.html, March 28th 2015) 24% of campaigns supported by the organization System data analysis provided by MeuRio. Meu Rio’s Pressure Pan Medium are successful. Average individual signature has 25% chance of Data analysis of 3.9 million users’ data from change. Change.org Medium generating a response. org through open API. World Bank analysis, 2015. EVALUATION TEAM | DEET 15 2016 World Development Report Background Paper CASE RESPONSE CRITERIA FOR CODING DATA SOURCE With nearly one million citizens contacted, the World Bank Development Report 2016 (draft). government has taken about 6000 administrative actions, mostly warnings but also formal apologies, with limited instances of suspension Punjab Proactive Medium and dismissal of civil servants. There is a systematic proportion of fake and incorrect citizen cell phone numbers entered by government officials, indicating constraints on senior manager oversight capactiy Reponses to reports sent by IPAB to authorities Interview with IPAB staff (e-mail Sunil Nair, 19 I Paid a Bribe Low described as “very limited”. December 2014). Limited to anecdotal data on MPs (rather limited) Berdeu & Abreu-Lopes, 2015. U-Report Low interest on U-Report data. 4% of reports fixed within one year of activity Website (https://www.chemikucha.ge/en/ , March 28 Chemi Kucha Low 2015) 11% response rate to reported school issues. Crowdsourcing Citizen Participation: The Check My School Low CheckMySchool 3G Experience, 2014. Government considers the 8% of reports System data provided by LAPOR team. LAPOR Low received got a response. Out of 314 messages sent, only six were responded Lee, & Schaefer (2014) MyVoice Low using the system, and eighteen were tagged for follow-up. Assessment shows that out of the 3,000 reports Bott & Young (2012) Huduma No submitted via SMS, email, and Twitter, none were resolved. EVALUATION TEAM | DEET 16 2016 World Development Report Background Paper CASE RESPONSE CRITERIA FOR CODING DATA SOURCE There was no evidence of responsiveness on the Interview with project staff (phone). IMCO No part of education authorities to school issues reported on the platform Despite study looking for evidence of Georgieva Andonovska, E. (2014), Madon (2014) institutional responsiveness, no clearly Karnataka Beneficiary Feedback Low documented evidence of government responsiveness on service delivery issues. Despite study looking for evidence of On Track Evaluation Report. institutional responsiveness, no clearly Keystone Accountability, 2015. e-Chautari Low documented evidence of government responsiveness on service delivery issues. Despite study looking for evidence of On Track Evaluation Report. institutional responsiveness, no clearly Keystone Accountability, 2015. Barrios Digital Low documented evidence of government responsiveness on service delivery issues. Despite interview with staff, no clearly documented Interview with project staff (e-mail). Sauti Za Wananchi Low evidence of government responsiveness on service delivery issues. EVALUATION TEAM | DEET 17 2016 World Development Report Background Paper The majority of platforms make their citizen feedback public (18 of 23). Out of the five that do not disclose the feedback, two are governmental and three involve donor agencies in collaboration with governments. Conversely, all of the CSO-driven initiatives publicise the input given by citizens. This finding makes par- ticular sense if one considers the directionality of accountability relations. User-feedback initiatives (i.e. not disclosed) are more likely to be implemented by governments or donors, where service providers are held accountable to a higher authority (upwards accountability). Conversely, given that CSOs have few means to hold providers directly accountable, they rely essentially on downwards accountability mechanisms, where the driving force of institutional responsiveness – at least hypothetically – is the exposure of the behaviour of service providers vis-à-vis citizens. No pattern seems to emerge when looking at disclosure of feedback and institutional responsiveness, however. In user feedback initiatives (where feedback is not disclosed and there is no collective action), the four cases are equally split between low and high levels of institutional responsive- ness. A similar pattern emerges when examining citizen engagement initiatives: public disclosure of feedback does not seem to lead – per se – to increased responsiveness from providers. In 14 cases, the provision of input through the dedicated platform is complemented by some type of offline action to prompt governments to respond and/or to monitor government responsiveness. This is the case, for instance, of the Rio Grande do Sul PB process, in which citizens are periodically elected to monitor the imple- mentation of investments prioritised through a voting process (Spada et al. 2015). In MajiVoice, the respon- siveness of the water service agency is actively monitored by the members of the Water Services Regulatory Board, which can trigger legal actions against service providers when they fail to meet pre-established quality standards (Belcher and Lopes 2016). Yet, offline action does not seem to ensure responsiveness by itself, as illustrated by the cases of e-Chautari in Nepal and Barrios Digital in Bolivia. However, among the 14 cases, the evidence is insufficient to verify that the intensity and regularity of these offline actions varies. In the category of civic action initiatives, where response involves online collective action, we find four different cas- es, with varying degrees of institutional responsiveness. The Rio Grande do Sul Digital PB process has a high level of institutional responsiveness, while the online petition platform Change.org and the Brazilian initiative Pressure Pan both have medium levels. A possible explanation of the different responsiveness levels is the difference in institutional design. Digital PB in Rio Grande do Sul is a governmental initiative mandated by state legislation. As such, all of the citizen-generated social investment proposals that are approved through the participatory process are officially included in the State’s budget, with a number of them effectively carried out by the state government. The other two initiatives are platforms that allow any citizen to initiate collective action to petition or exert pres- sure on the government to take an action towards any public agenda. This open-endedness means that the platforms can launch both some actions that trigger extensive uptake and mobilisations, and many that fail to generate follow-up. This potential for a large denominator, in terms of the total number of initiatives, would affect the overall percentage of petitions that trigger responsiveness. Indeed, some data seems to suggest the im- portance of mobilisation capacity: online petitions on Change.org are substantively more likely to be successful when sponsored by an organisation (World Bank 2015), and citizen campaigns through Pressure Pan are three times more likely to succeed when receiving mobilisation support from Pressure Pan’s staff. This evidence reso- nates with the proposition that the effectiveness of digital technologies in social mobilisation depends on offline structures of organisation and influence (Fung, Gilman and Shkabatur 2013). Finally, we find the widely-rec- ognised case of U-Report in Uganda, with low level of institutional responsiveness, which we shall discuss later. EVALUATION TEAM | DEET 18 2016 World Development Report Background Paper In terms of the institutional actors that drive the voice initiatives, 12 are led by CSOs, six by governments, and five by donors. Out of the seven initiatives with high levels of responsiveness, four are govern- ment-led and three CSO-led. Civil society and governments seem equally capable of creating plat- forms and processes that engender responsiveness. However, the three CSO high-response initiatives all share a common trait in that they involve partnerships with government. In other words, in all of the cases of high institutional responsiveness, the government is either leading the process or plays the role of a partner. However, not all of the initiatives involving government–CSO partnerships led to high levels of institutional responsiveness, as illustrated by the cases of I Paid a Bribe and Check My School, both of which had low percentages of issues raised by citizens that led to documented agency responses. Seen together, these findings seem to suggest that while partnership with government is not a sufficient condition for the responsiveness of CSO-led initiatives, it may well be an enabling one. Finally, while the initiatives showing medium and high degrees of institutional responsiveness involve both CSO and gov- ernment-driven efforts, we find no donor-driven platforms that led to institutional responsiveness. While we do not claim our sample to be representative and the results may be skewed due to the small number of donor-driven cases analysed, these patterns suggest future research paths, focusing on the role that different drivers may play in institutional responsiveness. Box 1. Whose voices are they? Whose voices are expressing themselves on ICT-enabled governmental service delivery feedback platforms? What kinds of bias may be involved? ICT platforms can potentially select for some kinds of responses over others. This can happen in at least two distinct ways – differential access to com- munication of feedback, and categorisation of user input that pre-selects for certain categories. First, the subset of citizens who engage with ICT systems may or may not represent the concerns of those citizens who lack ICT access, such as rural women or people without access to formal educa- tion. This is the case with UR’s U-Reporters, one quarter of whom are government employees (Mellon et al. 2015), and who under-represent the low-income, rural citizens who are most in need of public services. Indeed, the whole notion of user feedback suggests that the target group is limited to those citizens who ostensibly should have access but have problems in practice – such as those who have a water connection, but lack water. This implicit framing excludes those who are not included in water systems, clinics, schools or public security in the first place - and therefore not considered ‘users’. Second, as citizen concerns are input into government agency data systems for aggregation and transmission upwards to senior managers, administrative legibility requires them to be categorised into lists of preexisting categories, which may also select for some kinds of citizen priorities to the exclusion of others – as in the case of issues that are priorities for low-income urban women, as Ranganathan found in her study of e-redressal systems in Karnataka (2012). To sum up, the framing of the main questions addressed in this study – whether or not ICT service delivery feedback platforms lead to uptake, and whether or not such voice in turn leads to service delivery response – does not address two relevant questions: whose voice is projected, and how inclusive the feedback agenda is. EVALUATION TEAM | DEET 19 2016 World Development Report Background Paper When examining uptake, the results in Table 4 support our previous argument that citizen use of plat- forms (an output) should not be equated with institutional responsiveness (an outcome). This sample includes significant cases that combined high uptake with low responsiveness. The case of U-Report (UR), UNICEF’s social monitoring system for young Ugandans, provides compelling evidence for this point. Created in 2007, this SMS-based platform runs weekly polls with registered users on a broad range of is- sues (e.g. child marriage, access to education). To inform public debate, the results of the polls are widely disseminated through the project’s website and diverse mass media outlets, including a variety of formats such as newspaper articles, radio shows and even a documentary broadcast on major Ugandan TV chan- nels. Members of Parliament (MPs) are UR’s main policy audience. Aligned with a vision of real-time data collection to inform policymaking that goes beyond sending weekly newsletters with poll results to MPs, UNICEF also provides MPs with access to the platform to reach out to their audiences. The number of registered users (U-Reporters) has grown steadily since its launch, recently reaching more than 299,000 (Bayern 2015; World Bank 2015b). UNICEF describes UR as a “‘killer app’ for communication towards achieving equitable outcomes for children and their families” (UNICEF 2012). This enthusiastic view of UR has resonated in development circles, with the free SMS-based platform currently being rolled out in countries such as Rwanda, Burundi, the Democratic Republic of Congo, South Sudan, Nigeria and Mexico. Uptake is not a problem for UR in terms of numbers, and it leverages the potential of mobile phones as a means to ‘listen at scale’. However, 47% of UR participants have some university education and one quar- ter are government employees, raising questions about whose voices are being projected (see Box 1). Furthermore, until recently very little was known about the extent to which U-report’s take-up was translated into any type of institutional responsiveness. A new detailed evaluation of U-Report finds no systematic evidence of U-Report affecting policy, let alone MPs behaviour in terms of representation, legislation and oversight (Berdou and Lopes 2016). U-Report emerges thus as a significant case that illustrates the need to separate uptake (as an output), from institutional responsiveness (as an outcome). To conclude the discussion of these empirical findings, when examining the table above, one of the most noticeable patterns is the existence of numerous digital engagement initiatives that meet dead ends despite different pathways – at least in the relatively short run. The majority of the 23 cases studied led to low levels of institutional responsiveness, with 11 reporting medium to high levels (defined conservatively as leading to at least 20% response rates). Notably, the multiple dead ends do not seem to be motivated by the absence of any one specific factor. None of these variables appear to be a sufficient condition for institutional respon- siveness, suggesting that none of these factors can be a considered as a ‘magic bullet’. The findings suggest multiple pathways to institutional responsiveness – involving the convergence of multiple, mutually reinforcing factors. If one factor does stand out, however, it is government involvement, insofar as four of the six cases of government-led voice platforms were associated with high rates of service delivery responsiveness. EVALUATION TEAM | DEET 20 2016 World Development Report Background Paper Conclusion This study reviewed cases of ICT-enabled voice platforms where evidence of institutional response was available. As suggested in our introduction, in the ‘yelp’ feedback loop model, proponents tend to as- sume that user feedback to identify service delivery problems is sufficient to induce service providers to respond. This review of the evidence from 23 ICT-enabled platforms finds that this implicit market model, in which (individual) demands for good services produces its own supply, is not sufficient to leverage institutional response. This study organized the data from available empirical research in order to iden- tify broad patterns of user uptake, public access to user feedback data, and institutional arrangements, and provide an assessment of whether service providers respond to user feedback. This conclusion addresses some of the emerging issues that should be addressed in the future. Indeed, as the evidence base grows, more systematic explorations of the relationship between ICT-enabled citizen voice and institutional response should be possible. The findings from the 23 cases where both user uptake and institutional response data were available indicate mixed results on both counts. In eight cases, user uptake was high. Institutional response was high in seven cases, and intermediate in three. For the majority of cases, institutional response was low or non-existent. One reason for these mixed results, however, is that the umbrella category “ICT-enabled voice platforms” may have resulted in the selection of cases that are actually quite different from one another. Separating some of these approaches from one another may help to clarify the findings. Indeed, a similar approach has been used to unpack outcomes from the diverse initiatives that fall under the conceptual umbrella of “social accountability” (Fox 2014). What looks like “mixed results” at first glance may simply be a case of conflating apples with oranges. Since this research collected data on a diverse array of indepen- dent variables, patterns in citizen uptake and institutional response can be revisited, revealing patterns that would not otherwise be visible. This conclusion will highlight several variables that may be especially fruitful for future research on broad-based user feedback, civic engagement, and effective institutional response. I) Does the feedback platform contribute to upwards accountability, downwards accountability, or both? The institutional design of ICT-enabled voice platforms determines whether the role of citizen voice is limited to informing program managers and policymakers (i.e., upwards accountability), or whether voice is intended to contribute to public scrutiny and potential collective action, which in turn would create incentives for insti- tutional response (i.e. downwards accountability). Through processes of upwards accountability, ICT-enabled user feedback can help senior policy-makers to identify bottlenecks and address front-line service provision issues. For example, in one of the cases with the highest uptake – Punjab Proactive Feedback – citizen reports are not disclosed and there is no offline citizen engagement, so institutional response is left to the discretion of senior managers. However, there is evidence that many ICT-enabled voice platforms are also conducive to downwards accountability as well: User feedback is publicly disclosed in 18 of the 23 cases studied. In 12 of the 23 cases, ICT feedback was complemented by offline citizen engagement of some kind. While platforms that enable upwards accountability (e.g., large-scale opinion surveys) are associated with only modest levels of institutional responsiveness, there appears to be a relationship between platforms EVALUATION TEAM | DEET 21 2016 World Development Report Background Paper that are conducive to downwards accountability and platforms that produce greater responsiveness: Five of the seven high-impact platforms disclosed feedback. Six of the seven high-impact platforms involved offline citizen engagement. In all of the high-impact cases, government was present either as a driver (4 cases) or as a partner (3 cases). This suggests that for downwards accountability to work most effectively, both public disclosure of feedback and public collective action may be necessary. In other words, civic engagement, in addition to information, is what generates the civic muscle necessary to hold senior poli- cymakers and frontline service providers accountable. II) What institutional design features can influence the willingness and capacity of service providers to respond to citizen feedback? Another way to explore the role that citizen voice plays in driving institutional response is to explore the issue through the lens of a senior program manager. Their responsiveness to citizen feedback is deter- mined both by their willingness—intent and motivations—and their capacity—the institutional leverage they have to actually change practice. In some cases, institutional design and a strong sense of commit- ment to the organisational mission by high-level officials are enough to encourage a program manager’s willingness to respond. In these cases, the key role of ICT platforms is to bolster capacity to respond– as with MajiVoice’s water provision in Kenya. 5 Some program managers may have a strong sense of mis- sion, while others may be more concerned about the potential political risk associated with dissatisfied citizens. In either case, the systematic collection of citizen feedback can be a useful tool. In other words, the motivations for responsiveness do not appear to be directly influenced by ICT voice platforms. In contrast, the determinants of senior manager capacity to respond to citizen voice are directly affected by ICT platforms’ institutional and technical design. These features will determine the precision with which user problems are identified, which is crucial to identify which service providers are responsible. The cases studied suggest that it is crucial for user complaints to be routed to entities within the service pro- viding agency that have some incentive and capacity to respond. Specifically, experiences with the most high-impact platforms, such as the Dominican electricity agency and MajiVoice in Kenya, suggest that direct links between governmental feedback reception systems and internal work order systems greatly increase policymakers’ capacity to determine whether and how complaints have been resolved, which appears to be a necessary condition for effective institutional response. Similarly, two of the most suc- cessful CSO platforms – Por Mi Barrio in Uruguay and I Change My City in India – are connected to exist- ing governmental service provider complaint systems. These are examples of the institutional questions that play crucial roles as intervening variables that shape whether or not voice triggers teeth to act. The proposition that emerges here is that regardless of their motivations, policymakers with a commitment to bolstering institutional responsiveness should in principle have an incentives to: 1) institute tracking systems that directly link complaints to institutional responses and 2) to publicly disclose both citizen feedback and data regarding institutional response – in order to both inform and validate subsequent citizen action, and to potentially ‘name and shame’ non-performing units with their agency. 5 In the case of MajiVoice, degrees of responsiveness can be explained by the modality of contracts between government and service providers (renewable upon performance) as well as the creation of an oversight structure to monitor government response. For details, see Belcher and Lopes (2015). EVALUATION TEAM | DEET 22 2016 World Development Report Background Paper III) How can proactive listening systems broader outreach to citizens and project voice more widely? One of the relevant findings from this review of the evidence is that proactive listening systems are both rel- atively rare and yet quite significant. Two of the most well known cases of ICT enabled citizen voice—Punjab Feedback (which has the most uptake of any cases by far) and U-Reporters—involve proactive listening. Yet the evidence available indicates that neither of these platforms has triggered high levels of institutional response. While proactive listening in the Punjab Feedback case involves significant willingness by senior policymakers to respond to users, their capacity is constrained by both a limited complaint tracking sys- tem—citizen’s that have filed complaints often have their phone numbers misreported— and limited lever- age over civil service employees—their ability to sanction is limited by civil service rules. Indeed, the fact that user feedback is not made public could be interpreted as an indicator of the fragility of the system’s constituency within the government. Unlike the Punjab Feedback project, use of the U-Reporter system is not limited to users of basic services and its reporting bias towards urban, male, well-educated citizens suggests that its voice may not entirely representative of citizens. The most clear-cut case of a proactive lis- tening system with high levels of uptake and institutional response is the Dominican electricity system. This uneven pattern of uptake and responsiveness in such a diverse set of proactive listening cases suggests that more institutional experimentation and innovation is needed, with a strong emphasis on connecting the dots between incentives for citizens to express voice and the capacity of service providers to respond. IV) How can lessons from uptake of non-dedicated social media be applied to ICT-enabled service delivery platforms? The majority of cases where social media has enabled collective action concerned broad issues of civic concern, like corruption or authoritarian abuse, rather than specific service delivery issues. Moreover, these viral processes have been enabled by open, multi-purpose social media, rather than through dedicated, service-specific ICT platforms. The contrast between the track record of ICT-enabled civic engagement platforms dedicated to service delivery agencies, and that of much broader non-dedicated social media platforms, suggests that some of the lessons from the latter could be applied to the former. Crowdsourcing public grievances could, in principle, publicly legitimate citizen service delivery concerns, identify problem hotspots, and enable coordination for collective action to encourage service provider responsiveness. Yet, in practice, the evidence (especially from CSO-led, crowdsourced citizen feedback platforms) suggests that this has actually happened far less often than one might hypothesize. V) How can society-facing “targeted transparency” find synergy with government-facing “targeted citizen feedback,” to stimulate virtuous circles of mutually-reinforcing voice and teeth? Intuitively, one would expect citizens to be more likely to report problems with service delivery to providers if they have reason to believe that those service providers are likely to respond to that feedback. Conversely, non-responsiveness is likely to discourage citizen reports.6 This suggests the potential for encouraging vir- tuous circles of increased citizen reporting as agencies’ capacity to respond grows. It also underscores one 6 For empirical evidence of the effect of government responsiveness on levels of citizen participation, see Sjoberg et al. (2015). EVALUATION TEAM | DEET 23 2016 World Development Report Background Paper of the lessons from research on “targeted transparency,” which emphasizes the importance of embedding information disclosure and access in potential users’ everyday routines, in order to inform decision-making and potential collective action (Fung, Graham & Weil 2007). Yet, the limited institutional responsiveness achieved thus far by ICT-enabled citizen voice platforms suggests that perhaps the concept of embedded feedback should also be applied at the governmental “receiving end.” Indeed, while “targeted transparen- cy” usually refers to disclosure of relevant information to citizens, perhaps “targeted citizen feedback” is needed to help deliver information to government program managers in ways that embed it in official deci- sion-making processes (as in the case of MajiVoice, where citizen complaints are immediately attached to government work orders that can be tracked through the system). Final proposition for discussion To conclude, the empirical evidence available so far about the degree to which voice can trigger teeth in- dicates that service delivery user feedback has so far been most relevant where it increases the capacity of policymakers and senior managers to respond. It appears that dedicated ICT-enabled voice platforms – with a few exceptions – have yet to influence their willingness. Where senior managers are already com- mitted to learning from feedback and using it to bolster their capacity to get agencies to respond, ICT plat- forms can make a big difference. In that sense, ICT can make a technical contribution to a policy problem that to some degree has already been addressed. In summary, ICT platforms can bolster upwards accountability if they link citizen voice to policymaker capacity to see and respond to service delivery problems. This matters when policymakers already care. Where the challenge is how to get policymakers to care in the first place, then the question is how ICT platforms can bolster downwards accountability by enabling the collective action needed to give citizen voice some bite. EVALUATION TEAM | DEET 24 2016 World Development Report Background Paper References Bayern, J. (2015) Investigating the Impact of Open Data Initiatives: The Cases of Kenya, Uganda and the Philippines, Washington: World Bank, January Bayern, J. (2015), Citizen Feedback Survey Report, Washington: World Bank, January Berdou, E. and Abreu-Lopes, C. (2016) The case of UNICEF’s U-report (Uganda): Final Report to the Evaluation Framework for Digital Citizen Engagement, Digital Engagement Evaluation Team document, World Bank Belcher, M. and Abreu-Lopes, C. (2016) MajiVoice Kenya: Better Complaint Management at Public Utilities, forthcoming Digital Engagement Evaluation Team document, Washington: World Bank Bhatti, Z.K., Zall Kuseck, J. and Verheijen, T. (2015) Logged On: Smart Government Solutions from South Asia, Washington: World Bank Bott, M., & Young, G. (2012). The Role of Crowdsourcing for Better Governance in International Development, Praxis: The Fletcher Journal of Human Security 2(1), 47-70 Brockmyer, B. and Fox, J. (2015) Assessing the Evidence: The Effectiveness and Impact of Public Governance-Oriented Multi-Stakeholder Initiatives, London: Transparency and Accountability Initiative, http://www.transparency-initiative. org/reports/assessing-the-evidence-the-effectiveness-and-impact-of-public-governance-oriented-multi-stakeholder- initiatives (accessed 6 October 2015) Carothers, T. and Brechenmacher, S. (2014) Accountability, Transparency, Participation and Inclusion: A New Development Consensus? Washington: Carnegie Endowment for International Peace, http://carnegieendowment.org/ files/new_development_consensus.pdf (accessed 6 October 2015) Cornwall, A. (2002) Beneficiary, Consumer, Citizen: Perspectives on Participation for Poverty Reduction, SIDA Studies No. 2, Stockholm: Swedish International Development Agency Dhiratara, A. and M. M. Gibran Sisunan, LAPOR! Layanan Aspirasi Dan Penguadan Online Rakyat, presentation, n.d. Diecker, J. and M. Galan (2014) “’Creating’ a Public Sphere in Cyberspace: The Case of the EU,” in Carayannis, E.G., Campbell, D.F. and Efthymiopoulos M.P . (eds) Cyber-Development, Cyber-Democracy and Cyber-Defense, New York: Springer Downs, A. (1957) An Economic Theory of Democracy New York: Harper and Row Fox, J. (2007) The Uncertain Relationship between Transparency and Accountability, Development in Practice, 17(4), 663-671 Fox, J. (2014) Social Accountability: What does the Evidence Really Say? GPSA Working Paper No. 1, Washington: World Bank Global Partnership for Social Accountability programme Fung, A., Graham, M. and Weil, D. (2008) Full Disclosure: The Perils and Promise of Transparency, Cambridge: Cambridge University Press Fung, A., Gilman, H.R. and Shkabatur, J. (2013) Six Models for the Internet + Politics, International Studies Review 15(1), 30-47, doi:10.1111/misr.12028 Gaventa, J. and McGee, R. (2013) The Impact of Transparency and Accountability Initiatives, Development Policy Review 31(S1) s3-s28 Georgieva Andonovska, E. (2014), The Karnataka Beneficiary Verification System (BVS) – A Case Study. World Bank, December Gigler, S. and Bailur, S. (eds) (2014) Closing the Feedback Loop: Can Technology Close the Accountability Gap? Washington: World Bank Grandvoinnet, H., Aslam, G. and Raha, S. (2015) Opening the Black Box: Conceptual Drives of Social Accountability Effectiveness, Washington: World Bank Haikin, M.(2015). Impact of Online Voting on Participatory Budgeting in Brazil, Digital Engagement Evaluation Team, World Bank. Joshi, A. (2014). Reading the Local Context: A Causal Chain Approach to Social Accountability, IDS Bulletin 45(5), 23-35 EVALUATION TEAM | DEET 25 2016 World Development Report Background Paper Kalla, J., Mellon, J., Peixoto, T., Sjoberg, T. (2015) The Crowd Never Lies? Evaluating the Quality of Crowd-Sourced Data in Uganda, Digital Engagement Evaluation Team, World Bank LAPOR (2014). Facts Sheet, Jakarta: LAPOR Lee, P., & Schaefer, M. (2014). Making Mobile Feedback Programs Work: Lessons from Designing an ICT Tool with Local Communities, Washington: World Bank Lieberman, E., Posner, D. and Tsai, L. (2014) Does Information Lead to More Active Citizenship? Evidence from an Education Intervention in Rural Kenya, World Development 60(1), 69-83 Lindstedt, C., & Naurin, D. (2010). Transparency is Not Enough: Making Transparency Effective in Reducing Corruption, International Political Science Review, 31(3), 301-322. Lohmann, S. (2000). Collective Action Cascades: An Informational Rationale for the Power in Numbers. Journal of Economic Surveys, 14(5), 655-684 Madon, S. (2014) “Information Tools for Improving Accountability in Primary Health Care: Learning from the Case of Karnataka” in S. Gigler and S. Bailur (Eds.) (2014) Closing the Feedback Loop: Can Technology Close the Accountability Gap? Washington: World Bank Masud, M.O. (2015) Calling Citizens, Improving the State: Pakistan’s Citizen Feedback Monitoring Program, 2008– 2014, Princeton University, Innovations for Successful Societies http://successfulsocieties.princeton.edu/publications/ calling-public-empower-state-pakistan (accessed 13 October 2015) Mellon, J., Peixoto, T. and Sjoberg, T. (2015) The Crowd Never Lies? Evaluating the Quality of Crowd-Sourced Data in Uganda, Digital Engagement Evaluation Team, World Bank, unpublished Peixoto, T. (2013) “The Uncertain Relationship Between Open Data and Accountability: A Response to Yu and Robinson’s ‘The New Ambiguity of “Open Government,’” UCLA Law Review Discourse 60, 200-248 Prieto-Martín, P., de Marcos, L. and Martínez, J.J. (2011) The e-(R) evolution Will Not Be Funded, European Journal of ePractice 15. Ranganathan, M. (2012) “Reengineering Citizenship: Municipal Reforms and the Politics of ‘e- Grievance Redressal’ in Karnataka’s Cities,” in Desai, R and Sanyal, R. (eds) Urbanizing Citizenship: Contested Spaces in Indian Cities, Sage: Thousand Oaks and New Delhi Sjoberg, J. Mellon, T. Peixoto (2015) The Effect of Government Responsiveness on Future Political Participation, Digital Engagement Evaluation Team Working Paper 1. World Bank. Spada, P., Mellon J., Peixoto, T. and Sjoberg, F.M. (2015) Effects of the Internet on Participation: Study of a Public Policy Referendum in Brazil, World Bank Policy Research Working Paper 7204, Washington: World Bank Susha, I. and Grönlund, Å. (2014) Context Clues for the Stall of the Citizens’ Initiative: Lessons for Opening Up E-participation Development Practice, Government Information Quarterly, 31(3), 454 465 UNICEF (2012) U-report Application Revolutionizes Social Mobilization, Empowering Ugandan Youth, http://www.unicef.org/infobycountry/uganda_62001.html (accessed 13 October 2015) World Bank (2014a) Strategic Framework for Mainstreaming Citizen Engagement in World Bank Group Operations, Washington, DC: World Bank. https://openknowledge.worldbank.org/handle/10986/21113 (accessed 13 October 2015) World Bank (2014b) Survey Report: Citizen Feedback Monitoring Program, Washington: World Bank World Bank (2015) ‘Digital Analysis’, conducted by the World Bank’s Digital Evaluation Team, unpublished EVALUATION TEAM | DEET 26