Water and Sanitation Program: REPORT SLB CONNECT Mainstreaming citizen feedback on service delivery using ICTs Findings and lessons from ICT-based feedback surveys on water supply and sanitation services in Indian cities Vandana Bhatnagar, Nidhi Batra and Kanak Tiwari The Water and Sanitation Program is a multi-donor partnership, part of the World Bank Group’s Water Global Practice, supporting poor people in obtaining affordable, safe, and sustainable access to water and sanitation services. Using ICTs for citizen feedback surveys to mainstream demand side monitoring Acknowledgments This report is a synthesis of findings and outputs from the SLB-Connect program, which has been operationalized by the Water and Sanitation Program (WSP) under the aegis of the Service Level Benchmarks (SLB) program of the Ministry of Urban Development (MoUD), Government of India. Implementation of the program was made possible by the support received from MoUD along with the Departments of Urban Development of the Governments of Rajasthan, Madhya Pradesh and Uttar Pradesh. Valuable comments and excellent support were provided by the local government authorities in all the cities covered under the project, especially the Pimpri Chinchwad Municipal Corporation, which was a partner on this project from the pilot phase. Senior sector experts constituting the ‘SLB Connect Advisory Committee’ provided valuable inputs in setting the strategic direction of this initiative, as also our external partners including Dr. Renu Khosla (Center for Urban and Regional Excellence), Dr. Jagan Shah (National Institute of Urban Affairs), Dr. Manvita Baradi (Urban Management Centre), and Dr. Kaustuv Bandopadhyaya (PRIA). The team would like to thank William D. Kingdom (Lead Water & Sanitation Specialist), Manish Kumar (Senior Institutional Development Specialist), Luis Alberto Andres (Lead Economist) and Luiza A. Nora (Social Development Specialist) from the World Bank, for valuable insights and leads provided at critical stages of the project. The team appreciates the feedback and critique provided by the peer reviewers, Simon Carl O’Meally (Senior Governance Specialist), Shankar Narayanan (Senior Social Development Specialist), Craig P. Kullman (Senior Water & Sanitation Specialist), Shomikho Raha (Public Sector Specialist), Sudipto Sarkar (Lead Water & Sanitation Specialist) and Hyoung Gun Wang (Senior Economist). Thanks are also due to Vandana Mehra (Communications Officer), for support in the preparation of outreach materials and reports, and Ammini Menon (Program Assistant), for coordinating logistics during the project implementation. The project benefited from the committed support and effort of various consultants and implementing agencies, namely, Nidhi Batra, Dr. Sita Shekhar, Kedar Dash, Sanjay Sangal and Kanak Tiwari, PRIA (consultant for field surveys), pManifold Business Solutions (consultant for telephone and SMS surveys), and RMSI Private Limited (consultant for IT platform). Editorial inputs for this report were provided by consultants, Andrew Wright and Ananda Swaroop. Project implementation and preparation of this report has been led by Task Team Leader, Vandana Bhatnagar (Senior Institutional Development Specialist) from the Water Team in South Asia. Task Team Leader: Vandana Bhatnagar Peer Reviewers: Simon Carl O’Meally, Shankar Narayanan, Craig P. Kullman, Shomikho Raha, Sudipto Sarkar and Hyoung Gun Wang Editor: Andrew Wright and Ananda Swaroop Designed by: Roots Advertising Services The Water and Sanitation Program is a multi-donor partnership, part of the World Bank Group’s Water Global Practice, supporting poor people in obtaining affordable, safe, and sustainable access to water and sanitation services. WSP’s donors include Australia, Austria, Denmark, Finland, France, the Bill & Melinda Gates Foundation, Luxembourg, Netherlands, Norway, Sweden, Switzerland, United Kingdom, United States, and the World Bank. The findings, interpretations, and conclusions expressed herein are entirely those of the author and should not be attributed to the World Bank or its affiliated organizations, or to members of the Board of Executive Directors of the World Bank or the governments they represent. b Using ICTs for citizen feedback surveys to mainstream demand side monitoring Contents Abbreviations and Acronyms 3 Executive Summary 4 1. Background  7 2. What is ‘SLB-Connect’ and How Does it Work? 9 2.1 What Does SLB-Connect Measure?  9 2.2 Linkage Between Demand and Supply Side Metrics  10 2.3 System Components of SLB-Connect  10 2.4 Additional Modes of Collecting Feedback  13 3. Experience with Implementation of SLB-Connect  14 3.1 Pilot Phase  15 3.2 Scaled-up Demonstration  15 3.3 Additional Modes of Feedback – CATI Survey and SMS Polls  20 3.4 National Dissemination and Mainstreaming  22 4. Survey findings  24 4.1 Key Findings of MAPI Surveys in Pilot Phase  24 4.2 Key Findings of MAPI Surveys in Scaled-up Demonstration  28 4.3 Key Findings of CATI Survey and SMS Polls  41 5. Summary observations and lessons  43 5.1 Implementation Experience  43 5.2 Survey Findings on Service Levels  44 6. Recommendations for Design of Citizen Feedback Interventions  46 7. Way Forward  50 List of Boxes Box 1: About Service Level Benchmarks 7 Box 2: Social Accountability Approaches for Improving Service Delivery 8 Box 3: SLB-C Survey Process Followed in the Five Scale-up Cities 17 www.wsp.org 1 Using ICTs for citizen feedback surveys to mainstream demand side monitoring List of Figures Figure 1: Performance aspects and Citizen Service Experience Captured by SLB Connect 9 Figure 2: System Components of SLB-C 11 Figure 3: Screenshots of Survey Management Module 12 Figure 4: Implementation of SLB-C 14 Figure 5: Response of Municipal Functionaries to SLB-C 18 Figure 6: Process of Telephone Call Survey in Pimpri Chinchwad 20 Figure 7: Process of SMS Poll Survey in PCMC 21 Figure 8: Continuity of Water Supply: SLB-C Survey Findings Across the Five Scale-up Cities 31 Figure 9: Households Connected to Sewer 40 Figure 10: SLB-C Findings on Alternate Disposal Across the Five Cities 40 Figure 11: CATI Survey Findings on Continuity of Water Supply as Reported by PCMC Citizens 41 Figure 12: SLB-C Influences Top-down and Bottom-up Accountability 46 Figure 13: Comparison of Traditional Survey Practices to SLB-C 47 Figure 14: Comparison of Feedback Tools 48 List of Tables Table 1:  Comparison of Performance Metrics in the SLB-CONNECT (Demand-side) and SLB (Supply-side) Frameworks 10 Table 2: SLB-C System Functionalities 13 Table 3: City and Survey Sample Details 16 Table 4: Service Delivery Arrangements in Surveyed Cities 16 Table 5: Assessment of Responsiveness of Municipality 18 Table 6: Comparison of Three Feedback Surveys in Pimpri Chinchwad 22 Table 7: Survey Findings of MAPI Surveys in Pilot Phase 24 Table 8: Survey Findings of PCMC 2012 SLB-C Survey 25 Table 9: Survey Findings in Mehsana (2013 SLB-C Survey) 27 Table 10: Sample Profile Details 28 Table 11: Survey Findings in the Five Cities in the Scaled-up Demonstration – Water Supply 29 Table 12: Access to Water Supply: SLB-C Survey Findings in Five Scale-up Cities 30 Table 13: Incidence of Dirty Water Supply in the Last Three Months: SLB-C Survey Findings Across the Five Scale-up Cities 33 Table 14: Complaint Redressal for Water Supply Services: SLB-C Survey Findings Across the Five Scale-up Cities 34 Table 15: Satisfaction with Water Supply Services: SLB-C Survey Findings Across the Five Scale-up Cities 36 Table 16: Survey Findings in the Five Scaled-up Demonstration Cities – Sanitation 37 Table 17: Access to Toilet: SLB-C Survey Findings Across the Five Scale-up Cities 38 2 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Abbreviations and Acronyms AMRUT Atal Mission for Rejuvenation and Urban KUWSMP Karnataka Urban Water Supply Transformation Modernization Project APL above the poverty line lpcd liters per capita per day BPL below the poverty line MAPI Mobile Aided Personal Interview CATI Computer Assisted Telephonic Interview MoUD Ministry of Urban Development CEPT Centre for Environmental Planning and NGO nongovernmental organization Technology NPMC National Performance Monitoring Cell CSO Civil Society Organization PCMC Pimpri-Chinchwad Municipal Corporation CURE Centre for Urban and Regional SAc Social Accountability Excellence PRIA Society for Participatory Research In Asia GoI Government of India SMS Short Messaging Service GPS Global Positioning System SLB Service Level Benchmarks HH household SLB-C SLB Connect HHC Household Piped Connection TA Technical Assistance IBNET International Benchmarking Network for ULB Urban local Body Water and Sanitation Utilities USAID United States Agency for International ICT Information and Communication Development Technology UWSS Urban water supply and sanitation IT Information Technology WASH Water, Sanitation and Hygiene IVRS Interactive Voice Response System WSP Water and Sanitation Program www.wsp.org 3 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Executive Summary The water and sanitation sector in Indian cities is characterized • Generate more granular feedback on services at the intra- by weak accountability processes and lack of customer city level (ward/ zone) that could help identify problem orientation. The institutional focus of service providers tends hotspots and design area-specific interventions; and to be on creating new assets, rather than delivery of services • Highlight service inequities across user groups (for and performance efficiency. Moreover, in the absence of example, households in slum settlements), and strengthen reliable service data, the planning processes that inform the the voice of vulnerable groups in service delivery. creation of new assets are often not aligned with the needs and priorities of citizens. Through these processes, an enabling environment is created for strengthening accountability pressures on providers to In 2009, the Service Level Benchmarks (SLB) program deliver improved service outcomes. The SLB-C approach was introduced by the Government of India’s Ministry of leverages the use of Information and Communication Urban Development (MoUD) to strengthen the focus on service delivery. Under this program, service providers report Technologies (ICTs) for collecting feedback, quality performance on a standardized set of indicators for delivery assurance, real time analysis and public dissemination of of water supply, wastewater, solid waste management and findings. It also uses demand side metrics that are aligned with storm-water drainage services. While the SLB data have supply-side indicators, which facilitates ready integration gradually become an integral component of India’s urban with decision making. reform formulations, the availability and quality of this data So far, SLB-C has been implemented in eight cities across six have tended to remain poor. It is also seen to reflect only the states, representing a variety of service delivery arrangements service providers’ point of view, not the citizens’ perspective and implementation environments. Feedback has been on service delivery. collected from over 35,000 households using several methods, In recent years, various social accountability tools and of which the more predominant has been the Mobile Aided participation processes have also been introduced in India Personal Interview (MAPI) method, followed by telephone to strengthen pressures for improved public services and surveys and Short Messaging Service (SMS) polls. Drawing foster citizens’ involvement in planning processes. While from this experience, the SLB-C system and survey approach these initiatives have helped strengthen the overall focus on has been adapted for conduct of City Sanitation Rankings service delivery, basic services such as water and sanitation (‘Swachh Survekshan’) in 73 cities under the Swachh have largely remained outside their purview. Bharat Mission, wherein 80,000 citizen responses were also collected. The SLB-C approach has also been incorporated It was to address these gaps that the SLB Connect in the World Bank funded Karnataka Urban Water Supply (SLB-C) initiative was operationalized by the Water and Modernization Project. Sanitation Program, in partnership with the Ministry of Urban Development. Implemented as a complement to This report presents the findings, achievements and lessons the Ministry’s SLB program, SLB-C gathers and analyses that emerged from implementing the SLB-C initiative in feedback from citizens on water supply and sanitation Indian cities with varied contexts, and its subsequent use under services, in order to: a national urban program. It provides recommendations for future similar initiatives. • Facilitate tracking of service outcomes using citizen feedback; Findings • Provide a reality check for performance data reported by The feedback surveys conducted in eight cities provided service providers; concrete, relevant data which served as a useful reality 4 Using ICTs for citizen feedback surveys to mainstream demand side monitoring check to the SLB reports of service providers on their or not) could not be validated; fewer female respondents own performance. On questions related to infrastructure were reached; and respondents were reticent about answering provision, the survey data tended to be broadly aligned with questions on toilet use. the reports of service providers, while significant gaps were observed on several aspects related to service quality, such as Key Learnings duration of supply and incidence of dirty water supply. The As far as possible, demand-side metrics should be aligned granular data analysis helped reveal inequities between slum with supply-side indicators. Aligning the questions asked and non-slum areas, and also across wards with peripheral in citizen surveys with data reported by service providers areas of cities generally seeing poorer service levels. makes it easier to generate interest among stakeholders who are already familiar with service provider metrics, to track The survey results also drew attention to the inadequacy outcomes, strengthen monitoring, and hold service providers of existing formal complaint mechanisms, revealing that to account. Creating a common vocabulary of service metrics citizens’ use of telephone, SMS or online channels was from the demand and supply side, helps consumers and negligible. In most cities, only around 5 percent of citizens their representatives to dialog more effectively with service had lodged a complaint, mostly in person, and only a third providers. of complaints had been resolved. Designing surveys to be replicated at scale could help to The authorities in all cities accepted the survey findings, address local capacity constraints. Most local bodies lack though they subsequently displayed varying levels of interest the capacity to undertake citizen surveys. SLB-C suggests in acting on the concerns revealed by the surveys. Some cities how to make this easier: provides a default template which publicized the survey results on their municipality websites; is customizable with different questions from a question some integrated them in proposals for funding under national bank; enables surveys conducted in multiple locations to be urban programs such as the Atal Mission for Rejuvenation remotely monitored online in real time, making best use of and Urban Transformation (AMRUT), Smart Cities and a limited number of experts. It offers a platform which, if Swachh Bharat Mission. A few implemented interventions hosted by a nodal agency with requisite expertise, could serve to address service gaps. as a monitoring unit in the government, a regulatory agency, a research institution or civil society organization. The responsiveness of city functionaries was found to depend on several factors including their general comfort levels ICTs can improve impact by providing credible, with participatory processes and familiarity with ICTs; the transparent, immediately actionable information. Making local survey partner’s capacity and working relationship all the data accessible to stakeholders helped increase the with city authorities; the degree of cohesion among local transparency, and hence credibility, of the data collection political leaders; and the extent to which there were planning process. In contrast to traditional survey methods – which processes already underway for service improvements, into often take months to report, by which time the findings are which survey findings could feed in. dated – the system also enabled results to be made available to decision-makers immediately. The SLB Connect ICT platform enabled surveys to be conducted at scale in a short timeframe and with a limited Differing ways of obtaining feedback have different number of technical experts. It also proved able to cope with advantages and drawbacks. There are various ways of seeking diverse environments, reaching a significant percentage of citizen feedback – MAPI surveys, SMS polls, telephone respondents who are female, below the poverty line or living surveys, formal complaint mechanisms, and crowdsourced in slum areas. ratings – and they have different strengths and weaknesses. These should be viewed as complementary approaches rather Experience in Pimpri Chinchwad with the follow-up than as interchangeable substitutes. surveys using telephone and SMS found that, while cheaper and quicker than MAPI surveys, these methods had some How citizen feedback is expected to improve services drawbacks: household status (for example, whether in a slum should be clarified in advance. Citizen feedback can www.wsp.org 5 Using ICTs for citizen feedback surveys to mainstream demand side monitoring potentially be leveraged in several different ways to improve employing locals, such as students, rather than outsiders service delivery, such as informing service providers about to gather the data; providing granular data which is more gaps; mobilizing public opinion to generate pressure on relevant for local functionaries. service providers; or feeding into plans to improve policies or infrastructure. Clarifying in advance how feedback is The national government can help states and cities to adopt expected to be leveraged could help to decide the extent and citizen feedback processes in several ways. These include nature of civic mobilization and stakeholder engagement capacity building and advisory support; hosting or making required to accompany the survey. available ICT-based solutions; integrating these processes in government programs and formulations; and developing It may be difficult to generate popular demand for service orientation modules for citizen groups and political improvements. The survey results revealed low expectations representatives on how to interpret the resulting data and use on service delivery: despite poor service levels reported, 80- it for advocacy. 90 percent of respondents said they were partially or fully satisfied. This suggests there may be limits to the extent to The recent World Bank directive mandating inclusion of which civic mobilization is possible for service improvement. beneficiary feedback processes can give further impetus The surveys also showed that customers with complaints often to adoption of such processes in projects. These should address them to their local councilors, who could potentially however not be limited to self-reporting options such as play a greater role in putting pressure on service providers. helplines, SMS or online feedback. They should also include A useful metric in this regard could be the perceptions of mechanisms to collect feedback proactively, and identify whether services have improved or deteriorated compared to factors that may inhibit citizens’ use of formal grievance the previous year. redressal mechanisms. Support can be provided to clients in adopting these interventions in their operations and Way Forward decision making. It would also be useful to consolidate Advocacy and capacity building are needed for standardized learnings from SLB Connect with those from other World performance indicator frameworks to be adopted and Bank experiences in citizen engagement using ICTs, such as mainstreamed. Development agencies can play a critical Maji Voice (Kenya), Vozelectrica (Dominican Republic) and role in this, in partnership with the national government. Citizen Feedback Monitoring Program (Pakistan). There is Interest and buy-in of local functionaries can be increased potentially scope to reduce the costs of future such initiatives by involving them in the preparation of questionnaires; by creating a platform to share knowledge. 6 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 1. Background KEY POINTS • Availability and quality of performance data on service providers poor • Limited impact of past accountability initiatives on water and sanitation • SLB-C aimed at addressing existing constraints by leveraging emerging ICTs The water and sanitation sector in urban India is characterized by weak accountability processes and lack Box 1: About Service Level of customer orientation. Coverage is reportedly high, with Benchmarks approximately 70 percent households having access to SLBs are a standardized supply side indicator tap water, and 82 percent having access to toilet facilities. framework for measuring four basic urban services: However, the quality of services is often poor, resulting in water supply, wastewater, solid waste management and high coping costs, especially for the urban poor. Services are storm-water drainage. SLB indicator definitions for primarily provided by public providers that lack customer water supply largely draw from the globally accepted orientation, institutional capacity and accountability for International Benchmarking Network for Water and service outcomes. The institutional focus of these providers Sanitation Utilities (IBNET) indicator framework. tends to be on creating new assets, rather than delivery Of the 28 SLB indicators, 18 relate to water supply of services and performance efficiency. Moreover, in the and sanitation. Apart from indicator definitions and absence of reliable service data, the planning and design calculation methodology, the framework provides processes that inform the creation of new assets are often for a data reliability scale for each performance not aligned with the needs and priorities of citizens. indicator. Under a previous engagement, the Water and Sanitation Program (WSP) provided support Current status of performance reporting by cities for finalizing the indicator framework, and rolling In 2009, the GoI’s Ministry of Urban Development (MoUD) out a national pilot across 28 cities to demonstrate introduced the SLB program with the aim of facilitating a implementation of the framework. Further details shift in focus from infrastructure creation to the delivery on the SLB framework are available at: http://moud. of service outcomes (see Box 1). In 2010, the 13th Finance gov.in/policies/servicelevel Commission recommendations incorporated reporting on SLBs as a precondition for release of performance grants to Urban Local Bodies (ULBs). Since then, over 1,000 ULBs have been notifying their performance on the SLB increasingly mainstreamed into sector vocabulary and indicators annually. program formulations, helping to increase focus on service outcomes. More recently, this precondition has been retained for performance grants under the 14th Finance Commission However, the availability and quality of performance data (2015), and has also been integrated into the design of have often remained poor. They are also seen to reflect only the GoI’s urban investment program called the Atal the service providers’ point of view, not citizens’ perspective Mission for Rejuvenation and Urban Transformation on service delivery. One of the objectives of the SLB program (AMRUT), covering 500 cities. Several states – including was to foster dialog between citizens and municipalities on Maharashtra, Odisha and Rajasthan – have also referred issues related to service delivery, but this has not taken place to the SLBs when articulating their sector plans or as citizens have largely remained unaware of the SLB data strategies. Consequently, the SLB indicators have become reported by cities. www.wsp.org 7 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Current status of accountability initiatives in impact on water supply and sanitation. Channels available public services for citizens to provide feedback depend mostly on self- The GoI has started several initiatives aimed at making reporting by citizens (such as complaint reporting or online service delivery more ‘citizen centric’, such as laws on public feedback options) – and there is little incentive to use them, disclosure and the right to public services, and the National given service providers’ lack of customer orientation. There e-Governance Plan.1 These have however, tended to focus have been some efforts to proactively collect customer on public services which are transactional in nature, such feedback on service delivery through surveys, but these have as issuing certificates (birth, death, or marriage), property tended to be one-off initiatives, due to constraints including registration, utility connections or bill payments; basic lack of time, resources, local capacity and interest from services such as water supply and sanitation have largely decision makers. remained outside their purview. Moreover, while public participation is mandated as part of various urban programs, The SLB-C initiative was conceptualized and implemented local governments or service providers have little capacity to with the aim of addressing these constraints by leveraging undertake these processes, or clarity on how to do so. emerging ICTs. It also attempted to leverage the national government’s SLB program to strengthen the impact of While the above initiatives are helping to improve citizen voice and facilitate its integration into government accountability for some public services, they have had little decision making (see Box 2). Box 2: Social accountability approaches for improving service delivery Social Accountability (SAc) is an approach which seeks to stimulate citizen voice and client power to make the state and service providers accountable for meeting their service delivery obligations. Numerous SAc instruments have been developed in recent years, such as citizen report cards, community score cards, community monitoring of public service delivery, participatory budgeting and public expenditure tracking, public commissions, hearings and citizen advisory boards. These demand-side instruments differ from more conventional, supply-side mechanisms such as political checks and balances, accounting and auditing systems, administrative rules and legal procedures. Experience on the effectiveness of these tools has been mixed. They are heavily influenced by context. Recent studies draw a distinction between ‘tactical’ and ‘strategic’ approaches to SAc. The former tend to be localized, information-led approaches premised only on demand-side interventions. The more effective, strategic approach is however characterized by efforts to combine information access with an enabling environment, and coordinate with government reforms to encourage public sector effectiveness (What does the Evidence Really Say?, Jonathan Fox, GPSA Working Paper No.1, 2014; Opening the Black Box, Helene Grandvoinnet, Ghazia Aslam and Shomikho Raha, World Bank, 2015). 1 Enactments on the right to public services stipulate timeframes for public service delivery with penalties for deviation. The public disclosure law requires municipalities to publicly disclose organizational, financial and operational information. The National e-Governance Plan aims to provide Information Technology (IT) infrastructure and mobile platforms to allow citizens easier access to government services. 8 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 2. What is ‘SLB-Connect’ and How Does it Work? KEY POINTS • SLB-C addresses accountability gaps in delivery of water and sanitation services by connecting citizen feedback to service delivery and improvement processes • It captures feedback on various attributes of water supply and sanitation services that impact citizens directly SLB-Connect, as the name suggests, seeks to address Through these processes, the aim is to create an enabling accountability gaps in delivery of water and sanitation environment for strengthening accountability and increasing services by connecting citizen feedback to service delivery pressure on providers to deliver improved service outcomes. and improvement processes. It was developed as an extension of the SLB program to help mainstream demand-side Using the typology provided in the Strategic Framework for monitoring through ICT based citizen feedback surveys, and Mainstreaming Citizen Engagement in World Bank Group also help strengthen service providers’ supply-side reporting Operations released in 2014,2 the SLB-C approach corresponds under the SLB program. to interventions that entail ‘collecting, recording, and reporting on inputs from citizens’ and ‘citizen-led monitoring’. Within Citizen feedback surveys have been conducted using mobile- these categories, SLB-C’s defining characteristics are: (i) use to-web systems and other emerging technologies, and the of demand-side metrics that are aligned with supply-side results analyzed using publicly accessible online platforms. indicators, facilitating ready integration with decision making; Specific objectives of the SLB-C initiative are to: and (ii) use of an integrated and scalable ICT solution to enable zz Improve tracking of service outcomes using citizen feedback; transparent data collection, improved quality assurance, real- zz Provide a reality check for performance data reported by time analysis and public dissemination of citizen feedback. service providers; zz Generate more granular data on service performance at 2.1 What Does SLB-Connect Measure? the intra-city level (ward/zone) to help identify problems SLB-C captures feedback on various attributes of water and design interventions based on specific local needs; and supply and sanitation services that impact citizens directly, zz Provide data stratified by user groups (for example, as shown in Figure 1. The modular architecture of SLB-C households in slum settlements) to highlight potential also allows for the possibility of including measurement of service inequities that some of these groups may other services covered by the SLB framework, such as solid experience, and strengthen their voice in service delivery. waste management. Figure 1: Performance aspects and citizen service experience captured by SLB Connect Profile Water Supply Sanitation Feedback Age Access to water Access to toilets Satisfaction Gender Continuity Toilet usage Willingness for repeat Dwelling type Adequacy Access to sewerage survey Address Water quality network Contact number Income profile Complaint redressal Alternate disposal Suggestion to service Ease of bill payment system provider 2 Table 3.2, page 24 of the document. www.wsp.org 9 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Table 1: Comparison of performance metrics in the SLB-C (demand-side) and SLB (supply-side) frameworks Attribute SLB-C indicator definition [SLB indicator definition] WATER SUPPLY Access % of HHs reporting individual/shared household piped connections as primary source of water supply* [SLB: % of HHs with individual/shared household piped connection for water supply] Continuity Median value of responses on duration of supply* [SLB: Duration of water supply] Adequacy % of HHs reporting adequate supply to meet the needs of the family [SLB: Quantity of water supplied per capita (lpcd)] Quality % of HHs reporting no incidence of dirty water supply in the last three months [SLB: % of water samples meeting specified standards] Complaints % of HHs that lodged complaints reporting resolution in one day* [SLB: % of complaints resolved in one day] Bill Payments % of HHs reporting regular receipt of bills and finding the location and timing of bill payment to be convenient [SLB: Revenues collected as a percentage of revenues billed] Metering % of HHs reporting functional meters* [SLB: % of household connections with functional meters] SANITATION Toilet Access % of HHs reporting access to an individual, shared or public toilet* [% of HHs with access to individual, shared or public toilet] Toilet Usage % of HHs having access to toilets where all family members report use of toilets [No indicator] Access to Sewerage % of HHs reporting connection to sewer network* [% of properties with connection to sewer network] Alternate % of HHs reporting full or partial disposal of wastewater to any onsite facility Disposal [No indicator] *SLB-C indicators that are directly comparable with SLB indicators. Note: HHs: households; lpcd: liters per capita per day. 2.2 Linkage between Demand- and Supply-Side provides process and technical details of the MAPI surveys, Metrics while the other modes are discussed in later sections. SLB-C is aligned with the SLB framework on service attributes that directly impact users, as summarized in Table 1. The main components of SLB-C using MAPI systems are: (i) surveying sample households on service aspects using a 2.3 System Components of Slb-Connect mobile survey app; (ii) real-time monitoring of the survey Till now, data have been collected under SLB-C, primarily process using an online survey management module; and through enumerator-based household surveys conducted (iii) real-time analysis of survey results using a web-based using mobile applications, also known as Mobile Aided dashboard and data analysis tools. Survey results are thereafter Personal Interview (MAPI) systems. Additional modes of made accessible on a public online platform. The integrated, collecting feedback have also been tested, such as telephone workflow-based design of the ICT system distinguishes it surveys and Short Message Service (SMS) polls. This section from other solutions used for conduct of MAPI surveys. 10 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 2.3.1 Data Collection analysis. Users are assigned access and approval rights based An Android-based mobile survey application is used to get on their role in the survey process: functionaries from the citizen feedback on various service aspects. Enumerators using agency responsible for conducting the survey (for example, the mobile survey app collect the data in the local language the Society for Participatory Research In Asia (PRIA)) are through a detailed household survey, ensuring equitable assigned roles such as survey administrator, supervisor or representation of all sections of citizens geographically and enumerator, while other users such as WSP resource persons demographically, including feedback from residents living can be assigned rights to provide secondary oversight. in informal settlements. A Global Positioning System (GPS) in the cell phones tracks the location from where data are Quality control is exercised using predefined validation collected, thereby enhancing the reliability of data collection checks, exceptions flagging and various performance reports and enabling a granular analysis at the zone/ward level. These which enable the tracking of individual enumerators and functionalities are available on regular mobile phones in the supervisors. The availability of all records collected makes budget price range; the cost of handsets used in the SLB-C the survey results more credible to service providers. The project was approximately US$130 each. module is designed for replication of surveys at scale. The questionnaire can be customized using a ‘Question Bank’ and 2.3.2 Survey Management ‘Survey Form’ facility, while the ‘Survey Planning’ module A web-based survey management module enables survey allows sample design, user management and work allocation managers and sector experts to plan a survey and track its to be customized for each survey task. Previous survey forms progress in real time using remote monitoring. Predefined or survey tasks can be copied to readily initiate a new survey. workflow logic enables seamless process control – from data There is a facility to send SMS messages thanking respondents, collection, using the mobile app, to approval of records for where they have provided a contact mobile number. Figure 2: System components of SLB-C 1) Feedback collection 3) Dashboard for Results Analysis Mobile based HH Survey CATI (telephone) survey Pulse surveys - SMS Polling/ Missed call 2) Survey Management Module www.wsp.org 11 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Figure 3: Screenshots of survey management module Survey Questionnaire Module Survey Monitoring Module 12 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 2.3.3 Data Analysis and Dissemination comparison of SLB-C metrics across surveys conducted in All the data that pass through these quality assurance and different cities or in the same city at different points in time. approval steps feeds into a web-based ‘dashboard’, which The system features are summarized in Table 2. enables real-time analysis on the various service indicators through graphs, tables and maps. The dashboard is 2.4 Additional Modes of Collecting Feedback accessible online to decision makers, functionaries and SLB-C has tested two additional modes of collecting feedback: other stakeholders – including the public – once the survey Computer Assisted Telephonic Interviewing (CATI) and results have been finalized. The detailed analytics help SMS polls. In addition, citizen feedback was collected using service providers identify gaps and develop interventions, the Interactive Voice Response System (IVRS) under the either as part of a city-wide service improvement plan or MoUD-led city sanitation ratings. These interventions used routine operational reviews. Simple traffic-signal color codes off-the-shelf systems, and did not entail development of facilitate easy inferences on performance levels. Analysis is software tools. They were found to be quicker and cheaper possible for different user groups (for example, slum/non- than MAPI surveys, but with more constraints on data quality: slums) and geographic units (for example, zones/wards), with they captured a narrower range of data, and it was difficult to maps showing survey results down to individual responses assess the extent to which their findings were representative and dashboards enabling detailed analysis at the ward of the survey population. Based on the SLB-C experience, level. This granularity of information helps citizen groups the various modes of feedback – MAPI, CATI, IVRS and and Councilors to understand service gaps in their areas SMS polls – are seen as complementary approaches having and reference them when asking providers to take action. relevance for different kinds of citizen feedback applications. Comparison across wards can also generate competition Their relative advantages and disadvantages are discussed in to improve performance. Finally, the system provides for greater detail in later sections. Table 2: SLB-C system functionalities Mobile Interface Web Interface Purpose Feedback Collection Survey Planning Survey Management Dashboard Analysis Mobile-based household Question bank; survey Survey data Configuration module and survey planning, configuration management and dashboard to view survey for a specific survey task monitoring results √  Conduct survey in local language √  Develop a survey form for an area  √  View real-time survey data √  A ‘dashboard’ provided for with the help of a ‘question bank’, in tabular and spatial form each survey task, with the √  Conduct in ‘training’ and ‘live’ mode either editing an existing survey survey analytics in graphical, form or creating a new one √  Export survey data in tabular and spatial forms √  Geo-tagging of all locations user-friendly formats for √  Create ‘users’ and allocate them offline use √  Analytics available as SLB-C √   Can work in offline mode, requiring ‘work’ for the survey process metrics, as well as detailed connectivity only for syncing data √  Monitor and review survey analysis, stratified for √  Formulate survey sample and data based on system- location and user categories √  Records time and duration of survey methodology defined validation checks Functionalities √ Records pictures/video √ Dashboards available per √  Define up to three levels of spatial √ View various monitoring zone/ward √  Additional comments can be recorded hierarchy to plan for sampling and reports, performance analysis and progress reports for √ Option to customize √  Integration with ‘survey planning’ and each survey area and each analytics for a specific survey ‘survey management’ allows key functionalities enumerator and supervisor task of:  √  Define access levels to view - Work allocation survey results -  Regular monitoring and reporting  √  Option to upload additional docs/PPTs/ links onto a specific dashboard www.wsp.org 13 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 3. Experience with Implementation of SLB-Connect KEY POINTS • SLB-C was piloted in 2012 in Pimpri Chinchwad Municipal Corporation • Scaled up demonstration took place in five cities across three states • MoUD used the SLB-C platform to undertake City Sanitation Ratings as part of the Swachh Bharat Mission SLB-C has so far been implemented in eight cities across six zz Scaled up demonstration of MAPI surveys (2014-15), states, representing a variety of service delivery arrangements with implementation in five cities (Ajmer, Jabalpur, and implementation environments. These surveys have Jhunjhunu, Raebareli and Varanasi) using an upgraded been conducted at the behest of MoUD, with the exception ICT platform, with functionalities and capacity of Mehsana and Delhi. Feedback has been collected from enhanced to support scaling up; and about 35,000 households till now. For purposes of analysis, zz Repeat surveys using additional modes of feedback (2014- the implementation experience has been categorized into 15), conducted in Pimpri Chinchwad using CATI three phases: systems and SMS polls. zz Pilot phase of MAPI surveys (2012-14), encompassing the proof of concept in Pimpri Chinchwad followed This section provides details of the implementation by implementation in Mehsana and Delhi under approach followed in the various phases, the experience partnership arrangements. In Delhi, the survey was so far, and integration into a national urban sanitation not city-wide, but conducted at local level in two program and World Bank funded project. Survey findings slum settlements; are elaborated in the next section. Figure 4: Implementation of SLB-C PIMPRI-CHINCHWAD - Proof of concept 1.7 million population; 47 water zones Survey in 2012 - 5,200 HHs MEHSANA - 180,000 population; 14 wards Survey in 2013 - 1,400 HHs Partnership with Gates project Delhi- Community level (2 slum settlements) Survey in 2014 - 300 HHs & 600 HHs Partnership with CURE (NGO) PCMC -- Repeat exercise Scaled up demonstration Telephone survey in 2014 - Over 3000 HHs 5 cities in 3 provinces + SMS Pulse survey in 2015 Surveys in December 2014 to April 2015 Approx. 28,000 HHs Integrated with national urban program and World Bank funded project Note: HHs: households; NGO: nongovernmental organization. 14 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 3.1 Pilot Phase households were surveyed in less than a month, spread In 2012, SLB-C was piloted in Pimpri Chinchwad across 14 wards in the city. The survey scope was expanded Municipal Corporation (PCMC), a city with a population to cover sanitation services more comprehensively, and of 1.7 million as per the 2011 census. The city was selected findings were used to inform a service improvement in consultation with MoUD and after confirming that city plan under preparation for the city. In 2014, a survey functionaries were willing to participate. A basic mobile- was undertaken in two slum settlements in New Delhi to-web system was developed to conduct the survey in in partnership with the Centre for Urban and Regional alignment with the SLB-C framework. Throughout the Excellence (CURE), a nongovernmental organization design and implementation process, the Commissioner (NGO). About 900 households were surveyed in one and senior officials from the Water Department and month by volunteers from the local community itself. e-Governance unit were consulted for suggestions and feedback, to ensure their ownership of the initiative and The outputs of the pilot phase were: align the survey questions with the information they need zz Proof of concept for the SLB-C mobile-to-web system; for their decision making. The survey was implemented as a municipality initiative, with enumerators carrying IDs and zz A survey questionnaire on water and sanitation services letters of introduction from the municipal authority. Over and a framework for analysis of survey results, both 60 questions were included in the mobile survey app, with embedded in the mobile-to-web system; the focus primarily on water supply services. Over 5,200 zz A default sampling methodology and implementation households were surveyed within two months, spread approach for conducting surveys, including coordination across 47 water zones – approximately 100 households with city functionaries and political representatives; per zone. zz Limited components for a strategy of dissemination and The survey findings were presented to city officials, and also stakeholder engagement; presented at a workshop attended by elected representatives, zz A partnership model in which the SLB-C platform is local civil society representatives, citizens and the media. used by other agencies to conduct surveys under their Highlights were reported in the local press, and a link projects; and to the survey dashboard was posted on the municipality website. The municipality took on board the relatively zz Demonstrated applicability of SLB-C not just at a city poor feedback it received on water quality and customer level but also at more local level. complaint redressal processes, and intervened to address these issues. In 2013, the city launched a new helpline 3.2 Scaled-up Demonstration facility called SARATHI with more streamlined processes As a follow up to the pilot phase, a scaled-up demonstration for lodging complaints, including online, mobile app and was undertaken with the following objectives: call center, and provision for monitoring and follow up. Importantly, the launch was accompanied by intensive zz Operationalize an ICT system with a higher processing outreach to make citizens aware of the facility. Within six capacity and enhanced functionalities to handle scaled- months, around 150,000 citizens had used SARATHI’s up implementation; channels to lodge complaints or ask for information. zz Validate the scope of the questionnaire and analytical framework for water and sanitation services across a After the pilot in PCMC was successfully completed, wider cross-section of service delivery arrangements; interest expressed by other development agencies led to two more feedback surveys being conducted. In 2013, zz Assess the applicability of the SLB-C approach and a survey was implemented in Mehsana (Gujarat) in systems in urban environments with weaker institutional partnership with the Centre for Environmental Planning capacity and poorer economic profile of residents (as the and Technology (CEPT) University, and funded by cities in the pilot phase had been from more developed the Bill and Melinda Gates Foundation. About 1,400 states); and www.wsp.org 15 Using ICTs for citizen feedback surveys to mainstream demand side monitoring zz Demonstrate use of citizen feedback data to inform conducted the surveys in partnership with local NGOs. planning and project preparation processes. In most cities, the enumerators were youth from nearby colleges and universities, who were trained to use the The scaled-up demonstration took place in five cities across SLB-C tools. City-level dialogs were held to validate three states: Ajmer and Jhunjhunu in Rajasthan; Raebareli and disseminate survey findings, in partnership with and Varanasi in Uttar Pradesh; and Jabalpur in Madhya municipal officials and elected representatives. A flyer was Pradesh. The cities were selected in consultation with MoUD, also prepared, summarizing results in English and Hindi; which then solicited their participation. The choice of cities it was uploaded on the SLB-C website and the link was was guided by two main considerations. Firstly, they needed disseminated through bulk SMS. to be from economically weaker states with diverse service arrangements, to assess the applicability of the approach In parallel to these surveys being conducted, the SLB-C in such contexts. Secondly, they needed to have a project system was redeveloped to increase data processing capacities (existing or proposed) under a GoI-, state- or donor-funded and make them more robust, along with additional program which could demonstrate use of the feedback data functionalities for users to customize tasks. This caused some in project development or performance tracking. Details of challenges and delays in conduct of some of the surveys. the cities and surveys are shown in Tables 3 and 4. However, through close coordination across the teams, the problems were resolved and all survey tasks were completed Over 28,000 households were surveyed between December successfully. The process followed for each city in the scale- 2014 and April 2015. The implementing agency, PRIA, up phase is explained in Box 3. Table 3: City and survey sample details Jabalpur Varanasi Raebareli Ajmer Jhunjhunu City profile Total population 1,069,292 1,597,051 191,056 542,580 118,473 Population in slum areas 45% 19% 23% 20% 4% No. of wards 70 90 31 55 45 Sample details No. of respondents (HHs) 6,693 9,330 3,134 5,500 3,823 Respondents from slum areas 48% 22% 26% 18% 4% Table 4: Service delivery arrangements in surveyed cities City Agency responsible for water supply Agency responsible for sanitation Ajmer, Jhunjhunu Public Health Engineering Department (state agency) Municipal Corporation (Rajasthan) Raebareli (Uttar Pradesh) Municipal Corporation Municipal Corporation Varanasi (Uttar Pradesh) Jal Kal Department (the water department of the Municipal Jal Kal Department (Municipal Corporation) - sewerage Corporation) - Operations and Maintenance functions Municipal Corporation - septic tanks, etc. Jal Nigam (state agency) - capital works Jal Nigam (State agency) - capital works Jabalpur (Madhya Pradesh) Municipal Corporation Municipal Corporation 16 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Box 3: SLB-C survey process followed in the five scale-up cities Preparation zz Introduce the project to city officials and get them on board as partners; zz Conduct scoping assessment to understand status of service delivery, institutional arrangements, and obtain baseline information; and zz Confirm/refine standard SLB-C questionnaire to suit city context. Obtain feedback on questionnaire from city functionaries. Design survey plan and work allocation. Configure survey task, questionnaire and mobile app. Training of survey team zz Train enumerators and supervisors, including on system functionalities. Participation of city functionaries in training and oath-taking by enumerators, and in providing municipal IDs for enumerators. Implementation, monitoring and quality assurance of the survey zz Conduct survey, accompanied by real-time monitoring of data using survey management module. “Thank you” SMS sent to respondents on submission of survey data in mobile app; and zz Online monitoring at three levels: locally, by the supervisor; direct oversight by PRIA (Delhi); and secondary checks by the WSP team (Delhi and Bengaluru). Field validation/visits to monitor survey process and address quality concerns. Synthesis of survey findings zz On completion of the survey, review results on the dashboard and through offline analysis; check for anomalies or unexpected findings; synthesize key findings into a presentation. Sharing of survey findings zz Hold validation meeting with city officials, deliberate on key takeaways and areas requiring interventions. Discuss integration of data in improvement plans/project design; zz Conduct a dissemination workshop in partnership with city functionaries, to share findings with municipal councilors and other stakeholders (for example, NGOs, Civil Society Organizations (CSOs), housing societies, media). Coverage in local media. Flyers prepared synthesizing key findings in the local language; and zz SMS outreach to all survey respondents and councilors, providing a link to flyers and the website. 3.2.1  Engagement of Urban Local Body in the While a similar implementation process was followed in all Scaled-up Demonstration of SLB-C cities, and the mandate from the Ministry provided a valuable Local functionaries in the five cities were engaged at all entry point for initiating dialog, the level of involvement stages of the survey process. Before initiating the survey, and response of city functionaries and stakeholders varied the questionnaire was discussed with the service providers, considerably, as illustrated in Figure 5. In some cases (for their inputs were solicited and they were encouraged example, Ajmer, Raebareli), the level of involvement also to participate. Survey results were shared with them for shifted as implementation progressed. Factors influencing validation prior to public dissemination. City-level sharing the response included: was undertaken with the support of the municipality. Table 5 provides details of the various points of engagement and zz Extent to which the municipality was responsible and factors used to assess responsiveness of the municipality. accountable for service delivery; www.wsp.org 17 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Table 5: Assessment of responsiveness of municipality Points of engagement and assessment parameters for ULB responsiveness to SLB-C 1. Willingness to initiate feedback process 2. Ownership of and involvement in implementation of survey - Inputs for scoping study - Nodal officer appointment - Inputs on questionnaire - “Thank you” SMS to respondents on behalf of ULB including a helpline number of the municipality - Memorandum of Understanding with NGO/external agency conducting survey - Participation in training (oath taking) - Provision of ID cards to enumerators Points of engagement and assessment parameter for responsiveness of ULB to SLB-C 3. Interest in inputs received from citizen feedback - Validation meeting – participation and interest - City-level dissemination workshop – participation and ownership of findings - Providing inputs for flyer (details of development schemes, contact person) - Providing link to SLB-C findings on ULB website - Sending SMS disseminating survey findings to citizenry and officials - Exploring use of SLB-C findings for plans and proposals being developed by ULB - Exploring applicability of SLB-C findings for new accountability initiatives by ULB - Exploring applicability of SLB-C process for citizen engagement initiatives by ULB - Institutionalizing SLB-C process in ULB developmental framework zz Degree of cohesion or dissent among local political Figure 5: Response of municipal functionaries to leaders and parties, and their working relationship with SLB-C city administrators; JBP 80 zz Orientation of the city manager (Commissioner) and 70 local leaders to participatory processes; 60 50 40 zz Local survey partner’s capacity and working relationship JJN 30 VRN Willingness 20 with city authorities; 10 Ownership & 0 involvement zz Preparation of service improvement plans or proposals Interest in findings in the pipeline; and zz Familiarity and comfort of municipal officials with use AJM RBL of ICTs. Note: JBP: Jabalpur, VRN: Varansi, RBL: Raebareli, AJM: Ajmer, JJN: Jhunjhunu. 18 Using ICTs for citizen feedback surveys to mainstream demand side monitoring In particular, where service delivery is a municipal infrastructure interventions took place through a state agency responsibility (for example, Pimpri Chinchwad, Jabalpur), (for example, Varanasi). the pressures for accountability were more localized (for All cities accepted the survey findings. Two provided a link example, through municipal councilors) than where it is the on their municipality website to the findings, and most cities responsibility of the state department (for example, Ajmer). also integrated the findings in proposals/plans submitted The response was also greater in cities where service providers for funding under the national urban programs AMRUT, were better capacitated to take action (for example, Pimpri Smart Cities and Swachh Bharat. Jabalpur and Varanasi also Chinchwad) than those where weak local capacity meant initiated immediate measures to review and address specific there was greater dependence on the state government, or areas of concern, for example, poor water quality. Oath taking with Municipal Officials in Varanasi; enumerators conducting the SLB-C survey in the field. City-level sharing at Jabalpur and Raebareli with Mayor, Commissioner and Ward Councilors. SLB CONNECT JABALPUR CITIZEN FEEDBACK SURVEY ON WATER SUPPLY AND SANITATION SERVICES IN JABALPUR CITY, MADHYA PRADESH SLB Connect is an initiative of Ministry of Urban Water supply and Sanitation services in Development that captures citizen feedback on basic services Jabalpur city are provided by the Jabalpur using mobile to web systems. It complements the Ministry’s Municipal Corporation. Service Level Benchmarks programme according to which the city reports its performance levels of basic services. CITY PROFILE Population Households Slum households Wards Municipal area 10,69,292 2,22,613 96,754 79* 106.19 Km2 *SLB Connect survey is conducted in 70 wards based on recent delimitation in 2014 SAMPLE Sample Female Respondents willing Survey duration: Total sample Sample households households to give feedback in respondents March to April 2015 HHs: 6636 Non-slum 52% Slum 48% future 99% 45% OVERALL SCORE - WATER SUPPLY OVERALL SCORE - SANITATION 0 100 0 100 0 100 0 100 CITY SLUM CITY SLUM City Performance 59.4% 54% Slum Performance City Performance 64.4% 61.9% Slum Performance HIGHLIGHTS WATER SUPPLY SANITATION HHs connected to piped supply: 94% HHs have access to toilet facilities and 100% of them use toilets Use as main source 56% Access to toilet facilities in 40 wards Water supply adequate 78% is greater than the city average of 94% Convenience of timing 44% <2% HHs have sewerage connection HHs facing water scarcity during summers 43% 14% in open drains HHs drain toilet waste Never received dirty water 56% None of the HHs reported water meter 8% Complaints redressed within one day 7% Dissemination in the local media and results flyer. www.wsp.org 19 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 3.3 Additional Modes of Feedback – Cati Survey SLB-C parameters and additional aspects requested by and Sms Polls PCMC (for example, interest in receiving 24x7 water supply). Concurrent to the scaled-up demonstration of MAPI surveys The firm used a database of telephone numbers collected described above, at the request of PCMC, repeat surveys from respondents during the MAPI survey of 2012; due to a were conducted in the city to assess service levels after the sizeable degree of attrition, these needed to be supplemented first survey in 2012. These surveys were conducted using by additional numbers obtained from PCMC’s database. other modes of feedback: telephone calling surveys (CATI Figure 6 provides process details of the telephone call survey. systems) and SMS polling. The implementing agency, The use of the introductory SMS from the municipality pManifold Business Solutions Pvt. Ltd., implemented these helped familiarize respondents with the survey, and reduced in close coordination with PCMC’s e-Governance Cell and the time the call agent needed to spend on initial explanation. Water Department. While the telephone survey required fewer resources than the MAPI survey, some challenges were faced: The CATI survey was conducted in September and October 2014 by three to four trained call agents, collecting feedback zz It was difficult to validate respondents’ profile from 3,200 households spread across six administrative information, in particular their habitation status (slum zones.3 The questionnaire had 42 questions relating to the vs non-slum area); Figure 6: process of telephone call survey in Pimpri Chinchwad SMS from PCMC Introductory SMS sent by PCMC to respondents Call a contact number Permanent Failure Temporary Failure - If invalid number, or number does not exist, then stop. - If so, then attempt once again next day, if not repeat following day, then stop. If call is connected Valid Past Respondent Valid New Respondent from same Valid New Respondent (from Invalid Respondent Number - -Validate identity based on few HH PCMC area, but not a previous relocated out of PCMC area. questions from past Validate identity based on few - respondent HH) Stop. - Use questionnaire Q1 questions from past - Use questionnaire Q2 - Thank you SMS sent Use questionnaire Q1a - Thank you - - Thank you SMS sent SMS sent 3 The duration was longer than anticipated due to coordination delays in obtaining telephone numbers from the PCMC database, and testing of an incentive component which complicated the sampling protocols for the call campaign. 20 Using ICTs for citizen feedback surveys to mainstream demand side monitoring zz Questions on toilet access and use were not well received iii. City-wide, trend questions using fixed and rotating – call agents had to be specially trained to solicit the panels (one each). right response for these questions; and Responses were obtained in the form of SMS coded options zz A sizeable share of telephone numbers were on the or SMS free text. The process is show in Figure 7. Telecom Regulatory Authority of India’s Do Not Call registry, which has been set up to prevent unwanted Over 960 valid responses were received, representing a telemarketing calls and SMS. These numbers required response rate of 4-5 percent, which is in line with industry alternative arrangements as they cannot be called from norms for SMS polls; however, significant variance was commercial call centers without explicit permits. observed across the different polls. Useful learnings include: The survey feedback showed a few areas of concern, such as zz Most responses were received within the first 72 hours. poor water quality and problems in billing. To track these This suggested that the period for collecting responses further, a series of SMS polls were conducted in June and should be kept short – typically two to three days – as July 2015, using seven questions decided in consultation interest tends to flag thereafter; with the PCMC Water Department. These were translated zz There was a significant probability of consumers into the local SMS language (Marathi in English text), and not understanding a 1-5 rating scale. Better responses their length limited to 160 characters (including response could be achieved by using Y/N options or multiple options) to be accommodated as a single message. Twenty choice questions; polls were conducted (22,000 SMS messages sent) over two months. Polling was conducted in the following ways: zz For trend questions, rotating panels showed higher response rates than fixed panels. Where fixed panel i. Zone-specific, one-time questions (four); polls were used, response rates tended to drop after two ii. City-wide, one-time question (one); and rounds of polling; Figure 7: Process of SMS poll survey in PCMC Contacts selected Poll designed Reviewed Responses Poll question Welcome SMS collected sent to contacts sent to contacts Thank you Random follow- Analysis & SMS sent to up for feedback, reporting respondents checks www.wsp.org 21 Using ICTs for citizen feedback surveys to mainstream demand side monitoring zz Questions on consumers’ daily routine (for example, of the male head of house. Table 6 summarizes the points water quality, timing) seemed to elicit a higher response of comparison. than those on infrequent occurring events (for example, complaint redressal, bill payment); and Based on insights gained through the implementation experience, PCPM decided to operationalize a Customer zz To increase response rates, awareness creation efforts Services Monitoring Cell which could periodically undertake need to be made before and during the survey through such feedback processes, and analyze data generated from the outreach campaigns (via radio, posters, newspaper city’s customer complaints unit. advertisements, public events). ULBs have an important role to play in this. 3.4 National Dissemination and Mainstreaming Throughout the SLB-C program implementation, dialog The experience in PCMC allowed SLB-C to compare three was maintained with sector practitioners and partners and modes of feedback collection: the MAPI survey of 2012 an Advisory Committee was formed, consisting of senior (which provided detailed baseline feedback), the CATI representatives from the government, think tanks, NGOs survey in 2014 (a rapid follow-up assessment), and the and international development agencies. On completion SMS poll in 2015 (for further tracking of specific zones of the scaled-up demonstration, a national workshop was and service issues). One interesting observation was that held in November 2015 to share learnings and experiences: the share of female respondents fell from 67 percent in the “Smart Urban Services: Putting Citizens at the Center – 2012 survey to 22 percent in the 2014 survey; this could Improving Urban Water Supply and Sanitation Services be because mobile numbers provided were typically those through Citizen Participation”. Table 6: Comparison of three feedback surveys in Pimpri Chinchwad MAPI (2012) CATI (2014) SMS Poll (2015) Method Trained field surveyors Trained telephone surveyors conduct phone Scheduled SMS polls through conduct in-person interviews following initial SMS. SMS gateway system. Initial interviews. message sent by ULB. Respondent group Preselected group. Low drop-out Limited preselection. Higher drop-out rates. Uncontrolled group: participation rates. Female respondents: 67%. Female respondents: 22%. completely voluntary. Female respondents: unknown. Significance Used for in-depth data Used for limited data collection Used for real-time feedback collection on respondent within cost constraints. In-depth on specific aspects of service profile and on different study possible, subject to delivery. In-depth feedback service parameters. availability of prior information on not feasible. respondents. Questionnaire Out of 69 questions, Out of 42 questions, respondents Out of a long list of 37 respondents had to answer had to answer an average of 33 questions, 7 were shortlisted an average of 58 questions applicable to questions. Most profile questions by the ULB. them, based on questionnaire logic. were prefilled as respondents came from previous survey database. Interview time Average 12.5 minutes per Average 8.1 minutes per interview interview (median 9.0 (median 7.2 minutes) minutes) 35% less time required to complete surveys, compared to MAPI. Cost for respondent Nil Nil SMS charges as applicable. Duration of data Over 2 months, data Over 2.5 months, data collected on Polling done over a period of collection and peak collected on 45 days with 19 59 days with 4 surveyors at peak. 62 days. 0.25 surveyor surveyor strength surveyors at peak. used. Surveyor productivity 5,208 validated surveys 3,200 validated surveys collected. Average 965 valid responses collected. Average surveyor surveyor productivity of ~17 completed surveys collected (approximately productivity of ~12 per day (peak performance of 28).4 22,000 SMS messages sent). completed surveys per day 30-50% less effort required for similar completion (peak performance of 14). rates/duration, compared to MAPI. 4 Additional time and resources were required due to multiple sub-groups, which complicated survey planning and execution. 22 Using ICTs for citizen feedback surveys to mainstream demand side monitoring MoUD expressed interest in using the SLB-C platform event on February 15, 2016. To demonstrate transparency, to undertake City Sanitation Ratings, or the ‘Swachh the results dashboard and field observations collected for Survekshan’, as part of the Swachh Bharat Mission. Informed each city (including photographs) were made publicly by the SLB-C experience, two features distinguished this accessible online (https://gramener.com/swachhbharat/)7. round of city sanitation ratings from earlier exercises (i) The Ministry plans to undertake these city sanitation ratings extensive use of ICTs for data collection by independent annually, and expand the scope to cover 500 cities in the observers5 and results analysis; and (ii) inclusion of citizen next round (2017). The approach has also been extended to feedback as an additional, third part of the assessment the rural sanitation program, with a ratings exercise (Swachh process.6 Following SLB-C’s integrated ICT-based approach Survekshan - Gramin) conducted in August 2016 covering improved the speed, scale, accuracy, data integrity and 75 districts across the country (https://gramener.com/sbm/). transparency of the process. SLB-C also helped inform the Ministry’s thinking on the use of citizen feedback and SLB-C has also been integrated into the design of the demand-side metrics to assess service levels and facilitate World Bank-supported Karnataka Urban Water Supply citizen engagement. Modernization Project (KUWSMP), to help track service improvements in a more transparent and citizen-centric Data was collected from 73 cities in a record time of two weeks manner and thereby foster greater stakeholder trust in January 2016, including field observations and feedback in the project. This is in line with recent World Bank from over 80,000 respondents (at least 1,000 responses per guidelines mandating beneficiary feedback surveys in city) using IVRS. The ratings were announced at a national all projects. Active participation by sector experts, elected representatives, private sector representatives, service providers and government officials in the national workshop. 5 110 field assessors collected data from 3,000 locations across the 73 cities. The data were monitored by a 24x7 control room manned by a 20 member team. 6 The other two sources being data provided by the municipality (self-reporting), and data collected by independent observers sent to the city (field observations). Earlier rounds of city sanitation ratings had relied on field observations and data collected from secondary sources. 7 Further details on survey process and results available at http://qcin.org/swacch_survekshanp.php www.wsp.org 23 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 4. Survey Findings KEY POINTS • SLB-C surveys provided concrete and relevant data on the status of water and sanitation services • The data inform the cities in their preparation of proposals for service improvement under national and state urban programs • The ward-level analysis, along with spatial maps of survey results, used to identify areas with service gaps where improvements are needed Citizen feedback surveys under the SLB-C project have been used to identify localities where improvements are covered about 35,000 households across eight cities. These needed. The following sections summarize key findings provided concrete and relevant data on the status of water from these surveys. and sanitation services, which helped inform the cities in their preparation of proposals for service improvement 4.1 Key Findings of Mapi Surveys in Pilot Phase under various national and state urban programs. The ward- Table 7 summarizes the key findings of the pilot phase of level analysis, along with spatial maps of survey results, has SLB-C, in Pimpri Chinchwad, Mehsana and Delhi. Table 7: Survey findings of MAPI surveys in pilot phase OVERALL SCORES: Water supply PCMC (2012) Mehsana (2013) Delhi Slum (2014) Delhi Slum (2014) (Nursery basti) (Safeda basti) ACCESS (% with individual/shared connections) 83% SLB 91% SLB 0% 1% 78% 90% CONTINUITY (median value of municipal piped sources – hours x days) 3x7 6x7 1x7 1x7 2x7 2x7 ADEQUACY (% users of municipal piped and non-piped sources reporting adequate supply to meet household requirements) QUALITY (% users of municipal piped and non-piped sources reporting no dirty water supply in last 3 months) 52% 99% 78% 90% 34% 42% COMPLAINTS (municipal piped and non-piped sources) 19% 60% 5% 80% 0% 2% OVERALL SCORES: Sanitation TOILET ACCESS (% HHs reporting access to toilets, figure in brackets for individual toilets) 98% (75%) 90% 90% (86%) 80% 63% (4%) 78% (3%) ACCESS TO SEWERAGE (% HHs reporting connection to sewer network) 96% 79% 48% 70% 5% 18% ALTERNATE DISPOSAL (% HHs reporting disposal of wastewater to onsite facility) 6% N.A. 50% N.A. 17% 2% Note: As shown in Table 1, SLB and SLB-C performance indicator definitions are the same for access, continuity and complaint redressal, so scores are directly comparable. However, for adequacy and water quality, the definitions are different and, hence, the comparison of values is only indicative; HHs: households. 24 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 4.1.1 Survey Findings in Pimpri Chinchwad – partly attributable to greater dependence on stand zz Overall feedback: Feedback was favorable on access to posts. Users of stand posts reported higher incidence household piped water supply (83 percent) and adequacy of dirty water supply (73 percent) compared to users of supply (84 percent), but relatively poor for quality of of piped supply from house connections (48 percent), water (52 percent), complaint redressal (19 percent) and and lower complaint resolution (46 percent, compared ease of bill payments (52 percent). Most respondents (53 to 65 percent). Gaps were also evident in the feedback percent) felt services were the same as the previous year, on continuity. While the vast majority of respondents while about 30 percent felt it was better. for the city (92 percent) reported daily water supply, the figure was significantly lower in case of slum areas (79 zz Alignment with SLB (reported) data: Feedback showed percent). Wide variations were seen in feedback across alignment on access, adequacy and level of metering, but water zones. For example, values for duration of supply deviations were observed in case of continuity of supply, ranged from over five hours of daily supply (three zones), water quality, complaint redressal and bill payments. to four hours of supply twice a week (two zones); for zz  quity in service delivery: While overall service levels E water quality, values ranged from 100 percent (three were largely similar between slum and non-slum areas, zones) to almost zero (three zones). further analysis showed some differences. Dependence zz Specific feedback: Only about 15 percent of the on piped water from house connections was lower in respondents had made any complaint about water service slum areas (64 percent) compared to other areas (72- in the last year and about 74 percent of these had been 79 percent), while dependence on shared connections/ lodged through local councilors. Very few respondents stand posts was greater (see Table 8). Slum areas also were clear about billing frequency. Most (82 percent) reported higher incidence of dirty water supply (55 paid their bills at the municipal counter but only about percent) compared to non-slum areas (47 percent) 40 percent found its location convenient. Table 8: Survey Findings of PCMC 2012 SLB-C Survey Primary sources of municipal water supply for drinking and other purposes City Slum Non-slum HH connection % 72-78 64 72-79 Shared connection % 5 23 3-4 Stand post % 2 12 1 Feedback on complaint redressal systems Respondents who - lodged complaints 15% - reported resolution of complaints 65% Mode of lodging complaint Corporator -74% Personal/group visit – 18% No. of days to resolve (median) 4 Feedback on access to toilets City Slum Non-slum Own toilet % 75 18 80 Shared toilet % 8 4 8 Community toilet % 16 67 10 Open defecation % 2 11 1 Note: HH: household. www.wsp.org 25 Using ICTs for citizen feedback surveys to mainstream demand side monitoring PCMC survey process. The Water Department at PCMC was concerned about the was lower in slum areas (69 percent) than non-slum service aspects on which citizens’ feedback did not match areas (81 percent). their own assessment, water quality and complaint redressal. zz Specific feedback: As with PCMC, only 12 percent Granular analysis of SLB-C data (at the water zone level) respondents reported having lodged a complaint, of helped them pinpoint areas most affected by poor quality of which a mere 27 percent reported resolution (and only 5 water supply. The survey also highlighted that, despite the percent within one day). However, unlike PCMC, over availability of multiple complaint channels (helpline, SMS, 85 percent of the complaints had been lodged through a helpdesk), citizens preferred to use informal channels (mainly formal channel (mainly, the helpdesk). Of those lacking councilors). Recognizing the need to create awareness about and improve the responsiveness of the formal complaint a house connection, 35 percent indicated the main channels, the municipal administration decided to launch a reason for not having a connection was that they were new helpline facility, “SARATHI”, which received a good ‘not eligible’. response from the public. Sanitation zz Overall feedback: While the overall score on toilet 4.1.2 Survey Findings in Mehsana access was good (86 percent), access to sewerage was low Water Supply at 48 percent, with about 50 percent of the respondents zz Overall feedback : Survey feedback was once again reporting disposal of wastewater to onsite systems. relatively favorable on access to household piped water supply (91 percent) and adequacy of supply zz Alignment with SLB data: The gaps with reported (78 percent), but relatively poor for water quality (78 figures were substantial. While 86 percent of respondents percent), continuity (daily, one hour supply), and reported access to toilets, the corresponding figure complaint redressal (5 percent). Most respondents reported by the ULB was just 59 percent. Interestingly, (46 percent) felt services had improved compared to the citizen feedback data more closely approximated the the previous year, while about 23 percent felt it had 2011 census figures than the municipality’s SLB data. worsened. zz Equity in service delivery: Sharp inequities were evident zz Alignment with SLB data: Citizens’ feedback largely in access to toilet facilities. While citizens in slums matched the city’s reported levels of service for access, reported access at only 56 percent, the figure was 95 adequacy and continuity, but significant gaps were percent in non-slum areas; one-third of the respondents evident for water quality and complaint redressal. from slums were resorting to open defecation. Less than half of the toilets in these areas had a water connection zz Equity in service delivery: Access to household piped (compared to 88 percent in non-slum areas). water supply in slum areas (77 percent) was significantly lower than in non-slum areas (95 percent). Similarly, the zz Specific feedback: 70 percent of users of municipality- share of respondents indicating adequate water supply run community toilets found these to be ‘never’ 26 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Table 9: Survey findings in Mehsana (2013 SLB-C survey) Primary sources of municipal water supply for drinking and other purposes City Slum Non-slum HH connection % 86-89 68-73 92-93 Shared connection % 1-2 3-6 - Stand post % 4 16-18 1 Feedback on complaint redressal systems Respondents who   - lodged complaints 12%   - reported resolution 27% Mode of lodging complaint Corporator - 7% Helpdesk - 83% No. of days to resolve (median) 6 Feedback on access to toilets City Slum Non-slum Own toilet % 86 55 95 Shared toilet % 1 2 1 Community toilet % 3 10 1 Open defecation % 10 33 3 Note: HH: household. Survey process at Mehsana 2013. maintained, compared to 30 percent for toilets managed 4.1.3 Survey Findings in Delhi (Slum by private contractors. Despite high dependence on Settlements) onsite facilities for disposal of wastewater, almost half the The survey was conducted in two slums in Delhi, and respondents reported that these had never been cleaned. highlighted the service gaps experienced by those living in Of the rest, cleaning was reported to have been done impermanent and informal settlements. through informal arrangements such as local private zz Access to water supply: There was negligible access to contractors or labor. household piped water supply, with 78-86 percent of The survey findings were presented to the municipal respondents reporting primary dependence on public authorities, and helped them identify gaps in coverage and stand posts for drinking water. Primary sources for other service quality. The information was incorporated in the purposes were stand posts (50-60 percent) and hand preparation of a service improvement plan that was under pumps (35-44 percent). About 40 percent of respondents development at the time. reported that the public water point was more than 100 www.wsp.org 27 Using ICTs for citizen feedback surveys to mainstream demand side monitoring meters away, and entailed a waiting time of more than 15 the SLB-C system, covering 3,120 households spread across minutes. The two main reasons given by respondents for 34 slums under a United States Agency for International not having individual or shared connections were ‘lack Development (USAID)-funded project to develop a long- of eligibility’ and ‘lack of piped network’ in the area. term Water, Sanitation and Hygiene (WASH) program for low-income urban settlements in Delhi. zz Quality of service: 40-50 percent of respondents reported that supply timings were rarely predictable and Key Findings of Mapi Surveys in Scaled-up 4.2  33 percent found the timings to be inconvenient. Only Demonstration 34-40 percent respondents reported no incidence of dirty Under the scaled-up demonstration phase, MAPI surveys water supply in the previous three months, while 27-33 were implemented between December 2014 and April percent reported more than three such incidents. Over 2015 in five cities (Jabalpur, Varanasi, Raebareli, Ajmer 50 percent of respondents were impacted by scarcity and Jhunjhunu) spread across three states. Over 28,000 during the dry season, of which about 15-20 percent households were surveyed. was forced to shift to non-regular sources to meet their requirements. Only 4-8 percent of respondents had The profile of respondents varied across the five cities (see lodged a complaint for water supply problems, mostly Table 10). The share of respondent households with below through a local NGO or community leaders. the poverty line (BPL) status was highest in Jabalpur at 31 zz Access to sanitation facilities: 60-75 percent respondents percent, while it was 4-12 percent in the remaining cities. reported dependence on shared or community toilets, Similarly, the share of respondents from slum areas was while 22-37 percent resorted to open defecation. Of highest for Jabalpur (48 percent), while it was 18-26 percent those using community toilets, 40-60 percent reported in Ajmer, Varanasi and Raebareli; the figure was much smaller these to be more than 100 meters from their homes. in Jhunjhunu (4 percent). Over 90 percent of respondents in all cities lived in pucca houses, except Jabalpur where the The implementing agency (CURE) shared the survey findings figure was much lower (73 percent). Most of the respondents with the slum dwellers, through posters and local meetings, across the five cities (42-51 percent) were 30-49 years old. which they also used to inform their dialog with Delhi The share of female respondents was lower than male in all Jal Board on possible service improvement actions. More cities, ranging from a high of 45 percent in Jabalpur to 28-30 recently (in 2015), CURE undertook a larger survey using percent in Jhunjhunu, Raebareli and Varanasi. Table 10: Sample profile details Jabalpur Varanasi Raebareli Ajmer Jhunjhunu No. of respondents (HHs) 6,693 9,330 3,134 5,500 3,823 HHs from slum areas 48% 22% 26% 18% 4% Female respondents 45% 32% 23% 38% 28% Economic profile APL- 22% 90% 82% 91% 91% BPL- 31% 4% 8% 7% 7% No card- 47% 6% 8% 2% 2% Type of dwelling Pucca- 73% 99% 96% 99% 97% Kuccha- 26% 1% 4% 1% 3% Flat- 1% - - - - Note: APL: above the poverty line; HHs: households. 28 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Survey Findings for Water Supply in Scaled-up Demonstration 4.2.1  The key survey findings across the five cities are listed in Table 11. Table 11: Survey findings in the five cities in the scaled-up demonstration – water supply OVERALL SCORES Jabalpur Varanasi Raebareli Ajmer Jhunjhunu City level SLB-C Score 59.4 57.3 58.3 71 71.8 Slum SLB-C 54.0 50.2 56.3 69.3 11.8* Non-slum SLB-C 64.9 59.1 59.0 71.4 72.5 ACCESS (% with individual/shared connections) City level SLB 68 69 44 82 75 City level SLB-C 55.5 64.3 37.2 96.8 84.5 Slum SLB-C 44.8 46.6 32.8 92.4 1.5 * Non-slum SLB-C 65.3 69.4 38.7 97.7 87.5 CONTINUITY (median value of municipal piped sources; hours x days) City level SLB 2x7 10 x 7 3x7 0.75 x 7 0.5 x 7 City level SLB-C 2x7 4.5 x 7 4x7 1x4 0.4 x 7 Slum SLB-C 2x7 4.5 x 7 4x7 1x4 0.3 x 4 Non-slum SLB-C 2x7 5x7 4x7 1x4 0.4 x 7 ADEQUACY (% users of municipal piped and non-piped sources reporting adequate supply to meet household requirements) City level SLB 135 lpcd 275 lpcd 121 lpcd 100 lpcd 70 lpcd City level SLB-C 77 82 94.8 81 58.5 Slum SLB-C 70.4 79 93.8 80.3 28* Non-slum SLB-C 84 82.7 95.3 81.2 58.6 QUALITY (% users of municipal piped and non-piped sources reporting no dirty water supply in last 3 months) City level SLB 98 96 96 78 60 City level SLB-C 55.4 26.8 58.3 79.6 93.7 Slum SLB-C 51.5 23.7 51.5 77.8 92.9* Non-slum SLB-C 59.5 27.6 61.6 80 93.7 COMPLAINT REDRESSAL (municipal piped and non-piped sources) City level SLB 80 95 96 82 82 City level SLB-C 6.8 1.5 1 6.6 2 Slum SLB-C 5.4 2.6 2.2 12.4 0* Non-slum SLB-C 8.8 1.2 0 4.9 2 * These findings are not considered salient since the number of slum households covered in the Jhunjhunu survey was very small, due to a small share of slum households in the city; lpcd: liters per capita per day. Overall feedback and Varanasi. In the latter case, this was attributable zz The overall score for cities from Rajasthan was marginally to significant inequities in access levels and, to a lesser higher compared to the other cities, primarily due to extent, in scores for adequacy and water quality; high performance on the access indicator, which offset zz The performance levels reported by cities (SLB the relatively poor performance on continuity; scores) were roughly in alignment with citizen zz Overall performance for water supply was largely feedback (SLB-C scores) on most indicators, with the similar between slum and non-slum areas for Ajmer exception of quality and complaint redressal, where and Raebareli, but more differentiated in Jabalpur the gaps are substantial; www.wsp.org 29 Using ICTs for citizen feedback surveys to mainstream demand side monitoring zz Most respondents (63-85 percent) in all cities felt that Jabalpur, 25 percent for hand pumps in Raebareli, and the services had remained the same as in the previous negligible in remaining cities; year; 25-30 percent respondents in Ajmer, Jabalpur zz Eleven percent to 50 percent of respondents across and Raebareli felt that services were better than in the five cities reported primary dependence on private the previous year. Not much difference was observed covered sources (mainly bore wells) for drinking or in these responses between slum and non-slum other purposes, the figure being highest in Raebareli (50 areas; and percent) and Varanasi (34 percent). The spatial maps zz In terms of priority areas of service delivery, water quality for Jabalpur and Varanasi show greater dependence on scored highest (80-99 percent) in Varanasi, Raebareli private covered sources in the peripheral areas of the city; and Jabalpur, while pressure and adequacy were rated zz Roughly half the respondents reported incidence of water high in Ajmer and Jhunjhunu. scarcity in the summer months, with the figure being as More detailed observations on each of the service aspects are high as 67 percent in Ajmer and as low as 5 percent in provided below. Raebareli. During scarcity, respondents reported high dependence on private covered sources and government Access: The key findings are listed below and tabulated in hand pumps; and Table 12. zz Amongst those accessing municipal piped sources zz Respondents reporting access to water supply through outside the house (for example, stand posts) or non- household connections ranged from a high of 85-97 piped sources (for example, tankers), over half the percent in Ajmer and Jhunjhunu to 37% in Raebareli; respondents indicated willingness to take household connections. The figures ranged from 50-67 percent (the zz Feedback data showed access levels to be higher than exception being Jhunjhunu, where the figure was 100 reported (SLB) data for cities from Rajasthan, while it percent). The willingness to take household connections was the reverse for the other cities; was greater in case of slum respondents (for example, in zz Dependence on other municipal sources as a primary Ajmer, 75 percent slum areas versus 43 percent for non- source of water was low: 29 percent for stand posts in slum areas). Table 12: Access to water supply: SLB-C survey findings in five scale-up cities Primary source of water supply for drinking purposes Type of Source Jabalpur Varanasi Raebareli Ajmer Jhunjhunu Municipal HHC: 55% HHC: 59% HHC: 25% HHC: 97% HHC: 66% Stand post: 29% Hand pump: 5% Hand pump: 25% Non-municipal 14% (11%)* 34% (29%)* 50% (50%)* 2% 33% (11%)* Primary source of water supply for other purposes Type of source Jabalpur Varanasi Raebareli Ajmer Jhunjhunu Municipal HHC: 42% HHC: 61% HHC: 37% HHC: 94% HHC: 84% Stand post: 25% Hand pump: 15% Non-municipal 27% (23%)* 32% (28%)* 48% (47%)* 4% 16% (10%)* *(Private covered source figure in brackets). Note: HHC: Household Piped Connection. 30 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Continuity different parts of the cities, as observed from the zz Feedback indicated daily water supply in all cities except geographic spread of responses. Spatial maps (see Figure Ajmer, where citizens reported four days of supply per 8) reveal a clustering in the responses for days and hours week. On days when water is supplied, hours of supply of supply, suggesting inequities in the distribution of varied from 0.4 to 1 in Jhunjhunu and Ajmer, to as supply across the city, which is also reflected in ward much as four to five hours in Raebareli and Varanasi; level scores; and zz Comparisons with SLB data show some variations – zz In all cities, respondents reported high levels (85-97 most significantly in Ajmer (four days versus seven days percent) of predictability and convenience in the timing as per SLB data), and in Varanasi (4.5 hours versus 10 of supply. In Varanasi, the score for predictability was hours as per SLB data); lower (84 percent) than for convenience of timing (91 zz The responses for duration of supply differed between percent), while it was the reverse in the case of Ajmer (97 slum and non-slum areas by a modest 7-8 percentage percent and 86 percent, respectively). points except in Raebareli, where the difference was more marked – about 20 points. Variance exists across The survey findings for continuity are illustrated in Figure 8. Figure 8: Continuity of water supply: SLB-C survey findings across the five scale-up cities FREQUENCY OF SUPPLY (PIPED SOURCES) Days Per Week Hours of Supply Per Day 96.8 0 0.4 JBP 1.3 1 JBP 2 57.1 29.2 0.9 11.3 0.2 96.2 9.3 VRN 2.7 VRN 32.5 54.5 0.6 2.9 0.5 0.7 0 99.5 0.9 0.5 RBL 36.9 RBL 0 2.8 59.3 0.1 0 0 0 1.9 12.9 AJM 0.1 2.8 AJM 85.1 15 82.1 0.2 0.1 0.9 4.2 89.8 JJN 2.9 13.9 JJN 5.2 29.3 53 0.7 0 50 100 0 50 100 Over 20 Hours 10-20 Hours 5-10 Hours 7 Days 5-6 Days 3-4 Days 1-2 Days 2-5 Hours <2 Hours <1 Hours Note: JBP: Jabalpur, VRN: Varansi, RBL: Raebareli, AJM: Ajmer, JJN: Jhunjhunu. www.wsp.org 31 Using ICTs for citizen feedback surveys to mainstream demand side monitoring JABALPUR: Frequency of supply – hours of supply per day (piped sources) AJMER: Frequency of supply – days per week (piped sources) 32 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Adequacy (Varanasi, Raebareli and Jabalpur) reported 96-98 zz Overall about 80 percent of respondents reported adequate percent compliance with water quality standards. In supply of water from municipal sources – the outliers contrast, the cities with comparatively better feedback being Raebareli (95%) and Jhunjhunu (56 percent). The reported lower levels of compliance (79 percent for high score in Raebareli could be attributed to the relatively Ajmer and 60 percent for Jhunjhunu). The variance low share of the population reporting dependence on between reported performance and citizen feedback was municipal sources for water. The low score in Jhunjhunu comparatively lower for the cities from Rajasthan than is mirrored in the low quantity of water supplied to the for the other cities; city (70 liters per capita per day (lcpd)); zz The percentage of respondents reporting regular dirty zz While feedback on adequacy is similar for Varanasi water supply (more than three times, or always) was and Ajmer (81-82 percent), and the quantity of water as high as 48 percent in Varanasi and 22 percent in supplied (as per SLB data) differs widely (275 lpcd for Jabalpur; and Varanasi and 100 lpcd for Ajmer). This indicates the zz Variance across slum and non-slum areas was not high level of inefficiencies (water losses) and inequities marked, but was more evident when comparing across prevalent in the distribution system of Varanasi; users of different sources of water. In Jabalpur and Ajmer, zz There was more variance in adequacy levels across frequent incidence of dirty water supply (more than user groups in Jabalpur – between slum (70 percent) three times, or always) was reportedly less for household and non-slum (84 percent) areas, and also across users piped connection than other sources; however, it was accessing different sources of water (64 percent for higher than other sources in Varanasi and Raebareli. outside piped sources versus 85 percent for household piped connections); and Table 13 summarizes the incidence of dirty water supply in the preceding three months across the five cities. zz Ward-level scores showed considerable variation, ranging from 14-100 percent in Jhunjhunu, 35-98 percent in Complaint redressal Jabalpur, and 60-98 percent in Ajmer. zz Overall the share of respondents lodging complaints was Water quality low in all five cities, ranging from 5 percent in Raebareli to a maximum of 13 percent in Varanasi; the percentage zz Overall feedback for quality of water supplied from being 5-7 percent for the remaining three cities; municipal sources was poor in most cities. Only 27 percent of respondents in Varanasi reported no dirty zz Of those lodging complaints, only about a third of water supply in the preceding three months. The figures respondents reported resolution; the figure was just for Jabalpur and Raebareli were 55 percent and 58 17 percent for Varanasi, but higher at 47 percent for percent, respectively, as against 80 percent for Ajmer and Raebareli. One of the factors contributing to this was 94 percent for Jhunjhunu; the nature of the complaint – for example, broken hand pumps (the most common complaint in Raebareli) can zz The above was in marked contrast to performance be more easily addressed than water quality complaints reported by cities. The cities with the worst feedback (most common in Varanasi). Rates of resolution varied Table 13: Incidence of dirty water supply in the preceding three months: SLB-C survey findings across the five scale-up cities Jabalpur Varanasi Raebareli Ajmer Jhunjhunu Never 56% 27.1% 62.6% 79.8% 93.7% Less than three times 23.2% 25.7% 29.3% 14% 5.1% More than three times 15.5% 36.1% 16.5% 4% 1.1% Always 6.3% 12.3% 2.3% 2.5% -% www.wsp.org 33 Using ICTs for citizen feedback surveys to mainstream demand side monitoring between slum and non-slum areas, with no clear trend zz The preferred mode of lodging complaints varied evident; between respondents from slum and non-slum areas. In cities served by the municipal body, the preference for zz The two main types of complaint were irregular water channeling complaints through the municipal councilor supply and water quality (with the exception of Raebareli, was even greater for respondents from slum areas (80 where breakdown of hand pumps was the most common complaint). An overwhelming share of complaints (81 percent in Jabalpur versus 60 percent for respondents percent) in Varanasi was about poor water quality, which from non-slum areas; in Raebareli, 60 percent versus 23 mirrors the aforementioned low score on the water percent); and quality parameter (see Table 14); zz The reported time taken to respond to complaints varied zz In cities where services are delivered by the municipal as follows: corporation, a substantial share of complaints were 40 percent in Ajmer reported that responses took 1.  lodged through the municipal councilor (70 percent over seven days; in Jabalpur, 38-39 percent in Varanasi and Raebareli), 2. 37 percent in Jabalpur and 47 percent in Jhunjhunu compared to the cities where services are provided by reported a response within two or three days; the state department (Ajmer 10 percent, Jhunjhunu 3 percent). Other preferred modes of complaint were 50 percent in Raebareli reported a response in 3.  personal or group visits to the municipal/department between four and seven days; and office, which accounted for 82-88 percent complaints 63 percent in Varanasi reported a response between 4.  lodged in Ajmer and Jhunjhunu, and about 60 percent two and seven days. in Raebareli and Varanasi. The use of helplines, SMS or online channels for lodging complaints was negligible in Table 14 summarizes the key findings in respect to complaint all cities; redressal across the five scale-up cities. Table 14: Complaint redressal for water supply services: SLB-C survey findings across the five scale-up cities Lodging and resolution of complaints across five cities HHs made complaints in the last 1 year Complaints which have been resolved 8.4 29.3 JBP 9.5 JBP 27.6 7.3 33.7 12.4 17.3 VRN 11.6 VRN 25.7 12.6 15.4 6.1 46.5 RBL 8.3 RBL 56.6 5.1 30.6 7.8 34.4 AJM 10.4 AJM 34 7.3 34.6 4.6 22 JJN 7.5 JJN 0 4.6 22.1 % 0 5 10 15 % 0 20 40 60 City Slum Non-Slum City Slum Non-Slum Note: JBP: Jabalpur, VRN: Varansi, RBL: Raebareli, AJM: Ajmer, JJN: Jhunjhunu. 34 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Most common complaints across the five cities Jabalpur Varanasi Raebareli Ajmer Jhunjhunu Irregular water supply 71.3% 44% 12.9% 46.7% 85.3% Poor quality of water 25.2% 81% 30.7% 19.1% 11.3% Breakdown of hand pump - - 50.5% False meter reading - 0.4% - 3.3% - Billing errors 1% 5.5% - 19.6% 0.7% Others 23.3% 9.6% 18.8% 24.1% 9.3% Spatial representation of type of complaints lodged in Varanasi www.wsp.org 35 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Bill payments Satisfaction and comparison with previous year zz Billing is on a fixed charge basis in all cities except Ajmer, zz In all five cities, 90 percent or more respondents felt where 82 percent respondents reported availability of that, compared to the previous year, services had meters but only 52 percent of them reported the meters remained unchanged (60-80 percent) or improved (20- to be functional; 30 percent). Not much variation was evident between slum and non-slum respondents; zz The frequency of billing was reported to be once in two months for Ajmer and Jhunjhunu, while it was annual zz Adequacy of supply featured among the top three in the other cities; priorities for citizens in all five cities. More respondents from slum areas than non-slum areas prioritized zz The mode of bill payment varied across the cities. In adequacy. Water quality was the highest priority in Jabalpur and Varanasi, 93-96 percent of respondents Varanasi, Raebareli and Jabalpur; other important paid their bills at municipal/department counters, in parameters were adequacy and pressure of supply; and Raebareli 56 percent paid through municipal agents, while in Ajmer and Jhunjhunu 58-61 percent of zz Eighty to 90 percent of respondents were either ‘fully’ or respondents paid through the e-Mitra kiosk. Payment ‘partially’ satisfied with water supply services, with the through online channels was negligible; and exception of Jhunjhunu, where the figure was lower (67 percent). The share of ‘fully satisfied’ respondents was zz Over 90 percent of respondents found the location lowest in Varanasi (27 percent). In all cities, this figure of designated counters to be convenient for payment was lower for respondents in slum areas compared to of bills; there was marginal scope for improvement in non-slum areas. Raebareli and Ajmer. Counter timings were reported to be convenient by almost all (over 95 percent) respondents. The findings are summarized in Table 15. Table 15: Satisfaction with water supply services: SLB-C survey findings across the five scale-up cities Comparison of water supply services with previous year across five cities Jabalpur Varanasi Raebareli Ajmer Jhunjhunu Better than before 28% 20% 25% 31% 14% Same as before 63% 67% 73% 57% 85% Worse than before 7% 10% 1% 8% 1% Priorities related to water supply services as expressed by citizens top 4 priorities related to water supply 77.8 66.1 JBP 68.6 42.3 85.8 VRN 51.3 57.9 42.2 99.1 RBL 21.5 87.6 83.9 51 AJM 85.7 77.8 71.8 40.6 62.5 JJN 80.3 43.9 0 20 40 60 80 100 % Quality Pressure Adequacy Regulatory Note: JBP: Jabalpur, VRN: Varansi, RBL: Raebareli, AJM: Ajmer, JJN: Jhunjhunu. 36 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Survey Findings for Sanitation in the Scaled-up Demonstration 4.2.2  The key survey findings across the five cities are listed in Table 16. Table 16: Survey findings in the five scaled-up demonstration cities – sanitation OVERALL SCORES Jabalpur Varanasi Raebareli Ajmer Jhunjhunu City level SLB-C Score 94.1 97.7 82.3 96.2 95 Slum SLB-C 89.7 93.4 55.4 85.9 0.7* Non-slum SLB-C 98.1 98.9 91.9 98.3 98.5 ACCESS TO TOILETS (% reporting access to individual, shared or community toilets) City level SLB 71 82 74 93 65 City level SLB-C 94.1 97.7 82.3 96.2 95 Slum SLB-C 89.7 93.4 55.4 85.9 0.7 Non-slum SLB-C 98.1 98.9 91.9 98.3 98.5 TOILET USAGE (% reporting usage of toilets by all family members) City level SLB - - - - - City level SLB-C 99.6 99.5 99.6 99.7 99.3 Slum SLB-C 99.4 98.7 98 98.7 100 Non-slum SLB-C 99.7 99.8 100 99.9 99.3 ACCESS TO SEWERAGE (% with connections to municipal sewerage) City level SLB - 81 49 0.55 0.70 City level SLB-C 1.7 81.9 48.5 1 0.70 Slum SLB-C 0.5 64.6 16.8 0.6 0.70 Non-slum SLB-C 2.8 86.9 59.7 1.1 0.70 ALTERNATE DISPOSAL**(% reporting wastewater disposal to onsite systems) City level SLB-C 84.3 12.1 34.3 78.8 98.6 Slum SLB-C 77.7 23.5 66.4 78.7 - Non-slum SLB-C 89.5 9.1 32.3 78.9 98.6 *Sample size for slum households is very small **Not used for calculating overall score zz In sanitation, the overall scores for all cities are in a zz The performance levels reported by cities (SLB scores) similar range (94-98 percent) except for Raebareli (82 show wide variation from citizen feedback (SLB-C percent), where the lower score is primarily due to lower scores) on toilet access,9 but they are almost identical levels of access to toilets; for access to sewerage (SLB data are not available for Jabalpur). The level of toilet access reported by cities is zz Overall sanitation scores for slum and non-slum areas lower than indicated by citizen feedback. show divergence in Raebareli and Ajmer, primarily due to differences in toilet access. In Raebareli, the gap is More detailed feedback is provided below on specific widened by variance in access to sewerage between slum service aspects. (65 percent) and non-slum (87 percent) areas;8 and 8 J  hunjhunu has not been considered in this analysis since the number of slum households in the city is very small, and hence so is the corresponding sample size. 9 There is no SLB indicator for toilet usage. www.wsp.org 37 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Access to toilets in Jabalpur, 35 percent in Varanasi and 21 percent in zz Toilets at home are the most common type of toilet Jhunjhunu; facility accessed by respondents (82-95 percent). Less zz Community toilets in Jabalpur are mostly (51 percent) than 2 percent of respondents report use of shared or managed by the Municipal Corporation, and are community toilets (the figure is slightly higher for also mostly free of charge (86 percent). Feedback on Jabalpur, at 4 percent); maintenance is poor, with 59 percent users of community zz In Raebareli, 18 percent of respondents report lack toilets indicating that these toilets are rarely or never of access to any kind of toilet; the figure is higher (45 maintained; and percent) for respondents in slum areas. In other words, zz Of those not having access to a toilet, the main reason nearly half the respondents in slum areas of Raebareli given for not having a toilet at home was that it was have no access to a toilet facility. In other cities, 10-15 too expensive (50 percent in Varanasi to 84 percent in percent of slum respondents reported no access to toilet Raebareli; the figures were higher for slum areas). Other facilities; reasons included insufficient space (all cities except zz The main source of water in toilets is tap water inside Raebareli) and inadequate water supply (all cities except the toilet, with the figure ranging from 41 percent Jhunjhunu). for Jabalpur to 88 percent in Ajmer. A sizeable share of respondents also carry water by hand: 39 percent The findings are summarized in Table 17. Table 17: Access to toilet: SLB-C survey findings across the five scale-up cities Feedback on access to toilets Jabalpur Varanasi Raebareli Ajmer Jhunjhunu City Slum City Slum City Slum City Slum City Slum Own toilet at home 90% 84% 96% 92% 82% 55% 95% 85% 95% 1% Shared toilet 3% 3% 1% 0.5% 0.4% 0.5% 1% 1% 0.3% - Community toilet 1% 2% 0.4% 0.4% 0.2% 0.1% 0% 0% 0% - No toilet facility 6% 10% 2% 7% 18% 45% 4% 14% 5% - Jabalpur – Toilet access (lower access in eastern side of city) 38 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Rae Bareli – Toilet Access (pockets lacking access in north and east of city) Toilet usage (49 percent). The figures are almost identical to SLB zz Almost all respondents (over 99 percent) with access to data reported by the cities; toilets report that all family members use them. Among zz In Varanasi and Raebareli, slum areas report lower levels the few exceptions, it is mainly adult males or young of access to sewerage compared to non-slum areas (65 boys who are reported not to use toilets; and percent and 87 percent, respectively, in Varanasi; 17 zz Reasons for not using toilets range from lack of water percent and 60 percent in Raebareli). Spatial maps also supply to inadequate provision for disposal of wastewater, show that sewerage infrastructure is lacking mostly in the peripheral areas of cities; and lack of ventilation or fear that the toilet facility will fill up (onsite systems). zz Among those not connected, 24 percent of respondents in Ajmer and 33 percent in Varanasi report that a sewer Access to sewerage facility is available near their house. This shows ready zz Citizens report lack of access to sewerage infrastructure potential for improved wastewater collection in the city, in all cities except Varanasi (82 percent) and Raebareli without much need for additional investment. www.wsp.org 39 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Figure 9: Households connected to sewer Varanasi Raebareli Alternate disposal cleaned (30 percent in Ajmer, 55 percent in Jabalpur zz Disposal of wastewater from toilets through alternate and Raebareli, and 86 percent in Jhunjhunu), or are (onsite) systems is high in cities lacking sewerage not aware whether any cleaning had been done (24-33 infrastructure (78-94 percent), and substantial even in percent of respondents in Ajmer and Jabalpur); and Raebareli (48 percent); zz Of those getting their tanks/pits cleaned, municipality zz In Jabalpur and Ajmer, 14-18 percent of respondents services are used most often in Ajmer, Jabalpur and report disposal into open drains, while in Varanasi and Raebareli, followed by use of local labor. Ninety percent Raebareli 4-8 percent dispose into soak pits; either report frequency of cleaning to be less often than zz Despite the extensive dependence on onsite systems, once in two years, or are not aware of how frequently most respondents have never got their pits or septic tanks cleaning is done. Figure 10: SLB-C findings on alternate disposal across the five cities HHs using septic tank soak pit & Disposal of waste water from pit toilets as alternate sewerage disposal toilets to open drain 18.9 6.5 JBP 77.7 JBP 20.5 84.3 14 9.1 1.9 VRN 25.5 VRN 6.5 12.1 2.9 32.3 2.1 RBL 66.4 RBL 3.5 38.4 2.5 79.9 18.8 AJM 78.7 AJM 19.5 78.8 18.5 98.6 2.2 JJN 100 JJN 0 98.6 2.2 % 0 50 100 0 10 20 30 % Non-slum Slum City Non-slum Slum City Note: JBP: Jabalpur, VRN: Varansi, RBL: Raebareli, AJM: Ajmer, JJN: Jhunjhunu; HHs: households. 40 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 4.3 Key Findings of Cati Survey and Sms Polls complaint resolution continued to be low (12 percent). A 4.3.1 Telephone Survey in Pimpri Chinchwad significant shift was that only 28 percent of respondents had As a follow up to the MAPI household survey in 2012, a lodged their complaint through the councilor (as against 74 telephone survey was was conducted in PCMC in 2014. The percent in 2012), with a larger share (60 percent) routing findings showed a trend towards improvement in water and their complaints directly to the zonal/city engineer. sanitation services on some attributes, and a slight decline in others. Responses were stratified across six administrative Considerable variance was evident in performance across zones (as against 47 water zones in the 2012 survey), as zones. Analysis showed that zones C and B needed most telephone numbers obtained from PCMC were mapped to attention in terms of water quality, while zones C and administrative zones rather than water zones. E needed to improve the duration of water supply. Select survey findings are provided in Figure 11. The survey found that access to water through household piped connections had increased from 83 percent to 94 On the whole, 63 percent of respondents reported better percent, most likely due to PCMC’s campaign to provide service delivery compared to the year before, and about new connections between 2012 and 2014. Regularity 96 percent expressed willingness to provide feedback in of receiving bills and ease of payment were considerably the future. enhanced owing to provision of Citizen Facilitation Centers in all zones and initiatives such as SARATHI. However, the At the request of PCMC, the survey included a question phone survey showed a slight dip in the score for adequacy of on whether the respondent wanted 24x7 water supply, for water supplied, which could be attributed to the severe water which a project was under development. Feedback showed shortage faced by PCMC in the summer of 2014. that opinion was almost equally divided between those for and against it. Reasons given by those who were ‘not Feedback for quality of water showed some improvement interested’ in 24x7 (see Figure 11) included wastage of water. but remained poor (59 percent), emerging as a continued This highlighted the need for better communication of the area of concern for PCMC. Similarly, performance on advantages of continuous water supply. Figure 11: CATI survey findings on continuity of water supply as reported by PCMC citizens Number of Days of Water Supply Total Hours of Water Supply All ALL All ALL FF F F EE E E D D D D CC C C BB B B AA A 0% 0% 20% 20% 40% 40% 60% 60% 80% 80% 100% 100% 0% 0% 20% 20% 40% 40% 60% 60% 80% 80% 100% 100% Don't know 7 6 - less than 7 5 - less than 6 Dont’t know 4 - less than 5 7 - less than 4 3 26 - less - less than than 3 7 1 - less 5 - less thanthan 2 6 Above 8 - less 7- less than 88 66 - less - less than than 77 55 - less - less than than 66 4 - less than 5 3 - less than 4 2 - less than 3 1 - less than 2 less than 4 - less than 55 - less 3- less than 4 22 - less - less 33 than than 11 - less - less than than 22 less than 0.25 --less 0.25 1 than 1 notknow Donot Do know www.wsp.org 41 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Reasons for not wanting 24x7 water supply People will waste water Just give 1 hour water with full pressure People will waste water if they get it for 24 hours Getting sufficient water from PCMC 27x7 not needed Bill badh jayega Daily 2-3 hours of water supply will be sufficient Existing water supply is sufficient No need of 24x7 water supply 4.3.2 Sms Poll Findings in Pimpri Chinchwad from Zone C reported the lowest complaint resolution rates. The design of the SMS polls conducted in 2015 was Zone F consumers reported some inconvenience related to informed by the findings from the 2014 telephone survey. the location of bill collection centers, while a high share of The polls found the highest incidence of dirty water supply responses from Zone E indicated that bills were not received was in Zone F (13 out of a total of 57 incidents reported). on time. The open-ended questions received a poor response Overall, there was mixed feedback on adequacy of water rate (five out of 1,200). Trend questions with a rotating panel supply, with Zones D and F reporting lower rates. More than of respondents received consistent responses, whereas the 68 percent of the consumers who participated in the survey response rate to a poll repeated over time with a fixed set of were satisfied with the timing of water supply. Consumers respondents declined with each successive poll. 42 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 5. Summary Observations and Lessons KEY POINTS • Project implementation experience demonstrated the applicability of the SLB-C approach across diverse institutional environments and scales of implementation • The involvement of local functionaries in the preparatory process helped better planning of the survey, and greater ownership at the dissemination stage 5.1 Implementation Experience engage in the exercise among municipal functionaries, Outcomes and greater use of the data in plans; zz The feedback surveys were seen to provide concrete and zz The project implementation experience demonstrated relevant data on the status of water supply and sanitation the applicability of the SLB-C approach across diverse services which helped inform cities in preparation of institutional environments and scales of implementation, their plans and proposals for funding under various from city level to local community; urban programs such as AMRUT, Smart Cities and Swachh Bharat; zz The use of an integrated and modular ICT platform with built-in questionnaires, survey management zz The credibility of the process and acceptance of the functionalities and data analytics enabled findings were enhanced by provision of granular feedback implementation across multiple cities within a short at the ward level, spatial maps and access to detailed timeframe, with a limited number of technical experts. survey records with GPS and time-stamping. This Once the platform is available, the incremental cost for also helped generate more interest among councilors, each survey is limited to acquisition of mobile devices, highlighted service inequities, provided actionable survey planning, training, field work for data collection information and enabled prioritization of interventions; and survey monitoring, and dissemination of findings. zz The demand-side metrics provided a reality check for Time and costs associated with data input, analysis, and the SLB data reported by cities. While they validated report preparation are eliminated; the performance reporting on some metrics (for zz Field work was conducted by locally recruited manpower example, coverage, continuity), they also highlighted a (for example, students from local colleges, institutes), few significant gaps between reported performance and which helped to improve the optics of the exercise. It actual service experience (for example, water quality, was seen as an assessment by the city’s own residents complaints); and rather than outsiders, making the councilors and zz Government to citizen communication was activated functionaries more receptive to the survey findings. It through SMS messages sent at different stages of the also demonstrated local capacity to replicate such ICT- survey – “thank you” messages on collection of feedback, based exercises in future; and introductory messages for CATI and SMS surveys, and zz The multiple channels for collecting feedback were found dissemination of web links to summary survey findings. to be complementary. MAPI household surveys provided detailed feedback on services and helped to create a Program design profiled database of respondent contact numbers; CATI zz The alignment of SLB-C with the SLB program helped surveys provided a quicker (but less robust) means of in two ways. Firstly, it secured the involvement of the doing follow-up tracking on service aspects; and SMS MoUD in the rollout of the program. Secondly, use of polling helped to track specific areas of concern. The SLB terminology helped create greater receptivity to experience in PCMC showed that municipalities could www.wsp.org 43 Using ICTs for citizen feedback surveys to mainstream demand side monitoring use a mix of mechanisms to obtain feedback. The low which could serve as a reality check on reported share of complaints lodged through formal channels such data; enabling granular analysis of service levels, as helplines, SMS and online feedback also showed that which highlighted inequities within the city; these mechanisms need to be accompanied by proactive and helping to inform planning processes. These feedback collection to obtain a more complete picture of features made it more possible to hold providers citizen experience and facilitate engagement. accountable for delivery of service outcomes, and enabled integration of SLB-C in the national urban Stakeholder engagement and response program and the World Bank project. However, the zz Implementation of SLB-C in all cities was undertaken TA achieved limited success in facilitating citizen in close coordination with local functionaries. Their dialog on service delivery, which was seen to require involvement in the preparatory process helped better more sustained engagement for capacitating and planning of the survey, and greater ownership at the generating awareness among citizen groups, local dissemination stage. Dissemination activities in all functionaries, and other stakeholders, especially cities involved participation of the municipal mayor, in cities with weak service providers lacking in commissioner and head of the service department. customer orientation. Moreover, delivering service Municipality logos were used on the SLB-C online improvements on a sustained basis will require other dashboard, and several municipal websites provided enabling conditions such as role clarity, rationalized links to the SLB-C online dashboard; incentives, technical capacity, financial resources for improvements and citizen-oriented local leadership. zz The extent of engagement of local functionaries varied across the cities at different stages of implementation. 5.2 Survey Findings on Service Levels Some of the influencing factors which supported higher levels of responsiveness were the extent to which the city zz (Overall) Feedback for water supply tended to be better was responsible for service delivery; the technical and on access levels but poorer on service quality aspects. financial capacity of the service-providing unit; whether Poor water quality emerged as an important area of service improvements were being planned; and the concern for citizens in most cities. This also resulted extent of cohesion in the local polity; in high dependence on alternate sources of water (for example, private bore wells), even in cities otherwise zz As part of the implementation approach, councilors were having adequate water supply (for example, Varanasi, involved only at the dissemination stage. There were Jabalpur). Other areas of priority were adequacy and differing views on the need to involve councilors from the regularity of supply. Toilet access was reasonable in preparation stage onwards. The case in favor is that they most cities except Raebareli, but sewerage was lacking are closely linked to citizens on issues related to service in all except two cities (Varanasi and Raebareli), where delivery; the case against is that it could bring undue coverage was also poor. pressure on the survey implementation process; and zz (Alignment between SLB and SLB-C) There appeared zz Even while the feedback surveys captured the citizens’ to be greater alignment in supply- and demand-side data perspective and provided useful information to improve for infrastructure-based metrics (for example, household the dialog on service delivery, there were limits to the connections, toilet access, sewerage access, metering). extent of citizen engagement facilitated through this Significant gaps were observed in metrics linked to process. More sustained support is required to capacitate service quality (for example, water quality, complaint local citizen groups and functionaries to engage in a redressal and adequacy). This resonates with the sector’s meaningful dialog on service delivery and enable citizen accountability context, which emphasizes infrastructure participation in decision making. creation (requiring coverage data to be reported for preparation of projects and plans) over service delivery. In summary, the Technical Assistance (TA) achieved its stated objectives with respect to providing a zz (Alignment between SLB and SLB-C) For water systematic means to capture citizen feedback, supply access, performance as reported by the SLBs 44 Using ICTs for citizen feedback surveys to mainstream demand side monitoring was lower than corresponding SLB-C metrics for the Chinchwad, Jabalpur), citizens preferred channeling their two cities from Rajasthan, while the reverse was true complaints through councilors (more so for residents in for the other cities. On water quality, the reported slum areas). Conversely, most citizens in cities served by performance was more aligned with citizen feedback for state departments (for example, Ajmer and Jhunjhunu) cities from Rajasthan than other cities, where the gaps lodged their complaints directly with the helpdesk. In were substantial. Given that, in Rajasthan, water supply general, in the cities surveyed, respondents preferred a services are provided by a state department (rather than face-to-face interaction, with negligible use of helplines, city departments, as in other cities),10 it would useful to SMS or online facilities – a preference also mirrored explore if these institutional arrangements could have in modes of bill payment. Citizens’ preferred channel a bearing on data quality or degree of conservatism in of complaint reveals their perception of which set of reporting. Similarly, it would be useful to analyze why functionaries are most likely to be effective in creating reported data on toilet access is lower than feedback data pressures to improve service levels. in all cities, and whether there are any incentives for zz (Sanitation) Feedback data revealed that, despite cities to under-report on these. citizens reporting high levels of access to toilets, a zz (Equity) Granular analysis revealed service inequities sizeable share in the slum areas of several cities practiced between slum and non-slum areas, mainly on open defecation (for example, Raebareli, Mehsana). In infrastructure aspects (for example, house connections, the absence of sewerage, there is significant wastewater toilet access, sewerage access), primarily due to eligibility disposal into onsite facilities, which the feedback constraints. Inequities in quality of services (for example, indicated are not being cleaned regularly. The spatial adequacy, water quality) were typically attributable to maps help to highlight the sanitation hotspots in the city the type of source being accessed (for example, public for prioritization of interventions. stand post users reporting higher incidence of dirty zz (Satisfaction) Despite service levels being relatively water supply). Significant inequities were evident at poor, 80-90 percent of respondents were partially or the ward level, with peripheral areas characterized by fully satisfied with services (though the figures were lower poorer service levels. Service inequities could therefore for slum areas in all cities). Also, about 90 percent of be seen as a corollary to broader systematic issues such respondents felt that services were the same or better than as informal status of land ownership and poor quality of the previous year. Given the apparent low expectations urban planning. on service delivery, there may be limits to the extent of zz (Complaints redressal) The share of respondents lodging civic mobilization is possible for service improvement. A complaints was low (5-15 percent), of which typically a mitigating factor is that, in several cities, especially where third reported resolution. This could be a reflection of the service delivery is a municipal responsibility, councilors perceived (and, possibly, actual) inefficacy of complaint were sensitive to the feedback on “services compared to redressal mechanisms.11 In cities where service delivery the previous year”. Such political pressure points could was a municipal responsibility (for example, Pimpri be leveraged for improving services. 10 In the cities from Uttar Pradesh (Varanasi and Raebareli), the capex function is performed by a state utility, while operation and maintenance is the municipality’s responsibility. 11 This also resonates with the findings from the World Bank’s 2014 Review “Grievance Redress Mechanisms – On Paper and in Practice” – as referenced in the “Strategic Framework for Mainstreaming Citizen Engagement in World Bank Group Operations” 2014, Box 4.4 page 45. www.wsp.org 45 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 6. Recommendations for Design of Citizen KEY POINTS • Aligning demand-side metrics with supply-side indicators makes it easier to activate both upward and downward pressures of accountability • ICTs should be leveraged not just for speed and Feedback Interventions efficiency, but also for enhanced data integrity, transparency and impact Experience of the SLB-C initiative suggests some city administrators or municipal councilors.12 This is recommendations for the future design of citizen especially relevant in environments where data quality is feedback programs. weak or information is not readily accessible. zz Aligning demand-side metrics with supply-side Simultaneously, provision of performance metrics in a indicators makes it easier to activate both upward disaggregated form (for example, ward level) provides and downward pressures of accountability. Using a citizens with service delivery data they can relate to. framework of demand-side metrics and analytics that With appropriate dissemination, this could be used are similar to supply-side (reported) data makes it easier to generate bottom-up pressure on service providers to integrate feedback data into plans and decision- through political representatives, CSOs or media. making processes (see Figure 12). This helps government When providers and consumers share a common or funding agencies to strengthen monitoring of vocabulary of service metrics, they can engage more service providers and track service outcomes under effectively in dialog on service delivery. Given these various projects/programs (for example, as envisaged advantages, as an extension of the World Bank’s work for KUWSMP). Detailed analysis of these metrics on performance reporting and indicators, it could be also helps to improve accountability by reducing the useful to develop a more exhaustive list of demand-side information asymmetry between the provider and the metrics that could help capture service levels through overseeing agencies, such as the state urban department, such feedback processes. Figure 12: SLB-C influences top-down and bottom-up accountability Centre/state government programs, policies, regulations Top - Down guidelines Accountability Strengthening top-down accountability Accurate monitoring and reporting Service provider, ULB, utilities, Improving Service Delivery Strategic and informed decision making departments Increased responsiveness Bottom - Up Citizens, civil society Accountability Strengthening citizen engagement, therefore bottom-up accountability 12 W  here operationalized, regulatory agencies could perform the oversight functions and use such surveys to improve monitoring and accountability of providers. 46 Using ICTs for citizen feedback surveys to mainstream demand side monitoring zz Feedback surveys designed for replicability and Where replication of the platform is not feasible, implementation at scale could help to address capacity other ICT options could be considered to implement constraints that often hinder localized interventions. such feedback surveys, such as open source data Most local bodies lack the capacity to undertake citizen collection tools (Civil Society Organizations Open engagement activities and feedback surveys. There is Data Kit) integrated with data analytics products also limited availability of technical experts to design (Civil Society Organizations Tableau) configured for and oversee such surveys. Any tool for undertaking such default survey analysis. initiatives should therefore address the “how to” question zz ICTs should be leveraged not just for speed and and design for implementation at scale. Attributes which efficiency, but also for enhanced data integrity, enable this include: transparency and impact. As well as enabling A template-based approach for design and o  implementation at scale, integrated workflow-based implementation (for example, question bank, ICT systems can improve the integrity of data collection default questionnaire, sampling methodology – all processes and demonstrate transparency to decision- embedded in an ICT-based solution); makers and stakeholders. ICT functionalities can also o  Provision for remote monitoring of surveys at help deliver greater impact from the survey findings. multiple locations, to maximize use of limited This is made possible in the following ways: technical experts for survey monitoring, review and o Better quality control through built-in checks, flags analysis; and real-time monitoring reports; Modular design to address different geographic o  Greater transparency and data integrity, as all o  scales (for example, city, ward, locality) and the data collected is readily accessible – including different service areas (for example, water supply, rejected records, enumerator details, time stamps, sanitation); and geo-coordinates and photo images; Provision for implementation through partners o  More current and relevant findings, as results are o  (for example, an online platform hosted by a nodal available immediately after a survey – in contrast agency and used by multiple agencies to conduct to traditional methods, which often take five to surveys and disseminate results). seven months from survey to report, by which time findings could have become dated; These features were fully utilised by MoUD in the  implementation of the Swachh Survekshan survey across 73 cities. Potential for scaling up is greatest Figure 13: Comparison of traditional survey when the online platform is hosted by a nodal agency practices to SLB-C which can provide advisory and analytical inputs to help various other implementing agencies use the Survey Process SLB Connect Survey design Survey design system’s functionalities to conduct surveys. This role could potentially be performed by any entity with the Household survey Household survey + online data review, Replicable requisite credibility and technical expertise, such as a Data collation stringent quality control monitoring unit in the government, a regulatory agency, Data review a research institution or CSO. Such an approach could Data analysis also be considered in World Bank operations where Report preparation Dissemination - platforms such as SLB-C could be used to streamline 7-10 Publicly accessible (i)  1-1.5 Dissemination - dashboard implementation of feedback surveys on service delivery.13 workshop(s) months months (ii) Workshops 13 If the platform is based on open source software and available for replication, incremental system-related costs would be low (limited to installation and hosting services). These additional costs would be offset by the gains in speed, reliability and transparency of the survey process. www.wsp.org 47 Using ICTs for citizen feedback surveys to mainstream demand side monitoring Granular analysis, in the form of spatial maps and o  disadvantages (see Figure 14), it is more appropriate to analysis stratified across user categories/locations consider a mix of mechanisms for a feedback program, provides more actionable information by helping to depending on the context and objective of the program.14 identify hotspots and interventions required. It also For instance, while MAPI surveys are more resource makes the data more credible to decision makers to intensive, their dependence on citizen motivation is low, address service gaps and inefficiencies (for example, making them suitable for ‘low equilibrium’ contexts maps showing water quality hotspots could help to where incentives for self-reporting are low.15 MAPI identify leaks); and surveys are inclusive, reaching all groups and overcoming o  Survey findings can be rapidly and widely challenges posed by the digital divide. Feedback modes such as SMS polls and crowd-sourced feedback are disseminated through online dashboards, sending easier to implement and less resource intensive, but also bulk SMS messages with survey highlights or less detailed, inclusive and representative. A possible web links to results, or updates on social media approach could be to use MAPI surveys in poor service websites. This could be accompanied by traditional delivery contexts, or at the start of a project, to periodically dissemination methods such as workshops, flyers capture detailed feedback, and other channels for real- and coverage in local media. time follow up on specific aspects. Where service delivery zz The various modes of feedback such as MAPI surveys, arrangements are more mature, and digital access telephone surveys, SMS polls, IVRS systems, need is widespread, feedback could be tracked using just to be used in a complementary manner. Often these the lighter modes. Conversely, in contexts with poor are viewed as interchangeable modes for obtaining service quality and low capacity in particular, there will feedback. However, given their relative advantages and continue to be a need for face-to-face interactions such Figure 14: Comparison of feedback tools SLB Connect Mobile to Telephone SMS Polling Crowd sourced Complaint Weh HH Survey ratings/reporting monitoring Survey (CATI) Representative (respondent mix) Only customers who ..... Respondent mix uncertain ...... use formal channels Gender & income bias Mobile app based Quality of feedback (reliability, details) Fewer questions possible: interpretation errors likely Citizen motivation (Response rate) Use of informal “Why bother?” Difficult to sustain channels when provider response is poor Intra-city analysis Possible if Possible if geo-tagged; unlikely for geo-tagged SMS polling & online feedback Scale (No of Citizens reached) Resource requirement Note: HH: household. 14  ould also include mechanisms such as focused group discussions, panels, touchpoint audits. The aim here is not to describe the entire menu of feedback C mechanisms but to highlight their complementarity. 15  Low equilibrium is where the service provider has low capacity, poor customer orientation and low credibility, leading to customers having a low expectation of improvement/response, and giving them weak motivation or incentive to provide feedback. 48 Using ICTs for citizen feedback surveys to mainstream demand side monitoring as community-level meetings and town halls to ensure where service delivery is a municipal responsibility, inclusive and meaningful engagement. pressure would be relatively localized, at the city level; where service delivery is more centralized, higher levels of zz Clarifying how citizen feedback is expected to government (for example, state or national) would need improve services could help to decide the extent to be invoked to generate pressure for improved services. and nature of civic mobilization and stakeholder engagement required. Citizen feedback could be Also, where customers have low expectations of service leveraged in many ways to improve service delivery, such delivery, there could be limits to how much it is possible as mobilizing public opinion and generating pressure on to use civic mobilization to create pressure on service service providers to deliver improved services; informing providers. In such situations, local councilors (who often providers of service gaps and customer perception issues; act as intermediaries for problem resolution) could play a improving design of projects/plans under development; role in generating accountability pressure. Local officials or strengthening the tracking of service outcomes under also need to be engaged in implementation of feedback programs, especially where reported data lack reliability. processes, to get their buy in and improve their response to Not all of these require civic mobilization and extensive the feedback data. stakeholder engagement. When planning a citizen feedback initiative it is necessary Even where accountability pressures need to be to undertake a clear assessment of the context and develop strengthened, the means to do this may vary depending on a holistic strategy for how the feedback will be leveraged to the institutional context for service delivery. For instance, deliver maximum impact on service levels. www.wsp.org 49 Using ICTs for citizen feedback surveys to mainstream demand side monitoring 7. Way Forward KEY POINTS • Further strengthening and institutionalization of systematic citizen feedback processes is required • Government of India and development partners can help mainstream processes in projects and programs through capacity building, use of technology solutions and advocacy efforts Decision makers at various levels have found SLB-C to be The GoI could facilitate adoption of these processes by: a useful way of obtaining detailed insight into the on-the- zz Providing guidance, capacity building and advisory ground reality of service delivery, including identification support to interested states/cities; of service gaps and inequities within a city. It is seen to offer an approach for scaled-up implementation of feedback zz Hosting or making available ICT-based solutions for use surveys that can help to inform planning and program by interested state and local governments;16 monitoring processes. The SLB-C online platform has zz Strengthening incentives for adoption by integrating been integrated into the city sanitation ratings done for the these processes in government programs;17 and GoI’s Swachh Bharat Mission, and has been included in the project implementation processes of other development zz Developing orientation modules for citizen groups partners (CEPT and CURE). SLB-C has been integrated and political representatives on different feedback under the social accountability component of the World mechanisms, how to interpret demand side data/ Bank-supported KUWSMP project in Karnataka, and metrics, and use them for advocacy efforts on service is seen to offer an effective way to implement the World improvements. Bank’s recent guidelines on inclusion of citizen feedback mechanisms in projects. Several of these roles are envisaged in the scope of the proposed National Performance Monitoring Cell (NPMC), which Further strengthening and institutionalization of systematic could undertake these activities when it is operationalized. citizen feedback processes is required to improve tracking of Additional support efforts could include training and service outcomes and foster more demand-responsive service capacity building of civil society actors and networks that improvement planning. Accompanying interventions are could facilitate the interaction between citizens and service also required to rationalize institutional arrangements and providers. improve incentives for delivery of service outcomes. With a growing emphasis on transparency and citizen-centric service Institutions such as the World Bank can play an important delivery, the role of feedback and engagement processes is role in encouraging clients to adopt these interventions set to increase. Emerging ICT innovations will offer new in their operations and decision making, and supporting opportunities for making these processes more intelligent, their implementation. The recent World Bank directive inclusive and efficient. mandating inclusion of beneficiary feedback processes 16  ese could include ready-to-use cloud sourced solutions, multi-channel interfaces for ongoing feedback, and integration across multiple service Th areas for convenience of citizens. Adoption would depend on the extent of ICT penetration in the environment, institutional structures and providers’ service orientation. 17  o begin with, incentives could be for adoption of regular feedback mechanisms by local governments/service providers and their use in planning/ T review processes. Later these incentives could be linked to the feedback on service quality (that is, rewards for good performers and/or penalties for poor performers). 50 Using ICTs for citizen feedback surveys to mainstream demand side monitoring in projects can give further impetus. To make feedback experiences such as Maji Voice (Kenya), Vozelectrica processes more impactful, they should not be limited to (Dominican Republic), Citizen Feedback Monitoring self-reporting options such as helplines, SMS or online Program (Pakistan), SLB-C (India) and other emerging feedback, but also include mechanisms to collect feedback initiatives to help inform future interventions and make proactively. Feedback should encompass tracking of them more effective. To prevent duplication of effort and service delivery under a project, and not be limited to facilitate adoption, it would also be useful to explore a project implementation processes. Efforts also need to be virtual platform where these initiatives can be hosted for made to identify factors that inhibit citizens’ use of formal ready access by teams or clients in different countries. grievance redressal mechanisms, and initiate interventions Such knowledge-sharing efforts could reduce the cost of to make these more accessible and responsive. It would implementing citizen engagement interventions and help be useful to consolidate learnings from World Bank to obtain buy-in from clients. www.wsp.org 51 Notes Water and Sanitation Program The World Bank Hindustan Times House 18-20, Kasturba Gandhi Marg, New Delhi 110001, India Telephone: (91-11) 41479301, 49247601 E-mail: wspsa@worldbank.org www.wsp.org worldbank.org/water @WorldBankWater