THE WORLD BANK OPERATIONS EVALUATION DEPARTMENT 29356 This review finds that citizen report cards have been an important An Assessment of the Impact of Bangalore vehicle for civil society’s “voice” Citizen Report Cards on the Performance in Bangalore, of Public Agencies and that they have had a significant impact on the quality of public services. The review identifies success factors ECD Working Paper Series ♦ 12 and considers the replicability of the report Adikeshavalu Ravindra card approach. www.worldbank.org/oed June 2004 The World Bank Washington, D.C. Copyright 2004 Operations Evaluation Department Knowledge Programs & Evaluation Capacity Development Email: eline@worldbank.org Telephone: 202-473-4497 Facsimile: 202-522-3125 Evaluation Capacity Development (ECD) helps build sound governance in countries—improving transparency and building a performance culture within governments to support better management and policymaking, and to strengthen accountability relationships—through support for the creation or strengthening of national/sectoral monitoring and evaluation systems. A related area of focus is civil society, which can play a catalytic role through provision of assessments of government performance. OED aims to identify and help develop good-practice approaches in countries, and to share the growing body of experience with such work. The OED Working Paper series disseminates the findings of work in progress to encourage the exchange of ideas about enhancing development effectiveness through evaluation. An objective of the series is to get the findings out quickly, even if the presentations are somewhat informal. The findings, interpretations, opinions, and conclusions expressed in this paper are entirely those of the author. They do not necessarily represent the views of the Operations Evaluation Department or any other unit of the World Bank, its Executive Directors, or the countries they represent. Table of Contents Foreword ......................................................................................................................................... i Abbreviations................................................................................................................................. ii Executive Summary ..................................................................................................................... iii 1. CONTEXT AND PURPOSE OF THIS STUDY................................................................ 1 Citizen Report Cards ....................................................................................................................... 1 Purpose and Scope of the Study ...................................................................................................... 1 Summary of Major Findings of the First Two Citizen Report Cards on Bangalore........................ 2 2. SUMMARY OF INTERVIEWS.......................................................................................... 4 Public Service Agencies (PSAs) ..................................................................................................... 4 State Officials .................................................................................................................................. 9 Citizen Groups............................................................................................................................... 10 Media............................................................................................................................................. 12 3. CONCLUSIONS ................................................................................................................. 13 Bangalore Citizen Report CardsStrengths and Weaknesses...................................................... 13 Outcomes and Impacts .................................................................................................................. 14 Replication of Citizen Report Cards.............................................................................................. 16 Concluding Remarks ..................................................................................................................... 17 Annex AComments of the Public Affairs Centre...................................................................... 19 Annex BQuestionnaire for Public Service Agencies................................................................. 20 Annex CQuestionnaire for Senior State Officials ..................................................................... 21 Annex DQuestionnaire for Citizen Groups and NGOs ............................................................. 22 Annex EQuestionnaire for the Media........................................................................................ 23 BIBLIOGRAPHY ....................................................................................................................... 24 i FOREWORD The Operations Evaluation Department (OED) of the World Bank has a long-standing program of support to strengthen monitoring and evaluation (M&E) capacities in developing countries, as an important part of sound governance. As part of this support, OED has prepared a collection of resource material including case studies of countries which can be viewed as representing good-practice or promising-practice. This resource material is available electronically at: http://www.worldbank.org/oed/ecd/ Considerable interest has been expressed in the work of the Public Affairs Centre (PAC), an NGO based in Bangalore, India. The PAC has undertaken several citizen report cards in Bangalore, and the report card approach has now been undertaken in urban and rural areas in 23 other Indian states. Report cards have also been conducted in the Philippines, Ukraine and Vietnam. Interest in undertaking citizen report cards has been expressed by NGOs in many other countries. The Bangalore citizen report cards entail the collection of feedback from ordinary citizens on their perceptions of the quality of public services provided by the municipal government and its agencies. The feedback is collected through household surveys. The survey for each Bangalore report card cost $10-12,000 and took about 7 months to complete. In addition, the PAC devoted considerable time and effort to dissemination of the survey findings, persuasion of government departments about the need for change to their performance and operations, and direct support for several departments which asked for assistance. The World bank has been a strong advocate of citizen report cards; see, for example, World Development Report 2004: Making Services Work for Poor People, and the Bank’s participation website at: http://www.worldbank.org/participation/. This advocacy partly reflects the Bank’s support and encouragement of the “voice” of civil societyempowering NGOs and ordinary citizens. This reflects, in part, the priority for empowering the poor through fostering opportunities for citizens to provide their feedback on the quality of government services; such opportunities are rare in many countries. The priority for encouraging voice also rests on the understanding that lack of such opportunities is itself a type of poverty (see, for example, The World Bank, World Development Report 2000/2001: Attacking Poverty). Ideally, the benefits of a tool such as citizen report cards would extend beyond a one-way flow of feedback from citizens to governments; the hope is that this feedback will prompt governments to pay closer attention to citizen concerns, and to improve their delivery of public services. Citizen report cards thus have the potential to improve the quality of government. This paper presents the findings of an OED review of the experience with citizen report cards in Bangalore, focusing on their outcomes and impacts, and on the success factors underlying the Bangalore experience. The review found that the report cards have had a significant impact in Bangalore. Once the review was completed, the head of the PAC, Mr Sam Paul, was asked to provide comments on the review, and these are shown in Annex A. On the strength of the review’s findings, it was decided to include the Bangalore experience as one of 8 case studies in a recent OED report on Influential Evaluations. The review was conducted by Dr Adikeshavalu Ravindra, a former Chief Secretary, Government of Karnataka. OED’s task manager for this review was Keith Mackay. The support of Elaine Ooi, Michael Bamberger and Lydia Ndebele is gratefully acknowledged. The views expressed in this paper are solely those of the author, and do not necessarily represent the views of the World Bank. Patrick G. Grasso Acting Manager Knowledge Programs & Evaluation Capacity Development ii ABBREVIATIONS AusAID Australian Agency for International Development BATF Bangalore Agenda Task Force BDA Bangalore Development Authority BMC Bangalore Municipal Corporation BT Bangalore Telephones BWSSB Bangalore Water Supply and Sewerage Board CRC Citizen Report Card KEB Karnataka Electricity Board NGO Non-Government Organization PAC Public Affairs Centre PSA Public Service Agency RTO Regional Transport Office iii EXECUTIVE SUMMARY The Bangalore Citizen Report Card (CRC), pioneered by the Public Affairs Centre (PAC), provides an assessment of the satisfaction levels of citizens with regard to public services in Bangalore and ranks public service agencies (dealing with water, power, municipal services, transport, housing, telephones, banks and hospitals) in terms of their service performance. In conducting its first CRC in 1994, PAC carried out a random sample survey of households to assess various dimensions of public satisfaction with respect to staff behavior, quality of service, information provided, and corruption (speed money). The findings presented a generally pessimistic viewlevels of public satisfaction were low, the public agencies were not citizen- friendly, they lacked customer orientation, corruption was a serious problem, and the people paid a heavy cost for the inefficiency of the public sector. The second CRC survey revealed only a partial improvement. This review provides an independent assessment of the contribution of the first two CRCs (1994 and 1999) based on interviews with a sample of agency heads, senior state officials, citizen action groups and the media in Bangalore. The interviewees were generally appreciative of the CRC as a tool to obtain feedback on services. Following the findings, many of the agencies initiated reform measures; some senior agency officials perceived, however, that the CRCs did not reflect their achievements and the constraints on their performance. On the whole, the impact of the CRCs has been positive. They helped to increase public awareness of the quality of services and stimulated citizen groups to demand better services. They influenced key officials in understanding the perceptions of ordinary citizens and the role of civil society in city governance. Bangalore has witnessed a number of improvements following the CRCs, particularly the second one. The state government and public agencies launched a number of reforms to improve the infrastructure and services in the cityincluding via property tax reform through a self-assessment scheme, the creation of the Bangalore Agenda Task Force (BATF), and streamlining of agencies’ internal systems and procedures. There is now greater transparency in the operations of government agencies and better responsiveness to citizens’ needs. While a number of other factors have also contributed to the transformation of Bangalore, the CRCs acted as a catalyst in the process. The benefits to be derived from any CRC appear to depend to a large extent on several factors. First is the use of such information by the media and by civil society. The media can clearly play an important role in publicizing poor agency performance, and this in turn can provide a stimulus to civil society, to the agencies themselves, and to other key stakeholders within government. An active civil society can play an important role in continuing to press for needed reforms to agencies, and also in monitoring the extent to which reforms actually occur. The responsiveness of government agencies, particularly their leadership, is very important. Where senior officials are concerned with the performance of their agencies, and with serving ordinary citizenstheir customersas best as is feasible, this is likely to provide a much more fertile ground for action on the basis of CRC findings. Of course, to the extent that the civil service culture is not customer-oriented, and is not concerned with achieving high levels of performance, there will be important constraints on what can be achieved even when the most senior officials are committed to reform. The Bangalore experience illustrates what can be achieved when a dynamic organization, the PAC, is able to conduct rigorous surveys of the extent to which citizens are satisfied or dissatisfied with government performance, is able to ensure a high level of media coverage of the findings, and is able to both persuade and provide some support to government agencies to help them improve their performance. 1 1. CONTEXT AND PURPOSE OF THIS STUDY Citizen Report Cards The performance of government, especially the provision of public services, is increasingly coming under the scrutiny of civil society. It is widely believed that governments are not responsive to the needs of the people, more so in the case of the poor. The traditional instruments used to hold public servants accountable, such as audit, legislature oversight etc, have proved inadequate. Would citizen feedback be a more effective way of assessing the quality of public services and demanding accountability from those who govern us? The basic concept behind the Citizens Report Card is that citizens as users can provide useful information on the quality and adequacy of services and the problems they face with the service providers. They know better than anyone else how responsible or reliable an agency is or what are the costs attached to a service. Thus the Report Card is expected to “reflect the actual experience of people with a wide range of public services”. Such feedback will be valuable to judge the performance of a service provider and serve as an input to the government in policy making and reform. Purpose and Scope of the Study The Public Affairs Centre’S first two city level Report Cards on public services in Bangalore were completed in 1994 and 1999. (A third was completed late in 2003.) They reported on the feedback of households based on sample surveys of the quality of services provided by the main Public Service Agencies (PSAs). PAC has also brought out other specialized report cards. The purpose of this study is to independently assess the contribution of these Citizen Report Cards to the following five potential outcomes: 1. Improved quality of services. 2. Increased civic activism concerning public services. 3. Increased public awareness of the quality of services and the fact that citizens have the right and the power to demand better services. 4. Increased client orientation on the part of public service agencies. 5. Replication of the Citizen Report Card in other cities and countries. The assessment was done by: a) Conducting individual interviews with a sample of agency heads and senior staff of the major agencies covered by the Report Cards. A total of 19 individual interviews were conducted. This sample was not intended to be a representative sample of all senior staff in these agencies. Rather, the sample was selected in an endeavor to identify senior staff who were familiar with the CRC findings and who might be expected to be knowledgeable about any utilization of them. b) Interviews with a few senior state government officialsfive officials were interviewed, and these were also officials expected to be familiar with the CRC findings and utilization. 2 c) Interviews with representatives of seven NGOs or citizen action groups and four representatives of the media. The NGOs and citizen groups were selected on the basis of several factors, such as: their credentials; their active involvement in civic affairs; and their acquaintance with the report cards. The media representatives included two national, English-language newspapers (with local coverage) and one local newspaper, published in a local language (Kannada). Summary of Major Findings of the First Two Citizen Report Cards on Bangalore Before presenting the findings of the evaluation, it would be useful, as a prelude, to recapitulate the major findings of the two Citizen Report Cards. A. Citizen Report Card 1 (1994) The methodology adopted was a stratified random sample survey of nearly 1140 households (including the poor), focus group discussions, visit by investigators to offices of service providers to assess the extent to which information was provided or denied to the public, and interview of a limited number of lower and middle level staff from selected public agencies. The Public Service Agencies for which feedback was sought through this sample survey are listed in Table 1. The CRC assessment covered the following aspects: a) Overall public satisfaction (by agency). b) Dimensions of public satisfaction with respect to: • Staff behavior. • Quality of service. • Information provided. c) Speed money actually paid. d) The cost of compensatory investments made by citizens. After assessing different services, the agencies involved were ranked in terms of their service performance. The major findings were: none of the eight PSAs received a satisfactory rating from the respondents; the proportion of those dissatisfied far exceeded that of the satisfied; only 35% or less were satisfied with the behavior of the staff in PSAs; 14% of respondents paid speed money; most PSAs were not citizen friendly; and people paid a heavy cost for the inefficiency of the public sector. In brief, the conclusion was that the level of public satisfaction with the performance of PSAs was low, public agencies lacked customer orientation, and that supply shortages could not fully explain the prevalence of poor responsiveness to the public or corruption. 3 Table 1: Public Service Agencies for which Citizen Feedback Was Sought Public Service Agency Service Bangalore Development Authority (BDA) Housing (plots), Residential Layouts / Infrastructure Bangalore Municipal Corporation (BMC) Municipal Services Bangalore Water Supply and Sewerage Board Water and sewerage (BWSSB) Karnataka Electricity Board (KEB) Electricity Regional Transport Office (RTO) Driving license, Vehicle registration, etc Bangalore Telephones Telephones Banks Banking Hospitals Medical and Health Care B. Citizen Report Card 2 (1999) The second Report Card came after a period of five years. Two more agencies were included in the surveythe Police and Ration Shops. The methodology was also the same as that adopted for the first Report Card. The sample size was increased to 1339 for general households, and 839 for slum dwellers. The important findings were: citizens rated the quality of services of most agencies at fairly low levels; the respondents had to make an average of three visits to a public office to get a problem solved; not many agencies were efficient at solving problems; the usage of grievance redress mechanism was very poor; corruption was widely prevalent in most agencies; many users were willing to pay more for improvement in service delivery; and the satisfaction level and the usage of public services were much higher among the poor than among the general households. Comparison between the two Citizen Report Cards reveals that after five years: overall satisfaction had increased from 9% to 34%. While the proportion of people paying a bribe increased from 14% to 22% among general households, it had declined by 22% among the poor. The satisfaction ratings given by the urban poor were considerably higher, possibly because their expectations were lower. On the whole, while satisfaction levels continued to be low, there was some evidence of improvement. This, however, did not apply to the problem of corruption. The interviews reported below probed the factors behind these developments and the roles played by the Report Cards vis-à-vis the PSAs, and civil society institutions. 4 2. SUMMARY OF INTERVIEWS Public Service Agencies (PSAs) To begin with, four PSAs were identified for the purpose of interviewsthe Karnataka Electricity Board, the Bangalore Water Supply and Sewage Board, the Bangalore Municipal Corporation, and Bangalore Telephones. It was not possible to find the required number of persons familiar with Citizen Report Cards in Bangalore Telephones, which is part of a countrywide organization. As such, another agencythe Bangalore Maternity Homeswas chosen because the PAC had completed a separate Report Card on “Maternity Health Care for the Urban Poor” in 2000, soon after the second Report Card. However, one officer from Bangalore Telephones and one from Bangalore Development Authority were also interviewed to gain a wider perspective. The questionnaire used for interview of PSA officials is at Annex B. Karnataka Electricity Board Four officials were intervieweda former Chairman, former Directors of Finance and of Personnel, and a Chief Engineer. All of them had knowledge of the Report Card but to varying degree. One of them had a thorough understanding of both the CRCs and went on to introduce innovations at the State level later on. The others became familiar with CRCs when a presentation on the findings of the second CRC was made by the PAC in their organization. They had all interacted with PAC at some time. There was general agreement about the report card being a good feedback mechanism. It was considered, by and large, an “indicator of the state of affairs” and gave a “fairly accurate picture of what the user really faces”. It thus served as a wakeup call to the authorities to deal with the problems in their organizations. On the specific findings about the KEB such as quality of service, staff behavior, and speed money, all interviewees admitted to the prevalence of deficiencies but they also had a few comments on the methodology adopted for the survey. First, it was opined that the sample size was small for a city of the size of Bangalore. Secondly, the survey covered only the residential population and excluded other categories such as industrial and commercial establishments. Thirdly, some subjectivity in the responses could not be ruled out. What actions were taken to respond to the findings? All the respondents referred to the reforms introduced in KEB and the resultant improvements. Examples included a training program for staff aimed at behavioral change and skill development, creation of a citizen’s charter, and launching a website to provide information to customers. Specific measures leading to positive outcomes included computerization of billing to overcome wrong billing, electronic payment facility, adequate supply of application forms (there had been complaints of short supply), elimination of the requirement that applications routed through electrical contractors, setting up of reception counters to receive applications and complaints, introduction of voice recording systems to reduce response time, and creation of a mobile facility for receiving payments in remote areas. The issue of wrong billingidentified by the first Report Card as a key issueproved critical in prompting corrective measures. To what extent can these changes be attributed to the Report Card? The reforms in KEB were the outcome of a number of factors. Government’s policies and the Electricity Regulatory Commission’s prescriptions through a code for service delivery played a major role. Support from 5 the Bangalore Agenda Task Force (BATF) and pressure from ordinary citizens also strengthened the process. The Report Card had, however, helped to create awareness about KEB’s shortcomings and to accelerate the reforms. Subsequently, the KEB had on its own carried out a consumer satisfaction survey through a professional agency. What were the constraints in bringing about the desired changes? The KEB is a large organization responsible for transmission and distribution of power throughout the State. It had severe financial problems as it was not free to set its tariffs and relied on the Government for subsidy. It did not enjoy managerial autonomy because the Board is comprised mostly of Government nominees. Political interference was reported as aggravating the situation. One interviewee referred to the growing demand for power with about 1,000 customers being added every day. Bangalore Water Supply and Sewerage Board Four officers were intervieweda former Chairman, the current Chairman, and two Chief Engineers. An Executive Engineer was also present during one of the interviews. The former Chairman was familiar with the first Report Card. The others had interacted with PAC, and the two Chief Engineers had attended a presentation on the second Report Card. The general consensus was that the Report Cards had helped create awareness about the role of PSAs and how the people viewed them. The findings of Report Cards reflected the general state of affairs. One stated weakness of the Report Card was the small sample size for the survey. Another drawback was the exclusion of non-residential categories. A third stated weakness was that the survey was confined to customers while the PSA staff were ignored. Hence it was argued that the findings failed to present a comprehensive picture. All the interviewees were quick to point out the actions taken to effect improvements. BWSSB had taken several measures towards customer orientation; and staff behavior had improved following training programs. A good public grievance redress system had been developed. Water adalats (water courts) meet periodically in different localities to sort out consumer problems. Modern communication facilities such as e-mail and mobile phones are now used to reduce the response time. The various services provided by the agency, along with response time targets, are displayed in all the offices of the Board thereby promoting access to information. Services to the poor had improved through provision of individual water connections to their houses. However, all these changes could not be attributed to the Report Card. They were more the outcome of their own initiatives and government support, the Report Card acting as a catalyst. In some matters like wrong billing which created considerable dissatisfaction amongst consumers, the information contained in the Report Card was highly catalytic. Similarly, it was the Report Card that prompted the agency to start interacting with civil society. The role of citizen groups and resident associations in demanding better services was acknowledged. It was pointed out that BWSSB had also undertaken a feedback survey under an AusAID project. The problems faced by the agency included overall shortage in supply of water (Bangalore’s source of water is a river about 100 km from the city), shortage of staff at the lower levels (for example, valve men) and funds for operations and maintenance. It was, however, claimed that there had recently been improvements in these matters. Bangalore Municipal Corporation Five officers were intervieweda former Commissioner, the current Commissioner, a Deputy Commissioner, a Chief Engineer, and a Health Officer. The former Commissioner was well 6 informed about the Citizen Report Card system and was quite appreciative of its utility. The others felt it was a good tool to obtain people’s feedback. All of them had interacted with PAC on more than one occasion. One weakness of the CRC which was pointed out was the absence of any consultation with PSAs before commencing the survey. The interviewees generally agreed with the CRC findings and explained the actions taken to overcome the problems, which appeared simple but became complex in view of the nature of the organization. Some indications were given concerning how the CRC had acted as a catalyst by providing new information or by bringing pressure to take certain actions. It was specifically pointed out in the findings of the first CRC that the criteria and methods for the determination of property tax were not explicit and not known to most taxpayers. The public had accepted it as a negotiable tax and the BMC was losing money. The property tax system was rationalized following the CRC and was influenced by it. A new Self Assessment Scheme was introduced, and interviewees stated that this was simple and transparent, with clear criteria for determination of tax, and that they considered it to be “people-friendly”. This also helped in increasing tax collection and reducing corruption. It was generally known that garbage collection in the city was not satisfactory but it was the Citizen Report Card that provided information about the specific problems faced by the people56% saying that there was no regular collection in their areas and 20% pointing out that there were no dustbins near their houses. Subsequently solid waste management practices were studied more systematically and new measures such as door-to-door collection of garbage were introduced. Similarly, 28% of respondents to the CRC had said fused-out street lights were not replaced in time. After examining the problem, a decision was taken by the BMC to privatize maintenance of streetlights. A fund-based accounting system was introduced in partnership with the Bangalore Agenda Task Force. Time limits were also prescribed to attend to public complaints. Staff behavior had improved and there was greater interaction with citizen groups. Interviewees argued that there are several constraints in effecting changes in the BMC. Of all the PSAs in the city, this is the only democratic body with 100 elected Councilors. The executive powers vest with the Commissioner while the elected Mayor is more of a ceremonial head. There are conflicts between the deliberative and executive wings in the Corporation which affect the decision-making process as well as the pace of implementation of decisions taken. Inadequate resources and shortage of qualified and trained personnel are some of the other problems. The programs for the urban poor do not receive adequate attention. In view of its multifarious functions and the presence of elected representatives, BMC is exposed to the public in a big way. The interviewees admitted that public pressure does push them to better performance. There has also been increased civic activism in Bangalore in recent years, given the large number of NGOs and resident groups. BMC had in fact taken the initiative in setting up Swabhimana, a forum to work together with citizen groups. The Report Card had served as a good tool to keep up public pressure and increase accountability. Bangalore Maternity Homes Following the second Bangalore Report Card, a separate report card was prepared by PAC concerning the services provided by the Maternity Homes run by the Bangalore Municipal Corporation. The health facilities in these hospitals are used by relatively low income women and children. The study revealed that the overall satisfaction level of patients was low, the hospitals were poorly maintained, and there was widespread corruption. 7 Four officials were interviewedChief Health Officer, Health Officer (Mother and Child Health Care), an Additional Health Officer, and the Superintendent of one hospital. They were well acquainted with the Report Cards and the role of the PAC. While they were broadly in agreement with the findings, they felt that the phenomenon of speed money (bribes) was exaggerated. It was pointed out that it was a common practice for patients to generally offer money out of their own volition and it might not be proper to consider every payment as bribe. One of the stated weaknesses of the CRC was that it completely ignored the good aspects of the services rendered, and thus presented a one-sided picture. The interviewees felt that while the findings largely confirmed the general perception about public hospitals, the size and dimensions of the problems that emerged from the study constituted new information. For instance, the Citizen Report Card found that only 39% of patients received medicines free (over 70% had to pay a price), disposable syringes were used in 52% of the cases, and only 43% said the toilets were clean and usable. The Citizen Report Card also indicated the specific actions to be taken to solve the problems: a) An effective oversight mechanism to monitor the activities of maternity homes. It was suggested that a Board of visitors of 5-7 persons could play this role through quarterly meetings to review the operations, needs and plans of each hospital. b) A patients’ charter to be created for maternity homes. The services offered, time deadlines, terms of service, fees, patients’ rights and duties to be publicized. c) A system of patients sharing costs or user charges be introduced so that patients, particularly poor women, have a right to receive the services. The interviewees were enthusiastic in explaining the improvements brought about following the CRC. Action was taken on almost every finding in the CRC. To improve cleanliness, the cleaning function was entrusted to private contractors. Similarly, washing of linen was outsourced. This also helped in reducing the number of employees engaged in such work. Qualified nurses were appointed in place of untrained staff nurses, which made a difference in the quality of service. A Help Desk was set up in each hospital with the help of NGOs, and this guided the patients with respect to hospital procedures and services. User fees have been introduced and the revenue from this source is utilized by the same hospital. A Board of visitors comprising elected Councilors and some prominent citizens has been constituted for each hospital, and it is playing a useful overseeing role. Finally, a citizen charter has been created, and this provides information about available health services. On the whole, patients’ awareness had increased, and people seemed happy with the services. The majority of patients are poor, or lower middle class. The problems faced by the Corporation hospitals related mainly to vacancies of qualified persons and staff indiscipline at the lower levels. The latter has been overcome to a great extent by outsourcing services; filling up vacancies is a gradual process. What accounts for the changes that have been brought about? A novel feature of this particular report card on Maternity Homes was the follow-up to the study. In addition to specific recommendations made to improve the quality of services, the PAC collaborated with the management to help to implement them. This aspect was critical in the resultant improvement in hospital services. 8 Bangalore Development Authority (BDA) and Bangalore Telephones (BT) These two organizations are taken together as only one senior executive from each of them was interviewedthe former Commissioner of BDA and the present Principal General Manager of BT. In a way, these two agencies come out as the worst and the best in terms of rating in the Report Cards, and they present a study in contrast. The former Commissioner of BDA was fully familiar with the Report cards and had interacted closely with the PAC. According to her, the second Report card was almost a repeat of the first. She was critical of the methodology adopted by PAC and doubtful if the sample was representative. Secondly, she stated that the grouping of different service agencies in a comparative framework was not appropriate, as the parameters for evaluating performance would vary from agency to agency depending on the type of service provided. For instance, BDA was engaged in housing activity, which involved a lengthy process of land acquisition, development and sale of plots. All citizens were not clients of BDA as there are alternate service providers. On the other hand, agencies such as the Water Board or Electricity Board were monopolies. The Police had an entirely different role to play. Hence, she argued that comparing non-comparables, was bound to result in a distorted picture. The interviewee took note of the fact that the dissatisfaction of ordinary citizens with the services provided by BDA was strongly reflected in the Citizen Report Cardonly 2% were satisfied with the quality of services, 4% were satisfied with the information provided and 11% were satisfied with staff behavior. 95% of respondents visited its offices two or more times when they had a problem. The information about such a sorry state of affairs in the organization provided in the first Citizen Report Card spurred the BDA Commissioner to take a number of steps towards reform. These included: • Training programs for staff. The training was intended to improve staff behavior (in view of the numerous complaints about non-cooperation of staff in providing even basic information to members of the public), and to upgrade day-to-day skills of management, with a view to promoting greater efficiency. • Promoting transparency through greater interaction with citizen groups. • Measures to improve financial management (although this was not reflected in the second Report Card findings. The interviewee herself had sought the assistance of PAC and interacted with them for over a year in trying to implement the reforms. The Principal General Manager of BT was of the view that the Report Card was a good method of independent assessment. He also referred to the surveys carried out by the Telephone Regulatory Authority through external agencies. The marked improvement in telephone services in the years following the first Report Card was explained mainly by the market competition consequent to deregulation of the telecommunication sector in India. The entry of private operators had induced greater efficiency in the corporatized public sector. There was a greater thrust towards technology upgrading and computerization. But as deficiencies had continued, in areas such as staff behavior, various reforms were implemented in BT. These included: • Training for staff to achieve a greater customer orientation. 9 • Creation of internal feedback mechanismsa customer visiting the office is given a form to report his or her experience of dealing with the staff. Given the changed environment, customers’ expectations have also increased. State Officials The Citizen Report Cards on Bangalore assessed the performance of public agencies functioning at the city level. Hence, they did not generally attract the attention of state officials. But a few officials concerned with urban administration did take notice. The following five state officials familiar with the Report Card were interviewed: 1) Principal Secretary to the Chief Minister. 2) Additional Chief Secretary. 3) Principal Secretary, Infrastructure Development. 4) Principal Secretary, Food and Civil Supplies. 5) Secretary, Finance Department. Of the five, four had the advantage of having worked in the city PSAs and one had served as Principal Secretary, Urban Development. Their opinion was sought on the broader impacts and implications of the Report Cards initiative. The questionnaire used for interview is at Annex C. The impact of the CRCs was considered to be largely positive. Two interviewees acknowledged the influence of the Report Cards on them. The Principal Secretary to the Chief Minister who had earlier worked as Commissioner of the Municipal Corporation said that “the work done by the Public Affairs Centre, especially in the context of analyzing citizen satisfaction levels and feedback on the quality of public services, has had a profound impact on my work as a public manager”. He specifically referred to the major changes in Property Tax assessment in Bangalore as an outcome influenced by this process. He believed that “both public policy and its implementation are considerably refined by an energetic and comprehensive Report Card system”. The Finance Secretary of the State who had earlier worked as Director (Finance) in the Electricity Board said that the PAC enjoyed credibility as an impartial and independent organization and its Report Card “reflected what the user faces” and “helped to deal with the problems as seen by the users”. Interviewees also mentioned some limitations of the Report Cards, such as lack of wide publicity. The first Report Card particularly was not well known. One reason could be that the Report Card was in English and the local language translations were not prepared. Even in PSAs, the findings of Report Cards were known mostly to the top management. Moreover, the findings projected the PSAs in a negative light, ignoring their achievements. The interviewees referred to the proactive role of the state government during the last 3-4 years, i.e., since the second Citizen Report Card was made public. The leadership provided by the Chief Minister, particularly the establishment of the Bangalore Agenda Task Force (BATF), was particularly praised. The BATF acted as a bridge between service providers and the users. The PSAs started showing better performance, their heads received encouragement from the top, and they enjoyed longer tenures. 10 All interviewees agreed that a Report Card approach can be adapted to good effect in other contexts. One of the interviewees gave the example of how he successfully used this model in the Industries department where he was transferred from the Electricity Board. A regulatory framework was developed for the industries in Karnataka State, based on a survey to obtain feedback from industrialists. He followed the Report Card methodology, with support from PAC. Two senior state officials (who were not interviewed for this study) have stated publicly the significance and value of the Citizen Report Card approach. At the public release of the Millennial Survey on Kerala’s Public Services, held at Thiruvanathapuram on 26 November 2002, the Chief Minister of Kerala lauded the PAC for presenting the citizen feedback on government services without bias or expectation of favor, and promised action on the findings of the PAC study. The Chief Minister said: “I would like several Public Affairs Centres to be set up in this state. I need to get citizen feedback on government’s services without bias or expectation of favors. I want independent feedback that can be trusted. That it has been presented here without fear or bias is the merit of the PAC findings even though they are critical of our services. I propose to act on the findings provided by the PAC study”. At a public seminar organized by the Karnataka Minister for Industry with Dr. Samuel Paul, (the head of PAC) as the speaker at Bangalore, the Minister said that the feedback provided by the PAC study had helped the government to focus attention on the problems in the field and to think through the reforms needed to make government agencies more responsive to their customers. The Minister said: “We have in the past few months applied our minds to the feedback provided in your study. It has helped to focus our attention on the problems in the field that are not always given due attention. We have used the findings to think through the reforms we need to introduce to make the government agencies more responsive to their customers. You will see that the package of reforms we are taking to the Cabinet are radical and far reaching”. Citizen Groups Bangalore has a number of NGOs and citizen groups which have evinced active interest in civic issues. Representatives of seven of them were identified for the purpose of interview for this study. Selection of NGOs was based on several factors, including: their credentials; the need to elicit responses from a cross-section of interest groups; their active involvement in civil affairs; and their acquaintance with the Report Cards. The questionnaire used for interviews is at Annex D. The NGOs selected were: 1. Swabhimana. 2. CIVICCitizens Voluntary Initiative for the City. 3. AVASAssociation for Voluntary Action and Service. 4. CHCCommunity Health Cell. 5. APSAAssociation for Promoting Social Action. 6. CBPSCentre for Budget Policy Studies. 11 7. Resident Welfare Association (neighborhood groups). Swabhimana serves as an interactive forum between citizens and the local government, with a focus on community participation to resolve problems. CIVIC devotes attention to larger issues such as decentralization and urban planning. AVAS and APSA deal with problems of the urban poor, especially those of slum dwellers. CHC is involved in promoting and supporting community health action. CBPS is engaged in research and public policy including municipal budgets. The Resident Welfare Association is a neighborhood group concerned with civic problems of the residents of the locality. All of them were well acquainted with the Citizen Report Cards and the PAC, and were generally appreciative of the work done by the latter. They generally agreed with the CRC findings and believed that they reflected people’s perceptions. One of them said that the Report Card could serve as an "eye opener" to the PSAs. Another interviewee, however, had some misgivings about the methodology and opined that the responses could be subjective. How has civil society responded to the Report Cards? The leaders of the prominent citizen groups had no doubt welcomed the effort and some of them even praised it. The Report Card had helped them understand the processes of citizen-government interaction, especially how service providers respond to the needs of the people. A few of them were able to take up some of the issues raised in the Report Cards with the relevant agencies, such as redress of public grievances with BMC and BDA (Swabhimana and the Residents Association), and the setting up of ward committees with the government to press for decentralized governance (CIVIC). CBPS had participated in some of the meetings organized by PAC on the second Report Card. While the Report Cards had created greater awareness among those who had access to their findings, the interviewees felt that dissemination of findings was poor and did not percolate down the rungs of civil society. It was stated that the information from the CRCs had not been effectively used by citizen groups or the people. Knowledge of the Report Cards seemed confined to the English-speaking public in the absence of local language versions. The NGOs working in the areas of urban poverty reported that there were no conscious efforts on their part to pursue action on the specific findings of the Report Cards. One NGO representative was, however, involved in the study on Bangalore Maternity Homes, although she did not keep track of subsequent developments. Did the citizen groups see any improvements in PSAs following the Report Cards? One perception shared by all interviewees was that service providers had become more transparent. Following the second Report Card, PAC had organized a public meeting to bring together major public agencies and civil society, and this provided an opportunity to exchange ideas and experiences. The interaction between the PSAs and the citizens, especially the resident associations, increased, and officials were more prepared to share information. After the second Report Card, the state government also showed considerable interest in city affairs, and created the BATF. Improvements in the quality of service and the behavior of staff were visible in all the agencies though in varying degrees. Civil society on its own had shown greater activism in the last few years. There had been an increase in the number of NGOs and resident associations. There were increasing efforts to mobilize ordinary citizens to assert their rights and demand better services. Interviewees also referred to the role of PAC in educating citizens on voting rights and other civic issues such as the quality of roads, and municipal budgets. Government and PSAs had become more proactive and started collaborating with citizen groups, including PAC, in improving the quality of life. The Report Cards had served to focus attention on people's voice and to increase public awareness. 12 There were some suggestions to make the Report Cards a more effective tool. Citizen groups should be involved in the study so that people's needs are reflected more accurately. The surveys instead of being broad based could be targeted to certain key areas. The gap of five years between one survey and another was too long; an annual survey was seen as being more desirable. Clearly the cost implications of frequent surveys also have to be kept in mind. The citizen groups should be trained to make better use of the Report Cards and PAC should interact more frequently with them to achieve better results. One of the interviewees said there is need for more organizations like the Public Affairs Centre to impart credibility to civil society movements. Media The coverage of city issues in the Bangalore media has become prominent in recent years. Four representatives from the media were interviewedtwo from the Times of India (one is no longer employed there), one each from The Hindu and Kannada Prabha. The first two are English newspapers with countrywide circulation (the local news is confined to the State) and the third is a local language (Kannada) daily, a counterpart of the English daily, Indian Express. The questionnaire used for interview is at Annex E. All the interviewees opined that Report Cards were a very good effort at creating awareness about how the public service agencies fared in the eyes of the people. The findings were realistic, and in a way they "brought to light what existed". However, the study ignored the constraints faced by the PSAs in bringing about reforms. Another weaknesspointed out especially by the representative of the Kannada newspaperwas the non-availability of the Report Card in the local language. This had denied access to its findings by the non-English speaking population. In terms of impacts and outcomes, the general view of interviewees was that in the absence of self-evaluation by PSAs, the Report Cards served to highlight the problems faced by ordinary citizens and exert pressure on agencies to take corrective action. A number of factors accounted for the actual improvement in the quality of public servicesgovernment initiatives, people pushing for reforms, and media pressure. The BATF brought in expertise and resources from the private sector for city development. The increase in civic activism which was observed was mainly the result of the failures of the PSAs to deliver satisfactory services and a growing civic consciousness among the public. The media began to focus on the failures of PSAs and were goading them to action. Thus there were a number of parallel contributory factors, which influenced the changes in Bangalore. The Report Cards can be said to have accelerated the process. While the impact of the first Report Card was limited, the public meetings organized by the PAC after the second Report Cardwhere the media had also participatedcontributed to greater awareness. The media had reported the findings of the Report Cards but it was admitted by the media interviewees that there was no sustained effort on their part to track the action taken by the PSAs on their findings. However, wide coverage was given to any significant reform measure such as the Self Assessment Scheme of Property Tax, or the activities of the BATF. The journalists referred to the changing face of newspapers in recent years. There is increasing “local coverage” of the affairs of the city government and the day-to-day problems affecting citizens. In a way, the principles of citizen feedback have been adopted by newspapers (certainly in Bangalore) by devoting a separate column to readers’ views on topical issues. The interviewees were appreciative of the role played by PAC in promoting civic awareness. Apart from the Report Cards, it had carried out an analysis of municipal budgets (of the Bangalore Corporation) and shared the information in meetings with citizen groups. The PAC’s efforts to educate the citizens on voting rights (publication on “ABC of Voting”) and its 13 organization of candidate-citizen meetings on the eve of municipal elections were praiseworthy. Commenting on the role of PAC, the Times of India in its issue of November 8, 1999 had this to say: “Dr. Paul and his Public Affairs Centre have opened doors, even windows for a healthy tête-à-tête with our service providers ……. The public hearing into the bureaucrats and they responding not in anger, but with humility, admitting their faults and making very clear that they mean business: the welfare of the citizenry, the betterment in the quality of life, was uppermost on their agenda………. The Public Affairs Centre will, no doubt, keep up the pressure. And so should we.” To sum up, 31 persons were interviewed19 heads and officials of PSAs, 5 state official (4 of them also figured in the list of PSAs), 7 representatives of citizen groups, and 4 from the media. All of them supported the concept behind the Report Card, a substantial majority generally agreed with the findings, and some of them expressed reservations about the methodology used by PAC. They also offered suggestions to improve the design and use of Report Cards. 3. CONCLUSIONS The contribution of the Report Cards to the five potential outcomes and impacts identified for this reviewimproved quality of services, increased civic activism, increased public awareness, increased client orientation on the part of PSAs, and replication of CRCcan be assessed from the perspective of the objectives of PAC in initiating the CRC approach, and the environment in which the Report Cards were undertaken. The immediate objective of the CRC was to assess the satisfaction levels of citizens with regard to public services in Bangalore and to rank the PSAs in terms of their service performance. The Citizen Report Cards also benchmarked quality of service against a number of dimensions. It was hoped that public perceptions of service quality would stimulate citizen groups to exert pressure on service providers to take remedial action, and thus to improve the quality of their services. Bangalore Citizen Report Cards: Strengths and Weaknesses The Citizen Report Cards are perceived by the stakeholders who were interviewed as a valuable tool to obtain citizen feedback on the provision of public services in the city. The findings of the Bangalore Report Card were judged to be broadly accurate; and this was the first time that levels of citizen satisfaction had been measured in Bangalore. The findings of the two Report Cards generally presented a highly negative view of the performance of the city’s public services. The satisfaction levels of ordinary citizens were low, and only partial improvement was revealed by the second Report Card conducted five years later. Corruption had actually increased during this period, and the other dimensions of service such as staff behavior and problem resolution were not perceived to be acceptable. The concerns expressed by some of the intervieweess about the Report Card methodology need to be addressed. It is certainly the case that satisfaction surveys suffer from some drawbacks: responses can be influenced by subjective factors. And individuals may respond differently depending on, say, service levels in different locationswater may be supplied 8 hours a day in 14 one location and 2 hours in another. Even when service levels are comparable, levels of income or educational attainment may determine the nature of response. Thus satisfaction is determined not only by the actual service levels but by the respondent’s expectations. This is consistent with the finding that the proportion of poor households expressing satisfaction with quality of services was far higher than that of general households. Factors such as household characteristics, peer group effect, and equality of access to services are likely to influence the responses. In addition, the measurement of satisfaction response used in the Report Cards was based on a broad seven point scale, ranging from ‘very dissatisfied’ to ‘very satisfied’; this provides some scope for ambiguity. It is not easy to determine the relative magnitude of objective and subjective factors underlying citizen responses. But while it is reasonable to conclude that there was an element of subjectivity underlying citizens’ ratings, the feedback based on their perceptions nevertheless indicates their level of satisfaction or dissatisfaction with public services. The causes of satisfaction or dissatisfactionwhich are stated to be related to quality of service, staff behavior and corruptionidentify problem areas pertaining to a service or an agency. Secondly, some interviewees considered the sample size too small for a city of the size of Bangalore. There can be an argument for a larger sample but the sample size used does not substantially diminish the value of the study: the laws of statisticsspecifically the “law of large numbers”show us that it is the absolute size of a sample which is relevant for producing estimates which are statistically reliable (i.e., with high confidence levels), and not the sample size relative to the population being sampled. The survey was carried out by an independent and professional market research agency with the help of trained investigators. Sample size and design of the survey are usually planned keeping in mind the level of accuracy required. A third comment pertained to the study being confined to the residential households and the absence of feedback from industrial and commercial users who also benefit from civic services. This however seems to be a conscious decision of the PAC in defining the scope of the study. The focus of the Report Card is clearly on the household users including the poor. Another weakness of the Report Card that was stated by some interviewees concerned the PAC’s decision to restrict the Report Card focus to the customers or citizens, while not surveying the views or opinions of officials of the PSAs which provided the services. The poor quality of service may, at times, be due to factors beyond the control of the agencies. These can include shortages in the supply of water or power, outdated technology (telephones), failure by the State Government to release subsidy or grants in time (KEB, BMC), political interference, and short tenure of Chief Executives. While the purpose of the Report Card may be only to report what the people say and rate an agency on that basis, the perspective of PSAs may be of use in drawing conclusions about their performance in analyzing policy implications (last chapter in the Report Card). Repeat Report Cards could particularly capture improvements or positive aspects wherever users had noticed them. Outcomes and Impacts Improved Quality of Services The first Report Card in 1994 revealed that most citizens were dissatisfied with the quality of services they received. Following this, some action was initiated by PSAs to improve the state of affairs. The heads of two agenciesBWSSB and BDAhad taken note of the findings soon after the Report Card was published, and they interacted with PAC. They initiated training programs 15 (the latter with the assistance of PAC) with a view to improve the skills and customer responsiveness of the staff. In the BMC, a new property tax system intended to increase transparency and to reduce corruption was designed. KEB took some steps to improve its customer orientation, such as making bill collection easier through mobile counters. The performance of Bangalore Telephones improved considerably, mainly as a result of the use of modern technology, and in response to the competition it faced from private providers. The period following the second Report Card saw marked improvements in the quality of service provided by almost all agencies. There was greater awareness about the Report Card findings, and this had been facilitated partly by the presentations of them which PAC had made to each of the major public service agencies. These presentations helped to highlight the issues and challenges to the senior officials of each agency. Subsequently, PAC organized a public meeting to bring together the major PSAs and citizens. This provided an opportunity to the PSAs to respond to the Report Card findings and to inform the people of their own efforts to improve services. The publication of the second Report Card in 1999 coincided with major initiatives by the newly elected State Governmentsuch as the creation of the Bangalore Agenda Task Force, which consisted of prominent persons from different walks of life including industry and civil societyand reform measures undertaken by PSAs including efforts to create a citizen-friendly environment. These steps have contributed substantially to improvement in the quality of public services in Bangalore. Increased civic activism and public awareness Bangalore had an enlightened citizenry and a number of NGOs even before PAC came into being and the first Report Card was published. Issues relating to the urban poor, the environment, solid waste management, etc., had engaged their attention. It so happened that citizen groups, especially resident associations, increased in the years following the first Report Card. The PAC and the Report Card were created because the founder of the PAC was "deeply concerned about the deteriorating infrastructure and public services in the city, and the inability of the Government and its service providers to respond to the situation". The occasion was ripe for citizens to demand better services in a metropolis that was once called a garden city. A significant step was taken towards forging a partnership between public agencies and citizen groups, including PAC, with the setting up of a new forum, Swabhimana (self esteem), in 1995. Its goal coincided with that of the PAC and the Report Cardto stimulate thinking and action by Government and civil society towards improving the quality of public services and public accountability. It encouraged the formation of more Resident Associations or neighborhood groups to focus on local issues such as solid waste management, maintenance of parks, and other decentralized activities. PAC and Swabhimana have jointly organized meetings with civic groups on various city issues. The Bangalore Agenda Task Force has also served as a forum where civil society is represented and heard. The induction of Dr. Samuel Paul, Chairman of PAC, as a member of BATF is a recognition of the contribution of PAC and the Report Card in promoting civic consciousness. In fact, BATF has subsequently conducted its own report card based on citizen feedback, to assess the extent of improvements in service delivery. The role of the media in promoting civic activism has been significant. Various newspapers have carried the findings of the Report Cards. While they did not follow up by reporting the action taken on these findings, they have been active in reporting on civic issues and citizens’ activities. 16 The Government and PSA leaders are now more sensitive to what is reported in the media, and are seen to act on at least some of the shortcomings and failures which have been highlighted. Media coverage of matters relevant to citizens as well as events organized by citizen groups such as PAC have increased public awareness about issues relating to service delivery and accountability. Ordinary citizens have become sensitized to opportunities to assert their rights, especially in demanding information. There are now organized groups seeking information on budgetary details of BMC such as the funds allocated for public works in each ward, the expenditure on such works, etc. The efforts of PACsuch as its analysis and dissemination of BMC budgets, its campaign on electoral transparency, and its work to monitor road qualityhave made a significant contribution to this new activism. Increased Client Orientation After the first Report Card in 1994, there were efforts on the part of PSAs to train their staff to influence their attitudes to become more client oriented. Public grievance redress systems were put in place to deal with customer complaints. The assistance of some citizen groups including PAC was also sought. The results were more evident in Bangalore Telephones where competition had grown following the advent of cellular phones. But a beginning was made towards client orientation in all the agencies. After the second Report Card in 1999, and following a general improvement in services, PSAs have become more conscious of catering to the needs of their clients. Most public servicessuch as water, power, and municipal amenitiesare still monopolies, and all citizens constitute clients for these service providers. However, civic activism and proactive initiatives by the Government have induced PSAs to give the customer a more prominent place in their scheme of things. Citizen charters, the publication and supply of information under the Right to Information Act, the creation of PSA websites, documents explaining procedures and norms regarding availability of public services (for self assessment of property tax, sanction of building plans, issue of driving licenses, etc) are all examples of client orientation on the part of PSAs. The contribution of the Report Cards to these outcomes should not be viewed in isolation; these improvements reflected a congruence of Government initiatives, proactive civil servants, a civil society itching for improved governance, and a positive environment for reform. The Report Cards came at the right time to act as a catalyst. Replication of Citizen Report Cards The Report Card on Bangalore was the first of its kind in the world, and it has been replicated in other Indian cities such as Ahmedabad, Chennai, Delhi, Hyderabad, Mumbai and Pune (Iyer 1996; Paul and Iyer 1997; Balakrishnan and Sekhar 1998; and Paul 2002). PAC has supported this work in some of these cities with partner organizations which have also initiated a public dialogue and have advocated reform. The PAC has also undertaken similar report card assessment at the provincial and national levels, through a Millennial Survey of select states across India. The Industries Department in Karnataka has adopted the Report Card approach to get feedback from industrialists to improve governance in the industry sector. In Mumbai, a civic group called PRAJA has used PAC’s approach and produced a report card on the city’s services. PRAJA used the report card findings to jointly design a citizen charter on services with the City Municipal Corporation and made it operational. In addition, two cities in Ukraine and a social development project in the Philippines have preparedwith World Bank supportreport cards based on the PAC model. 17 The benefits flowing from a Report Card depend on the responsiveness of the PSAs and the use of the outputs by citizen groups and the media. A strong PSA with good leadership may initiate the required changes, while poor leadership may be unable to do so. In the Ukraine, city mayors are reported to have played a major role in acting on Report Card findings. There is also a role for civil society to play, to ensure dissemination of information through the media and other fora, and to exert pressure on PSAs to take remedial measures to improve the quality of their services. Does the generator of a Report Card have a role to play after the Report Card is published? The Bangalore experience suggests this is the case. PAC’s subsequent interaction with agencies, with a view to finding solutions for the problems identified, has shown tangible results. While an active role may not always be possible for civil society groups, any organization preparing a Report Card can play an enabling role in bringing the parties togetherthe PSAs and citizen groupsand initiating a dialogue on Report Card findings. A public monitoring system, which would need to be developed gradually, has the potential to strengthen processes of citizen feedback. Report Cards need not always cover all the public services or PSAs in a city in a single study. Although, such a study may give an overall picture of the city’s quality of services, the outcome may be limited in view of the difficulty in tracking actions taken by each agency. Report Cards can instead focus on a specific service such as water supply, public transport, or health: PAC’s Report Card on hospital services and its subsequent interaction with the Bangalore Corporation’s maternity homes has had an observable impact. Concluding Remarks How influential were the Citizen Report Cards in Bangalore? The answer varies from agency to agency: provision of report card information, however significant, is only one part of the story; how that information is used depends on factors such as leadership, resources, and the institutional environment facing each public agency. In the case of BMC, the information that lack of transparency in the determination of property taxes was responsible for corruption and loss of revenue proved critical in stimulating the agency to undertake property tax reform; this has yielded positive results. In KEB and BWSSB, the Citizen Report Card highlighted the problem of over-billing of consumers; in response both agencies took remedial action. With respect to the BDA, the high levels of dissatisfaction expressed by ordinary citizens with its services came as a shock to senior officials, who responded by initiating reforms. The Citizen Report Card findings have provided further impetus to ongoing efforts to improve service delivery. The finding that 20% of citizens had no dustbins near their houses pushed the BMC to focus attention to this aspect of solid waste management. The Report Card stimulated the KEB to undertake a survey on its own to obtain public feedback about its services. One significant issue that emerges from the Citizen Report Card work, and from the interviews conducted as part of this review, is that lack of information is a serious barrier that limits citizen access to public services. The emphasis on this aspect in the Citizen Report Card work has catalyzed the agencies to introduce greater transparency and customer orientation in their operations. It also led to greater interaction between the PSAs and citizen groups. Some general observations on the utility of citizen Report Card can be offered. The information which they provide may cover all users or some users such as households, or industries, or a particular category of consumer. Similarly, it may cover all public services in a city, or a specific 18 service such as water, power, transport or health. Such information can then be used to diagnose the problems relating to a service or an organization or a city. This could be done by the concerned agency itself or by the organization preparing the Report Card, if it has the capacity and resources to do so. The latter option may not always be feasible, however. Desirably, the agency would initiate a dialogue on the findings with citizen groups and professionals, and obtain inputs for taking remedial measures. In the case of the Bangalore Citizen Report Cards, the degree of impact varied between different agencies. With the first Report Card, largely at least two agencies (BWSSB and BDA) took the initiative to improve their performance, and they sought the advice of PAC. A third agency (BMC) took more time but eventually started consultations with citizen groups including PAC. This experience demonstrates that in a democratic setting, well-focused user feedback on services, made available by a civil society initiative, can stimulate at least some service providers to respond. In the case of the second Report Card, the PAC played a proactive role by making presentations of the findings to each PSA. This interaction created greater awareness of the Report Card within the agencies. Civil Society was involved more actively in open fora such as public meetings. This was the time when government also became proactive. Thus a climate for change was generated. While action came from public service agencies, the PAC had succeeded in stimulating thinking and providing some inputs. With the Report Card on Bangalore Maternity Homesa feedback from users of a specific servicePAC played a more proactive role. Meetings were held with the senior officers of the Corporation to discuss the findings and identify specific actions to be taken. The recommendations were implemented, and this led to improved service delivery. A progression in the influence of the Report Cards can be seen, from limited impact (with dissemination of feedback) to more impact (with dialogue and public pressure for change) to greater impact (with advice on reform), corresponding to the reactive, proactive and reformist roles of the PAC over a period of time. The skills, resources and organization required to play all three roles will not always be available to civil society organizations, however. In brief, the Bangalore Citizen Report Card was a pioneering effort to assess the satisfaction levels of ordinary citizens concerning the quality of public services and the performance of service providers. In spite of some weaknesses, it has contributed to increasing public awareness on civic issues, nudged some key officials and agencies to action, and stimulated citizen groups. Governments are not always able to monitor all aspects of services. Hence the importance of user feedback as an aid to monitoring and corrective action. The Public Affairs Centre has shown that independent civil society groups can play a useful role to provide such feedback which can act as a stimulus to reform. 19 Annex AComments of the Public Affairs Centre Dr. A. Ravindra’s assessment of Public Affairs Centre’s report cards is both comprehensive and balanced in terms of the stakeholders interviewed and the range of questions probed. The exercise was attempted almost a decade after the first report card on Bangalore was published. The lapse of time did indeed provide a sense of perspective on the intervention under study. Its limitation, however, is that the institutional memory of the report card in the public agencies involved has faded somewhat, some of the officials involved having left or retired. Tracing the sequence of events and influences has not been an easy task for the author under the circumstances. The report cards on Bangalore that Dr. Ravindra has described and evaluated in his paper started as a personal initiative by a concerned citizen. Some of the limitations of the report cards pointed out in the paper reflect the experimental nature of the project in its early stages when there was very little organizational and financial resources behind it. For example, the dissemination of the findings of report cards was largely confined to the press and a few seminars. Some of the newspapers were also in the local language and would have reached a large number of citizens who were not conversant with English. Communication and discussions were mostly with citizen groups and agencies who are conversant with English. Presently, it is standard practice to disseminate PAC’s reports in different languages. The author has commented on the limited impact of the report cards in galvanizing the organized NGOs in the city. But the fact is that these NGOs could not have become the champions of the kinds of civic issues being raised in the report cards. And PAC did not focus its attention on them. What happened subsequently is that a new breed of neighborhood resident associations began to emerge and play an active role along with PAC in mounting new civic advocacy efforts in the city. This process has over time led to a different style of advocacya shift from “shouting” to “counting”. It has led to a different way of using the power of information by civic groups on a variety of new issues. The measurement of satisfaction is a controversial subject. It is subjective in the sense that a person makes an internal assessment of his/her satisfaction with a service. Since different persons can have varying expectations and standards, the resultant measures cannot be claimed to be truly “objective”. On the other hand, if a person makes the assessment based on recent experience, this measure will contain information that cannot be easily obtained from any other source. It is no coincidence that business enterprises routinely gather such information from their customers. Further analysis of PAC data and user feedback gathered through similar studies show that satisfaction does measure important dimensions of service provision (Deichmann and Lall 2003). This means that subjectivity does not wipe out the significance and usefulness of the information. The report card focuses on user feedback as governments and monopoly service providers typically ignore such information. It is presented as “voice” of the people to which many public agencies are not exposed. Take the poor, for example. Their experiences and problems are usually articulated by their representatives, politicians, or NGOs, etc. In a report card, the poor speak directly and sampling ensures their representativeness. It is true that this information represents the users’ side of the story only and not of the provider staff. The agency staff also could be interviewed, but PAC did not feel that it should do it. In a few cases where PAC tried to interview some of the staff, it did not find them to be forthcoming. The service agencies have the resources to question them as the staff are under their control and they have the resources. PAC’s view is that the agencies should do their own internal studies and check the veracity of the report card findings. In fact, in Bangalore, several agencies have done this and have taken corrective action highlighted in the report cards. 20 Annex BQuestionnaire For Public Service Agencies 1. Please discuss the extent to which you are familiar with the activities of the Public Affairs Centre. 2. Comment on the two Citizen Report Cards on Bangaloretheir strengths and weaknesses. 3. What was the response to the findings concerning your agency? What actions were proposed to address the criticisms and concerns of the surveys? 4. What actions were actually taken? Give examples. What impact did they have on the way services were delivered and people / customers treated? 5. Was there increased civic activism concerning public services, such as asserting the right to demand better services following the CRCs? 6. What were the problems, if any, faced by your agency in implementing the proposed changes / reforms? 7. Were there any differences in the way your agency responded to the general public and the poor / slum dwellers? 8. A number of improvements have taken place in Bangalore since the two Report cards were published. To what extent would you attribute such improvements to the impact of the Report Cards? 9. Can the Report Card approach be replicated in other cities? 10. Any other information / opinions? 21 Annex CQuestionnaire For Senior State Officials 1. Please discuss the extent to which you are familiar with the activities of the Public Affairs Centre. 2. Comment on the two Citizen Report Cards on Bangaloretheir strengths and weaknesses. 3. What in your opinion was the influence of the CRC, on the following possible outcomes and impacts: a) Improved quality of services? b) Improved civic activism concerning public services (e.g., BATF)? c) Increased public awareness of the quality of public services, and the belief that citizens have the right have the right and the power to demand better services? d) Increased client orientation on the part of public service agencies? e) Replication of the Report Card approach in other cities and countries? 4. Give an idea of the broad impact of CRCs at the state level. Can the CRC approach be replicated / scaled up? 22 Annex DQuestionnaire For Citizen Groups and NGOs 1. Please discuss the extent to which you are familiar with the activities of the Public Affairs Centre. 2. Comment on the two Citizen Report Cards on Bangaloretheir strengths and weaknesses. 3. With which public service agency / services is your organization concerned? Do you agree with the findings pertaining to that agency / service? 4. What was the response of civil society groups to the Citizen Report Card? Did they (including your group) propose any action? 5. What were the actions taken by the public service agencies? 6. What impact did they have on the way the services were delivered and the people / customers were treated? 7. Did you notice any difference in the way the agencies responded to the general public and the poor / slum dwellers ? 8. What were the problems, if any, faced by the PSAs in implementing the proposed changes? 9. A number of improvements have taken place in PSAs and in Bangalore since the two Report Cards were published. To what extent would you attribute such improvements to the Report Cards? 23 Annex EQuestionnaire for the Media 1. Please discuss the extent to which you are familiar with the activities of the Public Affairs Centre. 2. Comment on the strengths and weaknesses of the two Citizen Report Cards on Bangalore. 3. What in your opinion was the influence of the Report Cards on the following possible outcomes and impacts: a) Improved quality of services? b) Improved civic activism concerning public services (e.g, BATF)? c) Increased public awareness of the quality of public services, and the belief that citizens have the right and the power to demand better services? d) Increase client orientation on the part of public service agencies? e) Replication of the Report Card approach in other cities and countries? 4. What role did the media play in disseminating information / keeping track of actions taken by public service agencies? 24 BIBLIOGRAPHY Balakrishnan, Suresh and Sita Sekhar. 1998. Public Services and the Urban Poor in MumbaiA Report Card. Bangalore: Public Affairs Centre. Deichmann, Uwe and Somik Lall. 2003. Are you satisfied? Citizen Feedback and Delivery of Urban Services. World Bank Policy Research Working Paper 3070. Washington D.C.: Development Research Group, The World Bank. Goetz, Anne-Marie and John Gaventa. 2001. Bringing Citizen Voice and Client Focus into Service Delivery. Sussex: Institute of Development Studies. Iyer, Anjana. 1996. A Report Card on Public Services in Madras. Bangalore: Public Affairs Centre. Paul, Sam. 1995. A Report Card on Public Services in Indian CitiesA View from Below. Bangalore: Public Affairs Centre. _____. 2002. Holding the State to Account: Citizen Monitoring in Action. Bangalore: Books for Change. Paul, Sam and Anjana Iyer. 1997. A Report Card on Public Services in Calcutta. Bangalore: Public Affairs Centre. Paul, Sam and Sita Sekhar. 2000. Benchmarking Urban Services: the Second Report Card on Public Services in Bangalore. Bangalore: Public Affairs Centre. Sekar, Sita. 1999. Public Services and the Urban Poor: A Comparative Assessment Based on Citizen Feedback from Five Indian Cities. Bangalore: Public Affairs Centre. _____. 2000. Maternity Health Care for the Urban Poor in Bangalore: A Report Card. Bangalore: Public Affairs Centre. 25 Other Papers in This Series #1: Keith Mackay, Lessons from National Experience. #2: Stephen Brushett, Zimbabwe: Issues and Opportunities. #3: Alain Barberie, Indonesia’s National Evaluation System. #4: Keith Mackay, The Development of Australia’s Evaluation System. #5: R. Pablo Guerrero O., Comparative Insights from Colombia, China and Indonesia. #6: Keith Mackay, Evaluation Capacity Development: A Diagnostic Guide and Action Framework. #7: Mark Schacter, Sub-Saharan Africa: Lessons from Experience in Supporting Sound Governance. #8: Arild Hauge, Strengthening Capacity for Monitoring and Evaluation in Uganda: A Results Based Management Perspective. #9: Marie-Hélène Adrien, Guide to Conducting Reviews of Organizations Supplying M&E Training. #10: Arild Hauge, The Development of Monitoring and Evaluation Capacities to Improve Government Performance in Uganda. #11. Keith Mackay, Two Generations of Performance Evaluation and Management System in Australia. Other Recommended Reading OED, Influential Evaluations: Evaluations that Improved Performance and Impacts of Development Programs, 2004. OED, 2002 Annual Report on Evaluation Capacity Development. OED, Monitoring and Evaluation: Some Tools, Methods and Approaches, 2002. Development Bank of Southern Africa, African Development Bank and The World Bank, Developing African Capacity for Monitoring and Evaluation, 2000. K. Mackay and S. Gariba (eds.), The Role of Civil Society in Assessing Public Sector Performance in Ghana, OED, 2000. Other relevant publications can be downloaded from OED’s ECD Website: >http://www.worldbank.org/oed/ecd/>