84371 The World Bank PREMnotes 2014 NUMBER 29 Special Series on Performance Management Can Improve Local Services in Developing Countries: The Service Improvement Action Plan Katharine Mark The Service Improvement Action Plan (SIAP) is a performance management tool developed to help local governments—especially those in developing or transition economies—improve their services. It has been used effectively by local governments with limited resources in a number of countries, including Albania, Honduras, Kosovo and Pakistan, among others. This note describes SIAP implementation to date, outlines some of the tool’s advantages and limitations, and considers the SIAP’s potential for replication. Local governments are well positioned to reap number of other cities in the United States that the benefits of performance management: their have used performance data in service delivery, decisions about services (informed by perfor- including Des Moines, Iowa, where performance mance data) can have rapid results that are easy data reported by citizens on handheld computers to see. In fact, local governments have been at the were transmitted to city computer systems and forefront in adopting performance measurement. the workload data were used to reorder service Local governments in the United States and the priorities. United Kingdom, for example, have been measur- Despite these promising trends, only a small ing outcomes since the 1970s, and local govern- fraction of the local governments who measure ments worldwide have increasingly been adopting performance actually use the data for resource performance indicators and monitoring outcomes allocation or services management. Leadership (Hatry 2008). Some local governments have gone in cities that have been successful in introducing further and have actively used the performance performance measurement continue to speak of data to improve local services. Perhaps the best “the difficulties of persuading managers to use known example is the “stat” system in the United performance data in their management strate- States, first pioneered in New York City as Comp- gies” (Sanger 2008). Obstacles include rigid Stat, in which detailed crime statistics were used budgeting practices, where those who know by the police department to reduce the crime rate. best how to reallocate resources do not have the In Baltimore, the CitiStat system was rolled out authority to do so, and centrally controlled civil across the whole of city government, with all de- service systems that restrict local governments partments reporting biweekly to review meetings from reallocating human resources. Also, if per- focused on selected performance indicators (Perez formance measurement is introduced top-down, and Rushing 2007). Sanger (2008) describes a it might be more difficult for local government FROM THE POVERTY REDUCTION AND ECONOMIC MANAGEMENT NETWORK staff to clearly see its potential, or see how straightforward and make improvements that could be seen and measured it can be to analyze and apply performance data to the by citizens, businesses, and city staff. The local govern- practical requirements of service delivery. ments in Albania—and indeed in most other countries Given the prevalence of this disparity between mea- where the SIAP was introduced—did not have any earlier surement and the use of resulting data, it is worth consid- experience with performance management, therefore ering the Service Improvement Action Plan (SIAP), which its practice emerged through the implementation of the has been effective in helping a substantial number of local SIAP. But in a number of Albanian municipalities, the ap- governments with limited resources implement perfor- proach led to some surprisingly rapid and visible results. mance management to achieve measurable improvements For instance, when the city of Pogradec implemented a in service outcomes. The following sections describe the SIAP to address problems with garbage collection and SIAP methodology, outline its strengths and weaknesses, street cleanliness, the percentage of households receiv- and conclude with discussion of its potential for replication ing regular garbage collection rose from 70 to 75 percent in other countries. within one calendar year, while the percentage of citizens satisfied with cleanliness in the city increased from 75 to The Service Improvement Action Plan 91 percent. At the same time, cost recovery for the clean- ing service increased from 54 to 76 percent (Cooley et The SIAP can be described as a bottom-up approach that al. 2007). Other Albanian cities, including Erseka and allows local government operational staff to be the prime Kucova, focused their SIAPs on the condition of streets actors in using performance management techniques to and sidewalks, public lighting, water supply, libraries, and do their jobs. The SIAP provides a semi-structured set of cost recovery: basic management steps that explicitly focus on outcomes, • The city of Erseka chose to tackle the maintenance of such as local roads in good condition, clean cities, or ac- streets and sidewalks. Both were in bad shape due to cess to drinking water. Local government working groups, deferred maintenance, lack of investment, and poor typically led by frontline managers, but often including practices such as parking on the sidewalks. Follow- community representatives, carry the SIAP forward, us- ing implementation of the SIAP, the percentage of ing performance data to allocate resources and inform people surveyed who rated maintenance of streets decision making to improve services. The key performance and sidewalks in their neighborhood as “good” or indicators measure outcomes that reflect not the process “very good” grew from 42 to 78 percent within of delivering the service, but the result. SIAP commonly one year. involves the following steps: • The city of Kucova identified poor street lighting—bro- 1. Select one service as the focus. ken or nonexistent lamp posts, missing bulbs, and poor 2. Establish a working group. maintenance—as a major problem because of its impact 3. Analyze the service, priority needs, and options. on safety. The percentage of citizens who said the city 4. Identify the specific outcomes that are to be improved. was “well lit” or “fairly well lit” rose from 34 percent in 5. Identify indicators to measure progress, including both 2004 to 65 percent in 2005 and 78 percent in 2006 output and outcome indicators. (Cooley et al. 2007; Vokopola 2013). 6. Identify data sources and collect baseline data. In the last decade, some of the same approaches have 7. Set targets for the selected indicators. been used in a number of other transition and developing 8. Develop an action plan to reach the targets. countries. In addition to Albania, the SIAP has been carried 9. Carry out actions and monitor the indicators. out in scores of local governments (most mid-sized, but The SIAP was first implemented in Albania in 2000. some quite small) around the world. Box 1 describes the At that time, Albania’s local governments were in dif- SIAP implementation in Honduras. ficult circumstances. Their country was impoverished after a long dictatorship and closed economy. The local governments were gaining new responsibilities through Key Elements in SIAP Methodology decentralization, but they had limited authority, little To date, the SIAP has been successfully implemented in experience, and scant resources. Moreover, they had inher- seven countries and introduced in at least four others ited a legacy of deferred maintenance in infrastructure and (box 2). In all countries, the SIAP followed the same steps a lack of trust from citizens. There was a strong desire to outlined above, although in many cases local governments improve services cheaply and quickly and to build public added their own approaches. This section describes the confidence: the SIAP became a means to that end. The nine major steps as they are usually carried out, along with SIAP helped local governments focus on priority services other specific applications. 2 PREMNOTE Box 1. SIAP Case Study: Yamaranguila, Honduras Yamaranguila is a small municipality of 16,000 in the southwest region of Honduras. Its population is scattered across an urban center (ca- becera) and about 100 rural settlements. Yamaranguila implemented a SIAP in 2005. The working group was made up of 12 representatives of community associations, patronatos, from both the rural areas and the urban center, along with six municipal employees. In prioritizing the service issues for the SIAP, the group convened an assembly of community groups. Reflecting the largely rural composition of the population, the working group decided to focus on an issue that mainly affects the rural population of the municipality: the maintenance and repair of the access roads to the smaller settlements. While deliberating on the problem, the working group observed that the high mountain area of the municipality was virtually inaccessible by road, which meant that mountain residents had to walk two to six hours to get to the town center. Residents with vehicles had to use alterna- tive routes to get to the town, traveling 12 instead of 5 kilometers (km). Farmers had difficulty bringing their products to market, and families were unable to access services in the town, especially health services. Even in the urban center, the streets were in such bad conditions that access to some of the neighborhoods was very difficult, especially during the rainy season. The municipal employees who participated in the SIAP recognized that “there was no budget allocation for maintaining urban streets or rural roads.” One of the community leaders put it more directly: “Before, street repairs were an act of God—when the central government decided to send equipment.” The working group used the Trained Observer Ratings methodology to measure the condition of the different roads and streets of the mu- nicipality, dividing it into six main areas. They measured the number of kilometers of road in each area that were in good, middling, bad, and very bad conditions. At the same time, they established as an overall goal to increase the 7 km of streets in “good” condition in 2005 to 14 km by 2007. These ratings were used to determine which sections of road were in worst condition and therefore prioritized for repair work. The action Results indicator 2005 (baseline) 2006 (target) 2007 (target) % of citizens who consider streets “accessible” 69 75 80 Kilometers of roads in “good” condition 7 10 14 Kilometers of roads in “very bad” conditions 3 1 1 plan presented to the municipal council by the SIAP group was based on the mobilization of community associations to provide repair and maintenance. The plan included estimates of costs per kilometer using community labor, which were substantially lower (less than half) than what the municipality had been paying to private contractors using heavy machinery. The plan was approved by the municipal council, which assigned a budget of 75,000 lempiras (approximately US$4,000) for the 2006 budget cycle. According to the community leaders in the SIAP group, approval by the council was partially due to the fact that the labor-intensive technol- ogy would help poor families generate income. Some of the council members questioned whether the repairs and maintenance would be of sufficient quality, but the plan also included a mechanism for program oversight to monitor work quality. Plan measures also ensured trans- parency in geographic prioritization and selection of beneficiaries, which helped forestall concerns about political favoritism or clientelism in choosing in which communities to work. The SIAP made it possible for the community and local government to agree, for the first time, upon priority infrastructure improvements, funds to be appropriated for street repairs, and the roads to be improved. A sustainable capacity for monitoring performance made it likely that oversight would continue. This was the first instance of collaboration between government and civil society around service improvement, and was seen by both sides as useful and positive. Source: Cartier 2005, and author’s personal knowledge. Step 1. Selecting a service input, as well as deliberations among city technical staff As noted, many cities have focused on a traditional service and leadership. 1 area or department—for example, roads or public lighting. Others have found, however, that the SIAP process can also Step 2. The working group address more complex problems or issues that do not neces- A cross-cutting working group is responsible for developing sarily reside within one city department. Service areas that and carrying out the SIAP. The composition of the working cities in the Russian Federation, Albania, and Kyrgyzstan group is critical because of the diverse functions it needs to have chosen include: juvenile delinquency, the avian flu, carry out, including identification of the SIAP focus area tourism, economic development, tax and fee collection, and desired outcomes; oversight of data collection; devel- and celebration of traditional local holidays (community opment and implementation of the action plan for service pride). Selection of service areas has been based on feedback improvement; and handling the budget implications of these from surveys identifying citizen priorities, city council activities. Typically the working group is led by a department PREMNOTE 3 is to move backward from the outcome (“clean streets”), Box 2. Countries Implementing SIAP rather than to start with an output (“we will build a Afghanistan landfill”). There may be more appropriate, cost-effective Albania ways to reach the outcome. In addition, focusing on an Georgia outcome that is important to all stakeholders usually Honduras Kosovo makes it easier to ensure full ownership and to get all Kyrgyzstan resources working together. Pakistan It is in that spirit that the working group should review Rwanda the service in question, noting problems, the priorities of Russian Federation citizens (sometimes using feedback from a survey) and Tajikistan different options for addressing problems, and then use Turkmenistan that review to identify first the principal outcomes sought Source: Author’s compilation. Note: For countries in italics, the SIAP was introduced, but results and then decide upon the actions needed to move forward are not known, or it is too early to tell. For example, in Afghanistan, (box 3). Depending on the complexity of the issue, the SIAPs were initiated in four municipalities in the first half of 2013, so results will not be known until 2014. knowledge of working group members, and the existence of prior analysis, it may not be necessary to undertake a major—and expensive—study. head, such as the director of public works, but it should The situation analysis is continually updated as the also include a representative of government leadership or SIAP process goes forward, and revisited at least once a year. higher management to ensure access and empowerment. Other actors to include would be those involved in planning Steps 4 and 5. Choosing outcomes and delivering the service, such as contractors (for example, and defining indicators a waste management company), if they are major players in Choosing the specific outcomes that will be the focus implementation. It can also be useful to bring in a member of the SIAP is an important step. The target outcomes of outside interest groups, such as citizen groups. Usually there is a smaller “core” group responsible for the majority of SIAP implementation, but other members can play an Box 3. Situation Analysis Questions and Facts important role in planning and in brainstorming ideas, Sample discussion topics for the situation analysis as well as in reaching out to other departments or groups. In the municipality of Kucova, Albania, the working • What do we know now about this priority area? group for improving the business environment included • What have citizens said they need? the Director of Revenues, the Director of Public Services • Would it help to bring an expert, local farmer, and the Urban Planning Specialist, as well as the Deputy teacher, or business person into the group as a Mayor. Additional members included the head of the member? Economic Commission, the Chairman of the Chamber of Selected characteristics to cover in the analysis Commerce, the directors of two banks, and a representa- • Geographic demographic information (population tive from the taxes directorate. For a SIAP in Russia that density, income, and education) relevant to this targeted juvenile delinquency, the working group included service. representatives from the schools, police, the health depart- • Characteristics of the service and how it is de- ment, and the community center. This broad-based support livered (whether by the local government, enter- enabled the group to propose a solution that might not have prise, or NGO). been conceived by any department independently to help • What are the costs? Are there trend data? Does achieve outcomes of interest to all, such as the safety and the right labor force exist locally to get the job well-being of youth. done? Step 3. Situation analysis and focus on outcomes • What services are currently offered to citizens (for example, number of collections per week, opening It is important to ensure that outcomes—and not outputs— hours)? are the SIAP’s focus. What citizens care about is not “to pave more of the local roads,” but “to have roads in good • What social or cultural traditions might affect or contribute to service delivery? enough condition so that children can get to school in the rainy season,” and therefore producing those results • Are there any legal issues to be considered? should be the government’s objective. The SIAP approach Source: Cooley et al. 2007. 4 PREMNOTE are usually identified during the situation analysis, and outcomes are most useful for planning and prioritization, it may be necessary to narrow the focus to just one or but during implementation, outputs are also tracked. For two, especially if resources are limited. For example, example, the outcome sought might be a reduction in both adult literacy and elementary school dropouts may the number of dropouts, but the number of teachers and be important to the community, but addressing them teacher absenteeism is also being tracked. The selection of simultaneously risks diluting the effort and reducing indicators is one area in which working group members effectiveness. At the same time, even in one specific area, may need training. The working group will need to identify there are many layers of outcomes, from “intermediate” outcome indicators that are truly measurable, practical and outcomes such as clean streets to “end” outcomes such actionable, as well as reliable over time.2 as more tourists spending money at local shops. SIAPs Step 6. Data sources usually track the various layers. Some working groups Cities considering the adoption of performance manage- consult with end users (for example, through focus group ment are often concerned about the cost of data collection. discussions or through nongovernmental organizations The most frequent data source for SIAPs has been local [NGOs] in the group) to ensure they have identified the government administrative records. Working groups are right final outcomes. often surprised at how much information already exists, Some working groups have developed “logic models” but have also had to struggle with data quality and improv- to help unearth the right outcomes and shed light on the ing data reporting processes to ensure data consistency and relationship between the activities and outputs of the lo- currency. The most valuable data for measuring outcomes cal government, such as organizing a community project are usually drawn from surveys, and many of the local to repair the roads, and outcomes such as improved road governments that have implemented SIAPs have tried to conditions and fewer accidents. Figure 1 provides a simple incorporate such data, from multiservice citizen surveys logic model with indicators for the road repair project or more narrowly focused user surveys. To minimize costs, carried out in Yamaranguila, Honduras. The boxes show cities have partnered with NGOs to carry out user surveys the inputs, outputs and outcomes, with their respective and have included surveys with bills and distributed them indicators below. through schools. A helpful data source that can be used to Some of the outcomes that have been targeted by SIAPs rate cleanliness, road conditions, and a host of other services in different cities include: draws from the practice called Trained Observer Ratings.3 • citizens are safer walking alone at night in the city The ratings can be used to identify and prioritize areas that (public lighting); need attention (for example, city areas that are least clean, • streets are cleaner and tourism rises (garbage service); segments of road that are in poorest condition) as well as • fewer traffic accidents (road repair); to monitor improvements over time. This approach is well • greater cost recovery (billings and collection); and suited to citizen involvement because it allows lay people to • bullying of school children is reduced (schools, health, assign scores using a predefined rating scale based on only and safety). brief training. In Georgia, for example, youth groups in A focus on outcomes does not mean that outputs more than 10 cities began rating street cleanliness, which and/or intermediate outcomes will be neglected. The end proved to be a valuable source of information for city clean- Figure 1. Logic Model with Indicators for Participatory Road Repair Project in Honduras provide construction materials citizens better road fewer quantity of inputs repair road condition accidents recruit and sign up citizens number of percentage of number of kilometers streets in traffic number of citizens repaired and good condition accidents signed up maintained Source: Author’s illustration. PREMNOTE 5 ing crews while broadening young people’s civic awareness action plan includes the work needed for performance (Mark 2008). measurement (such as data collection), but the heart of it defines the actions that are going to improve the service. For Steps 7 and 8. Targets and the action plan some local governments, simply writing down and formal- Targets play a critical role because of the importance of moti- izing those plans is a new process. In Afghanistan, after a vation to SIAP effectiveness. Targets are identified for every presentation of the action plan, a mayor said to the working indicator, including both outputs and outcomes. Local group: “You should be the mayor. You have a real plan for governments should always set the targets internally, with making this service better” (Reed 2013). In the action plan no involvement of central government or outside experts. for street cleanliness prepared in Lezhe, Albania (table 1), The targets are set in the context of a known baseline and explicitly planning a process to determine the best place- a specific action plan, which makes them more likely to be ment for garbage bins was the first step. realistic. However, if the targets are not ambitious enough, they are not likely to get city council support, which is espe- Step 9. Implement action plan cially important when additional resources are needed. In and monitor indicators some cases, the targets appear only in the SIAP document Once the SIAP has been approved, there are typically many itself and are known only by service managers and city lead- simultaneous actions that need to be taken, often by sev- ership. In other cases, the targets are made public, which eral different groups, for example, by the directorates for leadership may perceive as risky, but public dissemination Services, Finance/Budget, and Economic Development in can be useful in justifying a planned expense or in balancing the case of Lezhe. The diverse composition of the working revelations of poor past performance. Figure 2 presents an group proves useful at this stage, as the individual members excerpt from the budget presentation of Pogradec, Alba- will each need to keep the process moving by tackling dif- nia, which included the accomplishments of the past year ferent measures. The working group continues to meet pe- (2005) in comparison to set targets as well as the new targets riodically to keep informed on how each step is progressing. set for 2006, the upcoming budget year. In this case, their The city of Lezhe began to see positive results quite targets included not just increased satisfaction, but also quickly. A youth group carried out Trained Observer Rat- the percentage of cost recovered for the cleaning service. ings; between 2005 and 2006, the percentage of streets The action plan is developed by the working group, rated as “clean or very clean” grew from 0 to 47 percent. At often reflecting inputs from many different sources. The the same time, city collections increased: by 2006, 27 per- cent more citizens, 18 per- cent more businesses, and Figure 2. Budget Presentation from Pogradec, Albania 46 percent more institutions were paying the cleaning fee General Objectives — Targets than the previous year. The additional funds were then Survey Target Survey Change Target reinvested to improve the No. Outcomes 2004 2005 2005 (NB)a 2006 service. (%) (%) (%) ± (%) The cost of implement- 1 % of citizens ing SIAPs varies widely. In satisfied with cases where resources are 75 82 91 +16 93 cleanliness in the limited, SIAPs may include city only simple steps, realloca- 2 % of citizens satisfied with tion of resources, or citizen- 48 60 72 +24 75 fuelled activities. For larger cleanliness in the neighborhoods efforts, additional funds are 3 % of households required, either from mu- receiving regular nicipal budgets or in some 70 78 75 +5 78 garbage collection cases from external grants.4 service Many action plans in- 5 % of cleaning clude a formal link with the service cost 54 75 76 +22 85 recovery local budget process during Source: Budget presentation, Pogradec, Albania, 2006. implementation, especially a. Percentage point change in indicators between 2004 and 2005. if they are requesting sig- 6 PREMNOTE Table 1. Action Plan from Lezhe, Albania, for SIAP on Street Cleanliness What When Who Improve cleaning and garbage service fee collection by Continuously Local Revenues Directorate providing every family with a passbook Services Directorate Conduct cost analysis, issue request for proposal to procure July–August Budget Directorate metal and plastic bins, and determine the best placement 2005 Services Directorate across the city Include the output and outcome indicators from the SIAP in December 2005 Services Directorate the contract of the cleaning service Working group Present the SIAP indicators to the council as part of the December 2005 Services Directorate draft budget for 2006 Finance/Budget Directorate Purchase 150 metal bins and 50 plastic bins and place ac- February–March Services Directorate cording to the scheme prepared by the Services Directorate 2006 Contractor Establish Trained Observers Rating group and ensure rat- May 2006 and Services Directorate ings are conducted annually annually Working group thereafter Prepare and disseminate public awareness materials about Starting Services Directorate keeping the city clean and pay the cleaning and garbage February–March Economic Development collection service fee 2005 Directorate Review performance indicators to monitor the cleaning and July–September Services Directorate garbage collection service contract every year Source: Cooley et al. 2007. nificant funds or budget reallocation to reach a target. In However, the actual SIAP process is not dependent the example from Pogradec (figure 2), targets and recent on the personal involvement of city leaders. Unlike performance were provided in budget presentations. Some CitiStat, for example, which is a monitoring tool for cities in Albania began including indicators and targets in leadership, the SIAP is a tool for line managers to make their formal city budgets, linked with specific activities decisions and manage operations. Once established, and their cost. the SIAP can move forward effectively without requir- ing direct actions by senior leadership. Replication • Expertise. Required technical know-how is fairly nar- row, but still essential. The local governments need In most of the examples described in this note, SIAPs were someone with knowledge of performance manage- introduced with support from international donors. To ment—especially to help with understanding outcomes date, there are only a few documented cases of local gov- and developing indicators—and facilitators to help ernments carrying out SIAPs without external technical the working group with its first steps. Knowledge of assistance. To what extent local governments would be sampling is necessary to conduct surveys, but although able to initiate SIAPs independently is therefore still an surveys are useful, they are not essential for SIAPs. open question. Similarly, expertise in preparing survey instruments Factors to consider when deciding on whether to is useful, but model questionnaires are also widely introduce a SIAP include: available. Techniques such as Trained Observer Rat- • Leadership. It is clear that strong commitment by city ings require initial training, but are otherwise easily leadership is essential. The SIAP is not necessarily replicable. a daunting task, but it is a new way of doing things • Resources. The most significant cost for the SIAP is the and necessitates that senior staff have the time and time of the working group. Data collection costs could resources to follow through implementation. This be substantial if they include a wide survey, but much requires a time commitment from working group of the data collection can be carried out inexpensively. members, access to performance information, and The service improvements can range from very inex- the power to affect the way that services are delivered. pensive to major investments. PREMNOTE 7 • Computerization. When considering performance The two main challenges are first knowing some of the management, many local governments are concerned basic principles of performance management and having that insufficient IT capacity will be a significant confidence in the effectiveness of the approach, and second, impediment. But that is not necessarily true: in having enough freedom to take action to improve services. many instances, data collection and analysis can be Identifying the best indicators and finding good quality conducted easily without IT resources. Obviously data will continue to pose difficulties. Importantly, the computerization is needed for larger data sets and for SIAP informs decision making, but does not make decisions survey analysis, but many of the local governments automatic: there will still be hard choices to make between that have successfully implemented SIAPs had limited priorities, and complex thinking needed on options for or no IT capability. improvement. In some cases, data analysis will be simple, in • Authority. Local governments may not have the author- others it might require more sophistication, and therefore ity to determine priorities, or to reallocate finances results will require careful interpretation. from one sector to another. For example, in countries The magnitude of positive outcomes in many different where local government employees are employed by a countries suggests that the SIAP is a valuable tool that can be centralized civil service, cities may not be able to real- tailored for different country contexts. Experience indicates locate staff to different tasks. Accordingly, some local that this performance management tool can potentially help governments have been reluctant to go forward with local governments worldwide improve their services and a SIAP for this reason. As noted earlier, local govern- therefore the quality of life of their citizens. ments must have some freedom to take actions that affect service delivery for the SIAP. However, even Acknowledgments modest leeway can be enough to undertake some im- portant measures. The costs can be very minimal, and The SIAP was first developed by Sharon Cooley and some of the actions—moving the location of garbage Katharine Mark under a U.S. Agency for International bins, distributing information, organizing community Development (USAID) local government project in Alba- inputs—do not require major changes in approach or nia in 2000. It has since been used in a number of other additional investment. SIAPs have been successfully countries, also supported by USAID. The original design implemented in countries where the local governments benefited greatly from the work of Harry Hatry, and imple- had very limited autonomy, such as in Tajikistan in the mentation in Albania and subsequent countries was made early 2000s. possible by the efforts and skill of many others, including both international experts and local professionals in each Overall, it appears that the most important ingredient of the countries. The author thanks Keith Mackay, Anya is the political will to improve service delivery and ensure Reva, and Harry Hatry for valuable comments on earlier that the working group has a clear mandate and includes versions of this note. the right people. That makes the potential for SIAP replica- tion very promising. About the Author Conclusion Katharine Mark is Principal Research Scientist at the National Opinion Research Center (NORC) at the University of Chica- Hundreds of local governments with very limited resources go, and previously was Senior Research Associate at the Urban and constrained circumstances have been able to improve Institute. She has worked on local government for more than services using the SIAP. How has the SIAP managed to be 20 years, in Eastern Europe, Africa and Latin America, and so effective in delivering measurable service improvements? has helped national and local governments in more than Its chief characteristics are that it is practical, low-cost, and 10 countries introduce performance management and can be implemented by operational staff directly, giving performance-based budgeting. Email: mark-katie@norc.org. them full control over the process. Its primary strength is probably its motivational force. The structure encourages a local government to focus on a few priority problems. Notes Selecting indicators further helps focus on what needs to 1. Sharma (2011), in a note on the use of social accountabil- be done and how it will affect the outcome. Measuring ity tools, describes citizen report card surveys, which have progress motivates the working group, and through them, been used in many countries to identify the level of citizen in turn, motivates other relevant staff to get on board. Hav- satisfaction with different services and their priorities. ing different stakeholders agree to seek specific outcomes 2. See also Castro (2011) for a useful discussion on perfor- brings many resources to the table. mance indicators and targets. 8 PREMNOTE 3. Trained Observer Ratings are based on observations de Lancer Julnes, Frances Stokes Berry, Maria P. Aristigueta, made by trained laypersons using a standardized rat- and Kaifeng Yang. Sage Publications. ing scale. If properly done, the ratings can provide Mark, Katharine. 2008. “Experience with Trained Observers in Transition and Developing Countries: Citizen Engagement measurements that can be compared over time. More in Monitoring Results.” In International Handbook of Practice- information about Trained Observer Ratings can be Based Performance Management, ed. Patria de Lancer Julnes, found at the following Web site: http://www.urban. Frances Stokes Berry, Maria P. Aristigueta, and Kaifeng Yang. org/toolkit/data-methods/ratings.cfm, as well as in Sage Publications. Mark (2008). Mark, Katharine, and Ritu Nayyar-Stone. 2004. “Early Experi- 4. It should be noted that the SIAP process can be helpful ence with Performance Management in Hungary, Albania and Georgia: Assessing Its Potential for Local Service in obtaining grants because donor organizations or central Improvement.” In Improving the Quality of East and West funding may come with monitoring requirements. European Public Services, ed. Elke Löffler and Mirko Vintar. Aldershot: Ashgate. References Perez, Teresita, and Reece Rushing. 2007. The CitiStat Model: How Data-Driven Government Can Increase Efficiency and Effective- Cartier, William. 2005. “Case Studies of SIAPs Carried Out in ness. Center for America Progress. Honduras.” Project document. Reed, Barry. 2013. Information on SIAPs initiated in Afghanistan, Castro, M. 2011. “Defining and Using Performance Indicators and personal communication. Targets in Government M&E Systems.” World Bank PREM- Salihu, Agim, and Ginka Kapitanova. 2013. Examples of SIAPs notes Special Series on the Nuts & Bolts of Government M&E carried out in Kosovo, personal communication. Systems, No. 12, Washington, DC. Sanger, Mary Bryna. 2008. “From Measurement to Management: Cooley, Sharon, Katharine Mark, Sarah Polen, et al. 2007. The Breaking through the Barriers to State and Local Performance.” Service Improvement Action Plan: Using Performance Manage- Public Administration Review December (SI): S70–S86. ment to Improve Local Services; A Handbook for Albanian Local Sharma, R. 2011. “Use of Social Accountability Tools and Informa- Governments. Project publication, USAID Local Government tion Technologies in Monitoring and Evaluation.” World Bank and Decentralization in Albania. PREMnotes Special Series on the Nuts & Bolts of Government Hatry, Harry. 2008. “Emerging Developments in Performance M&E Systems, No. 7, Washington, DC. Measurement: An International Perspective.” In International Vokopola, Zana. 2013. Information on SIAPs carried out in Al- Handbook of Practice-Based Performance Management, ed. Patria bania, personal communication. This note series is intended to summarize good practices and key policy findings on PREM-related topics. The views expressed in the notes are those of the authors and do not necessarily reflect those of the World Bank. PREMnotes are widely distributed to Bank staff and are also available on the PREM Web site (http://www.worldbank.org/ prem). If you are interested in writing a PREMnote, email your idea to Madjiguene Seck at mseck@worldbank.org. For additional copies of this PREMnote please contact the PREM Advisory Service at x87736. This series is for both external and internal dissemination PREMNOTE 9