LINKING RESULTS TO PERFORMANCE: EVIDENCE FROM A RESULTS BASED FINANCING PRE-PILOT PROJECT IN KATETE DISTRICT, ZAMBIA 98265 DISCUSSION PAPER APRIL 2015 Collins Chansa Ashis Das Jumana Qamruddin Jed Friedman Akafwilangachi Mkandawire Monique Vledder Linking Results to Performance: Evidence from a Results Based Financing Pre-Pilot Project in Katete District, Zambia Collins Chansa Ashis Das Jumana Qamruddin Jed Friedman Akafwilangachi Mkandawire Monique Vledder April 2015 Health, Nutrition and Population (HNP) Discussion Paper The World Bank’s Health, Nutrition, and Population (HNP) Global Practice produces the papers in this series to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors and should not be attributed in any manner to the World Bank, its affiliated organizations, its Board of Executive Directors or the countries they represent. Citations and the use of material presented in this series should take into account this provisional character. For information regarding the HNP Discussion Paper Series, please contact the Editor, Martin Lutalo, at mlutalo@worldbank.org or Erika Yanick at eyanick@worldbank.org. © 2015 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. Health, Nutrition and Population (HNP) Discussion Paper Linking Results to Performance: Evidence from a Results Based Financing Pre-Pilot Project in Katete District, Zambia Collins Chansaa, Ashis Dasa, Jumana Qamruddina, Jed Friedmanb, Akafwilangachi Mkandawirec, Monique Vleddera a Health, Nutrition and Population Global Practice, The World Bank, Washington, DC, USA b Development Research Group, The World Bank, Washington, DC, USA c Ministry of Health, Lusaka, Zambia Abstract: Global interest in results based financing (RBF) as a means of strengthening health systems and improving service delivery coincides with heavy Zambian investment in its health sector—but with poor maternal and child health (MCH) outcomes. Determined to remedy the situation, Zambia pre-piloted an RBF project in Katete district aimed at testing how RBF can be adapted to the country context and its suitability in strengthening the health system and improving service delivery. This case study reviews the design and implementation of the Katete RBF Pre-Pilot and explores its impact on access and utilization of MCH services and health systems strengthening. The research team used a mixed-methods approach, collecting qualitative and quantitative data through semi-structured interviews, group discussions, document reviews and financial and health service delivery databases. Changes in performance indicators were estimated using an interrupted time series analysis complemented by a simulated modeling analysis. In Katete, RBF increased immunization coverage of children less than one year old by 4 percent and curative consultations by 14 percent between 2008 and 2012. These achievements were facilitated by some health system improvements such as increased managerial autonomy and local decision-making at service delivery levels, enhanced staff performance and teamwork, and community participation. Despite these gains, the project experienced some delays in RBF disbursements to health facilities and increased staff workload due to increased utilization of health services. The study revealed clients’ perceptions of high service quality despite low technical quality scores. RBF can be used to improve health system performance and service delivery quality. However, the implementation and effects of RBF are context-specific and the model has to meet certain key design criteria and requires basic inputs for RBF to work effectively. Keywords: Results based financing, maternal and child health, health systems performance, Zambia Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Jumana Qamruddin, Health, Nutrition and Population Global Practice, The World Bank, Washington, DC, USA. Email: jqamruddin@worldbank.org TABLE OF CONTENTS ACRONYMS ........................................................................................................................................... II ACKNOWLEDGMENTS ........................................................................................................................ III PART I – INTRODUCTION ..................................................................................................................... 1 PART II– METHODS............................................................................................................................... 3 STUDY DESIGN...................................................................................................................................... 3 STUDY SITE .......................................................................................................................................... 3 DATA COLLECTION ................................................................................................................................ 3 DATA ANALYSIS..................................................................................................................................... 4 PART III – RESULTS .............................................................................................................................. 5 THE KATETE RBF PRE-PILOT DESIGN.................................................................................................... 5 CHANGES IN SERVICE UTILIZATION......................................................................................................... 7 Quantitative Evidence ..................................................................................................................... 7 Qualitative Evidence ...................................................................................................................... 10 CHANGES IN THE HEALTH SYSTEM....................................................................................................... 10 Managerial Autonomy.................................................................................................................... 10 Data Quality ................................................................................................................................... 12 Quality of Health Care ................................................................................................................... 12 Increased Demand for Health Services......................................................................................... 13 Community participation ................................................................................................................ 13 CHANGES IN FINANCING....................................................................................................................... 14 Aggregate level of funding and composition of funding ................................................................. 14 Flow of funds to the rural health centers ....................................................................................... 15 PART IV – DISCUSSION ...................................................................................................................... 16 PART V – POLICY LESSONS.............................................................................................................. 19 REFERENCES ...................................................................................................................................... 20 ACRONYMS CHV Community Health Volunteer DMO District Medical Office D-RBFSC District RBF Steering Committee EHT Environmental Health Technician FGD Focus Group Discussion HCC Health Center Committee HMIS Health Management Information System ITS Interrupted Time Series MCH Maternal and Child Health MDG Millennium Development Goals MMR Maternal Mortality Ratio MOH Ministry of Health NHC Neighborhood Health Committee PMTCT Prevention of Mother-to-Child Transmission of HIV P-RBFSC Provincial RBF Steering Committee RHC Rural Health Center RBF Results Based Financing TBA Traditional Birth Attendant U5MR Under-Five Mortality Rate ii ACKNOWLEDGMENTS The authors are grateful to Chishimba Nakamba Mulambia and Easter Musanide Mwale for collecting and transcribing data from the Katete district focus group discussions. The authors are also indebted to Caesar Cheelo, Jos Dusseljee and Dr. Caroline Phiri for providing the earlier data on the external verifications and for the technical review of the Katete Pre-Pilot project. The authors commend Mambwe Kabaso, Brivine Sikapande, Teniford Shawa and Dr. Pricilla M. Zulu for facilitating the data collection process in Katete. Finally, the authors are grateful to the World Bank for publishing this report as an HNP Discussion Paper. iii PART I – INTRODUCTION Zambia has made recent strides towards the attainment of health-related Millennium Development Goals (MDGs). This progress is evidenced by the reduction of under-five mortality rates (U5MR) from 119 to 75 per 1,000 live births between 2007 and 2013 (CSO et al. 2014). During the same period, Zambia’s maternal mortality ratio (MMR) reduced from 591 to 398 per 100,000 live births (CSO et al. 2014). Despite these achievements, Zambia is unlikely to meet any of its health-related MDGs in 2015. For example, the percentage of children age 12-23 months who are fully vaccinated remained unchanged at 68 percent between 2007 and 2013/14 (CSO et al. 2014), and it is unlikely that Zambia will make gains large enough to close this gap in one year. Although at US$44 the total per capita health expenditure in Zambia is marginally higher than the average for Lower Middle Income Countries (US$43), service delivery is worse-off (Chansa et al. 2014). Zambia’s health sector struggles with ongoing inefficiencies in resource allocation and utilization, a high and increasing burden of communicable and non-communicable disease and poor quality of care (MOH, 2011). Other challenges include inadequate and poorly motivated human resources for health; erratic supply of essential medicines and medical supplies; limited autonomy in decision-making at decentralized levels of the health system; and a weak monitoring and evaluation system (MOH, 2011). The Zambian health system faces a human resources crisis due to a shortage of skilled workers, imbalanced skills mix, inequitable distribution, chronic absenteeism, tardiness and low morale (WB, 2009). In 2008, in an attempt to strengthen the health system and improve health service delivery, Zambia launched a results based financing (RBF) Pre-Pilot in Katete district financed by the World Bank through the Health Results Innovation Trust Fund. The Katete Pre-Pilot was designed to conceptualize and refine the RBF model in Zambia before countrywide roll out. The motivation to implement RBF in Zambia was driven by its success in several other African countries as a health financing innovation with the potential to strengthen health systems and improve health outcomes (Murray et al., 2007). With RBF, a principal entity provides a financial or in-kind reward conditional on the recipient undertaking a set of pre-determined actions or achieving a pre-determined performance goal. It encompasses output-based aid, provider or healthcare based incentives for performance and consumer incentives for behavioral changes (Eichler, 2006). By introducing incentives that reward results, RBF promotes greater accountability of service providers and improves management, efficiency and equity of service delivery and health information systems. An increasing body of evidence shows that RBF can strengthen health systems, help countries use limited resources effectively, improve staff motivation and morale and empower providers and beneficiaries (Rusa et al., 2009; Basinga et al., 2011; Soeters et al., 2011; Gertler and Vermeersch, 2012). Conversely, RBF approaches have the potential to create perverse incentives and unintended consequences (Eldridge and Palmer, 2009; Ireland et al., 2011; Miller and Babiarz, 2013). Miller and Babiarz (2013) argue that the use of incentives to improve performance has the potential to incline efforts towards contracted services at the expense of non-contracted services. They further argue that RBF may lead to heterogeneity in rewarding efforts across contracted services when health providers spend much of their time on the provision of services which yield the largest (net) marginal return (Miller and Babiarz, 2013). Another concern is that RBF approaches (which in most cases place emphasis on extrinsic incentives) may erode intrinsic motivation in the long run. Other authors (Ireland et al., 2011) view RBF as a financing mechanism rather than a strategic tool for reforming health systems in developing countries. Eldridge and Palmer (2009) further question the rationale of implementing RBF schemes in less developed countries where health systems are weak, contending that implementing RBF requires: (i) strong political and management support; (ii) room for change and innovation to maximize efficiencies; and (iii) strong health information and reporting systems (Eldridge and Palmer, 2009). Banerjee et al. (2008) add that sufficient capacity to enforce contracts, collect data and verify performance is necessary for pay-for-performance schemes to succeed. 1 Due to the unintended consequences of RBF approaches, some authors have requested more research on RBF in the health sector (Eldridge and Palmer, 2009; Miller and Babiarz, 2013). The consensus is that the growing body of research on performance incentives has not provided enough evidence on the design and underlying conceptual issues on performance incentives (Miller and Babiarz, 2013). The objective of this study, therefore, is to review the design and implementation of the RBF project in Katete, its impact on access and utilization of maternal and child health services, and usefulness of pre-piloting RBF projects. 2 PART II– METHODS STUDY DESIGN The research team used a case study design using a mixed-methods approach to evaluate the effects of the RBF Pre-Pilot project in Katete district. The choice of this design and data collection method was mainly due to lack of a counterfactual measure. This approach was particularly necessary given that the study tried to make sense of a complex and poorly controlled context and can be regarded as a “real- world research” (Rodson, 2002). Furthermore, given that one of the objectives of the study was to review the design and implementation of RBF in Katete, a case study design was appropriate because it enables researchers to understand the dynamics, processes and relations present within a specific setting (Eisenhardt, 1989; Yin, 2003). STUDY SITE The study was undertaken in Katete district in Zambia’s eastern province. Katete covers approximately 3,989 square kilometers of land (Figure 1). Katete’s population was about 238,807 at the time of the Pre- Pilot and growing at an annual rate of 2.8 percent (CSO, 2003). The study included twenty-five health facilities which were operational during the Pre-Pilot. These included six health posts, 18 rural health centers (RHC) and one urban health center. All 25 participating health facilities were coordinated and managed by Katete District Medical Office (DMO). The district also has one general hospital, St. Francis Mission Hospital, which was contracted by the DMO to conduct quality assessments at the health centers. DATA COLLECTION The study relied on a combination of qualitative and quantitative data-collection methods. To enable sufficient analysis, data covering the time period January 2006 to March 2012 was collected for the study. Data sources were semi-structured interviews, group discussions, document reviews and financial and health service delivery databases. In particular, health service delivery information was collected from the district Health Management Information System (HMIS) in line with the nine HMIS indicators used to incentivize performance during the Pre-Pilot project. Zambia’s HMIS has been in place since 1996 and is one of the better performing systems in Africa with 85 percent completion and data accuracy rates. Data on four of the nine RBF incentivized indicators was used for this study because Zambia’s 2009 HMIS restructuring led to changes in the definition of the other five indicators. As a result, there were only four comparable indicators before and after the Katete RBF Pre-Pilot project: institutional deliveries, fully immunized children under one year, outpatient consultations and antenatal care visits. The indicators were transformed to rates with relevant population level denominators to prevent overestimation of the effect of the intervention that might be influenced merely by a natural growth in population. The denominators were total estimated district pregnancies for institutional deliveries and antenatal care visits; total estimated district children under one year for fully immunized children under one year; and total district population for outpatient consultations. Aligning with the project payment cycle, data was transformed to quarterly rates. One of the potential shortcomings in linking payments to performance is the inherent risk of over-reporting by service providers. To mitigate this problem, internal and external verification mechanisms were put in place during the implementation of the RBF project in Katete. The Katete DMO conducted the first order substantiation of self-reported HMIS data by the health centers on a monthly basis. The University of Zambia—an institution not linked to the RBF implementation process—conducted two rounds of external data quality audits. As such, for the RBF implementation period, data for this study was collected from the externally verified audit reports and further triangulated with supervisory visits and performance assessment reports. On the financial side, data was collected from district and facility level action plans and financial reports. Additional information on the design and implementation of the Katete Pre-Pilot project was collected from the project manual, project-specific reports and minutes of the RBF steering committees at district and provincial levels. The research team obtained qualitative information through interviews with district health managers (n=7), as well as through group discussions with health facility staff (n=8) and community representatives (n=15) 3 between January and February of 2012. Community representatives included members of health center and neighborhood committees and volunteers. The interviews focused on provider and community representative perspectives on the effects of the RBF project in Katete. Interviews took an average of 45 minutes and group discussions 60 minutes. Interviews with the health managers and health facility staff were conducted in English, whereas those with the community representatives were conducted in Chinyanja, the local language. All the interviews were audio-recorded upon participants’ written informed consent. The qualitative interviews and discussions were conducted using semi-structured guides. DATA ANALYSIS The research team analyzed the quantitative service utilization data from the HMIS using interrupted time series (ITS) analysis. ITS analyzes data collected from several equally spaced time points before and after the intervention to observe the difference that the intervention might have induced other than the secular trend (Wagner et al., 2002). The time period under analysis was first quarter of 2006 (January to March, 2006) through first quarter of 2012 (January to March, 2012); in total there were 11 pre-intervention and 14 post-intervention time points for each indicator. The team undertook a simulated modeling analysis (SMA) recommended for short data streams (<30 data points per phase). The SMA software 1 provides more statistical power and is less prone to type I error than conventional statistics (Borckardt et al., 2008) and takes into account autocorrelation that is normally observed in time series data (Borckardt et al., 2008). Changes in levels and trends of the indicators were estimated. A level change signifies a rapid change in the volume of services through the difference in the indicator at the first intervention time point between the secular and intervention trends (Ramsay et al., 2003). A trend change signifies change over a longer period and is measured by the difference between pre- and post-intervention slopes (Serumaga et al., 2011). The recordings from qualitative interviews were transcribed and translated and the transcripts were coded in NVivo software. A thematic analysis of the transcripts was performed using inductive and deductive methods. Data generated from all sources were triangulated as appropriate. The research team obtained approval from the University of Zambia’s ethics committee to conduct this study. 1 SMA software was developed by Clinical Research Solutions 4 PART III – RESULTS THE KATETE RBF PRE-PILOT DESIGN Katete district’s (Figure 1) RBF Pre-Pilot ran from 2008 to 2011. The main aim of the Pre-Pilot project was to inform the content, process and overall design of the broader RBF pilot project, which was implemented between April 2012 and October 2014. The model adopted in the Pre-Pilot was a “fee-for-service” “performance based financing” initiative by “contracting-in” the public health system while adhering to the general principle of “separation of functions.” This entailed a quasi-split of responsibilities and tasks between the service providers, regulators, verifiers, purchasing agency and fund holder within the public health system as shown in Figure 2. Separation of functions and clear lines of responsibilities between different actors in the health system is one of the key characteristics of contracting in performance based arrangements (Mills and Broomberg, 1998). Figure 1: Map of Katete and its Health Facilities Source: Zambia National Health Facility Atlas Health Centers were the service providers and the District Medical Office and St. Francis Mission Hospital served as internal quantity and quality regulators, respectively; the District RBF Steering Committee (D- RBFSC) was regulator and verifier; the Provincial RBF Steering Committees (P-RBFSC) was purchaser; and the Ministry of Health (MOH) headquarters served as fund holder. The steering committees also included stakeholders from civil society and other allied non-health sectors. External verification was contracted out to the University of Zambia to: independently verify the accuracy of reported data; confirm that reported health services were actually provided; and measure the perceived quality of such services. 5 Figure 2: The Katete District RBF Pre-Pilot Project Model Source: Zambia HRBF Project Implementation Manual Nine health facility maternal and child health (MCH) indicators found in the HMIS were incentivized based on performance using a fee-for-service provider payment mechanism (listed in Table 1). The price of each indicator was based on baseline coverage and MOH targets and priorities. Consideration was made for the significance of the indicator in contributing to the attainment of programmatic and national MCH goals as well as the level of complexity in providing the service. For example, curative consultations were priced at US$0.2 per visit while institutional deliveries were priced at US$6.4 per delivery. Table 1: Incentivized Indicators and Unit Prices Indicator Unit Price (US$) 1 Curative consultation 0.2 2 Institutional deliveries by skilled birth attendant 6.4 3 Antenatal care (prenatal and follow up visits) 1.6 4 Postnatal visit 3.3 5 Full immunization of children under one year 2.3 6 Pregnant women receiving 3 doses of malaria IPT 1.6 7 Family planning users of modern methods at the end of the month 0.6 8 Pregnant women counseled and tested for HIV 1.8 9 Number of HIV pregnant women given anti-retroviral therapy 2.0 prophylaxis Source: Zambia HRBF Project Implementation Manual 6 To ensure that RBF interventions addressed both service coverage and quality, all health facilities were audited on a quarterly basis using a quality assessment checklist. The checklist incorporated international and national operating practices and guidelines in ten different areas deemed critical for quality improvement. At the end of the assessment, a composite quality score for each health center was computed. The weights and means of verification are presented in Table 2. The composite quality score for each health center was then multiplied by the total income from the “quantity” performance assessment to derive the final performance payment. This is shown in the formula below. where P = Performance Payment; Q = Quality Score; ai = Unit Price for indicator i; bi = Quantity achieved for indicator i. Table 2: Areas for Quality Assessment and Associated Weights Service Quality Index Weight Means of Verification 1 Curative Care 0.11 Direct observation 2 Antenatal Care 0.16 Direct observation Direct observation and 3 Family Planning 0.18 records review Direct observation and 4 Expanded Program on Immunization 0.09 records review Direct observation and 5 Delivery Room 0.20 records review 6 HIV Services 0.05 Direct observation 7 Supply Management 0.07 Direct observation Direct observation and 8 General Management 0.06 records review 9 Health Management Information System 0.06 Records review 10 Community Participation 0.03 Records review Total 1.00 Source: Zambia HRBF Project Implementation Manual CHANGES IN SERVICE UTILIZATION Quantitative Evidence As shown in Table 3, the rate of fully immunized children rose by 4 percentage points (17.21 percent pre- RBF vs. 21.23 percent post-RBF; see also Figure 3) whereas the rate of outpatient consultations increased by 14 percentage points (40.56 percent pre-RBF vs. 54.58 percent post-RBF; Figure 3) and the rate of institutional delivery went up by 3.5 percentage points (11.85 percent pre-RBF vs. 15.39 percent post-RBF; Figure 3). However, as determined by the SMA analysis, only two indicators (i.e. rate of fully immunized children [coefficient 0.643; p value 0.026] and outpatient consultations [coefficient 0.605; p value 0.036]) were statistically significant. The rate of antenatal care decreased by about 3.5 percentage points (76.17 percent pre-RBF vs. 72.84 percent post-RBF; Figure 3), but this reduction was not statistically significant. 7 Table 3: Effect of RBF on the Level and Trend of Key MCH Service Utilization Indicators Pre-intervention Post-intervention Variable mean (95% CI) mean (95% CI) Level change Trend change 2006Q1-2008Q3 2008Q4-2012Q1 Coefficient p Coefficient p Fully immunized 17.21 21.23 children rate (15.67-18.69) (20.1-22.4) 0.643 0.026 -0.053 0.888 Outpatient 40.56 54.58 consultations rate (38.5-43.12) (48.43-60.67) 0.605 0.036 -0.432 0.178 76.17 72.84 ANC rate (72.85-79.26) (69.01-76.74) -0.241 0.424 0.025 0.947 Institutional 11.85 15.39 delivery rate (9.6-13.98) (14.33-16.5) 0.514 0.155 0.25 0.517 Source: Authors There was no significant change in trends for any indicator, suggesting that the introduction of incentives had an immediate effect in terms of change in the levels (i.e. sudden increase in utilization) for those indicators that responded without a sustained increase afterwards through the end of the Pre-Pilot. For robustness, we moved the period of analysis forward by a quarter prior to the implementation to investigate any “anticipation effect” since publicity about the Pre-Pilot prior to actual RBF implementation may have spurred behavioral changes before the Pre-Pilot even began. If we do this, then we observe a significant increase in institutional delivery by 10 percent. 8 Figure 3: Trends in Incentivized Indicators Source: Authors 9 Qualitative Evidence Service providers attribute some of the increased healthcare utilization in the district to the RBF program. The increase in patient load is partly perceived to arise from health centers receiving patients beyond their catchment areas; additional patients came from neighboring communities, districts and countries, including Mozambique. The introduction of RBF has improved the services at the hospital and people in the village are more willing to go to the clinic knowing that they will be treated well. [Health Center Committee (HCC) member] When RBF was just starting most of the women used to deliver at home. But now you will find that as a midwife I am more occupied because people are delivering at the centers. [Midwife, RHC] Some outreach services such as under-five clinics were either introduced or organized more frequently to promote higher utilization rates. Health center staff and community members alike reported an increase in the utilization of institutional deliveries, counseling and treatment for HIV and AIDS and family planning services. However, there were also concerns about geographical access to the health centers, which the RBF model did not directly address. I have also seen that previously at our center we had five outreach under-five clinics but with the coming of RBF we now have six. [HCC member] It is a big task to help a woman who is HIV positive to look after their child. You know in rural communities there are so many myths and beliefs and the work is tough so the more they come to the clinic the more the women get informed and RBF has helped to bring more numbers of women though with some there is still the problem of distance. [HCC member] CHANGES IN THE HEALTH SYSTEM Managerial Autonomy The MOH disbursed RBF performance incentive payments directly into the health centers’ bank accounts. This mechanism is in contrast to the Government of the Republic of Zambia (GRZ) operational grant that the MOH disburses to the DMO, which then releases this money to district health facilities. In addition, health facilities involved in the RBF Pre-Pilot were given the discretion to utilize the performance-based payments according to their specific needs. The only condition was to spend a maximum of 75 percent of the RBF performance payments on staff bonuses/incentives and a minimum of 25 percent for activities at the health facilities and the communities. In contrast, health facilities had very limited discretion on the use of the GRZ operational grant. We were able to make budgets without necessarily consulting the district; the budgets were based on health center needs and priorities. The district would just come in to confirm if the budget was aligned to the priorities at the health center. I like that kind of autonomy because as a health center we know our needs and priorities. [Health Worker, RHC] In practice, health facilities used part of the RBF performance payments to purchase a number of goods and services, which assisted them to implement a variety of locally adapted activities (Table 4) in line with the quarterly Health Center Business Plans. Some health facilities recruited local data entry clerks and community health volunteers. One health center employed a retired midwife. Resources were also used to purchase medical equipment and supplies such as thermometers, weighing machines, mattresses, linens and mosquito nets. In some facilities “Mama Kits” (packages containing soap, napkins, petroleum jelly, baby powder and a blanket) were provided to women who delivered their babies at the health facility. Table 4 shows how the income from the RBF performance payments was utilized and the results observed. 10 Table 4: Utilization of the Income from the RBF Performance Incentive Payments Level Activity Results Accuracy and completeness in recording and Recruitment of data clerk reporting Reduced workload for the health workers and Recruitment of volunteers more focus on clinical care Recruitment of retired midwives Enhanced quantity and quality of institutional deliveries by skilled workers Hire of local transport for referral Prompt referral to next level of care Health Purchase of linens and Improved in-patient care, and client facility mattresses satisfaction Purchase of equipment, for Increased quality of care and health worker example, thermometer motivation Provision of Mama Kits (soap, napkin, pins, petroleum jelly, talcum powder, blanket) to women who delivered in the Increased client demand for health services health facility and facility-based utilization and deliveries Incentives to Traditional Birth Attendants (TBAs) and Increased client demand for health services Community volunteers to send women and and facility-based utilization of health children to health centers services Incentives to pregnant women for early registration and ANC Early pregnancy registration Source: Authors Both the ability to earn money according to the level of effort and flexibility in the use of income contributed increased staff morale and performance. Specifically, health workers and communities were appreciative of the ability to use the money to improve working conditions at the health facilities and service delivery in the communities, and for personal benefit. Perceptions from community representatives and health workers were that RBF contributed to improving the attitudes of health workers towards work and consequently performance. It also fostered teamwork among various stakeholders at the facility level. Considering that RBF performance incentive bonuses were shared among all the health workers at the health centers according to individual performances, it could have helped to enhance teamwork. We are happy that we have RBF in our district. It has increased our eagerness to work and even the performance itself. I can also say that job satisfaction was there because targets could be reached and in some instances we even exceeded our expectations for that indicator. [Health worker, RHC] What we have seen from the community is that the staff are really working well. They have put in extra effort. [Neighborhood Health Committee (NHC)member] It keeps us motivated to find out the ways and means to improve our situation. At the end of the day you are also raising money to solve your own problems. [Health worker, RHC] Though some health facilities were able to purchase some low cost medical equipment, it was not possible for others because low catchment populations translated to lower RBF income. Furthermore, small health facilities with one or two health workers could not raise adequate financial resources since their performance was limited by what the existing two staff members could achieve. Lack of retired health workers in the surrounding areas, for example, limited heath facilities from employing additional health workers even though they had the money to do so. 11 Data Quality The University of Zambia, an independent agency, was contracted to conduct two rounds of external verification to validate the accuracy of reporting of HMIS data at the health facilities. Subsequent audits conducted as the Pre-Pilot progressed showed improvements in record keeping and data accuracy (Cheelo et al., 2014). Over- and under-reporting of the HMIS information was reduced to about 10 percent. The general perception was that the recruitment of data entry clerks at the health facilities had contributed to the improvement in data recording. The employment of data entry clerks by most of the health centers also allowed the health center staff to devote more time to patient care. The system of filing and recording the clients is there at the center and the records are much more in order than before because of the data entry clerks. [Health Worker, RHC] A comprehensive review of project data also showed that the Pre-Pilot improved: data completeness through the use of patient registers instead of tally sheets; data accuracy through regular internal and external data audits; and timeliness by putting in place a schedule for data reporting through the audit/verification system, which is linked to results. In particular, strict guidelines for collecting, processing and transmitting data were put in place to ensure speedy verification and payments. However, while data quality (i.e. completeness, accuracy, and timeliness) improved, disbursement of RBF performance payments was poor. This is discussed further below. Quality of Health Care Employment of data entry clerks allowed the health center staff to devote more time to patient care. As a result, the health workers could earn more from RBF through additional effort towards quality of care and client satisfaction. Clients reported reduced waiting times and timely attention by the health care providers. We used to split tasks amongst health center staff and the volunteers such that we could use some volunteers to weigh the children whilst others would be outside taking the temperature for the children and the other one checking blood pressure levels before they can be screened. [Health Worker, RHC] Information from interviews on clients’ perceptions on quality of care at health facilities was further triangulated with reports from two client tracer surveys conducted by the University of Zambia during RBF Pre-Pilot implementation. The purpose of these client surveys was to confirm receipt of health services and to assess clients’ perceptions of various aspects of service delivery (Cheelo et al., 2014). Clients who could be traced during the initial and subsequent client tracing survey generally rated the health workers high on the "hospitality" and "technical" aspects of service delivery (Cheelo et al., 2014). About 89 percent reported that they received a friendly service at the facility; 83 percent received a clear explanation of the procedures and services; and 89 percent were informed of the results of the examinations conducted on them. Furthermore, 89 percent of the clients reported that they received medication during the visit to the health facility, and 97 percent of these clients indicated that the health staff explained how the medication was to be taken. Quality of care has improved because mothers are bringing their children and by nine months they are fully immunized. [NHC member] I think there was great improvement because protocols were followed. Previously you wouldn’t go to the clinic to measure blood pressure and these other things… now all the protocols are followed. …. And also the cleanliness of the centers because it is part of the quality assessment… [HCC member] The health workers are now able to provide a friendly service and explanations on the illness, procedures and services to be provided. [HCC member] However, contrary to the positive client perceptions on quality, results from the technical quality assessments on health center service provision conducted by the district hospital were not as positive. 12 Results show that the mean quality score for Katete district was 44.3 percent; only 7 out of the 25 health facilities, or 28 percent, scored more than 50 percent. Poor technical quality scores can be attributed to the structural nature of the quality assessment tool, which gives more weight to availability of inputs such as skilled birth attendant, drugs, medical equipment and health facility infrastructure than process quality. Health facilities had no control over their infrastructure or availability of drugs, essential medical equipment and supplies, which were procured centrally at either national, provincial or district levels. Quality auditors would continuously score a health facility poorly even if the ability to improve these structural elements was beyond their reach. For instance, the quality assessment penalized the health facilities if they had no skilled birth attendant despite that the recruitment of skilled health workers was the purview of the MOH headquarters. Although one RHC hired a local retired midwife, it was not possible for other health facilities due to lower RBF earnings or unavailability of retired health workers in the area. After applying the quality score, income from the RBF performance payments were often reduced. Thus, some health workers considered the quality audit as a punitive exercise. The district is aware that there is a staff shortage at health centers, and it is not the health centers’ fault. When marks are deducted because there are unqualified personnel it is very unfair. [Health Worker, RHC] Respondents expressed that reshuffling of skilled staff from health centers with excess capacity to needy health centers would benefit RBF project implementation. Increased Demand for Health Services As previously reported, stakeholders perceived an increase in the demand for health services among the communities—especially for institutional delivery, antenatal and postnatal care, and PMTCT services. Health centers engaged volunteers from the communities to sensitize those within the catchment area about available health facility services. Volunteers also encouraged pregnant women to deliver at health facilities and to go for post-natal care. They mobilized communities for child immunization days. At the same time, the health centers provided incentives to pregnant women for early registration at the antenatal clinics. Before, women were reluctant to move to the health centers even when they knew that they were about to deliver. But, now they are so willing to leave behind whatever work they have to find a place to be near the health center and make sure that the delivery takes place there. [NHC member] As a volunteer in our community, I have seen that patients are more willing to listen to advice and more willing to go to the clinic than before. [Community health volunteer] While intensified community mobilization increased client inflow to the facilities, it also increased the staff workload at the health facilities. Nevertheless some health workers felt the incentives attached to seeing additional patients mitigated the negative impact of the increased workload. Being at the rural health center I am forced to work 24 hours, but now with this RBF I don’t know what will happen. When people are sensitized they will be coming in numbers and I will fail to control. But on the part of the incentive I was quite happy. [Health Worker, RHC] Community participation Community participation in the planning and implementation of activities at the health facilities increased during the Katete RBF Pre-Pilot. Through the health center and neighborhood health committees, community representatives planned the strategies jointly with health center staff. Stakeholders revealed that committee meetings were held frequently and provided a forum for discussion, which increased interaction between the community and health workers. This was unlike the period before the RBF Pre- 13 Pilot, when community participation was initiated only after health workers had made a decision on a particular issue. The relationship even among ourselves at the center improved and this even extended to the community because we are working hand in hand. So there is teamwork both at the center and the community. [Health Worker, RHC] The mechanisms of achieving the results have given us enough power that we are able to plan and then supervise the staff during the time of rewarding the incentives. [NHC Member] Community leaders, including village headmen, also participated in monitoring and evaluation of activities at the health facilities. For example, some headmen maintained registration of newborns and used these registers to track their vaccination status. As part of the process of data verification, community representatives discussed the trends in indicators and achievements every month and were able to understand the data and to generate measures to improve performance. They were also involved in signing off the RBF provisional invoices before submission to the DMO. They (community representatives) used to look at the health center’s performance from the indicators, find out the gaps and tell us here we are not doing fine and the reasons are A, B, C and D. [District Medical Office, Katete] As a community person I am even able to provide data about the center in a community meeting because data management is now up-to-date and during data audit I know what happens. [NHC member] Some village headmen and other community leaders questioned why they did not receive some incentive considering they were also instrumental in mobilizing the communities for higher service utilization. A few community representatives expressed a desire to include the community level service providers, such as volunteers and traditional birth attendants, in a contractual agreement with the health facilities with clear guidelines for the price of each unit of service delivered at community level. CHANGES IN FINANCING Aggregate level of funding and composition of funding At the time of the Pre-Pilot, the main sources of funds at the district and health facilities were the GRZ and RBF. The GRZ was providing operational grants and staff salaries. The operational grant was a fixed amount based on the catchment population of the health facilities. During the period under review, the highest total income for all the health facilities combined was from GRZ for staff salaries (US$1,502,126) at 71 percent, followed by GRZ funds for operational grants (US$320,520) at 15 percent, and lastly RBF funds (US$302,225) at 14 percent. During the period under review, the level of GRZ funding for salaries and operational grants remained at almost static while RBF funding was increasing. The GRZ operational grant per health facility per quarter was constant at about US$1,068 during the RBF implementation period. Meanwhile, the RBF grant increased by 73 percent from US$535 per health facility per quarter during the initial stages of the project to US$923 per health facility per quarter by the end of the project. This increase in income suggests that the performance of health facilities improved over time. It also suggests that RBF performance payments were additional to the total income at the health facilities in Katete. The aggregate MOH disbursements to the health facilities for RBF performance shows that the minimum amount disbursed per capita per quarter was US$0.23 and the maximum was US$1.82. This excludes costs for supervision, capacity building, technical assistance and other administrative costs. Generally, health facilities with higher catchment populations made more RBF money and spent part of this money on staff motivation bonuses. The bonuses earned by each individual staff member were also dependent on the number of health workers at a health facility, i.e. the lower the number of staff, the higher the 14 motivation bonus. Thus, the ratio of the RBF staff motivation bonuses to the GRZ staff salary at health facilities was dependent on the catchment population, the number of staff at the health facility and the salary scale of the health cadres at the health facilities. On average, RBF staff motivation bonuses were between 2 and 56 percent of the salaries with a mean of 18 percent (median of 16 percent) with each health worker earning between US$8 and US$164 per month from RBF bonuses with a mean bonus of US$64 per month. Flow of funds to the rural health centers As earlier observed, improvements in the timeliness of data collection, processing and transmission did not translate into improved disbursement of RBF performance payments by the MOH, particularly during the start-up of the Pre-Pilot. The flow of RBF performance payments from the MOH to the health facilities was erratic and demotivated some health workers and volunteers. At some health facilities, some volunteers reduced their participation in the RBF project. Reduced volunteer coupled with enhanced demand for health services increased the workload of health workers in the affected areas. I think the challenge was that when we started implementing RBF, community volunteers worked at an increased pace. When there was a delay in payments some volunteers started shunning the program and there was a lot of work left for us health center staff. [Health Worker, RHC] Erratic disbursement of funds could also be why some health facilities were not always able to address their needs in a timely way even with the introduction of RBF. 15 PART IV – DISCUSSION This case study reveals that the RBF Pre-Pilot project in Katete was entirely managed through the Zambia public health system through a contracting-in arrangement. This design and implementation arrangement contrasts with other RBF models in Africa where external agencies such as NGOs are used as purchasers, verifiers or providers using contracting-out management arrangements (Toonen et al., 2009). The choice of this model and design may have been driven by a long history of performance based contracting in Zambia where structures at national, provincial, and district levels had been established as far back as 1996 to accommodate a split of functions and responsibilities between the MOH and the Central Board of Health (Chansa et al. 2014). The Katete Pre-Pilot project was able to contribute to strengthening the existing health system not only through the choice of this design but also through alignment with the existing government planning, and monitoring and evaluation structure. For example, the RBF internal verification process was built into the existing routine performance assessment system— although data collection and verification was conducted more frequently during the Katete RBF Pre-Pilot. In particular, the results show how the RBF Pre-Pilot project facilitated improvements in data quality. The design of the RBF Pre-Pilot project, which linked payments to verified results, prompted health facilities to hire data clerks, which led to improvements in data completeness, timeliness and accuracy. Health facilities were able to consistently use patient registers rather than tally sheets, while regular internal and external data audits contributed to improvements in data accuracy. External data verification by an independent firm, in particular, helped to validate the reported data and tracing of patients to confirm if the health services were provided (Cheelo et al. 2014). Use of a rigorous schedule for data reporting and verification also contributed to improved timeliness in data reporting and verification—demonstrated through prompt data collection, processing, and transmission according to the prescribed times and in readiness for timely decision making (USAID, 2004). However, improvements in timeliness of data collection, processing and transmission did not translate into improved disbursement of RBF performance grants from the MOH to the health facilities. This affected the implementation of the RBF Pre-Pilot project in terms of implementing all planned activities on time and sustaining the morale of staff and volunteers. The Katete RBF Pre-Pilot project was also associated with a higher utilization of health services, improved client perceptions on the quality of health care and overall health system strengthening. Improved managerial autonomy and local decision-making at service delivery levels increased staff performance and community participation. For example, provision of flexible funding and managerial autonomy stimulated the performance of the health system through increased motivation of health workers. Most of the health workers reported that they were motivated by the incentives, which enhanced teamwork and appreciation by the clients. The role of managerial autonomy in the management of health centers is critical to the success of incentive programs and a lack of autonomy among providers or healthcare organizations can restrict the range of behavioral responses (Miller and Babiarz, 2013). Toonen et al. (2009) further highlight the need for implementers to understand the implications of RBF schemes for health facility managers. They argue that it is important to divide the management options into actions on intrinsic and extrinsic motivation 2 factors. In Katete, health facility managers were able to innovate with the money earned from RBF by designing and implementing activities addressing the local needs of their health facilities and communities. This allowed for bottom-up and needs based planning for rural health centers as opposed to the traditional top-down approach. This, in turn, facilitated community participation in planning and implementation of activities at the health facilities. Health center and neighborhood health committees were able to hold meetings more frequently, which provided a forum for discussion and increased interaction between the community and health workers. The use of volunteers from the communities to sensitize people about the services available at the health facilities was another key strategy in enhancing community participation and increasing demand at health 2 Intrinsic motivation of a health worker can be triggered by moral duty or attachment to the goals of the employer organization, while extrinsic motivation can be brought about by monetary incentives. 16 facilities during the Katete RBF Pre-Pilot. This phenomenon is similar to the conventional RBF theory of change. The RBF approach, which focuses on performance, pushes health workers to be innovative and think out of the box in their quest to attract more clients to utilize health services at their health facilities (Miller and Babiarz, 2013). For example, hiring retired health workers and other casual workers such as data entry clerks contributed to the availability of health workers and improvements in health service delivery. The availability of data entry clerks enabled nurses and midwives to devote more time to patient care than data entry and other administrative duties. Involvement of the community in implementation and monitoring further enhanced accountability over financial resources, decision-making and management of the health facilities. Canavan et al. (2008) support this observation, indicating that community participation is vital in RBF approaches as it guarantees feedback on quality of care, which is critical to improving client satisfaction. We triangulated our results on clients’ perception on the quality of health care with a study by Cheelo et al. (2014), which traced patients and confirmed that health services were indeed provided. Similarly, this study revealed that the clients had rated health workers working in Katete during the RBF Pre-Pilot project high in the provision of quality health care. On the contrary, results from the technical quality assessments on service provision at health centers were poor. These poor technical quality scores could be attributed to the structural nature of the quality assessment tool, which gave more weight to availability of inputs. Several items on the quality checklist were beyond the capacity of the health workers at the health facilities to improve upon and it would have been beneficial to the running of the RBF Pre-Pilot project if the quality tool had been revised prior to the launch of the Pre-Pilot. The government provided the highest amount of funding through staff salaries and operational grants but this funding remained constant during the entire review period. Meanwhile, the RBF performance grant increased by 73 percent by the end of the project. This implies that health facility performance was improving over time. The cost of the RBF Pre-Pilot project at US$1.82 per capita per quarter was also within the realm of other RBF projects in Africa, which are estimated at US$2 per capita per quarter (Canavan et al. 2008). Health facilities with higher catchment populations earned more money, which presupposes that performance was dependent on the catchment population. Studies looking at the pay structures and composition of personal income for public health workers in four African countries including Zambia reveal that income from per diems and private practice accounts for about 10 percent and 0.1 percent of total personal income, respectively (McCoy, et al. 2008). The study further shows that several health workers in Zambia generate supplementary personal income from other sources, but this practice was more common in urban than rural health centers (32 percent and 9 percent of staff, respectively) (McCoy, et al. 2008). Given that the Katete Pre-Pilot project was operational in rural areas where generating extra income from other sources was limited, the income from RBF evidently boosted both operational and personal income at the health facilities. This income was essential for improving staff motivation and productivity, and strengthening the link between the health facilities and communities. Despite the lessons, successes and challenges this case study has revealed, there are several limitations to this study. First of all, there is no counterfactual to attribute change to the RBF. This was not feasible considering the fact that Katete was the only district selected by the MOH to pre-pilot RBF before national roll out to other districts. Nonetheless, the research team was able to look critically at the RBF design, potential effects and policy implications by considering the Pre-Pilot project as a “case” and using a combination of qualitative and quantitative research methods to argue for association rather than casual effect. The study further suggests that RBF is context specific and requires meeting key implementation and design conditions to achieve intended goals and objectives. For example, critical inputs such as human resources, equipment, drugs and medical supplies have to be sufficiently available. Some health facilities managed to use the RBF funds to make some of these inputs available but it was not possible for smaller health facilities because they could not raise enough money through RBF due to low catchment populations. Delays in disbursement of RBF performance grants to health facilities was another negative aspect experienced during the Pre-Pilot project and could be a reason why some health facilities were not 17 able to improve their situation or comprehensively address their most urgent needs even with the introduction of the RBF. Moreover, even if they had money, it was difficult for them to find unemployed nurses and midwives in their vicinities. There were also concerns about geographical access to the health centers, which RBF did not directly address. RBF helped to bring more patients to the health facilities but there was still a problem of access by those in extremely remote areas. Thus, in general, health facilities still had limited control over the availability of these inputs despite the introduction of the RBF Pre-Pilot, and could explain why the technical quality audits produced poor results. As expected, increased demand and utilization of health services during the RBF Pre-Pilot contributed to an increase in staff workload. Nonetheless, individual performance incentives that were attached to increased output were sufficient to compensate for the increased workload. Financial incentives are designed to motivate employees to improve their performance by producing better results through increased effort leading to increased outputs and better quality (Ude and Coker, 2012). In this regard, financial compensation through the RBF Pre-Pilot provided extra money to the health workers for their increased contributions. However, increased staff workload should be within the established output standards, that is, the amount of work that an average, well-trained employee working at a normal pace should be able to accomplish in a given period of time (Ude and Coker, 2012). Furthermore, the extent to which the RBF Pre-Pilot project could have improved the productivity of the health workers was dependent on the degree of underemployment before the implementation of the project. Lastly, the use of community volunteers to sensitize people about the services available at the health facilities also had a negative consequence in that some village headmen expected to be incentivized for their services but were not. Future RBF projects should consider incentivizing community level service providers with clear guidelines and more effective communication around the unit prices and services to be delivered at community level. 18 PART V – POLICY LESSONS This case study adds to the global body of evidence on the health system effects of RBF. The Katete RBF Pre-Pilot project contributed to the strengthening of the health system as measured by the utilization of health services, client satisfaction, community participation, staff motivation and managerial autonomy at health facility level. The RBF Pre-Pilot project also contributed to improvements in data quality. Nevertheless, there are several cautionary notes from this Pre-Pilot with respect to the RBF mechanism. Although RBF is poised to enhance health worker motivation (Eichler and Levine, 2009), it requires the availability of basic health system components such as qualified staff, equipment, supplies, etc. In the absence of these, the health worker might be further demotivated. In low- and middle-income countries, where a stretched health workforce is coupled with sub-optimal drug and equipment provision, the primary intervention should provide all the basic inputs such as health workers, drugs and medical equipment so that RBF can stimulate productivity by motivating existing health workers. As such, RBF is not an end in itself and should be used to catalyze performance in a health system were all necessary inputs are available but under utilized. The design of the RBF incentive package (including quantity and quality assessment checklists) should put adequate emphasis on the baseline status of the health system, such as staff availability, mix and distribution, so that the design takes into account the diversity of roles that the health workers play and consequently incentivizes them appropriately. On the other hand, the flow of funds in a RBF program should be predictable both in time and volume because RBF is very much dependent on the link between financing and performance. The RBF approach aims to maximize responsiveness of health workers to the incentives while considering context and other potential constraints critical as design elements to the successful implementation of RBF programs. If the RBF model does not meet these basic conditions health facilities and communities can lose faith in the RBF program. Furthermore, RBF can be implemented in such a way that it explicitly addresses both the supply and demand side. Some community representatives expressed a desire to include the community-level service providers, such as volunteers and traditional birth attendants, in a contractual agreement with the health facilities—with clear guidelines for the price of each unit of service delivered at community level. Finally, the study demonstrates that RBF is context specific and that piloting can be an important instrument for testing and adapting a model to the local context before nationwide scale-up. What we observe from Zambia is that RBF can be adapted and implemented through a “contracted-in” public health sector model. Most importantly, the Katete Pre-Pilot generated evidence that was necessary to inform policy about the feasibility of implementing RBF in Zambia. Having been tested and refined in Katete, the RBF project was subsequently scaled up to ten additional districts countrywide. Qamruddin et al. (2013) observe that the Katete RBF Pre-Pilot project was useful in: (i) establishing eligibility criteria for the inclusion of health facilities; (ii) refining the quality tool for assessing structural and process quality at health centers; (iii) revising the fees per indicator; (iv) incorporating a performance target for the distribution of clinical health workers (this could be achieved by encouraging managers at district level to reshuffle skilled staff from health centers with excess capacity to needy health centers); and (v) outlining contracting mechanisms. The RBF Pre-Pilot project also acted as a medium to create sufficient local capacities and to refine tools for evaluating the impact of the successor RBF project, which commenced in April 2012. 19 REFERENCES Basinga, P., P. J. Gertler, A. Binagwaho, A. L. Soucat, J. STURDY and C. M. Vermeersch. 2011. “Effect on Maternal and Child Health Services in Rwanda of Payment to Primary Health-care Providers for Performance: An Impact Evaluation.” Lancet, 377, 1421-8. Banerjee, A., E. Duflo and R. Glennerster. 2008. “Putting a Band‐Aid on a Corpse: Incentives for Nurses in the Indian Public Health Care System.” Journal of the European Economic Association, 6(2‐3), 487-500. Borckardt, J. J., M. R. Nash, M. D. Murphy, M. Moore, D. Shaw and P. O'neil. 2008. “Clinical Practice as Natural Laboratory for Psychotherapy Research: A Guide to Case-based Time-series Analysis.” Am Psychol, 63, 77-95. Canavan A., J. Toonen, and R. Elovainio. 2008. Performance Based Financing: An International Review of the Literature. Amsterdam: Development Policy and Practice, KIT. Chansa C., M. Sjoblom and M. Vledder. 2014. Achieving Better Health Outcomes Through Innovative Strategies and Results-focused Interventions. In: Adam, C., Collier, P., and Gondwe, M., eds. Zambia Building Prosperity from Resource Wealth. Oxford: Oxford University Press, pp. 399-431. Cheelo, C., M. Vledder, J. Qamruddin, C. Chansa, J. Friedman and A. Das. 2014. External Verification of a Health Results Based Financing Program: the Katete Experience in Zambia. Washington, DC: World Bank. CSO (Central Statistical Office. 2003. Population Projections Report. Lusaka: Central Statistics Office, Zambia. CSO (Central Statistical Office), MOH (Ministry of Health), TRDC (Tropical Disease Research Center), University Teaching Hospital, University of Zambia, and DHS Program. 2014. Zambia Demographic and Health Survey 2013-14 Preliminary Report. Retrieved online from http://dhsprogram.com/pubs/pdf/PR53/PR53.pdf on 10th November 2014. Eichler, R. 2006. Can "Pay for Performance” Increase Utilization by the Poor and Improve the Quality of Health Services? Discussion paper for the first meeting of the Working Group on Performance- Based Incentives. Washington, DC: Center for Global Development. Eichler, R. and R. Levine. 2009. Performance Incentives for Global Health: Potential and Pitfalls, Washington, DC, Center for Global Development. Eisenhardt, K. M. 1989. Building Theories from Case Study Research. Academy of Management Review 14 (4): 532-550. Eldridge, C., and N. Palmer. 2009. Performance-based Payment: Some Reflections on the Discourse, Evidence and Unanswered Questions. Health Policy and Planning; 24: 160–166. Gertler, P. J., and C. M. Vermeersch. 2012. Using Performance Incentives to Improve Health Outcomes. Impact Evaluation Series No. 60. Washington, DC: The World Bank. Ireland, M., E. Paul and B. Dujardin. 2011. Can Performance-based Financing be Used to Reform Health Systems in Developing Countries. Bulletin of the World Health Organization; 89: 695–698. McCoy, D et al. 2008. “Salaries and Incomes of Health Workers in Sub-Saharan Africa. Lancet Vol 371: 675–81. 20 Miller, G., and S. Babiarzk. 2013. Pay-for-Performance Incentives in Low- and Middle-Income Country Health Programs. NBER Working Paper No. 18932. Mills, A. and G. J. Bromberg. 1998. Experiences of Contracting Health Services: An Overview of the Literature. Macroeconomics, Health and Development Series – Technical Paper No. 33. Geneva: World Health Organization. Murray, C. J., J. Frenk. and T. EVANS. 2007. “The Global Campaign for the Health MDGs: Challenges, Opportunities, and the Imperative of Shared Learning.” Lancet, 370, 1018-20. Qamruddin, J., C. Chansa, A. DAS. 2013. Using Evidence to Scale Up Innovation: Insights from a Results-Based Financing (RBF) Project for Health in Zambia. IFC SmartLessons Brief Report No. 74056. Washington, DC: International Finance Coorporation. Ramsay, C. R., L. Matowe, R. Grilli, J. M. Grimshaw and R. E. Thomas. 2003. “Interrupted Time Series Designs in Health Technology Assessment: Lessons from Two Systematic Reviews of Behavior Change Strategies.” Int J Technol Assess Health Care, 19, 613-23. Rodson, C. 2002. Real World Research. Malden, MA: Blackwell Publishing. Rusa, L., D. Ngirabega JDE, W. Janssen, S. Van Bastelaere, D. Porignon and W. Vandenbulcke. 2009. “Performance-based Financing for Better Quality of Services in Rwandan Health Centers: 3-Year Experience.” Trop Med Int Health, 14, 830-7. Serumaga, B., D. Ross-Degnan, A. J Avery, R. A. Elliot, S. R. Majumdar, F. Zhang and S. B. Soumerai. 2011. “Effect of Pay for Performance on the Management and Outcomes of Hypertension in the United Kingdom: Interrupted Time Series Study.” BMJ, 342, d108. SMA (Simulation Modeling Analysis) Program for Short Streams of Time-series Data [computer program] 2014. Version 9.9.28. Knoxville (TN): Clinical Research Solutions. Soeters, R., P. B. Peerenboom, P. Mushagalusa and C. Kimanuka. 2011. “Performance-based Financing Experiment Improved Health Care in the Democratic Republic of Congo.” Health Aff (Millwood), 30, 1518-27. Toonen, J., A. Canavan, P. Vergeer and R. Elovainio. 2009. Learning Lessons on Implementing Performance Based Financing, from a Multi-country Evaluation. Amsterdam: KIT, in Collaboration with CORDAID and WHO. Ude U., and M. A. Coker. 2012. “Incentive Schemes, Employee Motivation and Productivity In Organizations In Nigeria: Analytical Linkages.” IOSR Journal of Business and Management. Volume 1, Issue 4, pp 32-39. UNICEF. 2010. Progress for Children: Achieving the MDGs with Equity. New York: United Nations Children’s Fund. USAID. 2004. A Manual for Strengthening HMIS Data Quality. Retrieved online from http://uphold.jsi.com/Docs/Resources/Training/Health/manual_for_strenghtening_hmis_data_quali ty.pdf on 20th September 2013. Wagner, A. K., S. B, Soumerai, F. Zhang and D. Ross-Degnan. 2002. “Segmented Regression Analysis of Interrupted Time Series Studies in Medication Use Research.” J Clin Pharm Ther, 27, 299-309. World Bank. 2009. Zambia Health Sector Public Expenditure Review: Accounting for Resources to Improve Effective Service Coverage. Washington: The World Bank. 21 Yin, R. 2003. Case study design and methodology. Thousands Oaks, Sage Publications. Zambia. Ministry of Health. 2011. National Health Strategic Plan 2011-2015. Lusaka. 22 Global interest in results based financing (RBF) as a means of strengthening health systems and improving service delivery coincides with heavy Zambian investment in its health sector—but with poor maternal and child health (MCH) outcomes. Determined to remedy the situation, Zambia pre-piloted an RBF project in Katete district aimed at testing how RBF can be adapted to the country context and its suitability in strengthening the health system and improving service delivery. This case study reviews the design and implementation of the Katete RBF Pre-Pilot and explores its impact on access and utilization of MCH services and health systems strengthening. The research team used a mixed-methods approach, collecting qualitative and quantitative data through semi-structured interviews, group discussions, document reviews and financial and health service delivery databases. Changes in performance indicators were estimated using an interrupted time series analysis complemented by a simulated modeling analysis. In Katete, RBF increased immunization coverage of children less than one year old by 4 percent and curative consultations by 14 percent between 2008 and 2012. These achievements were facilitated by some health system improvements such as increased managerial autonomy and local decision-making at service delivery levels, enhanced staff performance and teamwork, and community participation. Despite these gains, the project experienced some delays in RBF disbursements to health facilities and increased staff workload due to increased utilization of health services. The study revealed clients’ perceptions of high service quality despite low technical quality scores. RBF can be used to improve health system performance and service delivery quality. However, the implementation and effects of RBF are context-specific and the model has to meet certain key design criteria and requires basic inputs for RBF to work effectively. ABOUT THIS SERIES: This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author/s whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@ worldbank.org) or HNP Advisory Service (healthpop@worldbank.org, tel 202 473-2256). For more information, see also www.worldbank.org/hnppublications. 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org