THE SYSTEMATIC ASSESSMENT OF HEALTH WORKER PERFORMANCE: A FRAMEWORK FOR ANALYSIS AND ITS APPLICATION IN TANZANIA DISCUSSION PAPER AUGUST 2015 Kenneth L. Leonard Melkiory C. Masatu Christopher H. Herbst Christophe Lemiere THE SYSTEMATIC ASSESSMENT OF HEALTH WORKER PERFORMANCE: A Framework for Analysis and its Application in Tanzania Kenneth L. Leonard, Melkiory C. Masatu, Christopher H. Herbst, Christophe Lemiere August 2015 Health, Nutrition and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population (HNP) Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. For information regarding the HNP Discussion Paper Series, please contact the Editor, Martin Lutalo at mlutalo@worldbank.org or Erika Yanick at eyanick@worldbank.org. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. © 2015 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition and Population (HNP) Discussion Paper The Systematic Assessment of Health Worker Performance: A Framework for Analysis and its Application in Tanzania Kenneth L. Leonarda Melkiory C. Masatub Christopher H. Herbstc Christophe Lemiered a University of Maryland, College Park, USA b University of Dar es Salaam, Dar es Salaam, Tanzania c World Bank Group, Washington DC, USA d World Bank Group, Brussels, USA This paper was prepared as part of the World Bank’s Africa Region Human Resources for Health Program, which was partly funded by the Government of Norway. It is a product of the Health Service Delivery Global Solutions Group in the World Bank’s Health, Nutrition and Population Global Practice Abstract: This paper introduces a simple framework for understanding the dimensions and determinants of health worker performance based on the idea that there can be three different gaps affecting performance: a knowledge gap, the knowledge-capacity gap and the capacity-performance gap. The paper argues that performance is determined by a combination of competence, capacity and effort, and that any of these elements may lead to poor performance, and applies this framework to the measurement of health worker performance in Tanzania. Whilst discussing and highlighting key findings related to the assessment of health worker performance in Tanzania, the overarching objective of the paper is to offer a systematic way to analyze health worker performance through primary data collection and analysis to benefit researchers and countries beyond Tanzania. Keywords: Human Resources for Health, Health Worker Performance, Productivity, Health Service Delivery, Tanzania Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Christopher H. Herbst, Senior Health Specialist. The World Bank Group, Washington DC. cherbst@worldbank.org iii iv Table of Contents RIGHTS AND PERMISSIONS ...................................................................................... II 1 ACKNOWLEDGMENTS ................................................................................... VIII 2 INTRODUCTION ......................................................................................................1 3 CONCEPTUAL FRAMEWORK OF PERFORMANCE ......................................3 3.1 Performance as a Function of Capacity and Effort................................................... 3 3.2 The Three Gap Model of Quality ................................................................................. 5 4 HEALTH WORKER COMPETENCE....................................................................6 4.1 Patterns of Competence ............................................................................................... 9 5 HEALTH WORKER CAPACITY TO PERFORM..............................................10 5.1 The Role of Competence and Access to Infrastructure and Equipment in Determining Performance ...................................................................................................... 10 5.2 Measuring Capacity through Maximum Observed Performance ......................... 14 6 HEALTH WORKER PERFORMANCE AND DETERMINANTS OF PERFORMANCE ............................................................................................................15 6.1 Examining the Know-Do gap ..................................................................................... 16 6.2 Examining the Can-Do Gap ....................................................................................... 18 6.3 The Hawthorne Effect as Direct Evidence of the Can-Do gap ............................. 21 6.4 Determinants of Performance .................................................................................... 22 7 PRODUCTIVITY AND THE DEMAND FOR HEALTH SERVICES ...............23 7.1 Facilities with more Patients are more Productive.................................................. 23 7.2 Health Workers can Improve Productivity by Serving Patients Faster ................ 24 7.3 Productivity and Performance .................................................................................... 25 8 HEALTH WORKER MOTIVATION TO PROVIDE EFFORT ..........................26 8.1 A Field Experiment on the Determinants of Motivation.......................................... 27 The Laboratory Experiment ............................................................................................... 27 v The Encouragement Experiment...................................................................................... 28 8.2 Results ........................................................................................................................... 29 8.3 Sources of Motivation: Patients, Peers, and Organizations .................................. 34 Generosity towards patients.............................................................................................. 35 Peer Esteem ........................................................................................................................ 35 Organizations ...................................................................................................................... 36 9 POLICY IMPLICATIONS ......................................................................................37 An Operational Research Agenda ................................................................................... 40 10 CONCLUSION........................................................................................................41 11 REFERENCES .......................................................................................................43 vi Table of Figures Figure 1 Three Elements of Performance........................................................................................ 4 Figure 2 The Three Gap Model ........................................................................................................ 5 Figure 3 Certification and Competence to Diagnose ...................................................................... 7 Figure 4 Access to Health Facilities Competent and not Competent in Diagnosing the Causes of infant Diarrhea in Arusha, Monduli and Arumeru .......................................................................... 9 Figure 5 Competence, Capacity and Performance in a Sample of Clinicians ................................ 16 Figure 6 Graphical Demonstration of the Know-Do Gap in Tanzania .......................................... 17 Figure 7 Capacity and Performance (the Can-Do gap) .................................................................. 19 Figure 8 The Hawthorne Effect and Adherence to Protocol in an Outpatient Setting ................. 22 Figure 9 Time use, Meatu and Mwanga Districts Tanzania........................................................... 23 Figure 10 Outcomes categorized by the Average Quality of Care at Location of First Visit. ........ 24 Figure 11 Protocol Adherence Overall and for Generous Clinicians ............................................. 30 Figure 12 Changes in Quality from Peer Scrutiny and Encouragement ........................................ 31 Figure 13 Changes in Protocol Adherence with Scrutiny and Encouragement............................. 32 Figure 14 Generosity and Protocol Adherence Across Sectors ..................................................... 33 Figure 15 Changes in Protocol Adherence with Encouragement by Sector.................................. 34 vii 1 ACKNOWLEDGMENTS This discussion paper is a core output of the Human Resources for Health team within the Health Service Delivery Global Solutions Group in the Health, Nutrition and Population (HNP) Global Practice. The data for this paper and the technical content is drawn from the experience of four studies of health workers in the Arusha and Iringa regions and two studies of household health seeking behavior in these same regions. The studies, led by Kenneth L. Leonard and supported by the World Bank’s Africa Region Human Resources for Health Program, collected a substantial amount of primary data on different dimensions of health worker performance. The contribution of a long list of collaborators towards these studies has been acknowledged elsewhere. The Authors of this paper are particularly grateful to the Tanzanian Government, the University of Dar Es Salaam, and the Government of Norway for their collaboration and financial support. From the World Bank, particular thanks go to Son Nam Nguyen, Lead Health Specialist, and Akiko Maeda, Lead Health Specialist, for their invaluable comments towards earlier drafts of this report. The authors are also grateful for the support of Magnus Lindelow, Practice Manager for East and Southern Africa, HNP Global Practice, Trina Haque, Practice Manager for West and Central Africa, and Mickey Chopra, Global Solutions Lead of the Health Service Delivery Group in the HNP Global Practice. Finally, the authors are grateful to the World Bank for publishing this report as an HNP Discussion Paper. viii 2 INTRODUCTION One of the most pressing policy issues in developing countries is the significant mortality caused by preventable illness. This is particularly true in Tanzania, which has one of the highest infant mortality rates in the world, even though it has suffered neither war nor significant famine in its modern history. Estimates based on scenarios analyzed in Jones et al (2003) suggest that 100,000 child deaths in Tanzania could be averted by low cost interventions delivered through the existing health care system. Most Tanzanians have reasonable access to health facilities however the quality of care provided is poor). Policy makers in Tanzania (as in most countries) are focused on increasing the effectiveness of the existing health system. Tanzania has made important progress on making health care accessible to its population (Abel- Smith and Rawal, 1992, Klemick et al, 2009). It has addressed these issues by building health facilities and training cadres of health workers specifically to serve in these new primary and secondary care facilities. However, access does not mean that adequate care is provided. Seventy nine percent of the children under the age of five who died from malaria in Tanzania sought care from modern medical service providers before they died (de Savigny et al, 2004). The number of health workers is insufficient and their access to key equipment, medicines and health commodities is insufficient. Tanzania is plagued by malnutrition, malaria and HIV/AIDS, all of which make it difficult for a well-functioning health system to deliver effective health care. Unfortunately, in many respects, the health care system in Tanzania is not functioning well (Leonard and Masatu, 2005, 2007a). In this paper we focus on the performance of the health care workers in Tanzania, particularly the quality of care they provide for their patients. The objective of the paper is to report the findings of health worker performance assessments in Tanzania, presenting evidence to help the development of appropriate solutions to improve service delivery. A broader objective is to highlight a systematic way to analyze health worker performance through primary data collection and analysis to benefit researchers and countries beyond Tanzania. The data for this paper is drawn from four studies of health workers in the Arusha and Iringa regions and two studies of household health seeking behavior in these same regions. The studies, led by Kenneth L. Leonard and supported by the World Bank’s Africa Region Human Resources for Health Program, collected a substantial amount of primary data on health worker performance. The samples from these studies are small and are not nationally representative. However, by examining multiple dimensions of the performance of health workers across experimentally manipulated conditions, the data offer a rich view of the activities and motivations of health workers. Where possible we augment this view with the work of other researchers in Tanzania. Drawing on a simple but effective framework developed by the authors of this paper, we show that many health workers who are trained to provide the necessary level of care in Tanzania, do not in fact do so for their patients, and that this is linked to a number of intrinsic and extrinsic factors. We identify the determinants for the sub-optimal performance of health workers in Tanzania; therefore, we examine absenteeism, supplies of equipment, infrastructure, training and motivation as determinants of low quality. We find that all of these factors help to explain low levels of quality care. Tanzania health care workers have insufficient training, are asked to live in 1 remote locations and work in poorly maintained and supported buildings with irregular supplies of medicines and other commodities. Their salaries are frequently paid late and they receive few incentives from the supervision system to improve the quality of care. However, in this paper, we show that the easiest way to improve the system is to focus on health worker motivation. There is substantial evidence that the quality of care can be significantly improved without improving the other elements in the health care system. This is not to suggest that these improvements are not important, but may not be sufficient to achieve quality improvements. Conversely, the evidence suggests that improvements in training and infrastructure (in particular) alone without addressing motivation may not lead to any improvements in quality, hence motivation could be an essential component of performance. As we discuss in this paper, there are many factors determining motivation, including extrinsic (external) and intrinsic (internal) factors. Behaviors are influenced by economic and financial incentives as well as social/non-financial incentives. There is no question that extrinsic motivation is necessary for the performance of health workers in Tanzania: health workers need a salary that allows them to meet their basic needs and is sufficient to attract them to work in the sector. However, salaries for health workers in Tanzania have increased significantly recently and there is no evidence that these increases have improved quality (though they may have improved the willingness of health workers to enter the public sector). It is possible, even likely that extrinsic motivators such as promotions or salary bonuses, would improve quality, but the capacity of the health system to adequately monitor and reward quality is questionable. In this paper, we pay particular attention to intrinsic sources of motivation, referencing professional norms, altruism and responses to peer scrutiny and encouragement. We show that health workers work harder when they are observed by peers and work harder when they believe they can earn the respect or recognition of their peers. Again, this finding represents an opportunity for Tanzania, an opportunity to increase the quality of care without significant additional expenditure. The report is organized as follows: Section 2 introduces a simple framework for understanding the dimensions and determinants of performance based on the idea that there can be three different gaps affecting performance: a knowledge gap, the knowledge-capacity gap and the capacity-performance gap. The idea is that performance is determined by a combination of competence, capacity and effort and that any of these elements may lead to poor performance. Section 4 examines the capacity of health workers in this setting. Section 5 examines their capacity to perform and Chapter 6 examines their performance and the role of knowledge and capacity in this performance. Section 7 looks at the productivity of health workers which is a function of demand. We demonstrate that the demand for services is a function, not only of price and distance, but is also of the quality of care provided and that therefore quality and productivity are most often positively related. Section 8 examines the evidence on health worker motivation to provide effort. Section 8.3 looks in greater depth at the sources of motivation. Section 9 examines the policy implications of our views of health worker performance, and section 10 concludes. 2 3 CONCEPTUAL FRAMEWORK OF PERFORMANCE In this section we outline the conceptual framework for the analysis of health worker performance, which is discussed in this paper. 3.1 Performance as a Function of Capacity and Effort Presence, quality and productivity are three important measures of health worker performance. Clearly, in order to provide health care, a health worker must be present. This means they must be trained, posted to the facility and available to patients seeking care. In addition, the health worker must provide adequate quality care to those patients who do come. Finally, health workers should be productive at transforming inputs (including their own time) into outputs; the more patients a health worker can help, the better. What then are the determinants of performance in Tanzania, or elsewhere? Figure 1 represents a simple model of the determinants of performance. Three categories of health worker characteristics are presented: competence, capacity and performance. Each category is described with respect to its input, intermediary output and output in the health production process. Competence is the measurable output derived from health worker knowledge and skill. Education and experience determine health worker knowledge and skill. Although competence is an output of the health system, it also contributes to another output, capacity. Capacity is the measurable output produced by competence combined with infrastructure and supplies (equipment, medicines and medical consumables). Infrastructure and supplies, in turn, have their source in the workplace support and environment. Capacity, in addition to being an output of the individual health worker (or system), contributes to performance. Performance is a function of both capacity and effort. Effort, in turn is determined by the motivation of health workers. Many of the inputs and intermediary outputs in Figure 1 can be thought of as elements that could be addressed by policy, however not all of them are easy to measure, nor are they reflective of actual outputs. Inputs and intermediary outputs such as health worker motivation and skill or infrastructure and supplies for example can be measured directly, but are not, strictly speaking, final outputs of the health worker or health system. We could, for example, visit a health facility and inventory the medicines and supplies, including available support staff, but this by itself is not a useful measure of the capacity of a health worker; that would require knowing whether the health worker was qualified to use those medicines. Effort, on the other hand, though essential, is not directly measurable in health care (although it can be measured indirectly). However, we will make the case that the three outputs listed (competence, capacity and performance) are measurable and important outputs of the health system. Although performance is probably the most important output (as it measures what health workers actually do with their patients) capacity and competence also tell us important things about the health worker or health system. All else considered, we would rather higher levels of competence and capacity exist. 3 Figure 1 Three Elements of Performance Input* Intermediary Output output** Education, Health Worker Competence Experience Knowledge and Skill Workplace Infrastructure support and Capacity and supplies environment Motivation Effort Performance Source: Authors Note:*The term Input is used here for simplicity given the framework’s focus on the three categories. The inputs here can also be considered outputs of earlier policy inputs, such as interventions to improve the capacity of education systems, organizational management and functioning supply chains, and monetary and non-monetary incentives. **: Intermediary output here can also be considered a direct input to the three outputs, and referred to as such in relevant sections of this paper. In Chapter 4 we examine the evidence on the competence of health workers, turning, in Chapter 5, to their capacity to provide care and finally, in chapter 6 to performance. As indicated in Figure 1, motivation is the main driver behind the transformation of capacity into performance: health workers who are motivated to provide high levels of effort have high levels of performance when they have adequate capacity. However, motivation also can play a role through its link with workplace support and environment: sometimes, low levels of infrastructure and supplies (and therefore low capacity) can be caused by low levels of motivation, and sometimes low levels of infrastructure and supplies can cause low levels of motivation. There is interplay between the work environment and the motivation of health workers: sometimes a poor work environment for example is the product of poorly motivated health workers, and sometimes poorly motivated health workers are the product of a poor work environment. It will be important to keep our eye on this additional link between motivation and capacity as we examine the determinants of motivation. 4 3.2 The Three Gap Model of Quality Performance, capacity and competence can be measured on the same scale: performance, capacity to perform and competence to perform. Once they are measured in the same scale, we can advance our understanding of the determinants of performance by examining the interplay between these key outputs. Each health worker can be described by the triplet of competence, capacity and performance. This triplet might vary by task, but generally, it is easier to think of the triplet as describing the average of each measure over all the tasks the health worker is likely to be asked to perform. Competence measures the result of training (including initial training, on- the-job training and continuing education), and tells us what the health worker knows how to do. Capacity is less than knowledge, because capacity measures the ability of the health worker to use his or her knowledge in the current workplace. That work place might impose constraints due to lack of equipment or medicines, for example. A health worker with high knowledge is likely to have greater capacity if they were posted to a referral hospital than if they were posted to a rural clinic. Performance is the measure of what a health worker does with his or her regular patients. Performance cannot be greater than capacity, and might be significantly lower. Figure 2 The Three Gap Model Source: Authors Figure 2 shows these three measures in a two dimensional space. Knowledge and capacity are shown on the same axis and all three are measured in the same units with the same scale. The 45-degree line represents the “performance barrier” that performance cannot be greater than capacity (or knowledge). We can translate both knowledge and capacity into performance type measures: the knowledge to perform and the capacity to perform. These measures are shown on the Y axis, and result from translating the levels of knowledge and capacity using the 45-degree line. In addition, we have shown a performance target or a mandated level of performance. 5 Thus, Figure 2 shows four measures on the y-axis: the performance target, the competence to perform, the capacity to perform and performance. The differences between these four measures produce three gaps: the know gap, the know-can gap and the can-do gap. As we will show below, the performance target is set for each type of health worker, competence, capacity and performance can be measured and therefore the three gaps can be estimated for every health worker. The two dimensional representation of these three measures allows for more intuitive interpretation of some of the patterns in the gaps over a sample of health workers. Thus, for example we can plot the competence and performance of a sample of health workers to see the average pattern related the two measures. We could plot competence, capacity and performance simultaneously for a sample of health workers, but that would be harder to interpret visually. Graphs demonstrating the value of the three dimensional view will be shown below. 4 HEALTH WORKER COMPETENCE Leonard and Masatu (2007) report the results of measurements of competence from a sample of health workers in urban, peri-urban and rural Arusha region. Using case study vignettes they measured the competence of health workers to properly diagnose common outpatient conditions by comparing the actions they take to national protocol. The case study vignettes were performed using an actor, with another researcher present to record the activities of the clinician. All clinicians in the study knew they were being examined and therefore the measures of competence represent the best possible performance of clinicians, limited only by their knowledge of national protocol. There were four tests presented to one hundred and two clinicians. Each test had a maximum score of between 12 and 13 protocol items correctly identified. The average score per test was 4 and half items correctly identified with a maximum of between 8 and 10. In addition to variation in the organizations and location, there is sufficient variation in the credentials of the health workers to allow us to examine competence by training. The clinicians in the sample include nurses of various specializations, clinical assistants (assistants), clinical officers (officers), assistant medical officers (AMOs), and medical officers (MOs). Clinical assistants have an elementary school education and three years of medical training. Clinical officers traditionally have O level education (4 years of secondary school) and two years of medical training. AMOs are clinical officers with two additional years of training. MOs have both an A level education (6 years of secondary school) and 5 five years of university-level medical training. Nurses are not supposed to diagnose patients but in the rural areas they are frequently the only health personnel present and they do diagnose patients in these circumstances. Thus, the measures of competence for nurses represent a minimum competence for people with no training in protocol; they are not a measure of the competence of nurses in tasks for which they have been trained. Figure 3 shows the levels of competence (measured as the number of items of national protocol correctly identified compared to untrained personnel). All personnel trained in national protocol 6 have higher scores than those who are not trained in national protocol and doctors have much higher levels of knowledge than all other cadres. Note that all the cases examined with vignettes represent cases for which even assistant clinical officers have been trained. Thus, medical officers are knowledgeable about 2 and half more items out of an average of 5 and AMOs, COs and ACOs use about one more item correctly compared to untrained personnel. Figure 3 Certification and Competence to Diagnose Source: Authors Also shown on the graph is the probability that a health worker would correctly diagnose the illness presented in the case study vignette (compared to uncertified personnel). Again, medical officers are significantly more likely to properly diagnose the vignettes. Leonard and Masatu (2007) show that, when the certification of each clinician and the number of items correctly identified are examined as determinants of correct diagnosis, only the number of items correctly identified is significant. This means that health workers get the correct diagnosis because they use national protocol; MOs are good at diagnosing because they are good at using national protocol, not just because they are MOs. The regression underlying these results also examines the role of experience and finds that health workers with more experience are more likely to correctly identify elements of national protocol. Thus, we see that certification and experience increase the knowledge and competence of health workers to diagnose the kinds of illnesses that are likely to present in outpatient settings. Klemick, Leonard and Masatu (2008, 2009) use these same data to examine the distribution of competence in Arusha region. Presence is the most important feature allowing access to competent health workers; competence is only important among health workers who are present. 7 Table 1 shows the level of presence (and absenteeism) broken down by facility level, ownership, location and cadre (certification). Overall, 73 percent of scheduled health workers were present during random site visits. Hospitals and facilities in urban areas are more likely to have health workers present and MOs and AMOs have much higher rates of presence than other cadres. Health workers are more likely to be present in private facilities and the rates of presence are similar for public and NGO facilities. Clearly, location plays an important role in the rate of absenteeism; rural facilities are less likely to have heath workers present. There are fewer private facilities in the rural areas, and in general, very few MOs and AMOs are posted to the rural areas. Thus, most of the differences in absenteeism are driven by the location of the facility and NGO facilities and public facilities, which both serve similar populations, have similar rates of absenteeism. Table 1 Absenteeism in Facilities Accessible to Patients in Rural Arusha Broken down by # Scheduled Present Absent Facility Level Dispensary 28 66% 33% Health Center 19 52% 46% Hospital 46 84% 15% Ownership Public 63 71% 28% Private 15 80% 20% NGO 15 73% 26% Location Rural 19 63% 34% Semi-rural 38 74% 25% Urban 36 77% 22% Cadre MO 11 90% 10% AMO 15 87% 13% C. Officer 45 76% 23% C. Assistant 22 45% 55% Overall average 93 73% 26% Source: Klemick, Leonard, Masatu (2008) Figure 4 demonstrates access to competence in the study area. Each facility that has at least one health worker present on a regular basis is shown as a circle, representing the 5 kilometer radius catchment area. Roads are represented with various lines, representing the type of road. The study took place as part of a household survey and the estimated populations from the survey are represented with bars centered on the sub-village location. The survey did not sample households in Urban Arusha, but there is a large population located in the lower right area of the map. Importantly, the map shows access to competence, using the vignette for infant diarrhea as an example. Empty circles represent those facilities where there is no one present to properly diagnose and treat infant diarrhea and the solid circles represent those facilities where there is someone competent present. Although there are many facilities in the urban areas that do not have anyone competent present, the problem of access is much worse for households in the rural areas because the distances they must travel to reach a competent provider are much greater. 8 Figure 4 Access to Health Facilities Competent and not Competent in Diagnosing the Causes of infant Diarrhea in Arusha, Monduli and Arumeru Map of Arusha, Monduli and Arumeru districts showing roads, sampled subvillages and health facilities. Sampled subvillages are represented by black bars proportional to their population. Each facility is represented by a 5km-radius circle, indicating the accessibility by foot travel to the surrounding population. Striped circles signify health facilities with at least one clinician competent in diagnosing the causes of infant diarrhea (defined as an 80 percent or greater probability of correct diagnosis based on vignette scores), and empty circles are health facilities without any clinician competent in infant diarrhea diagnosis. Source: Klemick, Leonard and Masatu (2009, appendix) 4.1 Patterns of Competence Despite the reasonable access to health facilities, access to competent health care providers is relatively low in Tanzania. This lack of access is due in part to absenteeism and in part to low levels of training and experience. Although most households in Arusha region are near a health facility, in the rural areas, many health personnel are frequently absent, meaning there is often not someone present to deliver care. The rates of absenteeism, however, are similar across private, NGO and public facilities, which suggests that absenteeism is driven by the difficult realities of delivering health care in rural areas. The fact that Tanzania has expanded health care through the use of cadres of medical personnel with significantly less training than medical doctors is evident in the low levels of competence in Tanzanian facilities. There is a strong link between training and competence and many health workers have low levels of training. This problem is particularly manifest in rural areas because travel costs make it much more expensive for rural households to seek health care workers with greater competence. In fact, travel costs are so important in rural areas that improving road quality is one of the most important policy approaches to improving access to quality care (Klemick et al. 2009). 9 5 HEALTH WORKER CAPACITY TO PERFORM The capacity to provide quality health care is a function of competence combined with access to the necessary infrastructure and supplies. It is not a measure of what health workers actually do with their patients, but a measure of what they could do if they exerted full effort. It is possible to measure capacity directly; however, in the various projects that have been done in Tanzania to date we can only infer capacity indirectly. First, we examine the key elements that would contribute directly to capacity (competence, access to infrastructure and supplies and workload) and their roles in both competence and performance (both of which are measurable). This allows us to show that when certain types of equipment are missing, performance is low, even when competence is high, a finding that allows us to infer that low capacity can limit the performance of health workers. Second, we look at the best performance of health workers over a period of time and take that measure as a proxy for their capacity: if they demonstrate that they can perform a particular service, we can infer that they have the capacity to perform that service. This allows us to show that there is a statistical link between capacity and performance as well as a significant can-do gap. Both views of capacity concur that inadequate access to infrastructure and equipment do limit the performance of health workers in Tanzania. Both views also require us to look at performance as well as competence. The performance scores discussed here are similar to the competence scores except they examine the behavior of clinicians with their patients rather than with case study patients. For a series of presenting conditions, we compare the activities of health workers to the items required by national protocol for those same conditions. As with the items in the vignettes, we can assume that when health workers perform more of the required items, they are more likely to make the correct diagnosis and to avoid incorrect additional diagnoses. Unlike with the vignettes, we do not know the actual illnesses from which patients were suffering. 5.1 The Role of Competence and Access to Infrastructure and Equipment in Determining Performance Here we examine the data from the census of health facilities in an area of Arusha region, discussed above, and published in Leonard and Masatu (2005) and Leonard, Masatu and Vialou (2007). At each facility, the research team noted the state of infrastructure and existence of key supplies necessary for outpatient care. We examine those data here and their link to the competence and performance scores discussed in Leonard et al (2007). Table 2 shows the correlations between the location of a facility, the daily load and various equipment and infrastructure scores. The specific factors that make up the scores for consultation room, drugs, infrastructure, equipment, and waiting room are listed in Table 3. 10 Table 2 Correlations Component Scores of Infrastructure and Equipment room consultation drugs waiting room infrastructure equipment load daily patient setting: urban to main road setting: close consultation room 1.00 Drugs 0.80*** 1.00 waiting room 0.63*** 0.73*** 1.00 Infrastructure 0.60*** 0.68*** 0.61*** 1.00 Equipment 0.77*** 0.89*** 0.78*** 0.80*** 1.00 daily patient load -0.05 -0.01 0.11 0.15^ 0.15^ 1.00 setting: urban -0.09 -0.11 -0.18 0.43*** 0.02 0.23** 1.00 setting: close to main road 0.15^ 0.17* 0.24** -0.24** 0.12 -0.04 -0.74*** 1.00 setting: rural -0.08 -0.07 -0.05 -0.30*** -0.19* -0.31*** -0.45*** -0.27*** *** p<0.01, ** p<0.05, * p<0.10, ^ p<0.20 Source: Authors As we can see from Table 2, there is significant correlation among the scores that describe the equipment and infrastructure of the facility. In particular, the supply of drugs and the supply of equipment are correlated with a coefficient of 89 percent--the supplies of the consultation room and the availability of drugs have a correlation coefficient of 80 percent. This is not surprising, because we expect the supply of factors to be driven by the same underlying processes. Location, fee structure, ownership and facility management should all play a role in the availability of all of the various equipment and commodities. Indeed, note that a rural location is negatively correlated with all of the scores, though the correlation is not very high and is only significant for infrastructure and equipment. If, indeed all of the scores are driven by only a few underlying factors, we can determine this using factor analysis on the five scores that describe infrastructure and equipment. Table 3 shows the results of using factors analysis on these five scores. Although factor analysis does not provide an interpretation of the variables, we can see from inspection that one of the scores is more heavily weighted on factors that are permanent (infrastructure, condition of the waiting room and equipment, for example) and the other is more heavily weighted on factors that are temporary (drug supply and equipment, for example). We name these two factors the VARIABLE INPUTS and FIXED INPUTS factors to connote this sense. Note that the condition of the consultation room is more heavily weighed on the VARIABLE INPUTS factor than on the FIXED INPUTS factor, even though much of the equipment that should be available is more easily described as a permanent rather than temporary factor. This slight deviation in our understanding of the two factors underlying infrastructure and equipment suggests a second interpretation of the two factors: that the first describes things that are easier for individual health workers to control and the second describes things that require a broader institutional response. Under this interpretation, health workers will make sure they have 11 the right equipment in the consultation room and in the pharmacy, but the institution must ensure the quality of the infrastructure of the building and the supply of other equipment. Table 3 Fixed and Variable Factors of Facility Supplies VARIABLE Factor 1 Factor 2 VARIABLE INPUTS FIXED INPUTS consultation room 0.291 -0.070 Presence of:• At least one table and two chairs • an examination bed • a way to wash hands • functioning stethoscope • functioning thermometer • functioning sphygmomanometer • functioning otoscope • some spatula • functioning flashlight • gloves • new patient cards • adequate room lighting Drugs 0.455 -0.107 Drugs available: SP • Amodiaquine • Quinine Injection • Quinine tablets • ASA tablets • Paracetamol • ORS sachets • Cotrimoxazole Tablets • Cotrimoxazole Syrup • Penicillin G • Penicillin V tablets • Ampicillin tablets or capsules • Ampicillin syrup • Tetracycline • Metronidazole tablets • Mebendazole tablets • Tetracycline eye ointment • BBE • Multivitamin tablets • Chloroquine tablets • Chloroquine syrup • Chloriquine injection • Quinine Injection • Second line anti-malarial drug infrastructure -0.112 0.327 Presence of: At least one room for nursing activities • Nursing activities room in good general condition • Nursing activities room ventilated and well lit • Space or room to get injections in privacy • Room for patients to rest • At least one latrine • Latrine in good general condition • Piped water equipment 0.179 0.554 Presence of: Functioning scale for weighing • Method of determining a patient's height • Materials to prepare and administer ORS solution • Syringes and needles • Sterilizer and a stove • Broad spectrum antiseptic • Bandages • Plaster • Scissors • Forceps (dressing and dissection) • Sutures • Needle holder • Drug envelopes • A working microscope • Paint on the building in good condition • Roof in good condition • Grounds well kept waiting room -0.031 0.077 Presence of: Waiting room/ veranda • Place for patients to sit • Waiting room in good general condition • Waiting room ventilated and well lit Source: Authors We have advanced a view of capacity that includes both competence and access to infrastructure and equipment. To better understand the role of capacity we look at the link between competence and our two factors as well as the relationship between performance and the two factors (as well as competence). Table 4 shows the results of three regressions on the two factors that describe the many measures of infrastructure and supplies. The first regression shows the relationship between competence and the two factors and the second regression shows the relationship between performance and the two factors. If we assume that capacity is the combination of competence and equipment, we are looking to see if capacity is more closely related to competence or to performance. If competence and our measures of infrastructure and equipment are positively correlated then we can assume that capacity flows from competence: where doctors know what to do, they have the infrastructure and equipment necessary to provide it. 12 Table 4 Implied Relationship of Capacity to both Competence and Performance (1) (2) (3) VARIABLES COMPETENCE PERFORMANCE PERFORMANCE (quality) (quality) COMPETENCE 0.100^ [0.137] Factor: VARIABLE INPUTS 2.857 5.467*** 5.266*** [0.371] [0.010] [0.012] Factor: FIXED INPUTS 6.162** -0.0960 -0.733 [0.017] [0.953] [0.658] Constant 51.05*** 32.45*** 27.33*** [0.000] [0.000] [0.000] Observations 101 86 86 *** p<0.01, ** p<0.05, * p<0.10, ^ p<0.20 p-values in brackets Source: Authors Column one shows that competence is highly correlated with one of the scores: FIXED INPUTS. Thus, where competence is high we also find high levels of infrastructure and equipment: better doctors work in better equipped facilities, most likely because better equipped facilities attract better doctors. More importantly, the pattern suggests that capacity is not constrained by the levels of inputs measured by fixed inputs. On the other hand, column one also shows that competence is not well correlated with variable levels of inputs (drugs and the supplies that should be available in the consultation room). The coefficient for VARIABLE INPUTS is small and not significant. This suggests that when competence is high, it is not obvious that variable inputs are available, limiting the capacity of health workers to provide high effort. Thus, there is evidence of a gap between competence and capacity (the know-can gap) which can be caused by low levels of variable inputs (such as drugs). Further evidence of this gap can be seen in the relationship between performance (quality) and the two infrastructure and equipment scores (Column 2). Performance is not related to the FIXED INPUTS factor, but is strongly related to the VARIABLE INPUTS factor. This relationship suggests that performance is lower when variable in puts are lower and greater when variable inputs are, but that fixed inputs do not affect performance. Again this makes sense given the view of capacity seen in column one: fixed inputs do not restrain performance, because they are automatically present when health workers are also competent. Column 3 shows this relationship more explicitly. Our view of performance advanced in Section 2.1 is that performance is a function of capacity and effort, and that capacity is a function of competence and the availability of supplies and equipment. Column 3 shows that performance is a function of competence and variable inputs, but not a function of fixed inputs. This is not because fixed inputs are not important, but because fixed inputs do not constrain competence (shown in column one). 13 Thus, we can infer that capacity is limited by competence and access to variable inputs, but is not limited, in this setting, by access to fixed inputs. Since capacity is limited by access to variable inputs, performance will also be limited by access to variable inputs. 5.2 Measuring Capacity through Maximum Observed Performance Another way to view capacity is to examine the maximum observed performance exhibited during direct observation. If health workers do not usually perform a specific task (required by protocol) but do so occasionally, it can be inferred that they know how to perform that task and are capable of performing that task. It is not an ideal measure of capacity because a health worker may be capable of more than they ever exhibit and if supplies are short, they may occasionally use a particular medicine (for example) even if they cannot regularly use that medicine. Nonetheless, this is a proxy measure from which we can learn something about the relationship between capacity and competence, and later, between capacity and performance. Examining the sample of health workers discussed in Leonard et al (2007) and Leonard and Masatu (2005), we take the maximum level of performance displayed in consultations for presenting with a fever, cough or diarrhea. Table 5 shows a regression of this new variable, CAPACITY, on COMPETENCE, VARIABLE INPUTS and FIXED INPUTS. We can see that the capacity of a health worker is strongly related to the first factor, VARIABLE INPUTS, but not to either competence or FIXED INPUTS. This result is similar to the result we found above, except that we see no correspondence between capacity and competence. However, that is due, at least in part, to the way that we have measured capacity: because capacity and performance are measured on the same scale, the two scores are much more likely to be correlated with each other and capacity is less likely be correlated with competence. Both views of capacity suggest that it is important to understanding the outputs of the health system in Tanzania. Furthermore capacity is clearly constrained by access to variable inputs. It is not clear whether capacity is directly constrained by competence and fixed inputs, but some of the evidence suggests that it is so constrained. The Know-Can gap, on the other hand, is determined by access to variable inputs. We turn now to a more direct understanding of the determinants of health worker performance as measured by the quality of care. 14 Table 5 Capacity as a Function of Competence and Access to Infrastructure and Equipment (1) VARIABLES CAPACITY COMPETENCE 0.045 [0.536] Factor: VARIABLE INPUTS 4.676** [0.032] Factor: FIXED INPUTS 0.301 [0.863] Constant 40.34*** [0.000] Observations 82 *** p<0.01, ** p<0.05, * p<0.10, ^ p<0.20 p-values in brackets Source: Authors 6 HEALTH WORKER PERFORMANCE AND DETERMINANTS OF PERFORMANCE We turn now to health worker performance as measured by the adherence to protocol. Remember that in section 2.1 we advance the view that performance is determined by a combination of capacity and effort. Effort is not directly observable, but we can infer the level of effort by looking at the degree to which performance is explained by capacity and more directly by competence and the other elements that contribute to capacity. We measure performance by the degree to which health workers adhere to protocol as required by a patient’s presenting conditions. The research team visited each health facility at least twice over a two year period and observed the consultations of the patients who attended on the day of the visit. A member of the research team (always at least a clinical officer) sat in the room while the patient was being consulted and recorded whether or not the clinician performed a series of necessary items, given the patient’s presenting condition. In order to make the performance numbers roughly comparable across facilities, we standardized the scores by the presenting condition, focusing on cases in which the primary complaint was fever, cough or diarrhea. This data is discussed in more detail in Leonard and Masatu (2005) and Leonard et al (2007). Figure 5 shows the distribution of competence, capacity and performance for clinicians analyzed in this section. As expected, given the definitions of the terms, competence is greater than capacity which in turn is greater than performance. On average health workers demonstrate competence which is about 52 percent of all items required by protocol, capacity which is about 43 percent of the items required by protocol and performance which is about 33 percent of the items required by protocol. In this section, capacity and performance are measured on exactly the same scale and the range of scores is therefore similar. Competence, however, is measured 15 by using vignettes which are scored on a different scale. Although there is sufficient overlap in the two scales that we can accurately compare the averages, the range is much wider for competence than for either capacity or performance. Figure 5 Competence, Capacity and Performance in a Sample of Clinicians 80 percentage of items used correctly 20 40 0 60 competence capacity performance Source: authors 6.1 Examining the Know-Do gap Leonard and Masatu (2005, 2010) point out the existence of a consistent know-do gap (the difference between competence and performance) for the average clinician in the study area: the average clinician can demonstrate significantly greater knowledge of protocol than he actually follows in practice. Mliga (2000) examines data from Arusha region and Iringa region and shows a marked disparity between the performance of health workers in the charitable/NGO sector and the performance of health workers in the public sector. In addition, Mliga (2000) examines the various organizational features of these different health systems to try to understand why there are such important differences in the performance of health workers. He concludes that there are important differences in the management of public and NGO facilities and that these help to explain the differences in observed quality of care. Using the organizational indicators collected by Mliga (2000), Leonard, et al (2007) show that the size of the know-do gap is a function of the degree of decentralization in a facility. In other words, facilities where health workers and chiefs of post have greater control over important decisions (see table 6) are also facilities where performance is closely related to competence. Leonard and Masatu (2007) also look at the differences in the distribution of competence and performance in the rural and urban areas of Arusha region. They find that the competence of health workers in the rural areas is low (both in public and NGO facilities) but that the performance of health workers in the rural areas is much better in NGO facilities than in public facilities. 16 Together, these papers suggest that there is something about charitable organizations that differentiates them from public sector facilities in translating competence into performance. As illuminated in Table 6, these facilities differ from public facilities in a number of ways. The variables used include: a dummy variable indicating whether the chief of post can hire and fire personnel; the level at which salaries are set (national/regional/local); the degree to which the chief of post can (or must) use local funds to pay salaries and buy medicines; and the level at which choices about staffing are made (national/regional/local). These individual components can be shown to have a single underlying factor, which we call the decentralization score. The hypothesis advanced in Leonard et al (2007) is that decentralization increases the incentives to provide high levels of effort. Thus the combination of effort and competence leads to higher levels of performance, despite difficulties like rural location (Leonard and Masatu, 2007). Table 6 Factors Contributing factors to the Decentralization Score Variable Coefficient (std err) The ability to hire and fire personnel (yes/no) 1.245 (0.030) The level at which salary decisions are made 0.175 (0.007) The level at which financial decision are made 0.119 (0.019) The level at which staffing decisions are made 0.121 (0.010) Source: Leonard, Masatu and Vialou (2007) Figure 6 presents evidence of the know-do gap among clinicians in Tanzania (Leonard, 2010). The horizontal axis measures competence (the things that clinicians demonstrate they know how to do) and the vertical axis measures performance (the things that clinicians do with their patients). The solid line at 45 degrees from the axes represents the points where knowledge is the same as practice. Though some points are close to the line (and four are above it), most of the points lie below the line where competence and practice are equal, meaning they do less than they know how to do. These points, therefore, represent clinicians whose capacity is greater than their performance. The vertical distance between a point and the line is the know-do gap. The dashed line is the fitted curve through the data, and can be seen as the predicted relationship between capacity and performance. It is important to note that for much of the range of competence, increases in competence does not lead to improvements in performance. For example, by following the dashed line as competence improves from 30 percent to 40 percent, we see an approximate increase in performance of 7 percentage points (23 to 30 percent). But if competence increases from 50 percent to 80 percent, we see an increase in performance in only a few percentage points. At these higher levels of competence, additional training does not increase quality. However, these papers do not examine the capacity to deliver quality health care. The elements listed in Table 6 can also be seen as improving the availability of key equipment and supplies at the facility: if chiefs of post have decisions making authority over financial decisions, they probably can insure the presence of supplies. In this view, it may be that the know-can gap is determined by decentralization and that the can-do gap is more similar across organizations. If this is true, then effort is not the limiting factor, but supplies. In the end, the policy implications are similar: decentralization improves performance whether by encouraging effort or by insuring the supply of key materials. 17 We turn now to examining the evidence for a can-do gap in the performance of health workers. Figure 6 Graphical Demonstration of the Know-Do Gap in Tanzania 70 Performance (% of required items) 10 20 30 40 50 600 0 10 20 30 40 50 60 70 80 90 Competence (% of required items) Individual clinician's competence and performance Predicted quadratic relationship of competence to performance Performance = Competence Source: Authors 6.2 Examining the Can-Do Gap Figure 7 examines the graphical relationship between capacity (as defined in section 5.2) and the performance of health workers. Each point on the graph represents the combination of performance and capacity for a health worker in the sample. Also shown is the performance barrier. Many health workers in the sample are close to the performance barrier 1 , but most are below the performance barrier. Unlike the case of the know-do gap, there is a strong positive relationship between performance and capacity. The relationship between competence and performance is shown in Table 7 for reference, and it is clear that the relationship between capacity and performance is much stronger. In fact, whereas there is little evidence that competence increases performance, it is clear that capacity does increase performance. 1 Those few who have greater performance than capacity come from the fact that performance looks at all observed cases, where capacity examines only the cases with presenting conditions of fever, cough and diarrhea. 18 Figure 7 Capacity and Performance (the Can-Do gap) 65 Performance (Protocol Adherence) 25 35 45 15 55 15 25 35 45 55 65 Capacity (Protocol Adherence) Capacity Tendency, Capacity Tendency, Competence Performance Barrier Source: Authors Table 7 examines the relationship between performance and some determinants of performance, namely competence (COMPETENCE), health facility organization (DECENTRALIZATION) infrastructure and equipment (VARIABLE INPUTS and FIXED INPUTS), and the number of patients waiting to see the health worker (DAILY CASE LOAD). We examine these variables in order to understand the link between capacity and performance where capacity is determined in part by access to key equipment and supplies (as explained in Chapter 5. The first pattern that is clear across all four regressions is that the number of patients a health worker has to see, DAILY CASE LOAD, is not a factor in performance. This is similar to the findings of Maestad, Torsvik and Aakvik (2010) who also find that caseload does not reduce quality. DECENTRALIZATION is an important determinant of quality but not once we include the two factors that explain the infrastructure and equipment in a facility. Performance is related to VARIABLE INPUTS but not to FIXED INPUTS. Thus, column one shows that decentralization is an important determinant of performance but competence is not an important determinant. If we examine the roles of competence, caseload and the factors explaining infrastructure and equipment without decentralization (column two) we see that variable inputs and competence are both important, but fixed inputs are not important. Recall that in the previous section we showed that fixed inputs and competence are frequently found together meaning that even though fixed inputs do not look important to performance, they may be important through the measure of competence: when competence increases, the levels of fixed inputs are also high, so competence contains all the useful information. Column 3 shows that, when all of the variables are included at once, the only significant determinant of performance is VARIABLE INPUTS. Column 4 examines the patterns within government facilities. All data on other types of facilities is missing from this data. We can see that the underlying relationship between performance, capacity and supplies and equipment does not hold for public facilities. The coefficient for COMPETENCE has about the same magnitude and 19 is marginally significant, but the coefficient for other supplies and equipment is changes direction and magnitude. Thus, we see that in public facilities, competence might be a limiting factor but access to supplies and equipment is not. Table 7 Relationship Between Performance and Select Determinants (1) (2) (3) (4) VARIABLES Performance (quality) COMPETENCE 0.0776 0.0988^ 0.0854 0.114 [0.255] [0.146] [0.231] [0.204] DECENTRALIZATION 2.388** 0.965 [0.0405] [0.517] DAILY CASE LOAD 0.0911 0.0529 0.0845 0.0896 [0.560] [0.723] [0.591] [0.636] Factor: VARIABLE INPUTS 5.469** 4.330^ -0.0946 [0.0126] [0.122] [0.980] Factor: FIXED INPUTS -0.886 -0.524 0.197 [0.607] [0.773] [0.924] Constant 29.63*** 26.69*** 27.85*** 23.86*** [6.91e-11] [2.65e-09] [1.17e-08] [2.86e-06] Observations 86 86 86 55 *** p<0.01, ** p<0.05, * p<0.10, ^ p<0.20 p-values in brackets Source: Authors To understand the underlying reason for this apparent contradiction we turn to the flashlight problem in Box A. Thus, the capacity to provide health care appears to be one of the significant constraints to high quality care in this setting: health workers are limited by the availability of variable inputs like drugs and equipment in the consultation room but not by the availability of fixed inputs like infrastructure and other equipment. However, this link between equipment and performance does not exist within the public sector. In other words, high performance organizations have reasonable supplies of equipment and low performance organizations do not have reasonable supplies of equipment, but within low performance organizations, better equipment does not translate into better quality. We turn now to more direct evidence of a can-do gap in this setting: the Hawthorn effect. 20 Box A: The Flashlight Problem Using the data examined in Leonard and Masatu (2005) we find the following: Six and four percent of health workers in the government sector do not have a stethoscope or thermometer respectively, compared to three and two percent in other sectors. Clearly, it would be impossible to adhere to protocol without a thermometer or stethoscope and therefore we might interpret this to mean that the public sector is slightly more capacity constrained due to a lack of supplies. On the other hand, we might legitimately wonder why a health worker does not own a thermometer or stethoscope. We can understand this problem better by examining the availability of flashlights in health facilities. One percent of government health workers have a functioning flashlight available, compared to sixty percent of health workers in other sectors. A health worker who has a functioning flashlight has an average of 7 percentage points higher adherence to protocol than a health worker without a functioning flashlight (a result that is significant at the 1 percent level, even after controlling for competence). We highlight the example of a flashlight for the following reason: every health worker in our sample owns a flashlight at home, and every facility is within a five minute walk of a store that sells a flashlight or replacement batteries. There is no reason for a health worker not to own a flashlight. Even if the supply system is completely broken, flashlights are available. The problem with the flashlight supply is that government health workers do not feel the need to bring a flashlight to work when they are not available. This is a choice of the individual health worker. Thus, the supply problem brings us back again to the effort problem; it is not that many health workers cannot do their job properly, it is that that the public sector does cannot assure an adequate supply of flashlights and health workers s do not see a benefit to bringing a flashlight to work. 6.3 The Hawthorne Effect as Direct Evidence of the Can-Do gap The Hawthorne effect is clear evidence of the can-do gap in this setting. Leonard and Masatu (2006) examine a subset of facilities in Tanzania, collecting data from patients immediately after they leave the outpatient consultation room. Data were collected on three types of patients: patients seen before the research team arrived at a facility, those seen after the team arrived whose consultations were observed by the research team, and those seen after the team arrived but whose consultations were not observed by the research team. Patients in this third group visited doctors other than the doctor observed by the team but at the same facility. For the set of patients whose consultations were also observed by the research team, the correspondence between quality as measured by patient responses and quality as measured by the observing doctor on our research team allows us to validate the accuracy of collecting data from patients. Figure 8 shows the pattern of quality as estimated from patient responses for observed doctors and unobserved doctors at facilities visited. For doctors who were observed, the figure shows a significant jump in quality when the team arrives. However, for doctors who were never observed, there is no significant change in quality as estimated from patient reports. Figure 8 also shows that the Hawthorne effect is temporary; quality rapidly returns to levels similar to those found in the absence of a research team. 21 Figure 8 The Hawthorne Effect and Adherence to Protocol in an Outpatient Setting 70 Adherence (Percentage of Items Correct) 45 50 55 40 60 65 -10 -5 0 5 10 Number of Previous Consultations Under Observation doctor observed from t = 1 doctor never observed Source: Authors This clearly demonstrates the capacity to increase performance as well as the fact that clinicians in Tanzania are motivated by more than just pecuniary incentives. The Hawthorne effect involves no payment, punishment or reward; only the presence of a peer. Thus, it is clear that health workers have the capacity to provide higher quality health care, even when their performance is low. 6.4 Determinants of Performance Taken together, these views of performance demonstrate that effort constrains performance. Importantly, effort can be seen in two ways: First, effort matters in the consultation with patients, where it translated capacity into performance. This is most clearly visible in the Hawthorne effect, where we observe that health workers are capable of significantly improving their quality without any additional training or access to equipment. Second effort is important in assuring access to the variable inputs necessary to insure capacity: health workers need to exert effort within the system to insure that they have access to the necessary materials. There is naturally some question about whether this effort is even possible within the public sector. When we focus on public sector health workers we see no evidence of a link between variable inputs and performance, which might suggest that no public sector health workers try to insure access to materials, or it might suggest that no public sector health workers can insure access to materials. Either way, in other sectors, there is a greater link: this does not suggest that in other sectors access to materials is always present, but that when health worker exert effort to insure access they also provide greater quality care. 22 7 PRODUCTIVITY AND THE DEMAND FOR HEALTH SERVICES In this section, we look at the determinants of productivity in the health sector in Tanzania. Our data on productivity is indirect but, at a basic level, informative. In particular, we know which facilities serve a greater number of patients. Given that similar sized facilities have similar levels of inputs, serving more patients is equivalent to greater productivity. What do we know then, about facilities that serve greater numbers of patients? 7.1 Facilities with more Patients are more Productive First, we know that serving more patients does not lower the quality of care. Maestad, Torsvik and Aakvik (2010) show that performance (quality) is not constrained by caseload; in fact, facilities with greater caseloads have higher quality. Figure 9 shows that case load does not help explain quality either. How can it be that busier health workers do not provide lower quality care: does this mean that high quality does not take more time? In fact, it means that low quality health care workers are not constrained by time. Leon, Mwisongo, and Mcharo (2006) looked at time use for health workers in Tanzania and found that most of their assigned hours were not being used in any productive pursuits. Second, there is evidence that facilities with higher quality care attract more patients, precisely because patients seek higher quality care. Leonard, Mliga and Haile Mariam (2002) examine data from the Iringa region of Tanzania and show that households travel further in search of quality health care for illnesses that require high quality health care. In particular, households bypass lower quality facilities to seek care at higher quality facilities. Thus, lower quality facilities will have fewer patients (all other things equal) and higher quality facilities will have more patients. Leonard (2007) examines similar data but looks at health seeking behavior when new health workers arrive at health facilities. Households change their behavior (slowly) when better health workers arrive at health facilities and are willing to travel greater distances to reach higher quality facilities and visit for more severe illnesses. Figure 9 Time use, Meatu and Mwanga Districts Tanzania Source: Leon, Mwisongo and Mcharo (2006). 23 Figure 10 Outcomes categorized by the Average Quality of Care at Location of First Visit. 70% 62% 58% 60% 50% 40% 32% 28% 30% High Quality 20% Low Quality 7% 10% 4% 1%0% 2%5% 1%1% 0% Source: 1,399 health episodes from a household survey in rural Arusha region for which the patient has completed treatment matched with the average quality provided at the first facility visited for that illness. Quality is categorized by high quality (above average) and low quality (below average). Outcomes are cured; cured enough (saw marginal improvement); not cured (no improvement); died; went elsewhere (have since sought care at a different facility); and referred (were referred to another facility when they visited the first facility). 7.2 Health Workers can Improve Productivity by Serving Patients Faster This is not the only way to increase productivity at a health facility. Although we do not have data on this for Tanzania, one way to increase the number of patients at a facility is to lower the costs these patients face, or to locate the facility in the middle of a large underserved population. In that case, there will be a large caseload even if quality is low. In fact, productivity might be very high in exactly these cases. The following story is based on Tanzania: In rural Tanzania, Ms. M brings her nine-month-old to the local health clinic, carrying the child on her back. When she enters, Dr. K (an Assistant Medical Officer with O-level education and four years of medical training) asks her what the problem is. Still standing in front of his desk, she replies that her daughter has a fever. Dr. K fills a prescription for malaria based on this statement, even though he cannot see the child, much less observe her condition. The consultation and medicine are both free and Ms. M leaves the facility with the prescribed medicine. During the exit interview, a nurse on our team notes that the child is suffering from severe pneumonia. The health facility has the medicine to treat both malaria and pneumonia. Dr. K is trained in the diagnosis and treatment for these diseases and saw only 25 patients that day. Yet, but for the intervention of the nurse on our research team, the child would have died. (Das, Hammer and Leonard, 2008) The clinician described in this example had the potential to be highly productive because he was able to see many patients in a very short period of time. However, the quality of care was very low and, even though it might have been productive, this is not a good form of productivity. In addition, K. Leonard was told the following story about a busy urban clinic in Tanzania. The chief of post expected his clinical officers to see at least 30 patients during a 4-hour shift. One officer saw 60 patients per shift; on the outset, it seemed that he was exceptionally productive! 24 When asked how this was possible, the officer explained that since many cases were similar (that is, many infants had diarrhea, most children had a cough, many adults had malaria symptoms), he would write out prescriptions the night before his shift for the kinds of cases that he anticipated seeing the following day. During his shift, the officer would simply write the patient’s name at the top of a pre-written prescription. The clinical officer was able to see twice as many patients per shift compared to the other officers at the urban clinic. But did the care that he provided to patients meet quality standards? If an adult patient had a fever, would the officer automatically hand her a pre-written prescription without assessing her symptoms any further? This case eventually reached the Ministry of Health and after much deliberation the clinical officer was ordered to stop this practice. The amount of time that the officer spent with each patient increased and waiting times also increased. What can we learn from this? Without any information about quality, this practice can only be seen as productive, even though (according to the ministry of health) it is not appropriate care. Even in the case that it had been declared appropriate quality, few measures of productivity would take into account the time the health worker spends preparing prescriptions the night before. 7.3 Productivity and Performance The biggest opportunity to improve productivity in Tanzania will come from increase the time that health workers spend doing their jobs while they are on duty. The study Leon et al (2006) demonstrates very clearly that health workers are not spending too much time with their patients or poorly managing the flow of patients in a health facility: they are idle during the day. The anecdote recorded in Das et at (2008) shows that even when health workers have extra time, they often choose to spend very little time with patients, at a significant cost of quality. Thus, the greatest opportunity for gains in productivity will come from increasing the number of patients seen at a facility, paired with a health workers willingness to do the extra work that these extra patients would require. Why do some facilities have more patients than others? There are two obvious and major factors: cost and catchment area. All other things equal, patients will prefer to visit less expensive facilities and to travel a shorter distance to get there. However, the evidence contained in many studies of demand in Tanzania, show that patients also travel in order to obtain quality care. Thus, higher quality facilities will have more patients. This result is reflected in the fact that Maestad et al (2010) found quality to be positively related to caseload, not negatively related. Thus, increased quality will lead to increased patient load and therefore greater productivity. More importantly, efforts to increase productivity without addressing quality can be counterproductive. As noted in the anecdote about the rural clinician, there are many ways to increase the number of patients seen in a day and not all of them deliver high quality medicine. Even more importantly, we have shown that inadequate access to supplies of equipment can reduce capacity: increasing the number of patients at a facility can easily put strain on the available supplies of equipment, reducing capacity and therefore performance. 25 8 HEALTH WORKER MOTIVATION TO PROVIDE EFFORT Effort is clearly important in health care and we have seen evidence that difference in the levels of effort provided by health workers represent a large proportion of the differences in performance in this sector. How can we understand the determinants of effort level, or the motivations of different health workers? In this section, we examine the evidence for multiple sources of motivation. In particular we look at two types of health workers: altruistic and professional and two environments for motivation: organizational incentives and peer-based intrinsic incentives. Health workers can be motivated by intrinsic motivation, which refers to the possibility that people might gain satisfaction from acts of kindness or from the act of doing their job. This is in contrast to extrinsic motivation where people gain satisfaction from the material benefits they earn from doing their job. In the case of health workers, doing their job properly is an act of kindness, particularly in the public and charitable sector, where health workers often see patients who would not otherwise be able to afford high quality care. Thus, it is natural to ask “to what degree are health workers intrinsically motivated to their jobs?” or “are some health workers intrinsically motivated to do their jobs?” The question assumes that intrinsic motivation means that a person will always be more satisfied if they are helping the poor or doing their job. In addition to begin motivated by kindness, many health workers are motivated by a sense of professional accomplishment. This means that when the conform to professional norms, they receive immediate satisfaction, which is different than believing that their behavior will have long run material benefit (through promotion or reputation effects). In reality, it is also possible that professional and altruistic behavior is less a function of the characteristics of the individual and more a function of the characteristics of the environment in which the individual works. Even kind people need to be in an environment in which they feel that their efforts are appreciated or meaningful; very few people can give selflessly of themselves for long periods without any positive reinforcement. In addition, an individual with professional motivation may need to be in an environment where his behavior is supported by other professionals, or by the feeling of shared norms with these other professionals. Thus, the question of intrinsic motivation becomes not “are health workers are intrinsically motivated to do their job?”, but “under what circumstances do health workers earn intrinsic rewards from doing their job?” Perhaps more importantly “are there health workers who are potentially intrinsically motivated but who are not working in an environment where they can earn the intrinsic rewards from their effort?” The quality of health care in the public sector in Tanzania suggests that normal levels of intrinsic motivation are not sufficient to ensure quality. The average clinician does not provide adequate effort for his or her patients. On the other hand, some clinicians are doing an excellent job. Are these clinicians different from other clinicians; specifically are they intrinsically motivated? If we want to improve the quality of care, is it possible that the clinicians who provide low quality care are intrinsically motivated, but do not work in an environment that intrinsically rewards them for 26 their effort? This second possibility is one that should be particularly interesting to policy makers because it suggests an avenue for improving quality. 8.1 A Field Experiment on the Determinants of Motivation In this paper, we address these questions by looking at a sample of 102 clinicians who work in urban and peri-urban Arusha region in Northern Tanzania. With each of these clinicians we did four things: 1) We observed the quality of care that they normally provide to their patients measuring the degree to which they adhere to national protocol, 2) we observed the quality of care that they provide to their patients when there is a peer in the consultation room, 3) we observed the quality of care that they provide to their patients approximately 4 weeks after an encouragement visit from a doctor at a local research/education institute and finally 4) we invited all clinicians to attend a dissemination conference in which they were given 15,000 TSH and given the opportunity to share some of this money with an unidentified stranger outside of the room. From these four settings and activities we obtain four measures for each clinician: 1) baseline quality of care, 2) quality of care under peer scrutiny, 3) quality of care due to encouragement and study and 4) whether they are “generous,” in that they gave at least 7,500 TSH (half) to the unidentified stranger in the experiment at the dissemination conference. The Laboratory Experiment The laboratory experiment took place in Arusha, Tanzania in July 2010. The subject pool consisted of 71 clinicians and 78 non-clinician subjects. We ran one session each day for two days. We recruited non- clinicians subjects with printed advertisements distributed in major market areas in Arusha. While flyers were distributed to a variety of people, the group of non- clinician subjects was ultimately a convenience sample. All of the non-clinician subjects that arrived to participate each day were allowed into the experiment. Clinician subjects were given a per-diem of 35,000 Tsh in addition to what they earned in the experiment for participating. Non- clinician subjects received a show-up fee of 5,000 Tsh. One US dollar is approximately equal to 1,300 Tanzanian shillings. Clinician subjects gathered in a classroom and non-clinician subjects gathered on a lawn outside of the classroom, near enough that both groups could see each other but far enough that there was no communication or individual identification. This was done to preserve anonymity while ensuring that subjects understood the concept of being paired with another player. Subjects recorded decisions using paper and pen. Before each session, subjects read and signed the consent form. We provided a hard copy of the experimental instructions to each participant and read them aloud before the experiment began. The instructions explained the basic guidelines of the experiment and how earnings were determined. Subjects were given the chance to ask clarifying questions after the instructions were read. The experiment is a standard dictator game in which the dictator decides how to allocate a pool of money between him or herself and an anonymous partner. The dictator in each pair was always a clinician and he receiver was always someone drawn from the non-clinician pool. (We never used the terms doctor or patient in the experiment, but the clinicians knew they were in a group of clinicians). The only task for the clinician--dictator was to allocate 100 tokens between him- or 27 herself and the receiver, who had no choice to make but did understand how the game worked. To preserve anonymity, clinicians and their non-clinician partners never came into contact and were only able to see each other from a distance. Each token was worth 150 TSH, so that P1 was allocating 15,000 TSH (approximately 12 USD) between him or herself and the receiver. The Encouragement Experiment We collected data on clinician performance for 103 clinicians and 4,512 patients in the semi-urban area of Arusha, Northeast Tanzania. The field data collection ran from November 2008 until August 2010. 2 The sample includes public, private, and non-profit/charitable facilities. Clinicians were randomly sampled within each facility. We restrict our attention to clinicians because they are the primary health workers who provide the outpatient care in the area. They fill the role of “doctor”, though the majority of them do not have full medical degrees. The four cadres of clinicians include assistant clinical officer (ACO), clinical officer (CO), assistant medical officer (AMO), and medical officer (MO). We collected data on the quality of care on at least 7 (and not exceeding 9) separate occasions for each clinician, occurring in a time span of approximately two months per clinician. Start dates were staggered and the days on which we collected data for any given clinician were not announced in advance. On each day of data collection we interviewed all the patients the clinician saw in the 4-hour window that we visited the facility. Data were collected for each clinician at the patient level and consists of observations of the clinician with his or her patients as well as patient exit interviews using the Retrospective Consultation Review (RCR) instrument. The RCR is an exit interview intended for patients after their visit to the doctor has ended. It is a slightly modified version of the instrument used by Leonard and Masatu (2006) and it measures clinician effort. In this paper, we focus on four of the data collection visits and three additional visits by the research team. The first visit occurred before any data collected and was necessary to enroll clinicians in the study. The second (unannounced) visit was the first data collection visit and we use this visit to measure baseline quality because even though clinicians knew they were enrolled in a study, they did not know we were present and interviewing patients. The third visit usually followed the second visit immediately and is what we call the peer scrutiny visit. For this visit, a clinician on the research team entered the consultation room and observed the clinician working. Following this second data collection visit, the clinician was visited by a member of the research team in what we call the encouragement visit, outlined below. There were three more visit after the encouragement visit (which we do not examine here), and, at the end, two data collection visits after the research had effectively ended. It is these last two visits (approximately 4 weeks after the encouragement 2 We sampled 100 percent of the health care facilities in the area with outpatient departments, though some facilities were excluded based on convenience; they were either too difficult to reach for obtaining consent or had too small of a patient volume. 28 visit) that we use to see the medium term changes in quality due to the encouragement visit. The length of time between the different visits varied according to clinician schedule. Dr. Beatus Leon, a Tanzanian M.D. and lecturer at a health research institution, visited each clinician and read the following script. We appreciate your participation on this research study. The work that you do as a doctor is important. Quality health care makes a difference in the lives of many people. Dedicated, hardworking doctors can help us all achieve a better life for ourselves and our families. One important guideline for providing quality care is the national protocol for specific presenting symptoms. While following this guideline is not the only way to provide quality, we have observed that better doctors follow these guidelines more carefully. Some of the protocol items that we have noticed to be particularly important are telling the patient their diagnosis, explaining the diagnosis in plain language, and explaining whether or not the patient needs to return for further treatment. In addition it is important to determine if the patient has received treatment elsewhere or taken any medication before seeing you, and to check the patient’s temperature, and check their ears and/or throat when indicated by the symptom. For this research, we will look at clinician adherence to these specific protocol items. The goal of this visit was to test if we could duplicate the Hawthorne effect, but over a longer period. We chose specific items so that we could compare the performance on these items to performance on items not mentioned. The average clinician in the study increased the quality of care provided over the baseline and we were able to extend the Hawthorne effect to improve the quality of care for almost two months. 8.2 Results Here we outline the results of the research on motivation. Generosity (altruism) is high and is not a feature of the sector of employment Table 8 present how much clinicians gave to the stranger overall and according to the sector that the clinician works in. Most of the clinicians gave something to the stranger, only about 3 to 4% of the clinicians kept all of the money to themselves. The most common amount given was half, splitting the money between the stranger and keeping the money. This is a common amount in all settings. The money is, in effect, an unexpected gain and people often feel it only fair to split this with someone who otherwise would not receive any extra money. However, in all sectors, most of the clinicians gave some amount less than half. 29 What interests us is the clinicians who follow the norm of splitting the money between themselves and the stranger, those who gave at least 50 tokens to the stranger. A few gave more than 50 tokens, but we consider these generous as well. Thus, overall, 38 percent of clinicians are generous: 32 percent in the public sector, 47 percent in the private sector are 36 percent in the NGO sector. The differences between the sectors are not significant. Table 8 Laboratory Experiment Results, Summary of Giving Proportion N mean sd median mode Giving zero Giving Half all 63 35.03 19.77 40 50 3.17% 30.16% public 28 32.39 16.3 37.5 50 3.57% 32.14% private 21 38.9 22.63 49 50 4.76% 33.33% NGO 14 34.5 22.01 40 50 0.00% 21.43% Source: Authors Generosity (altruism) in health workers leads to improved quality of care for their patients One of the important design features of the experiment on motivation is that we can see the link between how people behave in the experiment and how they behave with their normal patients in the real world. Figure 11 shows the difference in the quality of care provided according to whether or not a clinician was coded as being generous. Clinicians who shared the money they were given provide higher quality care than clinicians who did not share the money they were given. This result partially validates the laboratory experiment because it shows that our measure derived from an arbitrary situation does indeed measure something real happening in the world. It matters that health workers are kind to others. Figure 11 Protocol Adherence Overall and for Generous Clinicians Source: Authors 30 The average health worker is capable of improving the quality of care and sustaining this increase for at least four weeks without any changes in training or access to infrastructure and supplies The original idea of the encouragement was to see if we could replicate the Hawthorne effect. It turns out that the encouragement effect is much larger than the Hawthorne effect seen in the scrutiny intervention. This is probably because the encouragement combined the impact of being studied with the impact of being observed. Figure 12 shows the changes in the quality of care due to the two interventions in our study, peer scrutiny and encouragement. On the horizontal x-axis we record the quality of care provided by each doctor in the baseline and on the vertical axis is the change in quality. Note that each doctor is represented by two points: the small triangle shows how much they increased their care when encouraged and the small circle shows how much they increased their care when they were under peer scrutiny. Figure 12 Changes in Quality from Peer Scrutiny and Encouragement Change in Percentage of Protocol Items Correct -20 -10 0 10 20 30 40 50 60 30 40 50 60 70 80 90 100 Percentage of Protocol Items Correct Peer Scrutiny Encouragement Peer Scrutiny Encouagement Source: Authors There are two important patterns to notice. First, most of the impact of peer scrutiny and encouragement is for clinicians who are at the lower end of the distribution of quality. For clinicians who are already doing very well, there is little change in quality. Second, the impact for encouragement is bigger than the impact for peer scrutiny. This can be seen in Figure 13 where we graph the average response to scrutiny, and encouragement for the items we specifically mentioned in the encouragement script. This was surprising to us; we expected clinicians to be more concerned about having a peer in the room than about being asked to work harder by someone who did not sit in the room and observe. 31 Figure 13 Changes in Protocol Adherence with Scrutiny and Encouragement Source: Authors Quality of care is higher in the NGO and Private sector and Generous clinicians are not better than the average clinician in these sectors Figure 14 shows the quality of care provided by clinicians in the different sectors as well as by type within the sector. Clinicians in the public sector provide much lower quality care than clinicians in either the private or NGO sector, reflecting the results described earlier (Mliga 2000, Leonard et al 2007). This result is not particularly surprising and it might be due to greater effort to greater capacity to provide effort in those sectors. However, we can test the role of generosity across these sectors. Generosity is most important in the public sector, where we see quality is much higher. Generous clinicians in the private sector provide slightly lower quality care than the average, and generous clinicians in the NGO sector provide slightly better care overall, however neither of these differences is statistically significant. Thus, in the public sector, generosity increases the performance of health workers, most likely due to increases in the effort these health workers are willing to exert on the behalf of their patients. However, even though there are generous clinicians in the other two sectors, high quality is not dependent on whether clinicians are generous. It would appear that health workers in the NGO and private sectors have good reason to provide high quality care, even if they are not generous. 32 Figure 14 Generosity and Protocol Adherence Across Sectors Source: Authors Health workers who are generous (altruistic) provide higher quality care, but are still capable of providing even higher quality care when encouraged Figure 15 shows the changes in the adherence to protocol as a result of the encouragement. As with the scrutiny intervention, these changes are all for the individual clinician and therefore show what is possible without any more medicines, training or equipment. The figure also shows the changes in each sector by whether or not a health worker is generous. Thus we can see three patterns: 1) the different responses to encouragement across the three sectors, 2) the different responses to encouragement of generous health workers and the average health worker and 3) the different ways that generous health workers respond to encouragement across the three sectors. First, we can see that the response to encouragement is essentially coming from the public sector health workers. There are responses in the other two sectors, but they are smaller and are not statistically significant. Second, we can see that there are no consistent differences to encouragement between the responses of generous health workers and the responses of the average health worker. Recall that generous health workers provide higher quality care in the baseline, so this graph suggests that after encouragement, generous health workers provide even more quality. This is somewhat surprising, because we had hypothesized that generous health workers would already be providing such high quality care, that it would be harder for them to improve. Third, we can see evidence that generosity and encouragement combine in different ways across the three sectors. Overall, generous health workers do not respond differently than average health workers in the public sector. However, in the private sector there is some evidence of a larger 33 response for generous health workers, and in the NGO sector there is some evidence of a smaller response. The difference between the private and the NGO sectors suggests that the core motivation of health workers in these sectors may be different. Generous health workers respond to encouragement in the private sector suggesting they are not receiving the right type of encouragement on a regular basis. However in the NGO sector, it is the average clinician who responds more to encouragement (all health workers have a positive response, overall). Figure 15 Changes in Protocol Adherence with Encouragement by Sector Source: Authors 8.3 Sources of Motivation: Patients, Peers, and Organizations There is a significant know-do gap in the quality of care provided in outpatient clinics in Tanzania, as seen by evidence from Arusha Region. Clinicians in this setting provide lower quality care than they are capable of providing. We show that this gap can be closed by peer encouragement. We interpret our results as being driven by two different types of intrinsic motivation. In one case, clinicians (like most people) have a sense of fairness towards others and are willing to share with or care for those who are less fortunate than themselves or in need. About a third of the doctors in our sample feel that they should share when they are given something that they have not earned and when someone who is paired with them will not earn anything if they do not give. Note that the anonymous stranger’s only chance to earn extra money was if the clinician gave them something. So these clinicians shared their gains with their partner even though it was costly to them and they knew they would not be thanked or compensated in the future. This type of behavior is similar to a sense that one should help poor patients by working in the public or voluntary sector. We call this generosity. A second type of motivation comes from a concern for the way one is perceived by peers. This can be thought of as esteem seeking. Clinicians care what other clinicians think of them and they know what is expected of clinicians because they have been trained and share norms with each 34 other. Thus, when they are engaged with a peer (either because the peer is in the room or because they know they are being studied by a peer) they will be more likely to follow norms, which in this case mostly means increased protocol adherence. Generosity towards patients What is the important difference between generous and non-generous clinicians? In the public sector, clinicians are not forced to do everything they can for their patients. They can choose to do much less and they know there will not be any negative consequences for them. Yet despite any clear extrinsic motivation, most clinicians do their job, seeing patients and providing them with some level of care. In addition, we see that clinicians who were generous in the laboratory experiment feel a need to work harder with their patients. Those who were not generous in the laboratory experiment did not feel the need to work harder and they did not have another strong reason to do so. Thus generosity improves the quality of care in the public sector. However, in the NGO sector and the private sector, even if the clinician is not generous he will provide higher quality care to his patients. The reason for this has to do with the incentives that come from his business (in the private sector) and his employer (in the NGO sector). Either his employer will encourage or force him to work harder, or he knows that working harder will earn him more money. Thus, generosity is important in the health sector only when there are no other good reasons to provide high quality care. It is important in the public sector and it therefore reveals a failure of the public system to incentivize necessary behavior. It does not mean that the public sector should hire more generous clinicians; it means it must find ways to encourage non-generous clinicians to provide high quality care. Peer Esteem Caring about what other clinicians think is a different thing than feeling that one should share with others who have been less fortunate. A clinician could be motivated by either of these concerns or both at the same time. Indeed, our evidence suggests that most clinicians care about what a peer thinks of them, and that generous clinicians also care about what their peers think. When a clinician increases his effort (or changes his behavior) because there is a peer in the room observing his behavior, it indicates that what he feels is expected of him is more (or different) than what he normally does. However, at the same time a clinician can care about what other clinicians think of him without changing his effort when observed. This simply indicates that what he believes is expected of him is the same as what he normally does. Thus, if a clinician is properly motivated to do what is expected of him, the presence of a peer (or the encouragement of a peer) will not change their behavior. It is possible that it changes the utility a doctor earns from his or her effort, but it does not change the level of effort. The clinicians in our sample who do not change their behavior when a peer enters the room are not the clinicians who provide low levels of effort, but the clinicians who provide high levels of effort. 35 When faced with peer scrutiny, the average doctor increases the quality of care. He has not been instructed to do anything, but having someone in the room makes him feel that he should be doing more. We have seen that the average clinician can do more, and therefore he does. The encouragement effect is more complicated. Three things are happening at the same time. First, they have been encouraged to do more by someone they respect. They would like to please him, but they are not sure that if they work harder they will in fact earn his respect. Many people come and ask them to work harder but never return and even if they return, would they know that the clinician has in fact worked harder? So encouragement is part of the story, but might not be all of it. Second, they have been given information about what is important. Maybe they didn’t know that it was important and therefore they are more knowledgeable after the visit than they were before. Third, they are being visited repeatedly by the research team. These are not visits from the peer, but they are visits from a team connected to the peer. The clinicians know the peer is involved and they are discovering that the research team is constantly engaged with them. This is very different from just one visit. In a sense a relationship is forming between the team and the clinician. We do not think the information is driving the behavior. We have tested to see if clinicians do more of the specific things we have asked them to do and found no difference. They do more of the things specifically mentioned, but no more than what they do of the things we never mentioned. It appears they are doing more of what they already know they should do. We think their reaction is a combination of the encouragement and the fact that they are being continuously studied. If we examine the impact of the encouragement we see that it gets larger each time we visit the clinician. This means that as clinicians discover that the team is serious and keeps coming back, they decide they care more and more about what the original visitor thinks of them, and they work harder. Thus, caring about what peers think of you is very important for clinicians. We can imagine that they care about other clinicians and they enjoy the process of interacting with them. Organizations Interestingly, in the NGO sector and in the private sector, clinicians do not do much more after being encouraged. This is not because they do not care. In the NGO sector, we think that clinicians are already exposed to their peers and they already have the opportunity to impress them. So when they find a new opportunity to do so, they may be happy, but they do not do more than they did before. In the public sector however, clinicians do not regularly have this opportunity, so they do more. In the private sector, the same thing may be happening, or it may be the case that private sector clinicians are more driven by extrinsic incentives. Adding more intrinsic incentives on top of strong extrinsic incentives is unlikely to change behavior because clinicians are already doing as much as they can. It could be that they do not care that they have been encouraged, but it is more likely that extrinsic incentives (money) are already sufficiently encouraging. 36 9 POLICY IMPLICATIONS This research was undertaken to understand the role of intrinsic motivation in the quality of care provided in a typical setting in Tanzania. Motivation is one of the important inputs into the performance of health care workers. Of course health care workers need to exist, they need to be trained and they need proper equipment to do their jobs, and in each of these aspects, there are important deficiencies. Yet, as the Hawthorne effect shows, health workers who are present, trained and in possession of the proper equipment often still do not do their job properly. Seventy nine percent of the children under the age of 5 who died from malaria in Tanzania sought care from modern medical service providers before they died (de Savigny et al, 2004). This fact, combined with the significant evidence for low performance suggests that at least part of the responsibility for these unnecessary deaths resides with low quality of care. Thus, we seek to understand the role of motivation in the low quality of care provided. It is important to understand the definition of intrinsic motivation advanced in this report. Just as an extrinsically motivated individual is one who will respond to proper extrinsic incentives, we see an intrinsically motivated individual as one who responds to the proper intrinsic incentives, not as someone who does the “right” thing no matter what incentives are provided. In the management literature (outside of health) the intrinsic rewards of the workplace are not from service to others (though this can be part of it) it is the utility that comes from the act of doing the job itself. Intrinsic motivation in this literature comes from the opportunity to do something of value in the workplace enhanced by feelings of autonomy, responsibility and esteem. People are intrinsically motivated when they have the power to do something, feel that it matters that it is done and believe that others care if they do it. However, these rewards only mean something to an individual who desires them and is willing to make sacrifices to earn them. Thus, it is also common to refer to intrinsically motivated individuals who care about the well being of others or who care about how they are perceived by others. These are the features of individuals which cause them to gain utility from intrinsic rewards. The key point is that neither characteristics of the job or features of the individual mean anything alone. Two individuals may have the same features, but work in jobs with different characteristics; they will therefore perform differently. Similarly, two individuals may work in similar jobs with similar intrinsic incentives but have different features and therefore perform differently. This distinction is important. It is often stated that because health care is an occupation that should have many intrinsic rewards, it is necessary to deliberately seek out intrinsically motivated people to do the job. This statement is true only if the health work place truly does provide intrinsic rewards; if we observe individuals who do not provide adequate care for their patients, it may be that there are insufficient rewards or it may be that the individuals are insufficiently motivated. To test this understanding of intrinsic motivation we examined two sources of intrinsic motivation: patient-based and peer-based motivation. Traditionally, when we talk about the special nature of health care delivery, we mean the responsibility that health workers have for the care of other individuals. However, equally important for health workers is their membership in a profession 37 that is held in high esteem. The sense of pride in membership comes both from the corporate responsibility for the health of others and the specialized training and knowledge that are part of the profession. Thus individuals may be motivated both by concerns for the well being of their patients and for the esteem and respect of their peers. Fortunately, in the end, both sources of motivation lead to the same end because the profession of health care is, itself, motivated by a concern for the well-being of patients. Either source of motivation, if offered proper rewards, will lead to higher quality care. We find that health workers who care more about the well-being of others provide higher quality care, and that most health workers, when put in a situation where scrutiny and encouragement increase intrinsic rewards to effort, increase their effort. Thus, we have examined the problem of intrinsic motivation from both perspectives. Generosity measures a characteristic of individual health workers and we find that a significant proportion of health workers are indeed generous. Scrutiny and encouragement, on the other hand, change the level of intrinsic rewards and we find that most health workers indeed respond to these rewards. It is useful to know that generous clinicians exist in all three sectors (public, NGO and private). These kinds of clinicians will almost always provide patients with valuable services. It is not clear if that the proportion of health workers who are generous is low. Perhaps all health workers should be generous. Health workers were told they were paired with another who would earn no money if they did not share some of the money that was given to them, unearned. Yet most health workers chose not to fairly share that money. Some of them gave a small amount to the person who was paired to them, but some even gave zero. On the other hand, the experiment was not conducted in a medical setting so it is not clear that it captures all elements of caring about the well being of strangers. In addition, generosity plays a limited role in the quality of care provided in the NGO sector. This is not because clinicians in the NGO sector are not generous; the rate of generosity is similar across the three sectors. The reason that generosity plays a smaller role in the NGO sector is that other forms of motivation are already encouraging doctors to provide high quality care. Despite the overall image of the NGO sector as one that attract health workers who care more about their patients, we see no evidence that health workers in this sector are different from health workers in other sectors. In our data, the sector with the greatest proportion of generous doctors is the private sector, not the NGO sector. What we see however is that health workers in this sector (both generous and non-generous) are given proper incentives (either extrinsic or intrinsic) such that they do the right things for their patients. We sought to understand the role of intrinsic motivation in the health care sector in Tanzania, using carefully designed interventions and instruments to see how clinicians change their behavior as their environment changes, and the role of generosity in these changes. The study also allows to make some preliminary policy recommendations: 38 Policy Implications 1: There is no excuse for poor performance in the public sector—quality can be improved immediately. Clearly the public sector is failing to provide sufficient motivation for health workers to use all of their skills for their patients. The design of the research, using the Hawthorne and encouragement effects clearly demonstrates a capacity to improve care without any change in patient mix, training, medicine availability or infrastructure. Public sector health workers in our study were able to improve the quality of care immediately and for long periods of time. Thus, our first policy relevant finding is that there is no valid structural excuse for the poor performance of clinicians in the public sector; quality can be improved without any additional investment in training or infrastructure. Low performance is often explained and excused by reference to training and infrastructure; this is no longer an acceptable answer because it is incomplete. This policy implication does not blame public sector health workers for this failing. In fact, if anything the Hawthorne effects and encouragement effects demonstrate that the public sector health workers want to do more and want to serve their clients better. Policy Implications 2: There is a lack of sufficient intrinsic motivation in the public sector Since exposure to peers increases the quality of the care in the public sector but does not increase the quality of care in the NGO sector, it is clear that health workers in the public sector do not have sufficient exposure to peer support or encouragement. This is despite the fact that most of the clinicians we studied said they received supportive supervision visits from their superiors. However, these visits were generally seen in a negative light, an opportunity for the supervisor to lecture the health worker rather than an opportunity for the health worker to demonstrate his ability or effort. It does not appear that supervisors watch the health workers during their work or converse with the health workers about their experience. A recent review of the role of supervision in the Cochrane review (Bosch-Capblanch, 2011) found no strong evidence that managerial supervision programs were effective at increasing the quality of care. Thus, the supervision visit serves a particular purpose but does not provide an opportunity for health workers to gain utility from peer-based intrinsic motivation. Thus, rather than fixing the supervision system, or asking supervisors to do multiple jobs, it is clear that an additional mechanism is necessary to give health workers the support they need and want to do their jobs better. Policy Implications 3: The NGO sector, not the private sector, is the best example for the public sector. The success of the NGO sector is important to understand. Health workers in the NGO sector in Tanzania are not recruited from different schools and they are not paid more than public sector health workers. Furthermore, they do not appear to be qualitatively different than public sector workers in their motivation. On the other hand, the management of NGO facilities is significantly 39 less centralized than that of public facilities (Leonard et al 2002, Mliga 2000). Thus, even though health workers are not paid more in the NGO sector they undoubtedly face greater extrinsic motivation to do their job than do health workers in the public sector. However, it is also possible that they face greater intrinsic motivation as well. Both the public and NGO sector provide supervision, but the nature of supervision appears to be qualitatively different. In a conference with doctors who participated in the study, health workers in the public and NGO sectors described the typical supervision visit. Both types of health workers were shocked to hear the descriptions of health workers in the other sector. Health workers in the NGO sector who were comfortable talking about it, described a positive and supportive relationship to their supervisors, something that surprised public sector doctors. It is not easy for the public sector to imitate the NGO sector. Health workers in the public sector are civil servants and it is therefore difficult to provide direct extrinsic incentives. In addition, fees collected at NGO facilities are generally large, larger proportions of these fees are available to facility managers and more discretion is awarded in their use. Changes such as these will not occur in the public sector. However, it is possible that much of the improved performance in the NGO sector comes from closing the know-do gap using intrinsic motivation. Health workers in the NGO sector may work harder because they perceive that their peers care that they work harder and they feel that their peers will notice their efforts; they have been scrutinized and encouraged. An Operational Research Agenda The research summarized in this report raises as many questions as it answers. In some cases, outlined below, answering questions can be an important element in moving towards more rational policy. We outline a few of the more important questions here. First, we have shown that health workers respond to scrutiny and expectations set by their peers. What are the important elements of peers that lead to this response? How do health workers in public facilities respond to the presence of health workers from other similar public facilities? It would seem unlikely that we can find a strong positive response in this context since almost all health workers are already working with other health workers and significant performance gaps remain. Would having peers visiting from other organizations change performance, or is it possible to induce performance improvements simply by setting up a peer visitation system within the public sector? Second, we have shown that it can be important to collect information from health workers as part of establishing expectations for quality. How do measurements and expectations interact to motivate health workers? Imagine for example, comparing a program in which peers visit a health facility and provide their own evaluation. This could be compared to a program in which health workers are provided with information about exit interviews with users of the facility and finally with a program in which peers visit a health facility and review information collected from exit interviews with users (similar to the experiment discussed above). Third, if measuring the quality of care is important, what is the most effective way to measure quality so as to induce performance improvements? Note that we normally evaluate measurement 40 tools by their accuracy, but this would be a different agenda. For example, do exit interviews with users generate a bigger response than household surveys of would be users? Fourth, what happens to patients if they are provided with information about the quality of care provided in the facilities that they might choose to visit? If measurement were simplified in uni- dimensional measure of quality (a one to five star system, for example) would this alter patients’ behavior in seeking facilities? Would it be important to compare facilities across the country by one metric, or to compare facilities differently according to local conditions (rural versus urban, for example), or to compare facilities to near-by NGO facilities? 10 CONCLUSION The motivation to provide high quality care to patients comes at least partially from social preferences. Health workers care about the well-being of their patients and how they are perceived by their peers. The study that forms the basis of this project sought to measure both the underlying sense of generosity towards patients and the way that health workers respond to the expectations of their peers. We found that health workers who demonstrate a strong fairness norm and are generous to strangers provide higher quality care than clinicians who are not generous to strangers. This suggests that many health workers are at least partially motivated to provide high quality care because they understand that their efforts will help patients. In addition, we find that health workers respond to the presence of their peers by increasing the quality of care that they provide. There is no evidence that these sources of motivation are substitutes. Health workers can be motivated by both of these concerns at the same time. Health workers who are generous increase their effort when they are faced with peer scrutiny or are encouraged by their peers. Similarly, health workers who are not generous increase their effort when encouraged by their peers. Thus the quality of care is at least partially dependent on the motivation of health workers to provide effort. Our study was able to augument the effort of health workers using simple interventions. These interventions—peer scrutiny and peer encouragement—do not represent policy relevant actions, but rather demonstrate the ease with which quality can be changed when health workers are not sufficiently motivated. This study has shown that health workers in the public sector are insufficiently motivated to do their jobs and suggests a series of policy recommendations to eliminate this shortfall. In particular, the current system of supervision in the public sector does not motivate health workers. Although clearly, more research is necessary, there is sufficient evidence of potential gains that the public sector should engage with the NGO sector to study the system by which health workers are motivated to provide higher quality care. This can take at least three forms. 41 First, qualitative studies of the supervision system in various NGO facilities should be undertaken to see if motivation within the NGO sector can be explained by different forms of supervision or peer support. Second, NGO supervisors should be contracted to visit public sector facilities with the sole objective of providing peer support. These additional supervision visits would serve no managerial purpose. This is similar to the encouragement visit performed by the member of our team, except it will be more intensive and would involve no data collection (except as part of the evaluation). This would be a direct test of the hypothesis that different forms of supervision can induce higher levels of effort. Third, increased communication between health workers within and across sectors needs to be encouraged. This communication would take place within the health facility and ideally while health workers are doing their jobs, providing opportunities for health workers to encourage each other to do better. Finally, any attempt to encourage or incentivize health workers in the public sector, such as pay for performance, needs to recognize the role of intrinsic motivation in the choices that public sector health workers can play. Successful pay for performance schemes not only transfer resources to health workers (which they do like), they increase the responsibility, autonomy and esteem that health workers feel in their posts. Talking with outside teams, searching for solutions to difficult problems and being reminded that you have not been forgotten are all important motivators, independent of the extra money. 42 11 REFERENCES Abel-Smith, B. and P. Rawal. 1992. ‘‘Can the Poor Afford ‘Free’ Health Services? A Case Study of Tanzania.’’ Health Policy and Planning, 7 (4), 329-41. Black, R. E., S. S. Morris, and J. Bryce. 2003. ‘‘Where and Why are 10 Million Children Dying Every Year?’’ Lancet, JUN 28, 361, 2226-2234. Bosch-Capblanch X., S. Liaqat, P. Garner P. 2011. “Managerial Supervision to Improve Primary Health Care in Low and Middle income Countries.” Cochrane Database Syst Rev. Sept 7; 9:CD006413. Brock, J. M., A. Lange and K. L. Leonard. 2011. “Activating Social Preferences: Evidence from a Field Experiment with Doctors in Tanzania.” Mimeo. Campbell, J. P, V. A. Maxey, and W. A. Watson. 1995. ‘‘Hawthorne Effect: Implications for Prehospital Research.’’ Annals of Emergency Medicine, 26 (5), 590-594. Das, Jishnu and Jeffrey Hammer. 2005. ‘‘Which Doctor?: Combining Vignettes and Item- Response to Measure Doctor Quality.’’ Journal of Development Economics, 78, 348-383. Das, J., J. Hammer and K. L. Leonard. 2008. “The Quality of Medical Advice in Low Income Countries,” Journal of Economic Perspectives, 22 (2). Foy, R., M. P. Eccles, G. Jamtvedt, J. Young, J. M. Grimshaw, and R. Baker. 2005. ‘‘What do We Know About How to do Audit and Feedback? Pitfalls in Applying Evidence from a Systematic Review.’’ BMC Health Services Research, 5 (50), 1-7. Gelijns, A., J. Graff Zivin, and R. Nelson. 2001. “Uncertainty and Technological Change in Medicine,” Journal of Health Politics, Policy, and Law, 26, 913-924. Gimotty, Phyllis A. 2002. ‘‘Delivery of Preventive Health Services for Breast Cancer Control: A Longitudinal View of Randomized Controlled Trial.’’ Health Services Res., 37 (1), 65-85. Grimshaw J. M and I. T. Russell. 1993. ‘‘Effect of Clininical Guidelines on Medical-Practice-A Systematic Review of Rigorous Evaluations.’’ Lancet, NOV 27, 342, 1317-1322. Grimshaw, J. M., L. Shirran, R. Thomas, G. Mowatt, C. Fraser, L. Bero, R. Grilli, E. Harvey, A. Oxman, and M. A O’Brien. 2001. ‘‘Changing Provider Behavior - An Overview of Systematic Reviews of Interventions.’’ Medical Care, 39 (8), II-2-II-45. Suppl. LC2. Jamtvedt, G., J . M Young, D. T. Kristoffersen, M. A. Thomson O’Brien, and A. D. Oxman. ‘2003. ‘Audit and feedback: Effects on Professional Practice and Health Care Outcomes (Review).’’ The Cochrane Database of Systematic Reviews (3). Jones, G., R. W. Steketee, R. E. Black, Z. A. Bhutta, and S. S. Morris. 2003. ‘‘How Many Child Deaths Can we Prevent this Year?’’ Lancet, JUL 5, 362, 65-71. 43 Klemick, Heather, K. L. Leonard, and C. Melkiory Masatu. 2008 ‘‘Present Doctors, Vanishing Care: Access to Quality Health Care in Rural Tanzania.’’ Department of Ag and Res Economics Working Paper 08-03, University of Maryland. Klemick, Heather., K. L. Leonard, and C. Melkiory Masatu. 2009. ‘‘Defining Access to Health Care: Evidence on the Importance of Quality and Distance in Rural Tanzania.’’ American Journal of Agricultural Economics. 21(2). Leon, B., A. Mwisongo and J. Mcharo. 2006. Health Worker Productivity in Public Health Facilities in a Poorly Staffed as Compared to a Better Staffed Rural District in Tanzania Mainland: A Time and Motion Study. Leonard, K. L., G. Mliga and D. Haile Mariam. 2002. “Bypassing Health Centers in Tanzania: Revealed Preferences for Observable and Unobservable Quality.” Journal of African Economies, 11 (4). Leonard, K. L. and M.C. Masatu. 2005 “Comparing vignettes and direct clinician observations in outpatient services: Tanzania.” Social Science & Medicine 61 (9). _____. 2006. “Outpatient Process Quality Evaluation and the Hawthorne Effect.” Social Science & Medicine 63 (9) 2006. _____. 2007. “Variations in the Quality of Care Accessible to Rural Communities in Tanzania.” Health Affairs, 26 (2). _____. 2010. “Using the Hawthorne Effect to Examine the Gap Between a Doctors Best Possible Practice and Actual Practice.” Journal of Development Economics. 93 (2): 226-243. _____. 2010. “Professionalism and the Know-Do Gap: Exploring Intrinsic Motivation Among Health Workers in Tanzania.” Health Economics 19 (12): 1461-1477. Leonard, K. L., M.C. Masatu and A. Vialou.2007. “Getting Doctors to do Their Best: The Roles of Ability and Motivation in Health Care Quality.” Journal of Human Resources, 43 (2). Leonard, K. L. 2007. “Learning in Health Care: Evidence of Learning about Clinician Quality in Tanzania.” Economic Development and Cultural Change, 55 (3) _____. 2008. “Is Patient Satisfaction Sensitive to Changes in the Quality of Care? An Exploitation of the Hawthorne Effect.” Journal of Health Economics, 27 (2). Leonard K. L., S. Adelman and T. Essam. 2009. “Idle Chatter or Learning? Evidence of Social Learning about Clinicians and the Health System from Rural Tanzania.” Social Science and Medicine. 69: 183-190 Maestad O, and G. Torsvik. 2008. “Improving the Quality of Health Care when Health Workers are in Short Supply.” Chr. Michelsen Institute: Mimeo. Maestad, O., G. Torsvik and A. Aakvik. 2010. “Overworked? On the relationship between workload and health worker performance.” Journal of Health Economics 29(5): 686-698 44 Mliga, G. 2000. “Decentralization and the Quality of Health Care.” in The New Institutional Economics and the Restructuring of Animal Health Services in Africa, edited by Leonard, D.K. Macmillan Press, London. Rowe, A. K., D. de Savigny, C. F. Lanata, and C. G. Victora. 2005. ‘‘How Can We Achieve and Maintain High Quality Performance of Health Workers in Low-resource Settings?’’ Lancet, SEP 17, 366, 1026-1035. de Savigny et al. 2004. “Care-Seeking Patterns for Fatal Malaria in Tanzania.” Malaria Journal 3(27). WHO. 2005. Bridging the ‘Know-do’ Gap: Meeting on Knowledge Translation in Global Health. Technical Report WHO/EIP/KMS/2006.2. 45 This paper introduces a simple framework for understanding the dimensions and determinants of health worker performance based on the idea that there can be three different gaps affecting performance: a knowledge gap, the knowledge-capacity gap and the capacity-performance gap. The paper argues that performance is determined by a combination of competence, capacity and effort, and that any of these elements may lead to poor performance, and applies this framework to the measurement of health worker performance in Tanzania. Whilst discussing and highlighting key findings related to the assessment of health worker performance in Tanzania, the overarching objective of the paper is to offer a systematic way to analyze health worker performance through primary data collection and analysis to benefit researchers and countries beyond Tanzania. ABOUT THIS SERIES: This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author/s whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@ worldbank.org) or HNP Advisory Service (healthpop@worldbank.org, tel 202 473-2256). For more information, see also www.worldbank.org/hnppublications. 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org