82666 SERVICE DELIVERY INDICATORS Education | Health Education and Health Services in UGANDA Data for Results and Accountability N OV E M B E R 2 013 Launch edition Waly Wane Gayle H. Martin Service Delivery Indicators: Data for Results and Accountability The Service Delivery Indicators provide a set of metrics for benchmarking service delivery performance in education and health in Africa. The overall objective of the indicators is to gauge the quality of service delivery in primary education and basic health services. The indicators enable governments and citizens to identify gaps and track progress within and across countries over time. It is envisaged that the broad availability, high public awareness and persistent focus on the indicators will mobilize policymakers, citizens, service providers, donors and other stakeholders into action. Inspired by the World Development Report 2004: Making Services Work for Poor People, the ultimate goal is to sharply increase accountability for improved quality of services toward the ultimate end of improving human development outcomes. The Service Delivery Indicators initiative is an Africa-wide program that collects facility-based data from schools and health facilities. The perspective it adopts is that of citizens accessing a service. The indicators can thus be viewed as a service delivery report card on education and health care. Complementing other sources that draw on citizens’ perceptions to assess performance, the indicators assemble objective and quantitative information from a survey of schools and health facilities. The SDI initiative is a partnership of the World Bank, the African Economic Research Consortium and the African Development Bank to develop and institutionalize a set of robust measures of service delivery. The measurement of these indicators is based on survey instruments underpinned by rigorous research and embraces the latest innovations in measuring provider competence and effort. The survey instruments were piloted in Tanzania and Senegal. Uganda is the second country where a full-fledged SDI has been implemented, following Kenya which was completed in July 2013. The SDI is now being rolled out in Mozambique, Nigeria, and Togo. More countries will follow in 2014. The major funders of the SDI initiative are The William and Flora Hewlett Foundation and the World Bank. More information on the SDI survey instruments and data, and more generally on the SDI initiative can be found at: www.SDIndicators.org and www.worldbank.org/SDI, or by contacting sdi@worldbank.org. November 2013 © 2013 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: +1 202-473-1000 Internet: www.worldbank.org This work is a product of the Service Delivery Indicators initiative (www.SDIndicators.org, www.worldbank.org/SDI) and the staff of the International Bank for Reconstruction and Development/The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: +1 202-522-2422; e-mail: pubrights@worldbank.org or sdi@worldbank.org Education and Health Services in UGANDA SDI: Data for Results and Accountability As stated in its Vision 2040 blueprint, Uganda aims FIGURE 1. Higher Spending on Services Doesn’t Necessarily to transform its society and provide citizens with a Improve Outcomes better standard of living by focusing on improving 100% the quality of health and education services. Kenya However, this is easier said than done. While Uganda has 80% Ghana School Completion Rate made significant progress in reducing poverty, enrolling more children in school, and raising the child survival 60% Uganda Malawi rate, serious challenges remain. These include ensuring Mauritania Senegal Mali that children learn basic language and mathematics skills 40% Ethiopia Nigeria in primary school, and that mothers and infants have Burkina Faso access to skilled health care, so that tragic and untimely 20% losses of life are prevented. Achieving these goals is not 0% only intrinsically important for all Ugandans, but also $0 $5 $10 $15 $20 $25 $30 $35 necessary for Uganda’s structural transformation. Expenditures per Student Source: World Development Indicators Service delivery outcomes are determined by the relationships of accountability between policymakers, service providers, and citizens. and inputs, (ii) the effort exerted by providers, and (iii) the The delivery of quality health care and education is knowledge of providers. As the surveys are being rolled contingent on what happens in clinics and in classrooms. out in more African countries, the SDI will soon provide a A combination of several basic elements is required for means to track service delivery performance in education quality services to be delivered at the frontline. Adequate and health across Sub-Saharan Africa and over time. The financing, infrastructure, human resources, material, indicators are unique in that they provide countries with and equipment are all required, while institutions and detailed and comparable data on important dimensions governance structure must provide incentives for of service delivery. service providers to perform. The impact of financial and Accessibility is another important feature of the material resources is substantially reduced when frontline SDI. The information is made publicly available, giving providers face a set of incentives that are not conducive citizens a factual, objective, and rigorous basis on which to performance. Therefore, simply increasing the level of to engage with governments and service providers on resources is not enough to address the quality deficit in how to improve service delivery. education and health. The SDI were published for Kenya for the first time in July The Service Delivery Indicators (SDI) aim to construct 2013, following pilot surveys in Tanzania and Senegal. a set of benchmarking metrics that captures critical This Uganda brief draws from the SDI Uganda Technical dimensions of service delivery, including the Report which contains more detailed information on the knowledge and effort of providers. The indicators fall survey methodology and results and is available online into three categories: (i) the availability of key infrastructure at www.SDIndicators.org. DATA FOR RESULTS AND ACCOUNTABILITY 1 Service Delivery Indicators for Uganda – Highlights The Service Delivery Indicators for Uganda are based on surveys of about 400 primary schools and 400 health facilities, and nearly 5,300 teachers and health providers. The indicators provide evidence that basic inputs and infrastructure—with the notable exception of textbooks and drugs—are largely available at schools and health facilities. But they also show that attention needs to be paid to the level of knowledge and effort among providers. Importantly, they uncover large regional inequalities in service delivery. What service providers know n There are significant gaps in provider knowledge among both public and private providers in health as well as education. – Only 35% of public health providers could correctly diagnose at least 4 out of 5 very common conditions (like diarrhea with dehydration and malaria with anemia). In health centers that only offer outpatient services (HC2), half (49%) of the providers could not identify more than one of these conditions. Worryingly, public providers followed only 1 out of 5 (20%) of the correct actions needed to manage maternal and neonatal complications. – Less than 1 in 5 (19%) of public school teachers showed mastery of the curriculum they teach. Years of education and level of teacher training were positively correlated with higher teacher scores. n The Northern region and rural areas consistently and significantly lagged behind the other regions and urban areas in measures of knowledge and competence of providers. What service providers do n In both education and health, the problem of low provider effort is largely a reflection of suboptimal management of human resources. This is evidenced by the findings that: – More than half (52%) of public health providers were not present in the facility. Sixty percent of this absence was approved, and hence potentially within management’s power to influence. – More than 1 out of 4 (27%) of teachers in public schools were not at work. Of those who were in school, about 1 in 3 (30%) were not teaching. The result is 40% of public school classrooms with no teacher teaching. n By extrapolation, the average public Primary 4 student in the North received only 50 actual days of teaching time during the school year, about 90 days fewer than her Kampala counterpart. What service providers have to work with n Schools and health facilities have some of the basic inputs and equipment to function properly, but a few serious challenges remain. No textbooks were used by students in 86% of the classes in public schools, and only 44% of the public health facilities had all 6 of Uganda’s essential drugs. The adequate availability of priority drugs for mothers and children remains a challenge with only 39% and 23% respectively available in public facilities. – While the observed use of textbooks by students in public schools was very low at 14%, public schools--contrary to expectations--actually fared better than private schools, where the use was virtually non-existent at 3%. – Within the public sector, rural health facilities had poorer equipment and infrastructure; however the availability of tracer drugs was higher in rural facilities. 2 EDUCATION AND HEALTH SERVICES IN UGANDA SDI in Uganda: Implementation and Scope The Uganda SDI surveys were implemented by for knowledge in English, mathematics and pedagogy. the Economic Policy and Research Center (EPRC) a In the health sector, in addition to noting the presence/ leading African think tank. Before implementation, absence of 1,507 health providers, 736 providers were there was an extensive consultation process involving administered seven vignettes1 (sometimes called ‘patient key stakeholders in education and health (technical case simulations’). Of the vignettes, five were on common officers in ministries, non-governmental organizations, tracer conditions2 and two on the management of and private sector) in Uganda to contextualize the maternal and neonatal complications. SDI instruments and discuss the survey design. Data collection in the field took place between June and 1 Clinical vignettes are a widely used teaching method used primarily to measure clinicians August 2013, with simultaneous data entry. (or trainee clinicians) knowledge and clinical reasoning. A vignette can be designed to measure knowledge about a specific diagnosis or clinical situation at the same time as it measures trainees’ skills in performing the tasks necessary to diagnose and care The SDI is representative of Uganda’s four regions and for a patient. According to this methodology, one of the fieldworkers acts as a case Kampala, covering 400 primary schools and 400 health study patient and he/she presents to the clinician specific symptoms from a carefully constructed script while another acts as an enumerator. The clinician, who is informed facilities across the country. Both public and private of the case simulation, is asked to proceed as if the fieldworker is a real patient. For each facility, the case simulations are presented to up to ten randomly selected health workers (for profit and not-for-profit) providers were included. who conduct outpatient consultations. If there are fewer than ten health workers who The enumerators noted the presence/absence of 3,783 provide clinical care, all the providers are interviewed. For more information on the methodology, see www.SDIndicators.org. teachers in the school (and also in the classroom) on an 2 Malaria with anemia, diabetes mellitus, acute diarrhea, pneumonia, and pulmonary unannounced visit; and 2,214 teachers were assessed tuberculosis. THE INDICATORS The SDI indicators are grouped into three categories: (i) What providers know (knowledge and ability). Teachers need to have at least a minimum level of knowledge of the subjects they are teaching and skills to transform their knowledge into meaningful teaching. Similarly, health providers need to be skilled and competent to manage the conditions they are presented with. (ii) What providers do (provider effort). A minimum requirement for service delivery, for example, is that teachers and health providers are present in the facility and working. (iii) What providers have to work with (availability of key inputs). These indicators deal with the service delivery environment, including the availability of teaching and medical equipment and supplies and school and health facility infrastructure. The annex provides a detailed description of the indicators. DATA FOR RESULTS AND ACCOUNTABILITY 3 TABLE 1: Uganda Service Delivery Indicators At-a-Glance HEALTH EDUCATION Rural Urban Rural Urban Uganda Public Private Uganda Public Private Public Public Public Public What providers know (ability) Diagnostic Accuracy 58% 56% 61% 50% 70% Minimum knowledge 20% 19% 20% 17% 25% Adherence to clinical Test score (English, 50% 48% 52% 43% 61% 45% 46% 45% 45% 47% Guidelines Maths, & Pedagogy) Management of maternal / 19% 19% 20% 19% 18% neonatal complications What providers do (effort) Caseload 6 10 2 10 5 School absence rate 24% 27% 14% 31% 19% Classroom Absence from facility 46% 52% 39% 52% 52% 53% 57% 40% 60% 50% absence rate Time spent 3h 2h 4h 2h 3h teaching per day 17 min 55 min 20 min 43 min 33 min What providers have to work with (availability of inputs) 3 Drug availability 48% 40% 55% 40% 46% Students per textbook 14 12 100 17 7 Equipment availability 82% 78% 87% 78% 88% Equipment availability 95% 94% 96% 94% 95% TABLE 1: Uganda Service Delivery Indicators At-a- Infrastructure Glance Infrastructure 3 availability 64% 48% 80% 45% 74% 56% 60% 42% 57% 66% availability Summary of the Findings to assessing content knowledge, teaching ability was assessed through a test of pedagogical knowledge. What do teachers and health Teachers’ knowledge of the subjects they teach was workers know? very low, and the pedagogical skills to transform their The share of teachers with minimum content knowledge knowledge into meaningful teaching were even lower. reflects the results of a customized teacher test On average, teachers scored 65 percent and 58 percent administered to Primary 4 mathematics and English on the mathematics and English tests (Figure 2), but teachers. The English test results were for teachers only 1 in 5 public school teachers scored at least 80 teaching English, and the mathematics test results were percent on a test based on the curriculum they taught. for teachers teaching mathematics. The tests were based Pedagogical skills were weak, as reflected in the average on items from the curricula being taught in Uganda. score of 26 percent on the pedagogy test, and only 7 The tests were also validated against the curricula from percent of teachers scored above 50 percent. Teachers 13 African countries including Uganda4. In addition with a higher level of education and more teacher Teachers and health workers had significant knowledge gaps. Only 1 in 5 English and mathematics teachers had mastery of the curriculum being taught. In health, only 1 in 10 providers diagnosed all five tracer conditions and half of providers in the lowest level health facilities correctly diagnosed only one condition (or none). 3 Availability of drugs, equipment, and infrastructure are defined in the annex. 4 See “Teaching Standards and Curriculum Review”, a background paper prepared for the SDI by David Johnson, Andrew Cunningham, and Rachel Dowling. 4 EDUCATION AND HEALTH SERVICES IN UGANDA training scored significantly better. Surprisingly, more in Kampala and the Central region scored slightly better experienced teachers and female teachers fared worse than those in the other three regions. Finally, teachers and on the assessment. Head teachers performed better on health providers in rural areas fared significantly worse the pedagogy assessment. than those in urban areas when it comes to knowledge. Quality in the health sector was assessed using two FIGURE 2. Teachers’ subject knowledge was low, and indicators of process (adherence to clinical guidelines in the capacity to transmit it (pedagogy) even lower. five tracer conditions and management of maternal and newborn complications—as measured in the vignette Score obtained in teacher assessment (percent correct) interviews); and one indicator of outcomes (diagnostic accuracy in the five tracer conditions at the end of the 65% 64% 67% vignette interviews). Three of the tracer conditions 58% 58% 58% were childhood conditions (malaria with anemia, acute diarrhea with severe dehydration, and pneumonia), and two were adult conditions (pulmonary tuberculosis and 26% 26% 24% diabetes mellitus). Two other conditions were included: post-partum hemorrhage, the most common cause of maternal death during birth; and neonatal asphyxia, the most common cause of neonatal death during birth. Uganda Public Private Public providers managed to correctly diagnose only Maths English Pedagogy slightly more than half (56 percent) of the tracer conditions. Further, only 3 in 10 public providers were able to correctly FIGURE 3. Share of health providers who could diagnose at least four out of five tracer conditions correctly diagnose… (see Figure 3). In public health centers providing only outpatient services, half of the providers diagnosed none or only one of the tracer conditions. Even when providers 30% correctly diagnosed a condition, there was no guarantee 26% they would recommend the full treatment. Indeed, the correct treatment was recommended in only 36 percent of the cases, reflecting weak provider knowledge. It 16% is alarming to note that although almost 9 out of 10 12% 10% (88 percent) providers were able to correctly diagnose pulmonary tuberculosis, nearly half (47 percent) did not 5% prescribe the correct treatment required. Knowledge and treatment also varied across conditions, malaria with All 5 Cases Exactly Exactly Exactly Only No Case anemia was the least likely to be correctly diagnosed and 4 Cases 3 Cases 2 Cases 1 Case less than 1 in 10 (8 percent) recommending the correct treatment (Figure 4). It is particularly worrying that so few health providers were able to even diagnose potentially FIGURE 4. There is a significant knowledge gap between deadly conditions such as malaria or diarrhea. diagnosis and treatment. Public-private and regional comparisons 36% All 58% Teachers in private schools or health providers in private health facilities did not consistently outperform their 19% gap 9% counterparts in the public sector in measures of ability. Malaria with Anemia 28% 46% gap Indeed none of the differences in the scores between Pulmonary TB 42% 88% these two groups was statistically significant, be it on 53% diagnostic accuracy, minimum teacher knowledge, Diabetes 62% 42% adherence to guidelines, or pedagogy assessment. Pneumonia 59% However, there were large differences across regions. In 26% Acute Diarrhea 45% particular, provider scores in the Northern region were significantly lower. Overall diagnostic accuracy was Full Treatment Correct Diagnosis significantly higher in Kampala. In education, teachers DATA FOR RESULTS AND ACCOUNTABILITY 5 What do providers do? FIGURE 5: Absence from school and absence from class: Percent distribution of teachers by SDI uses a standardized, internationally benchmarked absenteeism status methodology5 to measure absenteeism, namely unannounced visits. SDI consists of two visits to each facility; the first is announced in advance so as to Teaching outdoors increase the likelihood of being able to collect the data 2% underlying most of the indicators. The second visit, which happens during the 7 days following the first visit, In classroom teaching is unannounced and its sole purpose is to ascertain the 39% whereabouts of the providers. Providers who are not in Absent from school the facility because it is not their 24% Both health and shift are not considered absent. Health workers who are not in the In school not education systems facility because they are carrying in class In class not teaching 29% suffer from provider out outreach activities are likewise 6% not considered absent. absenteeism. In Absenteeism is high for both more than half of teachers and health providers. FIGURE 6: Reasons for absence in the health sector On average, 1 in 4 (24 percent) Uganda’s public teachers were not in school and schools, over 60% about the same share of schools Other 18% (26 percent) had absenteeism Outreach 3% of teachers were not rates higher than 40 percent. Not approved Absence 21% However, even for teachers who in the classroom were at school, the SDI shows that Approved absence 25% Official mission 10% teaching, while 1 in 3 were not in the classroom In training/seminar 15% teaching and therefore absent Sick/maternity 8% nearly half of health from class. For every 100 teachers, providers on active only 39 teachers were in class teaching, 29 were at school but providers. In contrast, health providers in rural areas were duty were not found not in the classroom teaching, more likely to be absent than their urban counterparts. and 24 were nowhere to be found in health facilities. in the school (Figure 5). Putting In the majority of cases, respondents gave a legitimate together the data on absenteeism reason for the absence of health workers—such as with data on time use within attending training or a seminar (15 percent), on official classrooms suggests that out of the official teaching day mission (10 percent), or other approved absence (25 of 7 hours 20 minutes, the average Primary 4 student percent). Like all other workers, teachers and health experiences only 3 hours 17 minutes of teaching and providers are sometimes absent for any number of reasons. learning time with her teacher. A typical expected absence rate is about 5 to 10 percent authorized leave. The rates in Uganda are substantially Which teachers were most likely to be absent from class? higher. From the citizen’s perspective, if almost half of It turns out that it was older teachers and those born in staff is not teaching students or attending patients there the district in which they teach. Female teachers were is a legitimate reason for concern even if every single significantly less likely to be absent. absence was sanctioned. Better management at the In the health sector almost half (46 percent) of the facility or higher administrative level could probably curb providers were not on the facility premises. Who was sanctioned absence by implementing tighter leave rules. most likely to be absent among health providers? There Another indicator for health provider’s level of effort is seems to be no specific pattern. All cadres appeared patient caseload. The indicator is defined as the average equally likely to be absent, the same holds for female number of outpatient visits a health provider attends and male providers, as well as for older and younger to per working day. It is computed as the number of outpatient visits recorded in outpatient records in the 5 See Rogers and Koziol “Provider Absence Surveys in Education and Health. A Guidance Note”. World Bank three months prior to the survey, divided by the number 6 EDUCATION AND HEALTH SERVICES IN UGANDA of days the facility was open during the three month FIGURE 7: Absenteeism across regions and differences period and the number of health workers who conduct between public and private schools patient consultations (i.e. excluding any staff who do not see patients). 24% Uganda The average health provider consulted with 6.1 53% outpatients per day, a surprisingly low number. Smaller Private 14% 40% facilities staffed with one or two health providers had Public 27% 57% the largest caseload with 11 outpatients per provider per 35% day, which is more than twice the load for facilities with Northern 69% 26% 3 to 5 providers (5.3 outpatients). Very large facilities with Eastern 60% 22% more than 20 health providers recorded a caseload of Central 47% only 2.1 outpatients. Western 18% 42% 11% Kampala 38% Public-private and regional comparisons School Absence Classroom Absence There is a lot of regional and public-private variation in providers’ level of effort in both sectors. Starting with the caseload, public health providers’ caseload (10 on classroom observation and in the case of students per outpatients per provider per day) was almost five times textbook the students were asked to show their textbook that of private providers (2.2 outpatients). Within the and the enumerator counted public sector, rural providers’ caseload was more than the number of textbooks in use. twice that of urban providers. Public health facilities The indicator is then simply the In nearly 90 percent with only 1 or 2 health providers were the busiest and number of textbooks shown received 18.6 outpatients on a daily basis. Western of Primary 4 English divided by the number of and Central regions had the highest absenteeism with students attending the class. and mathematics roughly 6 out of 10 health providers absent. In education, teachers’ absenteeism was especially prevalent in the Uganda does very well when classes surveyed, Northern and Eastern regions with more than 70 percent it comes to the availability of there was no and 60 percent respectively of the schools experiencing minimum teaching resources, absenteeism from classroom of 60 percent or more. As with nearly universal availability. textbook available. a result, Primary 4 students in the Northern region only Virtually all the classrooms had The average received 1 hour 52 minutes of teaching time per day out blackboards and students had of a 6 hour 56 minute teaching day. At the end of the pencil and paper to work with. Primary 4 class in school year a student in Kampala would have had over But in more than 90 percent the Eastern and three months’ worth of education more than a student of the Primary 4 classrooms in the North. observed there was no student Northern regions with a textbook, and nationwide had 2.5 and 3 times What are the inputs that providers there were about 14.4 students have to work with? per textbook—a high number by more students than any standard. Schools also did less Schools and health facilities need basic infrastructure its counterpart in well in terms of infrastructure: for and equipment to function properly. The SDI collects example only 70 percent of them Kampala. information on a few critical inputs to measure the had clean toilets. The student- quality of the environment in which providers work and teacher ratio was high at 45:1, deliver services. In schools, the “availability of teaching signaling overcrowded classrooms. resources” indicator measures (i) whether a Primary 4 classroom has a functioning blackboard and chalk; (ii) the The medical equipment indicator focuses on the share of students with pens; and (iii) the share of students availability of minimum equipment expected at a with notebooks. The “infrastructure availability” indicator facility i.e. (i) a weighing scale (adult, child or infant), (ii) captures whether a school has (i) functioning toilets a stethoscope, (iii) a blood pressure machine and (iv) a assessed as being clean, and private, and accessible; and thermometer. In each case the equipment needed to be (ii) sufficient light to read the blackboard from the back observed by the enumerator and assessed as functioning. of the classroom. In addition, the student-teacher ratio More than 4 out of 5 (82 percent) health facilities had all and students per textbook were assessed. Both depend the basic equipment required. In terms of health facility DATA FOR RESULTS AND ACCOUNTABILITY 7 infrastructure, ninety percent of facilities had access to FIGURE 8: There is considerable variation in number sanitation, more than ninety percent had access to clean of students per Primary 4 class across regions water, and close to three quarters (74 percent) had a source of electricity. This translates into an indicator of 58 “infrastructure availability” of 64 percent. 51 39 Drug availability is defined as the number of drugs of 32 which a facility has one or more available, as a proportion 29 of all the drugs on a list of 26 tracer medicines for children 20 and mothers identified by the World Health Organization (WHO). Enumerators needed to observe the drugs and the drugs were considered available only if at there was Kampala Central Western Uganda Eastern Northern at least one unexpired box/vial available. On average only 47 percent of this long list were available. However, the picture looks brighter if one looks at the six major FIGURE 9: Percent of Primary 4 classes where tracer drugs6 on Uganda’s essential medicines and health students use textbooks during English and supplies (EMHS) list. Indeed, on average, 79 percent Maths classes of the six tracer drugs were available at the facilities. However, less than half (49 percent) of the priority drugs for children and close to 1 out of 3 (35 percent) priority Uganda 11% drugs for mothers were available. Public-private and regional comparisons Private 3% There is a lot of regional as well as public-private variation Public 14% for the availability of inputs. Perhaps surprisingly, public schools did better than private schools on availability of 0 10 20 30 40 50 textbooks. In private schools, virtually none (3 percent) had textbooks observed in use by students compared to 14 percent in public schools—still a very low rate. rural facilities. Finally, the Northern and Eastern regions Public schools did better than private schools on were worse off in almost all dimensions except for tracer infrastructure, mostly because of the greater accessibility drugs, which were available in more than ninety percent and cleanliness of toilets. The student-teacher ratio was of Northern health facilities. significantly larger in public schools (50:1) compared to private (28:1) schools. Within the public sector, urban How does Uganda fare relative schools had a much larger Primary 4 class size than rural to other countries? schools: 58 students per teacher in urban areas versus 46 students per teacher in rural areas. Primary 4 teachers SDI Uganda used the same instruments and is fully in Northern region had classes twice as large as those of comparable to SDI Kenya. Teachers and students their colleagues in the Central region (62:1 vs 31:1). were assessed with the exact same questions on the assessments. The same questions were also put to heads Although public health facilities fared better on the of school across the two countries. Tanzania and Senegal availability of the six tracer drugs with 85 percent instruments also overlap a great deal with the ones for compared to 72 percent for the private facilities, the Uganda and Kenya although there are a few indicators private sector did better when it came to mothers’ and which are not comparable. children drugs. Private health facilities were also better equipped and had better infrastructure especially— In health, Uganda performed better than Kenya on input mostly because of electricity availability which stood indicators and adherence to guidelines. However, Kenyan at 90 percent in private facilities and only 73 percent health providers were 20 percent more likely to get the in public facilities. Within the public sector, rural health diagnosis right and were twice as likely to correctly facilities had poorer equipment and infrastructure; manage maternal and neonatal complications. Despite however the availability of tracer drugs was higher in a slightly higher caseload in Uganda (10 vs 9) Kenya’s public providers seemed to exert a higher level of effort 6 These are ACT, cotrimoxazole, measles vaccine, oral rehydration salt, Depo-Provera and Sulphadoxine pyramethamine. because of the very high absenteeism rate observed in 8 EDUCATION AND HEALTH SERVICES IN UGANDA TABLE 2: Comparative SDI At-a-Glance for Uganda, Kenya, Tanzania, and Senegal89 HEALTH EDUCATION   Uganda Kenya Tanzania Senegal   Uganda Kenya Tanzania Senegal Diagnostic Accuracy 56% 72% 57% 34% Minimum knowledge 19% 39% Not comparable Adherence to clinical Test score (English, Maths, 48% 44% 34% 22% 45% 57% Not comparable Guidelines & Pedagogy) Management of maternal / 19% 45% Not available neonatal complications Caseload 10 9 Not available School absence rate 27% 15% 23% 18% Absence from facility 52% 27% 21% 20% Classroom absence rate 56% 42% 53% 29% Time spent teaching 2h 2h 2h 3h per day 55 min 40 min 04 min 15 min 7h 5h 5h 4h Scheduled teaching time 20 min 40 min 12 min 36 min Student-teacher ratio 50 32 74 34 Drug availability 40% 67% Not comparable Students per textbook 12.4 3.1 Not comparable Equipment availabilitya 78% 76% 78% 53% Equipment availability 94% 95% Not available Infrastructure availability 48% 57% 19% 39% Infrastructure availability 60% 59% Not comparable a In the pilot for Senegal and Tanzania the presence of a weighing scale, a thermometer, and a stethoscope was used to compute the equipment availability indicator. Kenya and Uganda added a blood pressure machine on top of the three. Note also than in the pilot the availability was self-reported whereas in the full-fledged SDI availability and functioning status were physically assessed and verified. The indicator for the pilot is therefore biased upwards. Kenya significantly outperformed Uganda in both education and health. Doctors in Uganda performed at about the same level as Kenyan nurses on both diagnostic accuracy and the capability to provide full treatment. The proportion of Kenyan teachers who reached the minimum knowledge threshold was more than double that of Uganda’s teachers. Uganda. Except for absenteeism, providers in Senegal teachers performed better in every component of the and Tanzania performed poorly when compared to assessment: English, mathematics, and pedagogy. They Uganda and Kenya on diagnostic accuracy, adherence to were also less likely to be absent both from school and guidelines, and input indicators. classroom. Interestingly, because of a much longer official scheduled teaching time, the average Ugandan The quality of education of a country’s workforce as Primary 4 student had 15 minutes more contact time per captured by cognitive skills (proxied using scores on day with her teacher than the Kenyan student. Kenyan international tests) has been shown to be a strong students, however, outperformed Ugandan students on determinant of long-term growth, even more so than the SDI English and mathematics tests. educational attainment.7 Countries with a better educated workforce are more likely to be able to compete Do the SDI really matter? in an ever more globalized world. The quality of schools and teachers is also strongly positively correlated with While SDI measures knowledge, effort, and inputs, students learning outcomes. Kenyan public teachers at the end of the day these matter only so far as they largely outperformed their Ugandan counterparts. deliver the learning outcomes policy-makers and citizens The share of teachers with minimum knowledge was care about. To test the usefulness of the SDI, learning twice as large in Kenya compared to Uganda. Kenyan outcomes for Primary 4 students are also measured 7 See Hanushek and Woessman’s “Do better schools lead to more growth? Cognitive skills, economic outcomes and causation” Journal of Economic Growth (2012), 17: 267-321. 8 For the sake of comparability, results for public sector providers only are reported. 9 Note that for diagnostic accuracy vignettes for Senegal and Tanzania included pelvic inflammatory disease which has been replaced by diabetes mellitus in the Kenya and Uganda vignettes. The other four tracer conditions remained the same. The adherence to guidelines is computed similarly although in the case of Kenya and Uganda more questions were introduced. DATA FOR RESULTS AND ACCOUNTABILITY 9 during the survey. The relationship between the SDI from 180 to 90 deaths per 1,000 live births between 1989 and students’ performance is very strong. School and and 2011. This improvement is largely credited to vertical classroom absenteeism as well as student-teacher ratio programs but system-related indicators such as maternal are, as expected, strongly negatively correlated with mortality rate stagnated or slightly deteriorated. These student test scores. Teachers’ performance on English disappointing results are most likely related to the quality and mathematics, the share of teachers with minimum of service delivery. knowledge, and availability of teaching resources all are positively correlated with student performance. The only To achieve its ambitious but attainable Vision 2040, variable which seems uncorrelated with students test Uganda needs to accelerate economic growth. scores is school infrastructure. Building a healthy and educated labor force is critically important. The SDI shows that although teachers and For the health sector there are several outcome measures health workers function in a relatively decent working at the national level that can be used to assess whether environment, a number of challenges must be urgently they are correlated with SDI, such as under-five mortality addressed. Almost half of the providers were missing in rates or the maternal mortality ratio, but unfortunately action and not found in the classroom teaching students there is no indicator that can be linked directly to the or in the consultation rooms attending to patients. The service quality of individual facilities. However, one can SDI also uncovered significant knowledge gaps for both reasonably assume that the higher the prevalence of teachers and health workers. If not addressed, these misdiagnosis the worse the health outcomes. Also low service delivery failures will hamper Uganda’s effort to capacity to manage maternal and neonatal complications build a healthy and educated workforce. could be reasonably assumed to be negatively correlated with child and maternal health outcomes. The SDI also showed serious regional inequalities, which will also likely feed into greater income and welfare What does all this mean for Uganda? inequality in the future. This again hampers the shared Uganda has one of the youngest populations in the growth agenda and undermines the effort to build a world and has made great progress in expanding cohesive and prosperous Uganda. access to education. As of 2012, more than 8.4 million Over the past two decades, Uganda has made students were enrolled in its primary schools. However, tremendous progress on economic growth, poverty increased access to education was not accompanied by reduction, as well as on some human development improvement in learning outcomes. As several recent indicators. The results of the SDI, however, show that studies show, too many school-going children still cannot to sustain or accelerate this progress, Uganda will need properly read and count. Recent evidence shows that to focus on raising quality and efficiency in health and cognitive skills are much more important in promoting education. The discovery of oil could be transformational economic growth than number of years of schooling. for Uganda in terms of its vision for 2040, but only if the This puts the spotlight on the quality of schooling. quality agenda is addressed in health and education. Only The same picture holds in the health sector with steady then can natural resources revenues be translated into progress noted in under-five mortality, which fell by half long-term economic growth that benefits all Ugandans. 10 EDUCATION AND HEALTH SERVICES IN UGANDA Annex 1. The Service Delivery Indicators defined School absence rate Share of a maximum of 10 During the first announced visit, a maximum of ten teachers are randomly selected from the list of all randomly selected teachers teachers who are on the school roster. The whereabouts of these ten teachers are then verified in the second, absent from school during an unannounced, visit. Teachers found anywhere on the school premises are marked as present. unannounced visit. Classroom absence rate Share of teachers who are present The indicator is constructed in the same way as School Absence Rate indicator, with the exception that the in the classroom out of those numerator now is the number of teachers who are both at school and in the classroom. The denominator teachers present at school during is the number of teachers who are present at the school. A small number of teachers are found teaching scheduled teaching hours as outside, and these are marked as present for the purposes of the indicator. observed during an unannounced visit. Classroom teaching time (also known as Time on Task) Amount of time a teacher spends This indicators combines data from the Staff Roster Module (used to measure absence rate), the Classroom Observation teaching during a school day. Module, and reported teaching hours. The teaching time is adjusted for the time teachers are absent from the classroom, on average, and for the time the teacher remains in classrooms based on classroom observations recorded every 5 minutes in a teaching lesson. Distinction is made between teaching and non-teaching activities based on classroom observation done inside the classroom. Teaching is defined very broadly, including actively interacting with students, correcting or grading student’s work, asking questions, testing, using the blackboard or having students working on a specific task, drilling or memorization, and maintaining discipline in class. Non-teaching activities is defined as work that is not related to teaching, including working on private matters, doing nothing and thus leaving students not paying attention, or leaving the classroom altogether. Minimum knowledge among teachers Share of teachers with minimum This indicator measures teacher’s knowledge and is based mathematics and language tests covering the primary curriculum knowledge administered at the school level to all teachers of Primary 4. Textbooks per student Number of mathematics and language The indicator is measured as the number of mathematics and language books that students use in a Primary 4 classroom books used in a Primary 4 classroom divided by the number of students present in the classroom. The data will be collected as part of the classroom observation divided by the number of students schedule. present in the classroom Student/teacher ratio Average number of grade 4 pupils per The indicator of teachers’ availability is measured as the number of students per teacher based on the Classroom Primary 4 teacher. Observation Module, where the number of students are counted per teacher teaching. Equipment availability Proportion of schools with the Minimum teaching resources is assigned 0-1 capturing availability of (i) whether a Primary 4 classroom has following available: functioning a functioning blackboard and chalk, (ii) the share of students with pens, and (iii) the share of students with blackboard with chalk, pencils and notebooks, giving equal weight to each of the three components. notebooks. All three components Functioning blackboard and chalk: The enumerator assesses if there was a functioning blackboard in the are equally weighted. classroom, measured as whether a text written on the blackboard could be read at the front and back of the classroom, and whether there was chalk available to write on the blackboard. Pencils and notebooks: The enumerator counts the number of students with pencils and notebooks, respectively, and by dividing each count by the number of students in the classroom one can then estimate the share of students with pencils and the share of students with notebooks. DATA FOR RESULTS AND ACCOUNTABILITY 11 Infrastructure availability Proportion of the proportion Minimum infrastructure resources is assigned 0-1 capturing availability of: (i) functioning toilets of schools with the following operationalized as being clean, and private, and accessible; and (ii) sufficient light to read the blackboard from available: functioning electricity the back of the classroom, giving equal weight to each of the two components. and sanitation. Each component Functioning toilets: Whether the toilets were functioning was verified by the enumerators as being accessible, has the same weight. clean and private (enclosed and with gender separation). Electricity: Functional availability of electricity is assessed by checking whether the light in the classroom works gives minimum light quality. The enumerator places a printout on the board and checks (assisted by a mobile light meter) whether it was possible to read the printout from the back of the classroom given the slight source. Education expenditure reaching primary school Education expenditure reaching primary The indicator of availability of resources at the primary school level assesses the amount of resources available for services school to students at the school. It is measured as the recurrent expenditure (wage and non-wage) reaching the primary schools per primary school age student in US dollars at Purchasing Power Parity (PPP). Unlike the other indicators, this indicator is not a school-specific indicator and is calculated as the amount of resources reached per surveyed school, and then sample weights are used to estimate value for the population (of all schools) in aggregate. Quantities and values of in kind items were collected as part of the survey and when values of in kind items were missing, average unit costs were inferred using information from other surveyed schools. Sources for the number of primary school age children, broken down by rural and urban location, are the Ministry of Education and Vocational Training (2010) for Tanzania and ANSD (2008) for Senegal. Delays in wages Delays in wages The indicator captures the share of teachers who have wages due in excess of two months. The indicator measures whether or not teachers have received the totality of their expected wage from two months prior. Caseload per health provider Number of outpatient visits per The number of outpatient visits recorded in outpatient records in the three months prior to the survey, divided by the clinician per day. number of days the facility was open during the three month period and the number of health workers who conduct patient consultations (i.e. excluding cadre-types such as public health nurses and out-reach workers). Absence rate Average share of staff not in the Number of health workers that are not off duty who are absent from the facility on an announced visit as a share of ten facilities as observed during one randomly sampled workers. Health workers doing fieldwork (mainly community and public health workers) were counted unannounced visit. as present. The absence indicator was not estimated for hospitals because of the complex arrangements of off duty, interdepartmental shifts etc. Adherence to clinical guidelines Unweighted average of the For each of the following five case study patients: (i) malaria with anemia; (ii) diarrhea with severe share of relevant history taking dehydration; (iii) pneumonia; (iv) pulmonary tuberculosis; and (v) diabetes. questions, the share of relevant History Taking Questions: Assign a score of one if a relevant history raking question is asked. The number of examinations performed. relevant history taking questions asked by the clinician during consultation is expressed as a percentage of the total number of relevant history questions included in the questionnaire. Relevant Examination Questions: Assign a score of one if a relevant examination question is asked. The number of relevant examination taking questions asked by the clinician during consultation is expressed as a percentage of the total number of relevant examination questions included in the questionnaire. For each case study patient: Unweighted average of the: relevant history questions asked, and the percentage of physical examination questions asked. The history and examination questions considered are based on the Kenya National Clinical Guidelines and the guidelines for Integrated Management of Childhood Illnesses (IMCI). Management of maternal and neonatal complications Share of relevant treatment actions For each of the following two case study patients: (i) post-partum hemorrhage; and (ii) neonatal asphyxia. proposed by the clinician. Assign a score of one if a relevant action is proposed. The number of relevant treatment actions proposed by the clinician during consultation is expressed as a percentage of the total number of relevant treatment actions included in the questionnaire. 12 EDUCATION AND HEALTH SERVICES IN UGANDA Diagnostic accuracy Average share of correct diagnoses For each of the following five case study patients: (i) malaria with anemia; (ii) diarrhea with severe provided in the five case studies. dehydration; (iii) pneumonia; (iv) pulmonary tuberculosis; (v) diabetes. For each case study patient, assign a score of one as correct diagnosis for each case study patient if case is mentioned as diagnosis. Sum the total number of correct diagnoses identified. Divide by the total number of case study patients. Where multiple diagnoses were provided by the clinician, the diagnosis is coded as correct as long as it is mentioned, irrespective of what other alternative diagnoses were given. Drug availability Share of basic drugs which at the time Priority medicines for mothers: Assign score of one if facility reports and enumerator confirms/observes the facility has the of the survey were available at the drug available and non-expired on the day of visit for the following medicines: Oxytocin (injectable), misoprostol (cap/tab), facility health facilities. sodium chloride (saline solution) (injectable solution), azithromycin (cap/tab or oral liquid), calcium gluconate (injectable), cefixime (cap/tab), magnesium sulfate (injectable), benzathinebenzylpenicillin powder (for injection), ampicillin powder (for injection), betamethasone or dexamethasone (injectable), gentamicin (injectable) nifedipine (cap/tab), metronidazole (injectable), medroxyprogesterone acetate (Depo-Provera) (injectable), iron supplements (cap/tab) and folic acid supplements (cap/tab). Priority medicines for children: Assign score of one if facility reports and enumerator confirms after observing that the facility has the drug available and non-expired on the day of visit for the following medicines: Amoxicillin (syrup/ suspension), oral rehydration salts (ORS sachets), zinc (tablets), ceftriaxone (powder for injection), artemisinin combination therapy (ACT), artusunate (rectal or injectable), benzylpenicillin (powder for injection), vitamin A (capsules) We take out of analysis of the child tracer medicines two medicines (Gentamicin and ampicillin powder) that are included in the mother and in the child tracer medicine list to avoid double counting. The aggregate is adjusted by facility type to accommodate the fact that not all drugs (injectables) are expected to be at the lowest level facility, dispensaries./health posts where health workers are not expected to offer injections. Equipment availability Share of facilities with thermometer, Medical Equipment aggregate: Assign score of one if enumerator confirms the facility has one or more functioning of each stethoscope and weighing scale of the following: thermometers, stethoscopes, sphygmomanometers and a weighing scale (adult or child or infant weighing refrigerator and sterilization equipment. scale) as defined below. Health centers and first level hospitals are expected to include two additional pieces of equipment: a refrigerator and sterilization device/equipment. Thermometer: Assign score of one if facility reports and enumerator observes facility has one or more functioning thermometers. Stethoscope: Assign score of one if facility reports and enumerator confirms facility has one or more functioning stethoscopes. Sphygmomanometer: Assign score of one if facility reports and enumerator confirms facility has one or more functioning sphygmomanometers. Weighing Scale: Assign score of one if facility reports and enumerator confirms facility has one or more functioning Adult, or Child or Infant weighing scale. Refrigerator: Assign score of one if facility reports and enumerator confirms facility has one or more functioning refrigerator. Sterilization equipment: Assign score of one if facility reports and enumerator confirms facility has one or more functioning Sterilization device/equipment. Infrastructure availability Share of facilities with electricity, clean Infrastructure aggregate: Assign score of one if facility reports and enumerator confirms facility has electricity and water water and improved sanitation. and sanitation as defined. Electricity: Assign score of one if facility reports having the electric power grid, a fuel operated generator, a battery operated generator or a solar powered system as their main source of electricity. Water: Assign score of one if facility reports their main source of water is piped into the facility, piped onto facility grounds or comes from a public tap/standpipe, tubewell/borehole, a protected dug well, a protected spring, bottled water or a tanker truck. Sanitation: Assign score of one if facility reports and enumerator confirms facility has one or more functioning flush toilets or VIP latrines, or covered pit latrine (with slab). DATA FOR RESULTS AND ACCOUNTABILITY 13 SERVICE DELIVERY INDICATORS Education | Health sdi@worldbank.org www.worldbank.org/SDI www.SDIndicators.org NOVEMBER 2013 With support from The William and Flora Hewlett Foundation