103443 Health Worker Survey in Timor-Leste Final Report June 2015 Health Worker Survey in Timor-Leste Preface This report presents the results of the Health Worker Survey in Timor-Leste. The survey collected information on labour market dynamics, health worker preferences and concerns, and the skills, competence and motivation of doctors working in the public sector. The overall goal is to provide evidence for policy guidance on the health workforce in Timor-Leste. If you have any queries or need any clarifications please feel free to contact us using the information at the bottom of this page. This project was carried out by World Bank and Oxford Policy Management (OPM). Dili Institute of Technology (DIT) was the local partner. The Ministry of Health of the Government of Timor-Leste provided support at every stage of this project. If you have any queries please contact Xiaohui Hou, Senior Economist, World Bank (xhou@worldbank.org) or Rashid Zaman, Consultant, Health Portfolio, Oxford Policy Management (rashid.zaman@opml.co.uk). Full citation: World Bank and OPM (2015) “Health Worker Survey in Timor - Leste”, Dili: World Bank and Oxford: Oxford Policy Management (OPM). Oxford Policy Management Limited 6 St Aldates Courtyard Tel +44 (0) 1865 207300 38 St Aldates Fax +44 (0) 1865 207301 Oxford OX1 1BN Email admin@opml.co.uk Registered in England: 3122495 United Kingdom Website www.opml.co.uk © World Bank and Oxford Policy Management i Health Worker Survey in Timor-Leste Acknowledgements This report was produced and written by a task team from Oxford Policy Management (OPM) consisting of Rashid Zaman, Kay Engelhardt, Marc-Francois Smitz, Firdaus Hafidz, Sophie Witter and Tomas Lievens under the World Bank task team leader Xiaohui Hou (Senior Economist). World Bank staff including Christophe Lemiere (Senior Health Specialist), Eileen Sullivan (Senior Operations Officer for Health), Sarah Harrison (Health Consultant), Sara Maria Pereira (Consultant), Jennifer Shin (Consultant) and Cornelio Quintao de Carvalho (Team Assistant) provided technical inputs and logistical support. Toomas Palu (Practice Manager, Health, Nutrition and Population Global Practice) provided technical comments and overall supervision on this report. The team would also like to thank Franz Drees-Gross (Country Director for Timor-Leste, Papua New Guinea & Pacific Islands, East Asia and Pacific Region) and Bolormaa Amgaabazar (Country Manager for Timor-Leste) for their guidance and support. The authors would like to thank H. E. Maria do Céu Sarmento Pina da Costa (Minister of Health, Timor-Leste and H.E. Ana Isabel Soares (Vice-Minister of Health, Timor-Leste) for their overall supervision and guidance. The authors thank the staff members and external consultants of OPM including Juan Munoz, Katie McIntosh, Sourovi De, Mahmud Hassan and Wirya Wardaya for their support in sampling, analysis, survey management, data-entry programming and data management. The team would also like to thank Prima Ratrikaningtyas, Sutarman and Lukman Chandra, the medical researchers from the University of Gadjah Mada who joined the field teams. Dili Institute of Technology (DIT) was the local partner for this survey and the team included Estanislau Saldanha, Fernanda Julia, Afonso Almeida, Alvaro Amaral, Calisto Godinho Quiteria Fernandes, Joao Silva, Tiago Soares, Valenti Martins, Diamantino Soares, Francisco Godinho, Palmira Lemos and Jeferinu Ulan. The members of the data management team were Octovianus Pereira, Roberto Silva, Haryanto Santoso, Raul Amaral, Natalino Costa, Maria Amaral and Ticiana Caldas. The team would like to thank the Ministry of Health (MoH) officials who provided important support at various stages of this project, including Jose dos Reis Magno (Director General), Duarte Ximenes (National Director, Human Resources and Administration), Marcelo Amaral (National Director, Finance and Procurement), Amelia Barreto (Chief, Department of Quality Control), Lourenco Camnahas (Project Manager, National Health Sector Strategic Plan), Celeste Cham (Head, Department of Pharmacy), Odete Belo (President of Ad Hoc Commission, SAMES) and their colleagues. The report benefited from comments from Augustine Asante (Research Fellow, University of New South Wales), Joao Martins (Dean, Faculty of Medicine, University of Timor-Leste), Andre Cote (HR Specialist, National Health Sector Strategic Plan), Ian Morris (Consultant, World Bank), Sanjoy Nayak (Human Resources for Health Adviser, National Health Sector Strategic Plan) and peer reviewers Akiko Maeda (lead health specialist, World Bank), Christopher Herbst (Health Specialist, World Bank) and Armandina Gusmao-Amaral (Senior Health Officer, the Australian Department of Foreign Affairs and Trade). The team would also like to thank development partners and non- governmental organisations including the Department of Foreign Affairs and Trade (DFAT) of the Australian Government, the European Commission (EC), the World Health Organization (WHO), UNFPA, USAID, Marie Stopes and Health Improvement Project for their feedback during the stakeholder consultation workshop and individual consultations. Financial support for this work came from the Multi-Donor Trust Fund, which received funding from the Australian Government and the European Commission. © World Bank and Oxford Policy Management ii Health Worker Survey in Timor-Leste Executive Summary Introduction At the time of independence in 2002, Timor-Leste had a seriously weak health system with only a handful of doctors in the country. In this context, the governments of Timor-Leste signed an agreement and the Cuban Medical Brigade started to train medical students and deploy them in the country, particularly in rural areas. While the initial massive shortage has now been minimised, there are concerns over more complex issues including facility functionality, rural retention, motivation, preferences and competence of health workers. The objectives of this survey were to understand the labour market dynamics among health workers, to learn more about the preferences and concerns of health workers, and to assess the skills, competence and motivation of doctors. Methods This survey included four components: a health facility survey, a health worker survey, a discrete choice experiment (DCE) and direct clinical observations (DCO). A qualitative study was carried out prior to the main study. The health facilities and health workers were sampled using probability proportionate to size (PPS). Three field teams collected the quantitative survey data between July and August 2014. Data were double entered using a customised data-entry program. Associations between various predictors and outcome variables are presented through descriptive statistics and statistical tests. Further work could be done in future to examine the relationships between the three main datasets. The study has several limitations. One of them is the lack of demand-side information; the sample only included the health facilities and health workers and did not include any patients. To mitigate this limitation the survey methodology and instruments were carefully designed so that the results would not be biased based on health workers’ self -reporting on functionality or performance. Private health providers are not included in this survey, which is another important limitation. Private health care providers provide about one-quarter of the health services in Timor-Leste; including them may have increased the comprehensiveness of the survey. However, the primary goal of this survey was to inform policy-makers on government health workers. Findings Labour Market Dynamics: Findings from the Health Worker Survey The general health worker survey was administered to 443 health workers, including 175 doctors, 150 nurses and 118 midwives (about 20% of the health workers in the country). The majority (56%) of health care workers were women, including 58% of the doctors and all of the midwives. The only exception was nurses, where the majority (64%) were men. Close to 53% of all doctors interviewed were posted in rural facilities, while nurses were more concentrated in urban areas (70%). Twenty- one percent of doctors were working in Health Posts (HP), compared to 8% of nurses and 16% of midwives. Since there were only a few doctors in country in Timor-Leste at the time of independence, almost the entire cohort of doctors was newly trained; most of the doctors (96%) interviewed in the survey had less than five years of experience in the sector. The survey shows that health workers got opportunities to advance to higher types of facility with more experience. Motivation to join and long-term preferences: The vast majority of staff selected medicine as a career in order to help people and, generally health workers indicated high levels of satisfaction: © World Bank and Oxford Policy Management iii Health Worker Survey in Timor-Leste only 4% of the respondents indicated that they were ‘unsatisfied’ or ‘very unsatisfied’ with their work. Intrinsic motivation is also shown by the fact that eight in ten medical staff said that they would stay in the facility until the last patient was treated – even if they did not receive additional money for this. In the long run the majority of respondents would like to continue to work in the government/public sector, including 99% of the doctors. This was also the case for the broad majority of nurses and midwives, although 6% of nurses would prefer to move to the private sector. HPs were the least attractive stations for medical staff, with only 6% of doctors stating a HP as their long-term preferred facility, although 32% were content to work in Community Health Centres (CHC). Only 22% saw themselves working in rural sucos long term. However, 88% acknowledged that the MoH would determine where their next assignment would be. The majority of respondents (97%) were not looking for another job in the short term. Salary and financial benefits: Wage differentials within each cadre were relatively small, and income did not vary much by the years of experience, particularly for the doctors. On average, a doctor with more than ten years of experience earned just US$ 50 more than a newly joined doctor. More than half of the health staff interviewed believed their salary to be too low. However, they did not see other labour market opportunities that they would prefer. All medical staff receive their money through a direct deposit to their bank account and only very few (2%) have experienced any delays in receiving their money. Very few report private practice (but this may be under-reported). More non-financial benefits (such as housing and motorbikes) were reported by doctors and at lower-level facilities. However, only half of them reported that they get either sufficient fuel or funds to buy fuel for the motorbike, and more than half of all respondents also stated that these benefits were delayed. Workload: Although most respondents worked five days a week in HPs, CHCs and district/regional hospitals (63%, 66% and 54%, respectively), 22% to 36% worked six and sometimes seven days a week. The number of patients seen by the sampled doctors who participated in the DCOs was counted; the mean number of patients per day was 10.2, with the standard deviation (SD) of 7.5. In urban facilities the mean (SD) patient load was 11.5 (6.9) compared to 9.6 (7.8) in rural facilities. The number of patients also varied by level of facility, with higher tier facilities having more patients. Forty-five percent of respondents agreed with the statement that they have ‘too much work to do’. However, we found no clear correlation between workload and feelings of overload. Training: Almost all respondents believed that their training adequately prepared them to diagnose and treat clinical cases in Timor-Leste. When asked how many short-term training sessions (below 30 days) they had attended in the last three years, roughly half of the nurses and midwives reported doing three or more, while one-quarter did not attend any such training in this timeframe. At the same time, roughly one-third (35%) of the doctors attended three or more training sessions and around the same proportion (37%) attended none. Despite all the training opportunities, roughly half (52%) of respondents stated that ‘there is not enough opportunity to learn current medical knowledge’. Although a large number of doctors were in rural areas, it is interesting to see that only 19% underwent training on community health. Around 75% of doctors indicated that they require training on Integrated Management of Childhood Illness (IMCI) and 64% on Emergency Obstetrics and Newborn Care (EmONC). The doctors located in urban areas were found to benefit more: roughly half (51%) of them had attended three or more training sessions in the last year, as opposed to 26% in rural areas. © World Bank and Oxford Policy Management iv Health Worker Survey in Timor-Leste Older or highly satisfied doctors tended to be less interested in any kind of training, while females were keener than males to have the opportunities for visits from specialists. Workers in HPs and CHCs were more interested in specialisation. Highly satisfied doctors had a lower utility for all kinds of training. Older nurses and midwives, as well as those with more medical experience, tended to be more interested in any kind of training. Rural facility health workers were significantly more willing to finish their bachelor’s degrees. Supervision: Eighty-five percent of respondents indicated that they have a supervisor who is responsible for providing feedback on their performance. The majority of staff had supervisory meetings at least every three months, with urban-based staff having more meetings. Supervisors ’ activities were heavily biased toward activities that support technical staff development and quality control: supervisors observed the consultation, provided health instructions and asked knowledge assessment questions. Checking records, an administrative task, topped the list of supervisory tasks. There were no important differences between rural and urban facilities. 75% of all respondents stated that they felt the need to discuss difficulties with their supervisor within the last year; most of them (95%) actually discussed the issue with their supervisor and 66% noticed improvements afterwards. Challenges: Low salaries were identified as a problem (with 63% agreeing or strongly agreeing), followed by inadequate opportunities to learn (52%), lack of transport to see patients (50%), inadequate housing (48%), too much work (47%), security problems (39%), lack of supervision (30%), lack of feedback on performance (23%), and lack of motivation (20%). Absence from work: Eight percent of respondents reported being absent from work in the thirty days before the interviews due to sickness. The percentage was the same across all types of medical staff. Five percent of staff was absent in that timeframe because of personal reasons. In the event of an absence, roughly 13% of respondents stated that they were called by the facility head. In 5% of the cases, the supervisor discussed the absence with staff and 2% indicated that money was deducted from their salary as a reaction. Working Environment: Findings from the Health Facility Survey Sixty-nine health facilities in all thirteen districts of Timor-Leste, including six hospitals, thirty-three community health centres (CHCs) and thirty health posts (HPs) were visited. The data suggest that the national and referral hospitals were generally well equipped and that CHCs were moderately equipped. However, most of the HPs were poorly equipped and lacking basic facilities and services. The facilities located in rural areas were also considerably under-resourced compared to facilities in urban areas. The storage conditions of medicines were found to be of concern, especially in rural areas. Service availability was typically high in most of the areas except for treatment of HIV/AIDS, caesarean section and blood transfusion. The HPs offered fewer services than CHCs and hospitals. Health Worker Preferences: Results from the DCE In the DCE the preferences of health workers were explored by giving them the option to choose hypothetical jobs with different attributes. The findings revealed that the ‘probability of specialisation’ was the top-ranked most important factor for doctors. Visits from specialists, the availability of equipment, good housing, working in higher-level facilities and urban locations were the other significant attributes (by order of importance). Newly graduated doctors were totally neutral toward wages. For nurses and midwives, in-service training was valued most highly, followed by transportation, equipment, remote location and housing. © World Bank and Oxford Policy Management v Health Worker Survey in Timor-Leste Older doctors valued wages positively and the same was true for males, while females’ coefficient was negative and strong (-0.003 per dollar). Nurses and midwives valued a higher wage and factors associated positively with it were age, experience, being male, being a nurse (compared to a midwife), being married and owning a house, being in the same district as one’s family, working in a CHC in a rural area, not being provided with housing and having a relatively lower wage and working in a health facility with a better supply of drugs and equipment. Nurses and midwives would be ready to give up US$ 194 per month to access workshops, and US$ 113 to have a motorbike. Having medium scale of equipment availability ranked third place with a value of US$ 102. Competence of Doctors: Results from DCOs and Vignettes In total, 635 cases were observed, mostly in outpatient departments, with 40% in urban and 60% in rural areas. The vignettes focused on patients presenting with cough, fever and diarrhea. Knowledge and practice scores: The overall clinical performance, as measured by DCO, of general practitioners was very good in terms of attitudes (91% score) and moderate at history taking (57%), health education (50%) and treatment accuracy (69%). However, the average physical examination performance score was 28%. Lack of examination tools was suspected to be the one of the reasons they could not perform physical examinations as well as expected; some healthcare facilities did not even have any examination bed, blood pressure monitor (sphygmomanometer) or thermometer. The overall average performance score was 62%. When comparing observations with vignettes (to assess knowledge), there were significant differences in history taking, appropriateness of treatment and health education. The average scores for history taking and health education were higher in the vignette group, suggesting a ‘know–do’ gap. However, scores on the appropriateness of treatment were much better for DCOs than vignettes, which probably means that vignette cases were harder than daily practice cases. One worrying finding for both observations and vignettes was the low rate of hand-washing – slightly above 20% in observed cases and below 10% for simulated vignettes. Absence of facilities cannot explain the low rate in the vignettes. Determinants of clinical performance: HPs had a negative correlation with clinical performance when the clinical performance was distinguished by the type of health facility. As HPs were found to be comparatively poorly equipped, doctors might face difficulties in performing to ideal clinical practice. The location of the practice also had a significant effect on the clinical performance of doctors; doctors working in a rural area were less skilled than their counterparts in urban areas, which again may link to the absence of an enabling work environment. The clinical performance was also better when consultation periods were longer. The average consultation time per case was nine minutes. Doctors spent less time in DCOs (eight minutes) compared to vignettes (ten minutes), which was also statistically significant (p= <0.01). Conducting the vignettes after work may have permitted this longer focus on each case. Conclusion The study highlights some encouraging findings, including the gender balance of health staff (overall, if not within specific cadres), the concentration of doctors in rural areas, the high overall reported satisfaction of staff with their work and high intrinsic motivation, the positive intention to stay in the public sector, the feeling of being well prepared by training for work, the relatively © World Bank and Oxford Policy Management vi Health Worker Survey in Timor-Leste frequent and satisfactory supervisions reported, and the good attitudes to patients identified in observations and vignettes. However, some areas require more investigation and investment. The following recommendations are based on the evidence gathered from this survey: Ensure Policy Compliance at All Levels 1. Service availability should match the Basic Service Package (BSP): The survey found that the health facilities, particularly the rural ones often lacked services and facilities that they are supposed to have according to the BSP. Ensuring policy compliance to BSP will not only maintain the consistency of services across the sector, but will also improve the functionality of the rural health facilities which will eventually encourage health workers to stay motivated to work in rural areas. 2. Availability of medicines should comply with the Essential Drug List (EDL): The drug supplies were frequently inconsistent with the EDL recommendation. Complying with the EDL may improve service delivery by reducing the chance of stock-outs and misuse of drugs. However, it is possible that there is a functioning decentralised mechanism in place, which is effective, but not fully compliant to the central polity. This should be investigated before implementing this recommendation. 3. Clinical guidelines should be enforced: Enforcing clinical guidelines may increase the performance of the doctors. However, there are few related issues. First, changing practice usually requires a lot of effort. Second, since some health providers may lack elements of basic knowledge, the clinical guidelines, which are mostly practice oriented, may not be fully effective. And finally, the clinical guidelines in all relevant topics might not be available or updated. Strengthen the Health Human Resource Policy 4. Ensure salary progression for doctors: The data revealed that the salary difference between the junior and senior doctors was very narrow. This may demotivate experienced health workers and cause them to leave the public sector or engage in dual practice in future. A clear salary progression policy will keep health workers motivated to serve in the public sector and will also encourage better performance. 5. Provide necessary training for health workers: The overall low score on clinical competence and the narrow know-do gap indicates doctor’s lack of knowledge in Timor- Leste, especially in rural areas. Regular in-service training and visits from specialists are highly recommended to increase health worker’s competence. This strategy will also improve the confidence of young health workers and motivate them to stay in rural areas. 6. Improve the effectiveness of supervision: While the supervision visits were frequent, in most cases these visits were mostly focussed on checking official records. More efficient supervision and monitoring tools and exercises are recommended so that these visits are more meaningful and action oriented. 7. Adjust the expectations of health workers on their workload: Nearly half of the respondents reported to have a high workload. However, the average number of patients consulted, as observed during the survey, did not reflect that statement. It might be useful to better understand the reason behind this statement and adjust the workload expectation by establishing clear terms of reference for the work. © World Bank and Oxford Policy Management vii Health Worker Survey in Timor-Leste Improve the Functionality of Health Facilities 8. Improve the pharmaceutical chain of procurement and distribution: The survey results indicate a loosely functioning pharmaceutical chain with an incoherent distribution mechanism. A strong, demand-driven pharmaceutical procurement and disbursement system in which demand will be raised in a bottom-up process along with a sequential distribution mechanism is recommended. 9. Improve availability of medical supplies: The rural health facilities were frequently under-equipped and many health workers expressed this as one of the important attributes in deciding workplace. Improving availability of medical supplies in health facilities will improve the service delivery and at the same time will enhance the rural retention of the health workers. 10. Improve infrastructure and the access to water, electricity and communication: Many rural facilities lacked basic infrastructure including access to water, electricity and communication. Improving the infrastructure will optimise service delivery and rural health worker retention. An inter-sectoral initiative, involving the other concerned ministries, is recommended to identify and prioritize potential development areas, and improve the infrastructure of health facilities, including the accommodation of health workers. Further studies on assessing the training gaps, migration and exit to private sector, absenteeism and dual practice, pharmaceutical distribution, and location preference (urban vs. rural) of health workers will broaden the understanding on some of the survey findings. A follow-up survey, using similar methodology to monitor the trend over time is recommended. This survey highlights some important policy areas that can be addressed to better manage the health workforce in Timor-Leste and ensure better health service to the population, particularly in rural areas. The Ministry of Health can consider prioritising the above mentioned recommendations and group them into short, medium and long term initiatives. © World Bank and Oxford Policy Management viii Health Worker Survey in Timor-Leste Table of Contents Preface i Acknowledgements ii Executive Summary iii Introduction iii Methods iii Findings iii Conclusion vi List of Tables and Figures xi List of Abbreviations xiii 1 Introduction 1 1.1 Background 1 1.2 Health Systems 1 1.3 Health Workforce 2 1.4 Policy Issues 4 1.5 Objectives 5 2 Methods 6 2.1 Components 6 2.2 Sampling 7 2.3 Survey Implementation 8 2.4 Data Analysis 10 2.5 Ethical Considerations 11 2.6 Methodological Limitations 11 3 Labour Market Dynamics: Findings from the Health Worker Survey 13 3.1 Characteristics of the Sampled Health Workers 13 3.2 Study Choice and Long-Term Career Preferences 17 3.3 Workload 21 3.4 Salary and Financial Benefits 23 3.5 Training 26 3.6 Supervision 29 3.7 Challenges 32 3.8 Absence from Work 34 4 Working Environment: Findings from the Health Facility Survey 37 4.1 Characteristics of Health Facilities 37 4.2 Service Availability 38 4.3 Infrastructure and Equipment 39 4.4 Drugs and Supplies 41 5 Health Worker Preferences: Findings from the Discrete Choice Experiment 44 5.1 Setting-up the Experiment 44 Concept 44 5.2 Descriptive Statistics for Doctors, Nurses and Midwives 45 5.2.1 A Bias Against Job A? 48 5.3 Multivariate Analysis for Doctors 49 © World Bank and Oxford Policy Management ix Health Worker Survey in Timor-Leste 5.3.1 Basic Results 49 5.3.2 Multivariate Analysis 52 5.3.3 Modelling 54 5.4 Multivariate Analysis for Nurses and Midwives 55 5.4.1 Basic Results 55 5.4.2 Multivariate Analysis 57 6 Competence of Physicians: Findings from the Direct Clinical Observations and Vignettes 60 6.1 Characteristics of the DCO and Vignette Cases 60 6.2 Doctors’ Characteristics 61 6.3 Clinical Performance 61 6.4 Attitude 63 6.5 History Taking 64 6.6 Physical and Diagnostic Tests 67 6.7 Health Education 70 6.8 Determinant Factors of Clinical Performance 70 7 Conclusions 73 7.1 Policy Implications 73 7.2 Recommendations 76 7.3 Further Studies 77 7.4 Way Forward 78 References 79 Annex A Single Control Regressions for DCE 81 A.1 Single Control Regressions for Doctors 81 A.2 Single control regressions for nurses and midwives 84 Annex B Regression Models of DCO 87 B.1 Regression Model of Total Score Clinical Performance 87 B.2 DCO Regression Model of Total Score Clinical Performance 88 Annex C Additional Information on DCE Analysis 89 © World Bank and Oxford Policy Management x Health Worker Survey in Timor-Leste List of Tables and Figures Table 1: Health Cadres in Timor-Leste ............................................................................................ 2 Table 2: Proposed Sample .............................................................................................................. 7 Table 3: Distribution of Sampled Health Workers ......................................................................... 13 Table 4: Years of Experience in the Medical Sector by Occupational Group ............................... 15 Table 5: Total Experience in the Medical Sector by Facility.......................................................... 16 Table 6: Number of Facilities Worked in During Career by Occupational Group ......................... 16 Table 7: Current Facility by Occupational Group .......................................................................... 17 Table 8: Long-Term Goals by Occupational Group....................................................................... 19 Table 9: Number of Patients per Doctor per Day, by Types of Facility ......................................... 22 Table 10: Number of Benefits Received, by Occupational Group ............................................ 23 Table 11: Kinds of Benefit Received, by Occupational Group .................................................. 24 Table 12: Provision of Supplies for Motorbikes and their Timeliness ....................................... 24 Table 13: Respondent’s Share in Household Income ............................................................... 26 Table 14: Money Available per Household Member After Expenses per Month ...................... 26 Table 15: Number of Short-Term Training Sessions Undertaken in the Last Three Years ...... 26 Table 16: Number of Long-Term Training Sessions Undertaken in the Last Three Years ...... 27 Table 17: Training Sessions Attended in the Last Year, by Locality and Type of Facility ........ 28 Table 18: Supervisor by Occupational Group, Locality and Facility Type ................................ 29 Table 19: Frequency of Meetings with Supervisors .................................................................. 30 Table 20: Health Workers’ Perceived Problems – Average Scores.......................................... 34 Table 21: Number of Urban and Rural Facilities, by Facility Type ............................................ 37 Table 22: Mean and SD of Different Types of Health Worker, by Facility Type ....................... 37 Table 23: Availability of General Services, by Facility Type (%) ............................................... 38 Table 24: Readiness for Disabled People and Gender Equity, by Facility Type ...................... 39 Table 25: Water and Power Supply, by Facility Type................................................................ 40 Table 26: Infection Control Materials in Client Examination Rooms, by Facility Type ............. 41 Table 27: Availability of Antibiotics and Antimalarials, by Facility Type .................................... 42 Table 28: Availability of Supply Items, by Locality ..................................................................... 43 Table 29: DCE Sample .............................................................................................................. 45 Table 30: Proportion of Health Workers Choosing Job A Over Job B, by Choice Set ............. 46 Table 31: Percentage of Job Selection per Level of Attribute: Medical Doctors....................... 47 Table 32: Percentage of Job Selection per Level of Attribute: Nurses and Midwives .............. 48 Table 33: Logit Fixed Effect ....................................................................................................... 49 Table 34: Marginal Rates of Substitution for Doctors ................................................................ 50 Table 35: Urban MRS for Doctors.............................................................................................. 51 Table 36 Control Variable Descriptive Statistics ...................................................................... 52 Table 37: Logit Fixed Effect (Nurses and Midwives) ................................................................. 55 Table 38: MRS for Nurses and Midwives .................................................................................. 56 Table 39 Urban MRS for nurses and midwives ........................................................................ 57 Table 40: Non-Extremely Remote MRS for Nurses and Midwives ........................................... 57 Table 41: Nurses and Midwives: Control Variable Descriptive Statistics .................................. 57 Table 42: Characteristics of the DCO and Vignette Cases ....................................................... 60 Table 43: Patients' Primary Symptoms ...................................................................................... 61 Table 44: Medical Doctors’ Characteristics ............................................................................... 61 Table 45: Different Scores Between DCOs and Vignettes ........................................................ 62 Table 46: DCO Scores ............................................................................................................... 63 Table 47: Vignette Scores .......................................................................................................... 63 Table 48: Scores by Type of Facility.......................................................................................... 70 Table 49: Statistical Association Between Non-Quality and Performance Attributes ............... 71 Table 50: Proportion of Job A Selection, by Type of Health Workers....................................... 90 Table 51: Proportion of Choices That Were Rejected – by Type of Health Workers ............... 90 Table 52: Proportion of HW with Item Dominance .................................................................... 91 Table 53: Regression Results .................................................................................................... 92 Table 54: Regression Results for Nurses and Midwives ........................................................... 93 © World Bank and Oxford Policy Management xi Health Worker Survey in Timor-Leste Figure 1: National Health Service Configuration of Timor-Leste ................................................ 2 Figure 2: Occupational Type by Gender ................................................................................... 14 Figure 3: Occupational Type by Age Group ............................................................................. 14 Figure 4: Types of Medical Staff by Locality ............................................................................. 15 Figure 5: Professional Experience of Respondents, by Locality .............................................. 16 Figure 6: Time Spent in Rural Areas by Occupational Group .................................................. 17 Figure 7: Satisfaction with Current Job ..................................................................................... 18 Figure 8: Long-Term Preference of Doctors for Posting (by Current Work Location) ............. 19 Figure 9: Top Ten Statements on Motivation in Terms of Agreement ..................................... 20 Figure 10: Other Motivations ....................................................................................................... 21 Figure 11: Working Days per Facility Type ................................................................................. 22 Figure 12: Working Hours per Day per Locality .......................................................................... 23 Figure 13: Average Monthly Income by Occupation Type ......................................................... 25 Figure 14: Average Monthly Income by Years of Experience .................................................... 25 Figure 15: Training Sessions Undertaken by Doctors ................................................................ 27 Figure 16: Additional Training Required by Doctors ................................................................... 28 Figure 17: Time of Last Meeting with Supervisor ....................................................................... 30 Figure 18: Supervisor Activities in Rural Areas .......................................................................... 31 Figure 19: Supervisor Activities by Locality ................................................................................ 32 Figure 20: Problems in the Current Job ...................................................................................... 33 Figure 21: Personal Reasons for Absence from Work ............................................................... 35 Figure 22: Availability of General Services, by Locality .............................................................. 39 Figure 23: Communication Facilities, by Facility Type ............................................................... 40 Figure 24: Privacy of Client Examination Rooms, by Facility ..................................................... 41 Figure 25: Storage Condition, by Facility Type ........................................................................... 43 Figure 26: Average Clinical Performance Score by DCO and Vignette ..................................... 62 Figure 27: Doctors Attitude and Communication ........................................................................ 64 Figure 28: Fever-Related History Taking .................................................................................... 65 Figure 29: Cough-Related History Taking................................................................................... 66 Figure 30: Diarrhea-Related History Taking ............................................................................... 67 Figure 31: Physical Examinations ............................................................................................... 67 Figure 32: Fever-Related Physical Examinations and Diagnostic Tests ................................... 68 Figure 33: Cough-Related Physical Examinations and Diagnostic Tests .................................. 69 Figure 34: Diarrhoea-Related Physical Examinations and Diagnostic Tests ............................. 69 Figure 35: Health Education Provided to DCO and Vignette Cases .......................................... 70 Figure 36: Non-Quality vs. Lack of Knowledge........................................................................... 71 Figure 37: Non-Quality vs. Lack of Motivation ............................................................................ 72 Figure 38: Non-Quality vs. Lack of Equipment and Supplies ..................................................... 72 © World Bank and Oxford Policy Management xii Health Worker Survey in Timor-Leste List of Abbreviations CHC Community Health Centre DCE Discrete Choice Experiment DCO Direct Clinical Observation DFAT Department of Foreign Affairs and Trade DHS District Health Service DIT Dili Institute of Technology EC European Commission EmONC Emergency Obstetrics and Newborn Care FGD Focus Group Discussion HCW Health Care Worker HF Health Facility HP Health Post ILO International Labour Organization IMCI Integrated Management of Childhood Illness ISCO International Standard Classification of Occupations KII Key Informant Interview MoH Ministry of Health MRS Marginal Rate of Substitution OPM Oxford Policy Management PPS Probability Proportionate to Size SD Standard Deviation SISCa Servisu Integrado du Saude Comunidade STI Sexually Transmitted Infection TB Tuberculosis ToR Terms of Reference UNTL National University of Timor-Leste WHO World Health Organization © World Bank and Oxford Policy Management xiii Health Worker Survey in Timor-Leste 1 Introduction At the time of independence in 2002, Timor-Leste had a seriously weak health system and human health workforce with only a handful of doctors practicing in the country. In this context, the governments of Timor-Leste signed an agreement and the Cuban Medical Brigade started to train medical students and deploy them in the country, particularly in rural areas. While the initial massive shortage has now been minimised, there are concerns over more complex issues including facility functionality, rural retention, motivation, preferences and competence of the health workers. The objectives of this survey were to understand the labour market dynamics among health workers, to learn more about the preferences and concerns of health workers, and to assess the skills, competence and motivation of doctors. 1.1 Background Timor-Leste had a fragile health system when it became independent in 2002. During the war preceding independence more than 70% of the country’s health facilities were destroyed or seriously damaged and only approximately 20 doctors remained in the country (Cabral, et al., 2013). This radical shortage of health workers gradually started to improve as medical doctors trained in Cuba by the Cuban Medical Brigade began to return to Timor-Leste in 2004 (Asante, et al., 2012). However, there is still a shortage of health workers in all categories, including general doctors, specialists, nurses and midwives. In 2012, the health worker density in Timor-Leste was 1.3 per 1,000 population (Asante, et al., 2014). This ratio was below the threshold of 2.2 which was indicated by WHO as the minimum to have desired level of coverage of skilled birth attended – a proxy indicator widely used to understand the density of health workforce(WHO, 2006). With the recent, large influx of doctors trained by the Cuban Medical Brigade the critical shortage of health workers has lessened, but more complicated issues that are related to human resource management, retention in rural facilities and the quality of health workers have emerged. In addition, there are concerns over the readiness-to-serve and functionality of health facilities. 1.2 Health Systems The people of Timor-Leste have the constitutional right to health, medical care and a healthy environment. According to Article 57 of the Constitution, ‘All Timorese citizens are entitled to health care and the State has a duty to promote and protect this right free of charge, in accordance with its capabilities and in conformity with the law ’ (MoH, 2012). Timor-Leste has a three-tiered government health system, which includes a National Hospital at the tertiary level, district hospitals at the secondary level, and CHCs, HPs and outreach centres known as Servisu Integrado du Saude Comunidade (SISCa) at the primary level (MoH, 2012). According to the information provided by the MoH, there were five district hospitals, 66 CHCs, 205 HPs and 442 SISCas in country at the time of the survey. However, there are reports that not all of the SISCas were fully functioning (Martins & Trevena, 2014). © World Bank and Oxford Policy Management 1 Health Worker Survey in Timor-Leste Figure 1: National Health Service Configuration of Timor-Leste Source: Human Resource for Health, Timor-Leste (MoH, 2012) The National Hospital in Dili is the only tertiary care facility in Timor-Leste. It has inpatient, outpatient and emergency care facilities. It also has specialised services and enhanced laboratory facilities. The district referral hospitals also have outpatient, inpatient and emergency services but these hospitals only provide specialist care in four clinical areas: medicine, surgery, paediatrics and obstetrics-gynaecology. There are thirteen districts in Timor-Leste and five of them have a referral hospital. The CHCs offer outpatient and inpatient care. Some CHCs are larger and have dentistal and laboratory facilities. The HPs are the first level of health facilities and only provide outpatient care. The SISCas are monthly satellite clinics run by health staff from the HPs (MoH, 2012). The MoH plans to deploy three doctors in each CHC and one doctor in each HP (Asante, et al., 2014). The private health system is not very well developed in Timor-Leste. In 2011, there were twenty-six private health facilities in Timor-Leste, which covered about one-quarter of the country’s basic health service delivery, mostly in urban areas (Asante, et al., 2012). Government health workers are officially prohibited from private practice in Timor-Leste. 1.3 Health Workforce There are thirty-six health cadres from eight categories in government health services in Timor- Leste, as shown in Table 1. Table 1: Health Cadres in Timor-Leste Category Sub-category Cadre General practitioners General medical practitioner Medical practitioners Medical specialists Medical specialist Dentists Dentist SI Dental Nurse DIII Dental practitioners Dental nurses and technicians Dental Nurse SPRG Dental Lab Technician DI Pharmacy practitioners Pharmacists Pharmacist DIII © World Bank and Oxford Policy Management 2 Health Worker Survey in Timor-Leste Category Sub-category Cadre Pharmaceutical technicians/ Junior Pharmacy Technician (D1) assistants Pharmacy Technician (SMF) Nurse SPK/DI Nursing professionals Nurse D III Nursing and Midwifery Nurse SI practitioners Midwife DI Midwifery professionals Midwife D III Midwife D IV Public health generalists Public Health Generalist Public health specialists Public Health Specialist Nutritionist D I Nutrition professionals Non-medical public health Nutritionist D III practitioners Environment Health Officer DI Environmental health officers Environment Health Officer DIII Environment Health Officer SI Community health volunteers Promotores Saude Familia (PSF) Radiographer DI Radiographer DIII Medical imaging technicians Radiographer SI Electromedik DIII Medical technologists Laboratory Assistant SMAK/SPK Laboratory Assistant DI Laboratory assistants Laboratory Assistant DIII Laboratory Assistant SI Optometrists Optometrist DI Other health workers Physiotherapists Physiotherapist DIII Health service managers Health Service Manager Health management and Medical records technicians Medical Records Technician support staff Support staff Support staff Source: Human Resource for Health, Timor-Leste (MoH, 2012) Although, the International Standard Classification of Occupations (ISCO) of the International Labour Organization (ILO) categorises health professionals as ‘health professionals’ and ‘health associate professionals’ (ILO, 2008), there are practical challenges and standardisation issues in including all types of professionals under this definition. The health workforce is often defined as the aggregated total of physicians, nurses and midwives. The World Health Report 2006 used the aggregated total of these three professional groups to calculate the density of the health workforce and to compare across countries (WHO, 2006). Doctors, nurses and midwives were considered health workers in this survey. The majority of doctors in Timor-Leste are trained, either in Cuba or Timor-Leste, by the Cuban Medical Brigade (CMB). The CMB started enrolling students in 2004. Until 2007, medical students were trained simultaneously in Cuba and Timor-Leste. Students enrolled in or after 2008 are trained in Timor-Leste by CMB in collaboration with the University of Timor-Leste (UNTL). The program is a six-year Bachelor of Medicine course consisting of five years of medical schooling and one year of internship in the health facilities across the countries. The important characteristics of the Cuban medical training program are a commitment to community spirit and serving the people and the medical doctors sign an agreement with the government to work in public health sector for at least six years. In addition to the CMB-trained doctors there are some doctors who are © World Bank and Oxford Policy Management 3 Health Worker Survey in Timor-Leste trained in other countries, mostly in Indonesia. Most of the nurses and midwives are trained in either UNTL or in the Institute of Health Science (Asante, et al., 2014). 1.4 Policy Issues Lack and quality of infrastructure and limited resources at health facilities are longstanding problems in Timor-Leste. According to a recent report in The Lancet, there is no Magnetic Resonance Imaging machine in Timor-Leste and there is only one Computed Tomography scan machine, which is quite old (McCall, 2014). Facility functionality is worst in rural areas. A recent article reported that only half of the SISCas were functioning (Martins & Trevena, 2014). A scoping review and analysis of published literature concluded that access to and use of health services is one of the top priority areas for Timor-Leste. The review summarised the findings of the survey reports that 12% of households in Timor-Leste did not seek health care when a household member was ill and 25% of households were more than two hours away from their usual health care provider (Deen, et al., 2013). In most developing countries, the retention of health workers in rural areas is a major challenge. Timor-Leste is no exception and it also has some unique features. Various push and pull factors influence health workers’ choices to stay or not stay in rural health facilities. According to the strategic development plan of the Government of Timor-Leste, all HPs will have at least one doctor, two nurses and two midwives by 2020 (Government of Timor-Leste, 2011). Most of the Cuban-trained medical students were enrolled from rural areas and therefore have some incentive to go back to their areas of origin. However, because of the poor functionality of the rural health facilities, doctors may be unwilling to work in those facilities. There are also concerns regarding a lack of supervision and unclear career development paths for those working in rural areas (Asante, et al., 2014). While there is a strong political commitment to deploy health workers in rural areas, there are concerns over the utilisation of the services and the productivity of health workers posted in rural areas. A recent critical appraisal estimated that the patient load at the primary health care level in Timor-Leste was on average less than three persons per day (Cabral, 2010). However, this finding is based on some assumptions and modelling and not backed up by rigorous primary data. Since the literature suggests that the country’s rural facilities lack infrastructure and supplies, management function, supervision and opportunities to learn, there is a possibility that the health workers deployed in these facilities are not motivated. An article published on the tuberculosis (TB) services in Timor-Leste indicated that there was a lack of ownership by government health workers (Martins, et al., 2006). Finally, there are some concerns about the clinical competence and quality of care of the Cuban- trained doctors (Asante, et al., 2012). Although policy-makers believe that the competence of these doctors is adequate for the needs of the local health system, there have been no systematic independent reviews of the doctor’s performance in Timor-Leste (Asante, et al., 2014). A study of the use of medicine and adherence to the clinical guidelines by non-physician health workers (e.g. nurses and midwives) in Timor-Leste was conducted and showed favourable results (Higuchi, et al., 2012). However, there is no evidence about doctor’s clinical performance that could support or assuage concerns regarding the competence of doctors in Timor-Leste. All the above information indicates that there is a general lack of evidence in various aspects of the health labour market in Timor-Leste. This lack of information is mainly caused by the absence of an integrated information system and lack of properly designed research (Cabral, et al., 2013). A need for a properly designed sample survey on health workers in Timor-Leste was therefore © World Bank and Oxford Policy Management 4 Health Worker Survey in Timor-Leste identified, not only to gather evidence on some of the complex issues like motivation, preference and rural retention but also to systematically assess the quality of the doctors working in the country. 1.5 Objectives The primary objectives of this survey were to understand: 1. The labour market dynamics among health workers (especially the prevalence of dual job practice and the inflows/outflows between rural and urban areas); 2. The preferences and concerns of health workers, especially regarding their revenues and rural jobs; and 3. The skills, competence and motivation (i.e. the ‘know–do gap’) of doctors. It is expected that this survey will provide important information and evidence to health policy- makers in Timor-Leste to facilitate better understanding of the incentives and constraints of health workers in the country. © World Bank and Oxford Policy Management 5 Health Worker Survey in Timor-Leste 2 Methods This survey included four components: a health facility survey, a health worker survey, a DCE, and DCOs. A qualitative study was carried out prior to the main study. The health facilities and health workers were sampled using PPS. Three field teams collected the quantitative survey data during July and August 2014. Data were double entered using a customised data-entry program. Associations between various predictors and outcome variables are presented through descriptive statistics and statistical tests Ethical clearance was obtained from both international and national institutional review boards. The important limitations of this study are the absence of data on demand side (i.e. user) perspective and not including the private health sector. 2.1 Components The survey had four main components namely, general health facility survey, general health worker survey, discrete choice experiment (DCE) and direct clinical observations (DCO). A brief description of each of the four methods is outlined below. General Health Facility Survey This component aimed to gather basic information regarding the sampled health care facilities in order to understand the working environment. A structured questionnaire was administered to each of the sampled facilities. The respondents of this survey were the head of the facility, the person in-charge of the facility, or the most senior health worker responsible for client services who was present at the facility at the time of the survey. The questionnaire covered various dimensions of the health workers’ working environment, including inpatient service availability, processing of medical equipment, supplies and storage of medicines, user fees and sources of finance, sources of water and power supply, staff management and external supervision and disability and gender perspectives. General Health Worker Survey Data was gathered on health workers’ demographic characteristics and on various aspects on their profession. The field team members administered a structured questionnaire to the sampled health workers within the health facilities for this component. The general health worker survey questionnaire included modules on job history, preferences and views on the profession, current job, training, supervision and absenteeism. Discrete Choice Experiment (DCE) A DCE was carried out to understand health workers’ preferences. Qualitative research was carried out to identify the job attributes health workers take into account when deciding on rural postings. DCEs draw on Lancaster’s theory of values, which states that each choice is composed of multiple attributes, also known as the choice characteristics of interest (Lancaster, 1966). Assuming that the utility function of health workers follows the random utility theory, the observation of a large number of choices gives the relative preference of one attribute over the other across the sample of respondents, and is as such useful in terms of developing policy © World Bank and Oxford Policy Management 6 Health Worker Survey in Timor-Leste interventions. The DCE then consists in a set of choices between (usually) two options that differ in the value of their attributes. Prior to the main study a qualitative study was conducted to find out the trade-off between the number of attributes and their respective values (one attribute can assume more than one value, e.g. if salary is one attribute, then salary will assume different values in the DCE questionnaire). Based on the findings from the qualitative study, 16 choice sets were prepared and included in the general health worker survey questionnaire. Direct Clinical Observation (DCO) and Vignettes The skills, competence and knowledge of doctors were explored in the DCOs, in order to provide the basis for establishing the ‘know–do gap’. This quality of care survey combined a DCO to evaluate the doctor’s performance in a real clinical setting and vignettes to measure the doctor’s clinical knowledge and skill in an ideal outpatient setting using a standard simulated ‘patient’. Three medical researchers from the University of Gadjah Mada were recruited to administer the DCO and vignette tools. DCOs and vignettes were used for general medical doctors only. In the DCOs the medical researchers silently observed the entire clinical consultation of the patients by sampled doctors, using a structured tool to record the attitude of doctors, history taking, physical examination, diagnosis, treatment and health advice. At the end of the consultation period, for the same sampled doctors, the medical researchers observed three simulated cases (fever, cough and diarrhea) where the enumerators presented themselves as patients. The doctors recorded observations in a structured tool based on the attitude, history taking, physical examination, diagnosis, treatment and health advice of the simulated consultation. 2.2 Sampling The purpose of the sampling exercise was to obtain a sample of health facilities and health workers (in those facilities) with well-documented selection probabilities. The sample was drawn from six strata: two kinds of facilities (urban and rural) and three kinds of health workers (doctors, midwives, nurses). Community health workers, administration and support staff were excluded from the sample. Health facilities and health workers were sampled from all thirteen districts of Timor-Leste. The initial idea was to draw a sample of 480 health workers from 60 clusters, with the cluster size being eight. However, the following practical constraints were noted:  Many health workers were concentrated in a few large urban facilities; and  Many facilities (especially rural HPs) did not have eight health workers in total. To address these constraints the National Hospital and a few other referral hospitals were treated as if they were more than one cluster. In these facilities, more than two health workers of each kind were selected and the sample sizes adjusted accordingly. The target sample was complemented by a reserve sample, to account for contingencies. Table 2: Proposed Sample Location Doctors Nurses Midwives Total Facilities Urban 107 60 66 233 18 Rural 82 82 61 225 50 Total 189 142 127 458 68 © World Bank and Oxford Policy Management 7 Health Worker Survey in Timor-Leste The sampling was conducted using systematic random sampling with PPS. Based on the discussed sampling objectives and constraints, a final proposed sample included 458 health workers in 68 clusters (18 urban and 50 rural), as shown in Table 2. During the data collection 443 health workers were interviewed which means that the response rate was 97%. Additional subsamples were drawn for DCOs and vignettes. One doctor was randomly sampled (from the list of sampled doctors) from each facility for this component. The specialist doctors were excluded for this component, as their clinical skills are not comparable with general doctors. For DCOs, 10 patients were sampled using systematic random sampling during the fieldwork. If the anticipated patient number was less than 10 then all patients were observed. For the vignettes, each sampled doctor was presented with three simulated cases (fever, cough and diarrhoea). Weights Since the samples were drawn from various strata, the weights were used to correctly represent the survey population. The inverse probability was used to calculate the weights of the facility. To obtain weights for the health workers the clustered weight, i.e. the weight of each type of health worker within their cluster, was used. 2.3 Survey Implementation Qualitative Research In order to gain a contextual understanding and to select the attributes for the DCE qualitative research was conducted during February and March 2014. During this study, a series of key informant interviews (KIIs) and focus group discussions (FGDs) were administered. The respondents of the KIIs included the head of Human Resources Planning in the MoH, researchers and academics from the University of Timor-Leste (UNTL), medical students at UNTL and health service directors from three districts. Separate FGDs were carried out with groups of doctors, nurses and midwives. The attributes and the level of attributes that were explored during the qualitative research included location (including living close to family), facility types, supervision, health facility equipment, housing, motorbike, wage, training and workload. At the end of the qualitative study, seven attributes with 18 levels in total for the doctors and seven attributes with 19 levels for nurses and midwives were determined. These attributes and levels were analysed to prepare 16 choice sets for both doctors and nurses. Literature Review The existing literature was reviewed in order to strengthen an understanding on the context and topics. The documents included the Ministry of Health’s policy documents and guidelines, various survey reports and published journal articles. This review helped to tailor the data collection instruments and to compare the results with the recommended or existing policy. Survey Preparation Prior to finalising the design and survey instruments, an inception visit was carried out in April 2014 and various stakeholders were consulted during that visit. During this mission a workshop was © World Bank and Oxford Policy Management 8 Health Worker Survey in Timor-Leste organised at the World Bank office in Dili. Experts and officials from various organisations (including the MoH, the World Bank, UNTL, WHO, USAID, DFAT and EC) attended the workshop and provided comments and feedback on the design and data-collection instrument. The inception mission team also visited different levels of health facilities in one district. After the inception visit a series of activities were initiated including a literature review to gather knowledge from the published literature, sampling, developing a survey protocol, finalising the data-collection instruments, recruitment of medical researchers and field team members, development of data-entry programming and translation of data-collection instruments into Tetun. Most of the preparatory activities were carried out during May 2014. In June 2014, the data-collection instruments were pretested and the tools were finalised. During the latter half of the month the field teams received two weeks of training, which was followed by piloting in a health facility. Data Collection Three field teams were deployed for data collection. Each team was comprised of a supervisor, a medical researcher, two enumerators and a driver. The supervisors were responsible for leading the field teams and administering the health facility questionnaire. The enumerators conducted the interviews with the sampled doctors, nurses and midwives at the health facilities, and the medical researchers from Indonesia administered the DCO and vignette tools. The enumerators acted as ‘patients’ in the vignettes. Data were collected using a ‘paper and pen’ method. Each team interviewed up to eight health workers per day (four per enumerator). Data collection was completed in eight weeks during July and August 2014. Strong quality control procedures were put in place throughout the survey. There was a continuous presence of international consultants throughout the data collection period. They visited the field teams to ensure that data were being collected in accordance with the protocol. The field manager and the supervisors also regularly checked the completed questionnaires to ensure the data quality. Data Management Four customised data-entry programs were developed for the four different datasets (health facility survey, health worker survey including DCE, DCO and vignettes) using Visual Basic Version 6 and Microsoft Access. The data-entry programs had various built-in checks for range, outliers and internal consistency so that only valid responses could be entered. The data management team consisted of one data manager, four data-entry operators and two data editors. The data editors manually checked the questionnaires before they were entered. The data were double entered and any inconsistencies between the first and second entries were resolved by manually checking the questionnaires. Once the data were entered the data files were transferred into the statistical software Stata using the data transfer software StatTransfer. When the data were transferred into Stata, they were checked for missing values, ranges, outliers and internal consistency. © World Bank and Oxford Policy Management 9 Health Worker Survey in Timor-Leste 2.4 Data Analysis General Health Facility and Health Worker Surveys Descriptive statistics were primarily used to present the findings of the general health facility survey and health worker surveys. The statistical software SPPS and Stata were used to generate the survey results. Sampling weights were used to generate representative results. For cross-tabulations the health facilities were mostly categorised by locality (urban and rural) and by type of facility (hospitals, CHCs and HPs). Health workers were categorised by locality (urban and rural) and type (Cuban-trained doctors, non-Cuban-trained doctors, nurses and midwives). Where applicable, two-sample tests of proportions were carried out to check the statistical significance between two groups (e.g. urban vs. rural). DCE DCE analyses the relation between the levels of attributes and the choices made by health workers. Following the random utility theory we estimated a logit/probit model (with a maximum likelihood function) to analyse the dichotomous choices made by the respondents (McFadden, 1976). The relative importance of each coefficient then generated the Marginal Rate of Substitution (MRS) between the different attributes. The basic model for DCE analysis consisted of decomposing the choice made by health workers in factors from the DCE attributes. Following the utility theory backing the DCE approach, the ratio between the coefficients associated with each level of attribute reveals the relative importance of the attributes. The attributes can be included either in their difference or in their level. In this DCE many attributes were not in continuous scale (e.g. in-service training vs. visits from specialists) and so, in those cases, a categorical variable was used. However, for all attributes a base category was always determined as the lowest level of each attribute, i.e. extremely remote, no motorbike, HP as the facility, poor equipment and poor housing. The wage component was left to infer the value per dollar of each attribute (or rather its difference with its base category). The dependent variable was whether the job was selected or not, which took the value one or zero. To fit such a variable a probit model or a logit model was used, estimated by a maximum likelihood algorithm. DCOs and Vignettes In the DCOs and vignettes, the clinical competence of doctors was analysed by assessing their knowledge and skill. The descriptive statistics on doctors’ attitude, history taking, physical examination, treatment accuracy and health education toward real patients (in the DCOs) and simulated cases (in the vignettes) were analysed. The quality-of-care survey combined a DCO to evaluate doctors’ performance in a real clinical setting and vignettes to measure doctors’ c linical knowledge and skill in an ideal outpatient setting using a standard simulated ‘patient’. Thus the DCO tested the skill/performance of doctors and the vignettes measured the knowledge. The difference between these two is the “know-do” gap. In order to create a single score for each observation ( ), the total score was constructed from each section: attitude, history taking, physical examination, therapy appropriateness and health education. The response for each item ( ) is either yes or no and is scored as one or zero, while responses with item scale 1 (well performed/appropriate), 2 (below expectation/somewhat appropriate) and 3 (not performed/not appropriate), score one, 0.5, and 0 respectively. The © World Bank and Oxford Policy Management 10 Health Worker Survey in Timor-Leste average percentage of a score is defined as the score of observation divided by the maximum score observed: ∑ =0 ∑=0 ℎ ∑ =0 ℎ = + + max() max(ℎ ) max(ℎ ) ∑ =0 ℎ ∑ =0 ℎℎ + + max(ℎ ) max(ℎℎ ) Means were compared using an independent t-test and anova to analyse whether scores were significantly different between the variables of observations. In addition, regression analysis was used to explore the determinant factors of doctors’ clinical performance. 2.5 Ethical Considerations Ethical approval was obtained from both international and national institutional review boards. OPM’s Ethical Review Committee and the Human Research Ethics Committee of the Institute of Health of Timor-Leste both reviewed the research protocol and data-collection instruments and approved this study. All research participants provided written informed consent prior to participating in the study. The participation was fully voluntary and participants had the chance to withdraw at any point. Strict anonymity was maintained and all identifying information was removed from the datasets. 2.6 Methodological Limitations There are several limitations to this study; one of the most significant being the lack of demand- side information. The sample only included health facilities and health workers and did not include any patients. Naturally, it would have been possible to gather some important information on the functionality of health facilities and performance of health workers if patients had been included in the study design. To mitigate this limitation the survey methodology and instruments were carefully designed so that the results are not biased based on health workers’ self-reporting on functionality or performance. Private health providers were not included in this survey, which is another important limitation. About one-quarter of the health services in Timor-Leste are provided by private health care providers (Asante, et al., 2014). Including private health providers might have increased the comprehensiveness of the survey. However, the primary goal of this survey was to provide policy- makers with information on government health workers and therefore, private facilities were excluded from this survey. In our understanding the private sector is quite small and in some cases embedded in the public sector (i.e. dual practice) and therefore migration, from the public sector is not a significant issue as yet, particularly because most of the medical doctors have graduated in recent years. However, in future this may appear as another important health policy issue in Timor- Leste. The data on health worker absenteeism should be interpreted with caution, as the teams did not make unannounced visits to health facilities, even though they would have provided more reliable data on absenteeism. Since the primary objective of this survey was not to gather absenteeism data the decision was made to exclude unannounced visits, and avoid any potential delay in data collection. Finally, for the DCOs there were risks of an observer bias, known as the Hawthorne effect, which assumes that the performance of health care workers increases because of the presence of an © World Bank and Oxford Policy Management 11 Health Worker Survey in Timor-Leste external observer (Leonard, 2008). To minimise the risk of this bias, patients were sampled throughout the consulting hours for the sampled doctors. In addition, field team members were encouraged to keep a ‘low profile’ and to ask participants to perform as they would on a normal day. © World Bank and Oxford Policy Management 12 Health Worker Survey in Timor-Leste 3 Labour Market Dynamics: Findings from the Health Worker Survey This section outlines the findings of the general health worker survey that was administered to 443 health workers, including 175 doctors, 150 nurses and 118 midwives, and covered about 20% of the health workers in the country. The data show health workers to be intrinsically motivated. In the long run, the majority of respondents would like to continue to work in the government/public sector, including 99% of the doctors. This is also the case for the broad majority of nurses and midwives, although 6% of nurses would prefer to move to the private sector. Wage differentials within each cadre were relatively small, and income did not vary much by years of experience, particularly for the doctors. Forty-five percent of respondents agreed with the statement that they have ‘too much work to do’. However, there was no clear correlation between workload and feelings of overload. Almost all respondents believed their training adequately prepared them to diagnose and treat clinical cases in Timor-Leste. However, despite all the training opportunities, roughly half (52%) of the respondents stated that ‘there is not enough opportunity to learn current medical knowledge’. Five percent of respondents were absent for personal reasons in the month preceding the interview. Supervision was found to be frequent and appeared to have positive outcomes in general. 3.1 Characteristics of the Sampled Health Workers In the health worker survey, 443 medical staff including 175 doctors, 150 nurses and 118 midwives were interviewed. Since the samples were drawn from various strata, the analysis was weighted according to the total number of different staff types and the locality of their work (urban or rural). Approximately 20% of the health workers in the country were sampled. Table 3 describes the distribution of the sampled health workers compared to the total working in Timor-Leste at the time of the survey. Table 3: Distribution of Sampled Health Workers Number of % within the Total number in sampled health % sampled sample country* workers Doctors 175 40% 612 29% Nurses 150 34% 1,095 14% Midwives 118 27% 540 22% Total 443 100% 2,247 20% Based on data received from the MoH prior to the survey Of the 443 respondents, 178 (40%) were male and 265 (60%) were female. Of them, 242 (55%) were working in urban health facilities and 201 (45%) in rural facilities. Of the 175 doctors, 160 were trained by the Cuban Medical Brigade either in Cuba or in Timor-Leste. The remaining 15 doctors were trained in Indonesia. Roughly 70% of both the nurses and midwives were trained in Timor-Leste and approximately 29% in Indonesia. The remainder of the reported findings are based on analyses of the weighted dataset. © World Bank and Oxford Policy Management 13 Health Worker Survey in Timor-Leste Figure 2: Occupational Type by Gender 100% 80% 36% 56% 58% 60% 100% 40% 64% 20% 44% 42% 0% Total Doctors Nurses Midwives Male Female Figure 2 shows the gender distribution across the different occupational types. The majority (56%) of the health care workers were women, including 58% of the doctors and all of the midwives. The only exception was nurses, where the majority (64%) were men. Figure 3: Occupational Type by Age Group 100% 2% 6% 14% 24% 80% 43% 29% 42% 60% 36% 40% 29% 34% 55% 20% 20% 28% 19% 18% 0% Total Doctors Nurse Midwife 20-29 years 30-39 years 40-49 years 50+ years As shown in Figure 3, the majority of staff were between 25 and 39 years old – the overall average age of all respondents was 37.8 years. The doctors were particularly young: 98% were less than 40 years old and the mean age was 29.5 years. The other medical staff were relatively older: only 18% of the nurses or midwives were below the age of 30 and roughly half of them were 40 years or older. One in four nurses (24%) were above 50 years old. These findings were probably a result of the fact that, in the past, these types of medical personnel were recruited (and trained) in Timor-Leste and remained in the country after independence. However, that is not the case with doctors, who were always scarce in the country. After © World Bank and Oxford Policy Management 14 Health Worker Survey in Timor-Leste independence the majority of the Indonesian-educated doctors left the country, stripping the sector of most of its medical capacity. The medical education of doctors was envisaged to improve the access and quality of health, especially in rural areas. The policy seems to have been successful as close to 53% of all doctors interviewed were posted in rural facilities. Figure 4: Types of Medical Staff by Locality 100% 80% 70% 60% 60% 53% 53% 47% 47% 40% 40% 30% 20% 0% Total Doctors Nurses Midwives Urban Rural Only 30% of nurses were interviewed in rural areas, implying that the majority of them work in urban facilities, while the number of midwives appears to be almost equally split between urban (53%) and rural (47%) areas. This reflects the MoH’s policy to provide better coverage especially for maternity health in rural Timor-Leste. Table 4: Years of Experience in the Medical Sector by Occupational Group Total Doctors Nurses Midwives <5 years 40% 96% 20% 21% 5–9 years 5% 2% 5% 6% 10–14 years 21% 2% 28% 28% 15–19 years 13% 0% 15% 24% ≥20 years 21% 0% 32% 21% Since there were only a few doctors in Timor-Leste at the time of independence, almost the entire cohort of doctors is newly trained – most of the doctors (96%) interviewed in the survey had less than five years of experience in the sector. On the other hand, about half of the nurses and midwives (45%) had 15 or more years of experience in their area of work. © World Bank and Oxford Policy Management 15 Health Worker Survey in Timor-Leste Table 5: Total Experience in the Medical Sector by Facility Referral National Total HPs CHCs Hospitals Hospital < 5 years 40% 62% 33% 38% 46% 5–9 years 5% 6% 6% 3% 2% 10–14 years 21% 18% 25% 11% 20% 15–19 years 13% 6% 11% 22% 17% ≥20 years 21% 8% 25% 26% 15% The level of experience of staff seems to be connected to the placement of staff: 32% of the respondents who work in HPs had more than 10 years of experience (see Table 5). In higher-level facilities (CHCs, referral hospitals or the National Hospital), the number of staff with that level of experience was much higher (61%, 59% and 52%, respectively). It therefore appears that staff with comparatively less experience started their career in HPs. Figure 5: Professional Experience of Respondents, by Locality 100% 21% 21% 80% 10% 15% 60% 13% 26% 6% 40% 3% 49% 20% 34% 0% Urban Rural <5 years 5-9 years 10-14 years 15-19 years >=20 years The survey shows that health workers with more experience got opportunities to advance to higher types of facility. This also explains the findings from Figure 5, which shows that almost half of the medical staff with under five years’ experience (49%) were based in rural Timor-Leste, whereas in urban areas the same category comprised one-third (34%) of the workforce. Table 6: Number of Facilities Worked in During Career by Occupational Group Total Doctors Nurses Midwives One health facility 53% 74% 46% 43% Two health facilities 32% 21% 37% 34% Three health facilities 11% 5% 11% 19% Four or more health facilities 4% 0% 5% 4% © World Bank and Oxford Policy Management 16 Health Worker Survey in Timor-Leste Approximately three out of four doctors (74%) were still working in their first facility (see Table 6). Fewer nurses and midwives were still working in their first workplace (45% and 42%, respectively). Thirty-seven percent of the nurses and 34% of the midwives were working in their second facility and roughly one-quarter (23%) of midwives have worked in three or more facilities during their career to date. This finding is not surprising because, as mentioned above, in general nurses and midwives had longer careers than the doctors in the survey. Figure 6: Time Spent in Rural Areas by Occupational Group 100% 18% 12% 10% 11% 80% 6% 12% 39% 41% 36% 60% 23% 40% 46% 46% 49% 20% 41% 0% Total Doctors Nurses Midwives Nil <3 months 3-6 months 7-11 months ≥12 months A little more than half of the respondents (54%) stated that they had spent some time in rural facilities: 59% of the doctors, 54% of the nurses and 51% of the midwives (see Figure 6). Table 7 shows that 21% of doctors were working in HPs and 46% in CHCs. On the other hand, 8% of nurses and 16% of midwives work in HPs. Table 7: Current Facility by Occupational Group Total Doctors Nurses Midwives HP 13% 21% 8% 16% CHC 55% 46% 61% 54% Hospital 32% 33% 32% 30% 3.2 Study Choice and Long-Term Career Preferences When asked about their motivation to study medicine, almost every respondent (98%) referred to their intention to help people, which speaks of a high level of intrinsic – and not so much monetary – motivation. Four in ten doctors trained in Cuba (38%), received a letter of recommendation from the village head (known as the Chefe de Suco). It was often part of the agreement with the Chefe that these doctors would go back to their suco for work. However, only 44% of the doctors stated they would © World Bank and Oxford Policy Management 17 Health Worker Survey in Timor-Leste go back to their village in the future – this means that the majority of them do not feel obliged to do so and would prefer to go elsewhere. Figure 7: Satisfaction with Current Job Total 49% 48% Total Rural 58% 39% Urban 44% 53% Total 51% 45% Doctors Urban 49% 50% Rural 52% 41% Total 44% 53% Nurses Urban 37% 59% Rural 61% 38% Midwives Total 61% 38% Urban 60% 38% Rural 62% 38% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Very Satisfied Satisfied Unsatisfied Very Unsatisfied Compared to the other types of staff, midwives were by far the most satisfied with their work (see Figure 7): 61% of them consider themselves to be ‘very satisfied’ and an additional 38% to be ‘satisfied’ with their work. However, even though nurses (53%) and doctors (51%) indicate themselves ‘very satisfied’ (the highest level) comparatively less often, generally speaking Timor- Leste’s health workers indicate high levels of satisfaction. Only 4% of the respondents said they were ‘unsatisfied’ or ‘very unsatisfied’ with their work. Doctors in urban areas were more satisfied than doctors working in rural areas: 99% of urban doctors were either ‘very satisfied’ or ‘satisfied’ compared to 93% in rural areas. In contrast, the nurses and midwives working in rural areas were more satisfied: 98% of the nurses and all of the midwives working in rural areas were satisfied compared to 96% and 98% of their urban counterparts respectively. In the long run, the majority of respondents would like to continue to work in the government/public sector, including 99% of the doctors. This was also found to be the case for the broad majority of nurses and midwives, although 6% of the nurses would prefer to move to the private sector. This was especially true of those working in HPs, in urban areas. © World Bank and Oxford Policy Management 18 Health Worker Survey in Timor-Leste Table 8: Long-Term Goals by Occupational Group Total Doctors Nurses Midwives Government/public sector 95% 99% 93% 96% Private sector 4% 1% 6% 2% NGO 0% 0% 0% 1% Others 1% 0% 1% 0% Most doctors have a desire to move to higher tier facilities later in their career (see Figure 8). This hints at the challenge in relation to retaining doctors in rural primary health facilities. Figure 8: Long-Term Preference of Doctors for Posting (by Current Work Location) 100% 96% 90% 80% 60% 60% 37% 40% 31% 30% 27% 25% 25% 24% 20% 16% 10% 8% 6% 0% 1% 2% 2% 3% 0% 2% 1% 1% 0% 0% 0% 2% 2% 0% 0% 0% Total HP CHC Referral Hospital National Hospital HP CHC Referral Hospital National Hospital Others Don't know Figure 8 shows that, in terms of professional mobility, HPs were the least attractive stations for medical staff, with just 6% of doctors citing them as their long-term preferred facility. Thirty percent of the respondents working in HPs would like to stay there in the long run, 27% would like to join a CHC, and 40% would prefer to work in a referral hospital or the National Hospital. While the majority of staff currently working in CHCs (60%) would also like to work there in the future, one in four (25%) would like to move on to a district hospital. Once they have reached a district hospital or the National Hospital, staff seem likely to stay there. More interesting, however, was the finding that very few respondents considered ‘moving back’ to a lower level (i.e. from a CHC to a HP). That is a clear indicator that the work in HPs is considered by most to merely be a starting point in the medical sector, not a place where one might wish to stay. The preference for working in an urban area was further underlined by the fact that half of the doctors would like to work in a district town. An additional 28% would prefer to work in Dili. Only 22% see themselves working in rural sucos. Figure 9 shows that medical staff felt that there was a limit to their control over their career: 85% acknowledged that the MoH will determine where their next assignment would be. The sense of a © World Bank and Oxford Policy Management 19 Health Worker Survey in Timor-Leste lack of control is further exhibited by the 70% ‘agreeing’ or ‘strongly agreeing’ with the statement ‘I am not clear about my career path’. Eight in ten medical staff exhibit a high level of motivation by saying that they would stay in the facility until the last patient is treated – even if they would not receive additional money. Proximity to one’s family was also important (70% believed it is important to work close to where their family are). While only 40% indicated that they could be close to ‘immediate family members’ in rural areas. Figure 9: Top Ten Statements on Motivation in Terms of Agreement I like helping others 99% Opportunity to help the poor 93% My next job will depend where the Ministry of Health wants 85% me to go health staff should be paid more when they work in 84% disadvantaged or very remote I stay in the facility until I have taken care of all patients 82% Always find a job with this qualification 79% More opportunity to take on management responsibility 73% I am not clear about my career paths 71% Very important for me to work close to where my family 70% lives Rural work gives me more exposure to a broad spectrum of 66% diseases 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Figure 10 reveals that a sizeable portion of respondents was not strongly against living in rural areas (42% declared it enjoyable and 47% believed that life in rural areas appeals to them). The sample size did not allow for an in-depth analysis of the underlying drivers of motivation. Nonetheless, it appears from the other analyses that a general career goal is to work in urban areas. Urban health facilities present a better work environment with more career development opportunities. © World Bank and Oxford Policy Management 20 Health Worker Survey in Timor-Leste Figure 10: Other Motivations I chose my profession because other people value it 59% I can see myself working overseas in the future 56% Rural areas are less stressful than working in the city 47% Working in rural areas is better for the career 46% The kind of life you have in rural areas appeals to me 46% The social life in rural areas is enjoyable 42% It is important to me to be able to choose where I go 42% Working in rural areas means I live close to my immediate… 40% Bringing up children in rural areas is difficult 39% Living in a city is stressful 36% Quality of life in rural areas is very good 32% Working in rural areas means no support from… 28% Working in a rural position gives you more financial reward 25% I chose my profession because I can earn a lot of money 22% I will have another job in the next 5-10 years 15% I will have another job in the next 2-3 years 10% I can see myself leaving the medical profession in the future 6% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% The majority of respondents (97%) were not looking for another job in the short run, although the desire to change jobs in the longer term (5–10 years) was found to be slightly higher (8%). This can be explained by staff preferences: the majority of them would like to stay in their current medical area and only very few (4%) have ambitions to leave their clinical position and move into management (1% of doctors and 6% of nurses). 3.3 Workload Approximately one in five doctors (18%) believed they had more patients than they could take care of. On the other side of the spectrum, 19% believed that they could handle ‘a lot more’ patients per day. Around 60% of respondents, however, think they have just enough patients. Among the midwives, 24% indicated that they could handle more patients. It appears that the perceived workload was greater in higher tier facilities. Twenty-six percent of respondents in HPs reported that they could handle more patients, whereas only 3% in the National Hospital agreed with the statement. The number of patients seen by the sampled doctors who participated in a DCO was counted; the mean number of patients per day was 10.2, with an SD of 7.5. In urban facilities the mean (SD) patient load was 11.5 (6.9) compared to 9.6 (7.8) in rural facilities. The number of patients also varied by level of facility, with higher tier facilities having more patients. © World Bank and Oxford Policy Management 21 Health Worker Survey in Timor-Leste Table 9: Number of Patients per Doctor per Day, by Types of Facility Mean SD Minimum Maximum Hospital 13.2 9.2 4 26 CHC 10.9 7.2 2 37 HP 8.5 7.4 1 28 Although most respondents work five days per week in HPs, CHCs and district/regional hospitals (63%, 66% and 54%, respectively), 22% to 36% work six and sometimes seven days a week. Figure 11: Working Days per Facility Type 100% 32% 33% 30% 80% 36% 60% 45% 40% 59% 66% 63% 20% 22% 9% 0% 3% 2% Total Hospital CHC Health Post 1-4 days per week 5 days per week 6-7 days per week In the National Hospital, 36% of respondents stated that they work five days a week and 43% that they work six to seven days per week. Four in 10 staff there reported having one day off from work per week (if at all). This highlights the danger that, in the medium to long term, some medical staff might suffer from exhaustion. © World Bank and Oxford Policy Management 22 Health Worker Survey in Timor-Leste Figure 12: Working Hours per Day per Locality 100% 2% 8% 12% 80% 60% 87% 80% 97% 40% 20% 6% 8% 0% 2% Total Urban Rural <8 hours per day 8 hours per day >8 hours per day Looking at the number of working hours per day, it becomes apparent that working in a rural area usually means an eight-hour workday (97%). In urban areas, on the other hand, the pattern is less clear; 12% of respondents work more than eight hours and another 8% less than that. 3.4 Salary and Financial Benefits All medical staff received their money through a direct deposit to their bank account and few of them (2%) had experienced any delays in receiving their money. Ninety-seven percent of all medical staff surveyed stated that they do not receive any additional money or presents from their patients or from drug companies, although 4% of doctors stated that they had received ‘large gifts’. Only 7 out of 443 respondents (among them only one doctor) stated participating in private practice, while 7% of nurses and 5% of midwives stated that they also work in a non-medical profession. However, these data need to be carefully interpreted as private practice is not legally permitted in Timor-Leste and therefore, some of the respondents may not have accurately reported participation. Table 10: Number of Benefits Received, by Occupational Group Total Doctors Nurses Midwives Hospitals CHCs HPs None 63% 33% 74% 73% 88% 54% 41% One 29% 41% 24% 25% 9% 37% 41% Two 8% 26% 1% 2% 2% 9% 18% Three 1% 0% 1% 0% 2% 0% 0% © World Bank and Oxford Policy Management 23 Health Worker Survey in Timor-Leste Doctors received more non-salary benefits than nurses and midwives. As is shown in Table 10, two-thirds of them receive at least one benefit, while this was only found to be the case for around one-quarter of the nurses and midwives. Table 10 also shows that benefits were more widespread in CHCs and HPs than in hospitals, indicating that there are some existing incentive mechanisms in operation at lower-level rural facilities. Table 11: Kinds of Benefit Received, by Occupational Group Total Doctors Nurses Midwives Hospitals CHCs HPs Housing 17% 36% 6% 20% 11% 15% 38% Housing benefit 2% 4% 1% 1% 4% 1% 0% Motorbike 27% 53% 22% 8% 3% 38% 40% In terms of the type of benefits, 53% of doctors were provided with a motorbike and 36% of them were provided with housing. Again, this is lower for midwives and nurses, which suggests these benefits were specifically created for doctors (see Table 11). Three percent of all staff working in hospitals were provided with a motorbike as opposed to 38% of those working in CHCs and 40% in HPs. It can be inferred that most of these motorbikes are for use in rural areas. Table 12: Provision of Supplies for Motorbikes and their Timeliness Total Doctors Nurses Midwives Got sufficient fuel/ funds for fuel 60% 49% 73% 56% Always on time 23% 23% 26% 0% Usually on time 21% 19% 20% 40% Usually delayed 34% 37% 32% 21% Always delayed 23% 21% 22% 40% When it comes to the maintenance of these motorbikes, however, doctors and midwives were particularly critical. Only half of doctor and midwives reported that they receive either sufficient fuel or funds to buy fuel for the motorbike. More than half of all respondents also stated that these benefits were delayed, with just one in four respondents stating that these were provided ‘always on time’. This could mean that medical staff must pre-finance (if not pay for) the fuel for their work- related trips, which could put a strain on their personal income. On the other hand, most medical staff only have one motorbike which they also ride for personal use. Tracking actual work-related, as opposed to personal use, and the amount of fuel used might be difficult. It would therefore be helpful to ascertain the amount of work-related travel medical staff undertake and provide fuel funds accordingly. © World Bank and Oxford Policy Management 24 Health Worker Survey in Timor-Leste Figure 13: Average Monthly Income by Occupation Type Total $505 Total Rural $514 Urban $500 Doctors Total $622 Rural $609 Urban $637 Total $453 Nurse Rural $452 Urban $454 Total $491 Midwife Rural $479 Urban $502 $400 $450 $500 $550 $600 $650 $700 As shown in Figure 13, doctors had an average income of US$ 622, nurses US$ 453, and midwives US$ 479. With the exception of nurses, incomes were slightly higher for those who worked in urban areas. This could also be attributed to the fact that more senior health workers worked in urban facilities. Figure 14: Average Monthly Income by Years of Experience ≥10 years $657 5-9 years $636 4 years $631 Doctors 3 years $635 2 years $612 1 years $625 <1 year $607 ≥20 years $475 15-19 years $448 Nurse 10-14 years $426 5-9 years $403 <5 years $475 ≥20 years $517 15-19 years $500 Midwife 10-14 years $496 5-9 years $485 <5 years $451 $400 $450 $500 $550 $600 $650 $700 © World Bank and Oxford Policy Management 25 Health Worker Survey in Timor-Leste Figure 14 shows the monthly incomes of health worker category by years of experience. The data indicate that income did not vary much by years of experience, particularly in regard to doctors. On average, a doctor with more than ten years of experience earns only US$ 50 more than a new doctor. This could lead to dissatisfaction regarding wages and benefits among the more long- serving doctors. Table 13: Respondent’s Share in Household Income Total Doctors Nurses Midwives All (100%) 47% 49% 50% 35% 66-99% 14% 9% 15% 18% 50-65% 28% 36% 22% 36% <50% 11% 6% 14% 11% The analysis of the respondents’ share of household income shows that roughly half of the health workers were the sole contributors. That is the case for all occupational types except midwives. In addition, in 42% of the cases health workers provided at least half of the household income. Only one in nine respondents (11%) contributed less than half (see Table 13). Table 14: Money Available per Household Member After Expenses per Month Total All doctors Nurses Midwives Less than US$ 50 26% 2% 47% 34% US$ 50–100 19% 9% 23% 28% US$ 100–200 22% 30% 15% 20% US$ 200 or more 33% 59% 15% 18% Table 14 shows the breakdown of available money per household after all expenses. About 60% of the doctors had more than US$ 200 available per household member, whereas for half of the nurses and one in three midwives the available money amounted to US$ 50 or less. 3.5 Training Almost all respondents believed that their training had well prepared them to diagnose and treat clinical cases in Timor-Leste. Table 15: Number of Short-Term Training Sessions Undertaken in the Last Three Years Total Doctors Nurses Midwives None 28% 37% 24% 25% One short-term training session 10% 11% 9% 9% Two short-term training sessions 17% 17% 19% 13% Three or more short-term training sessions 45% 35% 48% 52% When asked how many short-term training sessions (less than 30 days) they had attended in the last three years, roughly half of the nurses and midwives said they had undertaken three or more (48% and 52%, respectively). On the other hand, one-quarter of them had not attended any such © World Bank and Oxford Policy Management 26 Health Worker Survey in Timor-Leste training in this timeframe. At the same time, roughly one-third (35%) of doctors had attended three or more training sessions and around the same percentage (37%) none at all. Long-term training sessions (30 days or more) had been attended by roughly one-quarter of the doctors in the last three years, a finding that is not surprising given that most of them had just completed their medical training. On the other hand, 41% of nurses and 35% of midwives had undertaken long-term training in the last three years. Table 16: Number of Long-Term Training Sessions Undertaken in the Last Three Years Total Doctors Nurses Midwives None 64% 73% 59% 65% One long-term training session 10% 7% 11% 11% Two long-term training sessions 9% 7% 10% 7% Three or more long-term training sessions 17% 13% 20% 17% Doctors were asked about some specific training sessions and dates of attendance. Figure 15 shows that the majority of doctors (57%) had been trained on TB diagnosis and treatment. However, other training sessions were much less common: only around one in three doctors (36%) were trained in HIV/AIDS care and management and 29% were trained on labour and delivery, of which 5% were trained more than one year ago. Although a large number of doctors are located in rural areas, it is worth noting that only 19% of these doctors have undergone community health training. In addition, only 11% had received training on hypertension, 7% on diabetes and 5% on management. Figure 15: Training Sessions Undertaken by Doctors TB diagnosis and treatment 55% HIV/AIDS care and management 30% Labor and delivery 24% IMCI (Integrated Management of… 27% Malaria 25% Pre/post-natal care 19% Community Health 17% EPI (Expanded Programme of… 14% Mental Health 13% FP (Family Planning) methods 11% EmONC (Emergency Obstetric and… 11% Infection Control 11% Hypertension 9% Diabetes 5% Management training 5% 0% 20% 40% 60% 80% 100% Less than 1 year ago More than 1 year ago © World Bank and Oxford Policy Management 27 Health Worker Survey in Timor-Leste Table 17 details the number of training sessions that respondents received in the last year. One in five respondents (19%) received no training. The majority of these respondents work in rural health facilities. At the other end of the spectrum is a percentage of staff that attended three or more training sessions. Not surprisingly, the respondents were mostly hospital staff (60%). Staff working in rural areas (mostly in CHCs and HPs) received considerably less training than those located in urban areas (and who work in hospitals). Table 17: Training Sessions Attended in the Last Year, by Locality and Type of Facility Locality Facility type Total Urban Rural Hospital CHC HP No training 19% 13% 25% 10% 23% 27% sessions One training 23% 20% 26% 15% 30% 20% session Two training 20% 17% 23% 16% 25% 15% sessions Three or more training 38% 51% 26% 60% 22% 38% sessions Only two of the doctors stated that they would not require any further training, which means that almost every doctor interviewed believed they require additional training for their present job. Figure 16: Additional Training Required by Doctors IMCI 81% 77% EmONC 59% 74% TB diagnosis and treatment 65% 64% HIV/AIDS care and management 63% 63% Malaria 69% 53% FP methods 51% 59% Pre/post-natal care 59% 49% Labor and delivery 59% 46% EPI 49% 55% Hypertension 46% 46% Management training 41% 48% Community Health 44% 44% Infection Control 41% 46% Diabetes 38% 49% Mental Health 43% 39% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rural Urban © World Bank and Oxford Policy Management 28 Health Worker Survey in Timor-Leste As shown in Figure 16, more than three-quarters of doctors indicated that they require training on IMCI and more than 59% on EmONC. Even though 55% of doctors stated that they received training on TB in the last year, roughly two out of three think they require additional training on that subject. Community health, diabetes and mental health were the least desired training topics, nonetheless more than 40% of the doctors requested them. There were only small differences in training requests between doctors in rural and urban areas. As mentioned above, almost every doctor believed their training prepared them to diagnose and treat diseases in Timor-Leste. Interestingly, more than 75% still believed they require more training on IMCI to do their job and more than half believed they require training on malaria. Furthermore, even doctors who had recently undergone training wanted more training in the same subject. These findings may seem paradoxical, but a possible explanation may be that the request for further training relates to subject-specific matters, whereas the overwhelming affirmation to the statement that a doctor’s training prepares them well to diagnose and treat diseases is general in nature. The findings outlined in this section should be followed up to better understand why such a sizeable portion of doctors believe additional training is required for their daily work – and if such training is actually needed or is merely desired by the doctors. 3.6 Supervision The vast majority (85%) of respondents indicated that they have a supervisor who is responsible for providing performance feedback. This proportion is more or less the same across all professional groups, localities and facility levels. While more than half (57%) of the medical staff in rural areas were supervised by the District Health Service (DHS), only 30% of staff located in urban areas were supervised by DHS. In hospitals, specialist physicians supervised 36% of the health workers; 53% of those working in CHCs reported to the District Medical Officer. Staff in HPs was either supervised by the District Medical Officer (41%) or by the Head of the Health Facility (38%). Table 18: Supervisor by Occupational Group, Locality and Facility Type Total Doctors Nurses Midwives Urban Rural Hospitals CHCs HPs Staff from 40% 42% 39% 41% 30% 57% 18% 53% 41% DHS Head of 6% 6% 4% 8% 2% 11% 0% 1% 38% Facility Specialist 13% 14% 12% 16% 19% 4% 36% 3% 4% Physician General 12% 9% 13% 12% 11% 13% 10% 14% 7% Physician Others 29% 29% 31% 22% 37% 16% 36% 29% 11% Roughly eight out of ten respondents (83%) reported that they had meetings with their supervisor at least every three months, with the majority of staff – except for midwives – having monthly meetings. © World Bank and Oxford Policy Management 29 Health Worker Survey in Timor-Leste Table 19: Frequency of Meetings with Supervisors Total Hospitals CHCs HPs Never 2% 3% 2% 2% Every year 9% 6% 11% 9% Every six months 4% 5% 4% 7% Every four months 1% 2% 1% 3% Every three months 27% 13% 33% 35% Six times per year 0% 0% 1% 0% Eight times per year 0% 1% 0% 0% Every month 55% 71% 49% 44% Location plays a role in the frequency of supervisory meetings. Seventy-one percent of respondents located in hospitals had monthly meetings. In other facilities, such as HPs or CHCs, this percentage was reduced by half. Still, despite the challenges in organising these meetings in more rural areas, the majority of staff had supervisory meetings at least every three months. Roughly one in 10 respondents stated that they only had one supervisory meeting per year and roughly 2% reported that they did not have any supervisory meetings. Figure 17: Time of Last Meeting with Supervisor All 71% 9% 11% 9% Doctors 73% 11% 9% 7% Nurses 71% 10% 10% 9% Midwives 70% 5% 13% 12% Working in urban facility 71% 8% 12% 9% Working in rural facility 70% 11% 9% 10% Staff working hospital 71% 12% 10% 8% Staff working CHC 70% 7% 13% 11% Staff working in HP 76% 13% 4% 7% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Within last 30 days Within last 31-90 days Within last 3-6 months More than 6 months This finding is further verified by respondents’ answers to the question of when the last meeting with their supervisor took place: while 71% of all respondents reported that a meeting happened in the preceding 30 days, 9% indicated that a meeting happened more than six months ago. Staff in HPs seemed to meet with their supervisors most often, with 76% of respondents having met a supervisor in the last month. © World Bank and Oxford Policy Management 30 Health Worker Survey in Timor-Leste To understand the quality and contents of the interaction with the supervisor, respondents were asked: ‘what did your main supervisor do during the last supervisory interaction, for example did he or she bring something or ask questions? ’ Respondents were allowed to spontaneously answer the question without category prompts. The responses were recorded with the appropriate code. Only when the respondent had finished speaking were all remaining items read out and the respondent asked whether or not their supervisor participated in the additional activities. These answer types were coded separately. The findings on supervisor activity in rural areas are shown in Figure 18. Figure 18: Supervisor Activities in Rural Areas Checked records 39% 33% Observed consultation 37% 31% Provided health instruction 36% 30% Asked knowledge assessment questions 16% 36% Brought supplies 30% 14% Checked finances 14% 28% Provided administrative instruction 11% 28% Reinforced skills / provided on the job training 4% 30% Provided instruction in filling Health Management 2% 28% Information System (HMIS) form 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Unprompted Prompted In rural areas, respondents highlighted the following supervisory activities: the supervisor checked records, observed consultations and provided health instructions as well as supplies. These thus appear to be supervisors’ main activities. However, only around one in seven (16%) of the respondents spontaneously spoke about the supervisor asking knowledge assessment questions or checking finances (14%). After prompting, the percentage of respondents mentioning these categories rose, however, it can be concluded that these are mostly secondary aims of supervisory visits. The same is true for providing administrative instructions (11%) and, to a much lower extent, checking finances (4%) or giving instructions for filling out the Health Management Information System (2%). Therefore, most supervisory visits in rural areas revolve around the checking of records and observing the (young) doctors at work, with administrative work taking a backseat. To understand whether the type of supervision differs between rural and urban areas, the prompted and unprompted responses were added up. In Figure 19, a similar picture appears across the two areas: © World Bank and Oxford Policy Management 31 Health Worker Survey in Timor-Leste Figure 19: Supervisor Activities by Locality 72% Checked records 79% 68% Observed consultation 77% 66% Provided health instruction 75% 44% Brought supplies 46% 52% Asked knowledge assessment questions 59% 39% Provided administrative instruction 45% 34% Reinforced skills / provided on the job training 57% 42% Checked finances 30% Provided instruction in filling Health Management 30% Information System (HMIS) form 42% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rural Urban It appears that the nature of supervisory visits is by and large the same in rural and urban areas, although ‘checking finances’ seems to be more prevalent in rural areas whereas ‘on the job training’ happens more in urban settings. 3.7 Challenges Figure 20 outlines the reported key problems in the current job as reported by respondents. ‘My salary is too low’ was the statement that a clear majority of health workers agreed with. For roughly half (52%) of the respondents, a key problem appears to be that ‘there is not enough opportunity to learn current medical knowledge’. This issue surfaced earlier (see Section 4.5) and might be connected to how the training that is available is perceived. It may also have something to do with the overall quality of education that respondents received, as they may feel they do not have enough up-to-date medical knowledge. Again, closer follow up on these findings is recommended so as to better understand the drivers behind the sentiments. Inadequate housing and transportation were also issues for roughly half of the respondents. Again, these findings would require follow up to understand the nature of the issues involved and their overall priority, as well as possible solutions should they be required. © World Bank and Oxford Policy Management 32 Health Worker Survey in Timor-Leste Figure 20: Problems in the Current Job My salary is too low 17% 46% 31% 6% I do not have enough opportunity to learn current medical 10% 42% 37% 12% knowledge There is inadequate transportation for me to see my 13% 37% 35% 15% patients The housing provided to me is not adequate 13% 35% 41% 10% I have too much work to do 8% 39% 45% 8% We face security problems in the facility 10% 29% 37% 24% In the facility there is a lack of supervision 5% 25% 55% 15% There is a lack of feedback on performance from my 5% 18% 61% 16% supervisor I do not feel motivated 3% 17% 52% 27% 0% 20% 40% 60% 80% 100% Strongly agree agree disagree Strongly disagree Despite the problems mentioned by medical staff, the statement with the lowest level of agreement was ‘I do not feel motivated’. Nonetheless, around 25% agreed or strongly agreed with this statement. To understand the reasons for this lack of motivation, a correlation analysis was conducted. Although all of the statements except for one1 set out in the list correlate significantly2 with the item, the overall correlation coefficients were very low, with the highest being .271 (‘I have too much work to do’). Therefore, as before, this item should be followed up with staff interviews in order to understand the drivers for this statement. When analysed by staff type, differences in two items were found: on a scale from 1 (strongly agree) to 4 (strongly disagree), nurses and midwives score an average of 2.3 and 2.5 points, respectively, to the question on transportation. Therefore, on average, nurses and midwives were less satisfied with transportation than doctors (score of 2.8).The other issue is with salary: here the midwives’ response was most critical, with an average of 2.0 as compared to doctors’ score of 2.6. 1 That item is ‘We need more staff to take care of all patients ’. 2 This has a significance level of 0.00 on a two-tailed test. © World Bank and Oxford Policy Management 33 Health Worker Survey in Timor-Leste Table 20: Health Workers’ Perceived Problems – Average Scores I do not have enough opportunity to There is inadequate transportation The housing provided to me is not We need more staff to take care of performance from my supervisor We do not have enough supplies We face security problems in the learn current medical knowledge We do not have enough proper In the facility there is a lack of There is a lack of feedback on I have too much work to do for me to see my patients I do not feel motivated My salary is too low and drugs here supervision all patients equipment adequate facility Total 2.9 2.5 2.5 2.5 3 1.6 2.5 1.8 1.8 2.8 2.3 2.7 Hospitals 3.0 2.7 2.9 2.6 3.3 1.6 2.6 2.0 2.0 3.0 2.5 2.7 CHCs 2.9 2.4 2.3 2.5 2.9 1.6 2.5 1.8 1.8 2.8 2.1 2.8 HPs 2.7 2.4 2.5 2.6 3.0 1.6 2.3 1.6 1.6 2.5 2.3 2.5 Doctors 2.9 2.7 2.8 2.6 3.1 1.6 2.6 1.8 1.8 2.8 2.6 2.8 Nurses 2.9 2.5 2.5 2.5 3.0 1.5 2.4 1.9 1.8 2.8 2.0 2.8 Midwives 2.8 2.4 2.3 2.4 3.0 1.5 2.3 1.8 1.8 2.8 2.3 2.7 Urban 2.9 2.6 2.5 2.5 3.1 1.5 2.6 1.8 1.8 2.9 2.2 2.8 Rural 2.9 2.4 2.5 2.6 2.9 1.6 2.5 1.8 1.8 2.7 2.3 2.7 Scale: 1: strongly agree, 2: agree, 3: disagree, 4: strongly disagree Seventy-five percent of all respondents stated that they felt the need to discuss these (or other difficulties) with their supervisor within the last year. Most of them (95%) then actually discussed it with their supervisor, with 66% noticing improvements afterwards. With that backdrop it is not surprising that the majority of respondents (77%) were ‘(very) satisfied’ with the last supervisory meeting and that their motivational effect is also high: 86% indicate that they ‘always’ feel motivated afterwards. Overall, it appears that these sentiments exist across all categories, with no significant differences between the type of staff, locality or the facility type the respondent works in. 3.8 Absence from Work As was mentioned in Section 2.6, unannounced visits to health facilities, such as would have allowed us to capture more reliable data on absenteeism, were not made. In the survey, only 8% of respondents reported being absent from work in the 30 days prior to the interviews because they were sick. The same was true across all types of medical staff. However, more than 33% of doctors stated that they had attended training and were therefore absent from work, although this ratio was lower for nurses and midwives (with 17% and 28% indicating they had attended training). © World Bank and Oxford Policy Management 34 Health Worker Survey in Timor-Leste Figure 21: Personal Reasons for Absence from Work 22% Sick relatives/family 20% 22% I was sick 18% I was asked by my supervisor to pick up 15% salary/drugs/equipment 8% 9% I have another job to attend (paid) 2% 4% I have another job to attend (unpaid) 3% 3% I had household chores 3% 2% Travelling to work took too long 4% 1% I must care for children 4% 0% I was tired from previous day 3% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Rural Urban Most staff said that they had not been absent in the last 30 days due to personal reasons. To clarify the type of personal reasons that result in absence, staff were asked ‘regardless of when that was, why were you away from work for a personal reason ?’ This question was open, meaning that no answer categories were provided. One in five (20%) staff stated that they did not go to work because relatives or family members were sick and roughly the same percentage reported that they were absent when they were sick themselves. In rural areas, another main reason (15%) was being asked by their supervisor to pick something up, like salary, drugs or equipment. That reason was less prevalent in urban areas (8%), possibly because the supply situation is better in urban areas. Other reasons, such as ‘doing household chores’, ‘taking care of children’ or ‘tiredness from the job’, were each mentioned by roughly 3% of respondents. Interestingly, however, 9% of rural doctors mentioned that they were absent because they had to do another paid job at the same time. When they were absent, roughly 13% of respondents stated that their absence resulted in a call from the facility head. In 5% of these cases, the supervisor discussed the absence with the staff member and 2% indicated that money was deducted from their salary as a result. Overall, however, it appears that absence for personal reasons is often not followed up or does not have any consequences. In seven out of ten cases (69%) the respondent does not report any reaction from the supervisor in the event of their absence. © World Bank and Oxford Policy Management 35 Health Worker Survey in Timor-Leste Although it could be argued that due to the nature of rural areas, supervisors might not be aware of absences, there is no evidence of this to be found in the dataset. Indeed, it appears that staff members in the National Hospital were those least likely to be contacted by supervisors in the event of an absence. This finding could mean various things: it could mean supervisors generally do not interfere when staff are absent, that they might not be aware of the staff member’s absence because they do not have much contact with each other, or simply that there is a mechanism in place that makes the supervisor’s follow-up obsolete (e.g. staff call in to report absences). To better understand staff absence and supervisors’ reactions, follow up on these findings is recommended. © World Bank and Oxford Policy Management 36 Health Worker Survey in Timor-Leste 4 Working Environment: Findings from the Health Facility Survey This section provides a snapshot of health facilities and their service availability to assess the working environment of health workers. The field teams visited 69 health facilities in all 13 districts of Timor-Leste, including 6 hospitals, 33 CHCs and 30 HPs. The data suggest that the hospitals were generally well equipped and that CHCs were moderately equipped. However, most of the HPs were still poorly equipped and lacking basic facilities and services. The facilities located in rural areas were also considerably under-resourced compared to facilities in urban areas. Storage conditions for medicines were found to be particularly concerning, especially in rural areas. 4.1 Characteristics of Health Facilities The field teams visited 69 facilities in all 13 districts of Timor-Leste. Of them, 21 (30%) were urban and 48 (70%) were rural. All three types of health facility – i.e. hospitals, community health centres (CHC) and health posts (HP) were visited by the field teams. Approximately one-quarter of the fixed health facilities in the country were visited (see Table 21). It should be noted that the health facilities have been classified as hospitals, CHCs and HPs for brevity: hospitals include the National Hospital in Dili and five referral hospitals; CHCs are the intermediary-level health facilities with or without inpatient services; and HPs are lowest tier health facilities providing outpatient services and are typically found in rural areas. Table 21: Number of Urban and Rural Facilities, by Facility Type Hospitals CHCs HPs Total Urban facilities visited 6 13 2 21 Rural facilities visited 0 20 28 48 Total visited 6 33 30 69 Total facilities in country* 6 66 205 277 % of facilities visited 100% 50% 15%3 25% * Based on data received from the MoH All 6 sampled hospitals, but none of the 30 sampled HPs, had inpatient services. Among the 33 sampled CHCs, 18 (55%) had inpatient services; the remaining 15 (45%) did not. Table 22 presents the number of health workers by type of health facility. The National Hospital was excluded from this table, as the number of health workers is considerably higher than those in other hospitals (the National Hospital had 91 general physicians, 18 specialist physicians, 74 nurses and 61 midwives at the time of the survey). Table 22: Mean and SD of Different Types of Health Worker, by Facility Type Hospitals CHCs HPs Mean SD Mean SD Mean SD 3The proportion of sampled health posts is relatively smaller than the other types of health facilities as the HPs are usually smaller facilities with few staffs in post. Since the health workers were sampled by probability proportion to size (PPS) the larger facilities were more likely to be sampled. © World Bank and Oxford Policy Management 37 Health Worker Survey in Timor-Leste General physicians 9.0 3.2 4.9 5.6 1.1 0.1 Specialist physicians 3.6 2.0 0.1 0.4 0.0 0.0 Nurses 30.5 11.4 7.7 6.1 0.7 0.7 Midwives 11.8 2.2 4.8 4.8 0.8 0.6 As expected, table 22 (above) shows the numbers of all health worker types were higher in hospitals than in CHCs and HPs. The large SD of health workers (especially in CHCs) indicates that health workers were not found to be evenly distributed across the different types of facilities. The results presented above are unweighted as they refer to the distribution of the sample population. Other results presented in subsequent sub-sections are weighted and are representative at the country level. 4.2 Service Availability General service availability was found to be in an acceptable condition. As shown in Table 23, the availability of general services usually followed the MoH’s recommended Basic Service Package (MoH, 2007). Table 23: Availability of General Services, by Facility Type (%) Availability* Hospitals CHCs HPs All Child vaccination services A, B, C 52% 100% 75% 80% Curative care for <5 children A, B, C 100% 100% 93% 94% Family planning services A, B, C 100% 100% 88% 90% Antenatal care A, B, C 100% 100% 93% 94% Malaria – diagnosis/ treatment A, B, C 100% 100% 86% 89% Diagnosis or treatment of STI A, B, C 100% 93% 38% 49% Diagnosis or treatment of TB A, B, C 100% 100% 55% 64% HIV counselling and testing A, B, C 100% 91% 63% 68% Minor surgical services A, B, C 100% 85% 65% 69% Delivery and/or newborn care A, B, C 100% 100% 83% 86% 24-hour staff availability A, B 100% 96% 37% N/A HIV/AIDS treatment A 100% N/A N/A N/A Caesarean section A 100% N/A N/A N/A Blood transfusion A 79% N/A N/A N/A * According to the Basic Service Package 2007, these services are supposed to be available at hospitals (A), CHCs (B) and HPs (C) Only half of referral hospitals reported providing child vaccination services as these services were being provided by lower tier facilities and by SISCas. Blood transfusion services and caesarean section services were only available in hospitals and a few CHCs. The HPs were typically found to provide fewer services. © World Bank and Oxford Policy Management 38 Health Worker Survey in Timor-Leste The availability of general services was consistently higher in urban facilities than in rural facilities, as shown in Figure 22. Figure 22: Availability of General Services, by Locality 100% 92% 100% 100% 99% 100% 92% 100% 100% 100% 100% 95% 100% 87% 86% 90% 82% 80% 74% 70% 59% 61% 60% 50% 53% 46% 50% 42% 40% 34% 30% 21% 20% 3% 6% 3% 10% 1% 0% Urban Rural Most of the CHCs and HPs were not user-friendly for people with disabilities, as shown in Table 24. Table 24: Readiness for Disabled People and Gender Equity, by Facility Type Hospitals CHCs HPs All Ramps available 100% 44% 3% 11% Handrails available 100% 15% 3% 6% Signs for visually impaired patients 39% 4% 3% 3% Guidelines for domestic violence 100% 50% 27% 32% Treatment of domestic violence 100% 80% 46% 53% Maternity leave for staff 100% 98% 98% 98% Table 24 also shows that all of the hospitals have guidelines and treatment provision for domestic violence. However, the CHCs and HPs frequently lack this service. Almost all the facilities provide maternity leave for their female staff. 4.3 Infrastructure and Equipment Access to clean water remains a significant problem, particularly in rural areas, and 18% of health facilities did not have a fixed source of water. Of the facilities that had a primary water source, more than half did not have a water source on their premises. Although all the hospitals had water sources on their premises, more than half of the CHCs and HPs did not (see Table 25). Most of the facilities (61%) had a piped connection, followed by tubewells or boreholes (7%), protected dug wells (6%) and protected springs (5%). Three percent of health facilities were using unsafe water sources, in the form of unprotected wells (2%) and surface water (1%). © World Bank and Oxford Policy Management 39 Health Worker Survey in Timor-Leste Table 25: Water and Power Supply, by Facility Type Hospitals CHCs HPs All Water source on the premises 100% 49% 48% 48% Water source within 500 metres 0% 45% 33% 35% Water source beyond 500 metres 0% 7% 19% 16% Water available at the source 100% 68% 64% 65% Connected to electricity grid 100% 92% 66% 72% Other source of electricity 79% 59% 31% 37% Electricity always available 72% 52% 34% 39% All hospitals and most CHCs had access to electricity, but nearly half had experienced interruption during the seven days preceding the survey and some of them did not have any alternative sources of electricity. Access to electricity among HPs remains an issue. Table 25 also shows that 72% of health facilities were connected to the electricity grid. Facilities that were not connected to the grid had alternative sources for electricity and some were connected to the grid and had an alternative source as back up. Among all facilities, 39% had had uninterrupted electricity during the seven days preceding the survey. Figure 23: Communication Facilities, by Facility Type 2% HP 2% 29% 2% 0% CHC 61% 57% 22% 70% Hospital 100% 100% 100% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Functioning internet Functioning computer Functioning cellular Functioning landline All hospitals were equipped with functioning landlines, cellular phones and computers and 70% had an internet connection. However, a considerable proportion of CHCs and HPs did not have any functioning means of communication (see Figure 23). This has given rise to certain communication challenges and affects the reporting of urgent cases and shortages of medical supplies and pharmaceuticals. On the other hand, the data show cellular coverage to be quite high even for HPs, which offers the potential to use mobile phone technology for inventory monitoring and other communications. The privacy of patients in client examination rooms met the required standards in hospitals. In contrast, 58% of HPs did not have any audio or visual privacy in the client examination rooms (see Figure 24). Forty-five percent of urban health facilities lack privacy in client examination rooms, compared to 56% in rural facilities, which is not a statistically significant difference (p-value=0.49). © World Bank and Oxford Policy Management 40 Health Worker Survey in Timor-Leste Figure 24: Privacy of Client Examination Rooms, by Facility HP 36% 6%0% 58% CHC 48% 6% 7% 39% Hospital 91% 0% 9% 0% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Private Room Audio and Visual Privacy Visual Privacy No Privacy The availability of infection control materials in client examination rooms, by type of material and type of facility, was widely varied. For example, more than 90% of the facilities had alcohol-based hand rub and disposable syringes, but only 7% had eye-protecting goggles. More than half of the facilities did not have any infection control guidelines in the client examination room. Table 26: Infection Control Materials in Client Examination Rooms, by Facility Type Hospitals CHCs HPs All Running water 100% 76% 60% 63% Hand soap 100% 90% 68% 72% Alcohol-based hand rub 100% 92% 91% 91% Waste disposal with lid 100% 78% 41% 49% Other waste disposal 100% 65% 72% 71% Sharps container 100% 94% 88% 89% Disposable gloves 100% 93% 81% 84% Disinfectant 100% 93% 74% 78% Disposable syringes 100% 100% 93% 95% Medical masks 100% 95% 64% 71% Gowns 100% 64% 46% 50% Eye protection 52% 15% 5% 7% Infection control guidelines 79% 50% 45% 46% All the hospitals and CHCs were equipped to sterilise metal equipment, compared to 43% of the HPs – a statistically significant difference (p-value<0.01) 4.4 Drugs and Supplies The availability of antibiotics and antimalarial drugs is shown in Table 27. The table identifies gaps in the availability of drugs according to the recommendations of the 2010 Essential Drug List. The © World Bank and Oxford Policy Management 41 Health Worker Survey in Timor-Leste data showed that the availability of drugs was inconsistent across levels of facilities. For example, none of the HPs should have Ceftriaxone, which is a third-generation injectable cephalosporin drug and should only be administered in the case of severe infections. HPs had significantly fewer of the drugs mandated for all levels. Table 27: Availability of Antibiotics and Antimalarials, by Facility Type Availability* Hospitals CHCs HPs All Amoxicillin tablet A, B, C 100% 72% 54% 58% Amoxicillin syrup A, B, C 100% 100% 66% 73% Ampicillin injection A, B, C 100% 78% 29% 40% Artemisinin-based Combination Therapy A, B, C 41% 56% 49% 51% Azithromycin tablet n/a 44% 64% 24% 32% Azithromycin syrup n/a 24% 14% 0% 3% Benzylpenicillin injection A, B, C 71% 41% 17% 23% Cefixime tablet n/a 44% 48% 5% 14% Ceftriaxone injection A 91% N/A N/A N/A Ciprofloxacin tablet A, B, C 100% 54% 46% 48% Co-trimoxazole tablet A, B, C 100% 78% 90% 88% Co-trimoxazole suspension A, B, C 100% 77% 44% 51% Doxycycline capsule A, B, C 100% 88% 50% 58% Erythromycin tablet A, B, C 100% 76% 64% 67% Erythromycin suspension A, B, C 59% 72% 43% 49% Gentamycin injection A, B 80% 68% N/A N/A Metronidazole tablet A, B, C 100% 94% 61% 68% Metronidazole injection A, B, C 79% 54% 12% 21% Quinine tablets A, B, C 44% 68% 43% 48% Quinine injection A, B, C 50% 13% 2% 4% Tetracycline capsule A, B 12% 26% N/A N/A Tetracycline eye ointment A, B 80% 89% N/A N/A * According to the Essential Drug List 2010, these drugs are supposed to be available at hospitals (A), CHCs (B) and HPs (C). All of the hospitals and CHCs and 93% of the HPs had the facility to store medicines at the health facility; however, the storage conditions of these medicines were not ideal in the majority of the facilities. Based on visual observation by the survey teams, there was evidence of rodents or pests in the medicine storage room at 50% of the health facilities, and in 41% of the facilities the storage room was not well ventilated (see Figure 25). © World Bank and Oxford Policy Management 42 Health Worker Survey in Timor-Leste Figure 25: Storage Condition, by Facility Type 100% 90% 80% 70% 64% 60% 53% 47% 50% 38% 40% 29% 30% 18% 17% 19% 20% 11% 9% 10% 10% 0% 0% Medicines on floor Sun light may fall on Evidence of rodents or Not well ventilated medicine pets Hospital Community Health Centre Health Post Table 28 highlights significance differences between facility levels in terms of the availability of certain medical supply items. There were particular shortages of supply items in HPs. Ten percent of the HPs had oxygen cylinders and available nebulizer machines; peak-flow meters and pulse oximeters in HPs ranged between 5% and 7%. Table 28: Availability of Supply Items, by Locality Hospital CHC HP All Adult scale 100% 98% 97% 97% Child scale 100% 92% 79% 81% Infant scale 100% 94% 56% 64% Stadiometer 100% 93% 88% 89% Thermometer 100% 98% 82% 85% Stethoscope 100% 99% 79% 83% Blood pressure meter 100% 93% 74% 78% Nebulizer machine 100% 61% 7% 18% Peak-flow meter 60% 62% 5% 17% Pulse oximeter 100% 48% 6% 15% Oxygen cylinder 100% 62% 10% 21% Disposable syringes 100% 100% 93% 95% Infusion set 100% 94% 70% 75% Cannula 100% 88% 66% 71% Latex gloves 100% 98% 71% 77% © World Bank and Oxford Policy Management 43 Health Worker Survey in Timor-Leste 5 Health Worker Preferences: Findings from the Discrete Choice Experiment In the discrete choice experiment the survey team explored health worker preferences by giving them the option to choose hypothetical jobs that varied by mixed attributes. The findings revealed that the ‘probability of specialisation’ was the top-ranked most important factor for doctors. Visits from specialists, availability of equipment, good housing, working in higher-level facilities and urban location were the other significant attributes (by order of importance). Newly graduated doctors were totally neutral toward wages. For nurses and midwives, in-service training was valued most highly, followed by transportation, equipment, remote location and housing. 5.1 Setting-up the Experiment Concept Discrete choice experiments draws on Lancaster’s theory of values (1966), which states that each good/option is actually composed of many attributes, or characteristics, which are valuable. Assuming that utility function of health workers follows the random utility theory (McFadden 1974), then, the observation of a large number of choices will give the relative preference of one attribute over the other. The DCE then consists in a set of choices between (usually) two options that differ in the value of their attributes. In this study, the two options were two jobs which had various attributes (characteristics or dimensions) and each of these attributes had different levels. Selection of the Attributes and Levels The attributes of a DCE are the features of each job. A qualitative study was conducted to prepare the attributes and levels for DCE. The qualitative study involved key informant interviews (KII) with national and district level managers, in depth interviews with public health researchers, in depth interviews with medical students at UNTL, focus group discussions (FGDs) with midwives/nurses and FGDs with medical doctors. All the qualitative data was recorded and analysed. The attributes and their levels that were used for the DCE analysis are listed below. Box 1: Attributes and Levels for DCE Doctors Nurses and midwives Location: Urban, Remote, Extremely remote Location: Urban, Remote, Extremely remote Facility type: CHC, HP Facility type: CHC, HP Equipment: Poor, Medium, Good Equipment: Poor, Medium, Good Housing: Poor, Good Housing: Poor, Good Transportation: None, Motorbike Transportation: None, Motorbike Income: $610, $732, $854 Income: $450, $540, $630 Training: None, workshop, specialization Training: None, workshop, bachelor degree Creation of the Experiment The goal in building the choice set was to obtain as much information as possible with the smallest amount of choices possible. Indeed, respondents may become tired and bored quickly and will then put less effort into selecting the best option. A maximum of 16 choices is recommended. In © World Bank and Oxford Policy Management 44 Health Worker Survey in Timor-Leste order to get the most of these observed choices, the lowest variance possible (the D-efficiency), is required. For practical reasons it was decided that all health workers would face the same 16 choices, instead of 72 as would be optimal for the D-efficiency criteria. This reduced efficiency to 97%. In addition, some restrictions were included to avoid unrealistic jobs, such as a CHC in a very remote area. This further lowered efficiency to 93%. The job attribute levels were same in one choice (Job A) of all 16 sets, while the other choice varies in its attributes levels. The creation of all possible sets, as well as the optimisation to find the best combination of 16 choices was made using the macros for SAS. More details on the design of the choice set can be found in Annex C. Analysis of the DCE Data If the Random Utility Theory holds, one can estimate a logit/probit model (with a maximum likelihood function) to analyse the dichotomous choices made by the respondents Choice (0/1)= B*X where X is a vector containing the differences between the level of the attributes of each option. The relative importance of each coefficient will give the Marginal Rate of Substitution between the different attributes. MRS = B1/B2 5.2 Descriptive Statistics for Doctors, Nurses and Midwives The discrete choice experiment (DCE) consisted of 16 job choices that were made by health workers: each choice is called a set. The sets presented to doctors were different from those presented to nurses and midwives in regard to levels and the number of levels within each attribute, but the attributes themselves were the same. Only two health workers did not fill in the DCE section. The sample is composed of 173 doctors, 150 nurses and 118 midwives. Each health worker made 16 choices, so a total of 7,056 choices were made among the 14,112 single jobs proposed to the health workers. This is the level of observation for the DCE analysis. Table 29: DCE Sample Number of health worker Number of sets Number of jobs Doctors 173 2,768 5,536 Nurses 150 2,400 4,800 Midwives 118 1,888 3,776 Total 441 7,056 14,112 The health workers made choices between two hypothetical jobs: job A and job B. They were then asked whether they would refuse both jobs. The table below describes for each set the percentage of health workers selecting job A over job B, as well as whether or not the health workers would refuse both jobs. © World Bank and Oxford Policy Management 45 Health Worker Survey in Timor-Leste Table 30: Proportion of Health Workers Choosing Job A Over Job B, by Choice Set Doctor Nurses Midwives Set Choice A Refuse Choice A Refuse Choice A Refuse 1 37.6% 14.9% 62.7% 62.7% 63.6% 63.6% 2 42.2% 16.8% 55.3% 7.3% 46.6% 16.1% 3 23.7% 16.2% 57.3% 9.3% 58.5% 19.5% 4 34.1% 20.2% 54.0% 12.7% 60.2% 23.7% 5 46.8% 20.8% 48.7% 13.3% 52.5% 22.0% 6 37.6% 23.1% 54.0% 10.0% 55.9% 20.3% 7 51.4% 20.2% 64.0% 15.3% 59.3% 19.5% 8 31.2% 17.3% 56.0% 16.0% 51.7% 16.9% 9 35.8% 13.3% 64.7% 14.0% 50.8% 22.9% 10 41.6% 15.0% 45.3% 16.0% 50.8% 22.9% 11 56.6% 17.3% 58.0% 10.0% 50.8% 22.0% 12 26.6% 16.2% 52.7% 17.3% 48.3% 25.4% 13 48.0% 14.5% 42.0% 13.3% 45.8% 18.6% 14 26.0% 20.8% 52.7% 15.3% 49.2% 13.6% 15 42.2% 15.6% 50.7% 14.7% 53.4% 22.9% 16 42.8% 15.0% 54.7% 12.8% 51.7% 17.8% Total 39.0% 18.8% 54.5% 16.3% 53.1% 23.0% An average of 39% of the medical doctors would choose job A: the minimum is 23.7% for set number 3 and the maximum is around 56.6% for set 11. The nurses and midwives had averages of 54.5 and 53.1 respectively, and their ranges were [42%–65%] and [46%–64%]. The fact that health workers were found to be split over the choices indicates that there was some balance in utility between the jobs proposed and that some trade-offs were made: the data should carry a lot of information. The refusal rate is 19% (of all health workers pooled). The team also verified that individual health workers made trade-offs and did not have monotone preferences: only 3.6% picked only A jobs or B jobs (see Annex C). The ‘both jobs refusal’ rate was also kept in mind for regression analysis (2% would refuse all jobs: see the ‘both jobs refusal’ analysis in Annex C). A typical DCE evaluation is to check whether respondents always selected a response based on the levels of only one attribute. A large proportion of doctors’ choices show dominance in regard to education, which is understandable and not abnormal. The ultimate goal of a DCE is to understand the relation between the levels of attributes and the choice made by health workers. The table below shows the job selection rate when features of the job are present or absent, as well as the correlation between the job selections. If the number of a set was large enough and the array perfectly orthogonal, the percentage of job choice would be a good indicator of the utility derived from that attribute. However, given that the number of sets is small and that a correlation between the levels of all attributes could not be avoided, it is not possible to directly infer utility from this table. Nonetheless, © World Bank and Oxford Policy Management 46 Health Worker Survey in Timor-Leste it is still an indicator of preferences and the reader might be interested in these descriptive statistics. Table 31: Percentage of Job Selection per Level of Attribute: Medical Doctors Corr N of appearance of the % of job selection by with levels of attributes doctor when attribute is choice Job Prese Job A Job B - Absent diff A&B nt Urban 0 8 8 0.04 59.8% 46.7% 13.04% Location Remote 16 3 19 -0.03 44.1% 58.6% -14.40% Very remote 0 5 5 -0.01 56.6% 48.8% 7.81% Poor equipment 0 6 6 0.01 54.6% 48.9% 5.67% Equipment Medium equipment 16 4 20 -0.02 45.2% 58.0% -12.86% Good equipment 0 6 6 0.02 61.5% 47.4% 14.12% No training 0 4 4 -0.03 50.7% 49.9% 0.84% Workshops 16 3 19 -0.08 43.0% 60.2% -17.22% Training Visits from 0 5 5 0.06 61.8% 47.8% 13.94% specialists Higher probability of 0 4 4 0.08 67.8% 47.5% 20.34% specialisation HF type CHC 0 6 6 0.05 62.9% 47.0% 15.93% Housing Good housing 0 8 8 0.04 62.9% 45.7% 17.16% Transport Motorbike 16 8 24 -0.02 46.9% 59.2% -12.25% Wage 1 0 6 6 0.06 60.2% 47.6% 12.55% Wage Wage 2 16 4 20 -0.08 43.0% 61.6% -18.59% Wage 3 0 6 6 0.08 63.0% 47.0% 16.05% Overall, these statistics show what could be expected: better health facilities, educational prospects and housing were more associated with job selection, while the relation with motorbikes, wages and locations was less clear. The results for the nurses and midwives were much less contrasted. The categories low wage level, good housing, no training and bachelor’s degree, good equipment and very remote location were mildly negatively associated with job choice. On the other hand, remote location, medium equipment, workshops, CHC location and better wages were preferred (with robust lower weights for the constant job option). Table 32 shows the percentage of selection of jobs for different levels of attributes for nurses and midwives. © World Bank and Oxford Policy Management 47 Health Worker Survey in Timor-Leste Table 32: Percentage of Job Selection per Level of Attribute: Nurses and Midwives N of appearance of % of job selection by % of job selection by the levels of nurses when attribute midwives when attribute attributes is is Job Prese Absen Prese Absen Job A Job B diff diff A&B nt t nt t Urban 0 6 6 48.0% 50.5% -2.4% 49.1% 50.2% -1.1% Location Remote 16 6 22 52.1% 45.5% 6.6% 50.9% 48.1% 2.8% Very remote 0 4 4 41.7% 51.2% -9.5% 46.6% 50.5% -3.9% Poor 0 6 6 47.8% 50.5% -2.7% 49.7% 50.1% -0.4% equipment Medium Equipment 16 4 20 51.4% 47.7% 3.7% 51.3% 47.8% 3.5% equipment Good 0 6 6 47.6% 50.6% -3.0% 45.9% 50.9% -5.0% equipment No training 0 6 6 45.0% 51.1% -6.1% 46.8% 50.7% -3.9% Workshops 16 4 20 52.6% 45.7% 6.9% 51.7% 47.1% 4.6% Training Bachelor 0 6 6 46.3% 50.9% -4.6% 47.4% 50.6% -3.1% degree HF type CHC 0 7 7 50.7% 49.8% 0.9% 52.0% 49.4% 2.6% Good Housing 0 9 9 49.1% 50.4% -1.3% 47.6% 50.9% -3.3% housing Transport Motorbike 16 7 23 50.9% 47.8% 3.1% 50.1% 49.7% 0.3% Wage 1 0 6 6 42.0% 51.8% -9.8% 45.2% 51.1% -5.9% Wage Wage 2 16 4 20 53.0% 45.0% 8.0% 51.4% 47.7% 3.6% Wage 3 0 6 6 48.0% 50.5% -2.4% 50.3% 49.9% 0.4% 5.2.1 A Bias Against Job A? For the doctors, the striking fact is that the items that were most present (because they are in job A) were usually not preferred. This reflects the fact that only 38% of the doctors picked job A over job B, and that preference for job A had a very high weight in the computation of these means. However, it is puzzling that all the lowest level of the attributes were systematically preferred to the higher levels if the higher levels were in job A: very remote is preferred to remote, poor equipment is preferred to medium equipment, no training at all is preferred to workshops, no motorbikes were preferred to jobs with motorbikes, and jobs with income set at US$ 610 were preferred to jobs with income set at US$ 710.4 When considering explanations other than a genuine utility difference, one can only speculate that there is a bias against job A. However, the jobs seemed balanced and job A reflects very well the situation of many health workers: only the income is a bit higher. It is possible that the doctors 4To investigate a bit further, the same table was created for subsamples : removing the ‘specialization’ dominant items, working with only HP –based doctors or excluding those working in HPs and those who choose only one option. This did not change the results. However, if we keep only job B, or if we give job A a lower weight (1/16), the table is very different and the numbers are more coherent with regard to expectations concerning utilities (see working excel file). © World Bank and Oxford Policy Management 48 Health Worker Survey in Timor-Leste responded strategically; they might have thought that they were going to be given job A, which might be of lower quality than their present position. 5.3 Multivariate Analysis for Doctors 5.3.1 Basic Results The Regression Results The first basic regression had many positive coefficients, as it was decided to keep the less valued category as the base category for the regression. The largest and most positive coefficient is on the higher probability of specialisation, which means that training is the attribute with the largest range between its best level and its lowest level. Beside specialisation, specialist’s visits to the doctor’s facility are also a valued type of training. Good and medium-level equipment were also important to doctors, and they preferred to be in a CHC rather than in a HP. Urban location was preferred to remote and extremely remote, but it seems that an extremely remote location does not carry a different value to a remote location in the eyes of the interviewed doctors. As for the personal advantage preferences of the doctors, good housing was first, while wages and having a motorbike did not seem to be important at all: the coefficients were negative, but non-significantly different from zero. These results are robust to the exclusion of those who systematically picked either job A or job B. Further robustness checks were performed, as described in Annex C. Table 33: Logit Fixed Effect Logit fixed effect Base category Ranking All sample coef se Location: remote -0.092 0.065 11 Location: extremely remote Location urban 0.212*** 0.057 7 Equipment medium 0.250*** 0.053 6 Equipment: poor Equipment good 0.411*** 0.053 3 Training workshops 0.082 0.068 8 Training visits 0.573*** 0.062 No training 2 Training specialist 0.723*** 0.064 1 HF type: CHC 0.321*** 0.048 HF type: HP 5 Housing: good 0.396*** 0.042 Poor housing 4 Transport: motorbike -0.031 0.040 No motorbike 10 Wage -0.000 0.000 NA 9 N 5,536 Note: p-values: 0.01 - ***; 0.05 - **; 0.1 - * Reading the MRS Table The full MRS table can be found below (based on the all-sample regression). The horizontal rows of this table represent the items that are being traded in (compared to their base category), while the vertical columns represent the items that are traded for. The coefficient in each cell gives the quantity of how much the column item needs to be changed in order to get one row item. © World Bank and Oxford Policy Management 49 Health Worker Survey in Timor-Leste A MRS that is equal to -1 between A and B means that you exchange one A for one B. A MRS more negative than -1 indicates that more than one column item needs to be relinquished in order to get one row item. On the contrary, a MRS that is between -1 and 0 means only a portion of the column item is relinquished to get the row item. If the MRS is above 0, the signs of the coefficients are different. In that case, one item is considered as bad and the doctors would have to ‘accept’ that bad item in order to get the row item. Take the second row for example, staff can ask how much doctors are willing to pay in order to go from an extremely remote location (base category) to an urban location. Traded for itself (second column), the MRS is equal to -1, which is logical: they are worth the same. As for the third column, it gives the MRS with good equipment, which has ‘poor equipment’ as the base category. The coefficient is -0.850, which means that the ‘urban’ item is worth 0.85 ‘good equipment’. If we believe the utility theory on which this method is based, we can say that for, the doctors, the utility of a move from an extremely remote location to a urban location equals only 83% of the utility that they would obtain if they had medium-level equipment instead of poor equipment. We can also observe the corresponding reverse MRS: row 3 is medium equipment and its MRS with urban location (column 2) is 1.18: thus, doctors would trade the medium equipment for 1.18 times the benefit they get from an urban location. Table 34: Marginal Rates of Substitution for Doctors Equipment: Equipment: workshops Transport: motorbike specialist Location: Location: Housing: Training: Training: Training: HF type: medium remote urban Wage visits good good Location: remote -1 0.432 0.367 0.223 1.123 0.160 0.127 CHC 0.285 0.231 -3.001 -282.455 Location urban 2.317 -1 -0.850 -0.517 -2.602 -0.370 -0.294 -0.661 -0.536 6.955 654.553 Equipment medium 2.726 -1.176 -1 -0.608 -3.061 -0.436 -0.345 -0.778 -0.630 8.183 770.076 Equipment good 4.486 -1.936 -1.645 -1 -5.037 -0.717 -0.568 -1.280 -1.037 13.463 1266.981 Workshops 0.891 -0.384 -0.327 -0.199 -1 -0.142 -0.113 -0.254 -0.206 2.673 251.551 Training visits 6.259 -2.701 -2.296 -1.395 -7.028 -1 -0.793 -1.786 -1.447 18.785 1767.910 Training specialist 7.892 -3.405 -2.895 -1.759 -8.861 -1.261 -1 -2.252 -1.824 23.685 2229.059 HF type: CHC 3.505 -1.512 -1.285 -0.781 -3.935 -0.560 -0.444 -1 -0.810 10.518 989.896 Housing: good 4.326 -1.867 -1.587 -0.964 -4.857 -0.691 -0.548 -1.234 -1 12.982 1221.788 Transport: motorbike -0.333 0.144 0.122 0.074 0.374 0.053 0.042 0.095 0.077 -1 -94.111 Wage -0.004 0.002 0.001 0.001 0.004 0.001 0.000 0.001 0.001 -0.011 -1 In the regression presented above, it is noteworthy that remote location (compared to extremely remote), having a motorbike (compared to not having one) and wage all have negative coefficients. This is counterintuitive and these items are considered as bad. As a result, the MRSs are positive when compared to items considered desirable. For example, the MRS between motorbike and medium equipment is 0.122: the counterintuitive interpretation of that is that doctors would have to be compensated with 0.12 ‘medium equipment’ in order to accept the ‘bad’ item motorbike. The fact that wage is not valued is problematic. In the literature it is commonplace to evaluate the value © World Bank and Oxford Policy Management 50 Health Worker Survey in Timor-Leste of each item of a DCE by computing its marginal substitution rate with the monetary term: if this one is null or negative, all the MRSs are upside down, since it implies that workers then would pay money to get bad things and need to be paid to have the items they value. The interpretation of these results is therefore rather complicated. Policy Implications Based on this MRS table, policy-makers are able to understand the trade-offs between the various attributes of jobs. If policy-makers want to change health workers’ behaviour using market forces, this reveals exactly the level of other attributes that would make health workers adopt that behaviour. The focus of the survey was to find out which incentives would encourage doctors to move to remote or extremely remote areas. It is therefore good news to see that this coefficient is not very high in the regression. The ‘urban’ row demonstrates that it is worth only 85% of a medium equipment level and only 52% of good equipment. Therefore, improving facility levels and the availability of drugs and medical equipment would be largely enough to compensate for an urban location with a low equipment level. As for training, visits from specialists or a higher probability of specialisation would also incentivise doctors to work in non-urban settings as opposed to urban settings where they would not have such an opportunity. Practicing in a rural CHC would also give a 51% increase in satisfaction, compared to doctors in an urban HP (MRS CHC-urban = -1.51). Good housing would increase satisfaction by 86%. The table below highlights the urban column and adds the statistical significance. The p-value gives the mean estimate of the population that would pick the urban attribute, over the other attribute listed. Providing higher probability of specialisation or visits from specialists would apparently make 99% of the doctors select a non-urban workplace. Good housing is more important for 95% of the doctors. A medium level of equipment would convince two-thirds of the doctors to move to a remote or extremely remote area, while 32.5% would rather stay in an urban area. Table 35: Urban MRS for Doctors Attributes MRS % pick urban Location: remote 0.4315227 99.99% Equipment medium -1.1764915 32.51% Equipment good -1.9356426 3.53% Workshops -0.3843092 97.35% Training visits -2.7009414 0.94% Training specialist -3.4054663 0.35% HF type: CHC -1.5123242 16.92% Housing: good -1.8665987 4.56% Transport: motorbike 0.1437788 100.00% Policy Implications Of course, policy-makers might not want all doctors to move to rural areas. Designing a policy that would provide good training, good facilities and good housing in remote areas would largely exceed the objective and thus create imbalances. Therefore, based on the policy-makers’ © World Bank and Oxford Policy Management 51 Health Worker Survey in Timor-Leste objective, further simulations can be carried out to create incentives that would make a certain percentage of doctors want to opt to move to remote or extremely remote areas. Policy-makers must arbitrate based not only on effectiveness and targeted objectives, but also on cost. The implementation of each attribute should be evaluated so that a cost can be associated with objectives and policy changes. Policy implementation decisions can then be weighed against the costs. Some policies also have system-wide positive externalities (such as good equipment), while others are exclusive to recipients (such as good housing). These externalities should also be considered while designing policy. Moreover, policy-makers can revise current policies based on these findings. For example, motorbikes appear not be valued. The results from this survey seem to indicate that this policy could be discontinued.5 5.3.2 Multivariate Analysis The basic regression does not allow us to distinguish the effect of doctors’ specific characteristics, although this could be very relevant to policy-makers, as some subgroups may be more sensitive to some attributes. Observing the preferences inferred from the DCE based on doctors’ current situation regarding the DCE attributes is also important. A few control variables related to doctors’ personal situation were selected including, their studies, their medical experience and their medical situation (see Table 36). Stated preferences were avoided, as they appear in the same field as the DCE and the preference effect might be diluted between the job choice and the stated preferences. Table 36 Control Variable Descriptive Statistics Variable Min Mean Max sd N Age 24 28.948 53 3.813214 5,504 Female 0 58.38% 1 0.4929695 5,536 Married 0 63.58% 1 0.4812378 5,536 Own house 0 55.49% 1 0.4970203 5,536 Live in different district than family 0 21.97% 1 0.4140491 5,536 Live in other district than urban family 0 9.83% 1 0.2977007 5,536 N years of medical practice 0 1.8902 14 2.233571 5,536 HF type: HP 0 15.61% 1 0.3629539 5,536 HF type: CHC 0 41.04% 1 0.4919516 5,536 HF in rural area 0 39.88% 1 0.4897048 5,536 Ever worked in HP 0 8.09% 1 0.2727443 5,536 Ever worked in CHC 0 27.75% 1 0.4477842 5,536 Ever worked in hospital 0 5.78% 1 0.2333925 5,536 Trained in Cuba 0 92.49% 1 0.2636482 5,536 Received a letter from village authorities 0 41.41% 1 0.4926509 3,168 High or very high satisfaction with job 0 49.71% 1 0.5000368 5,536 5 However, it should be noted that any conclusions drawn depend on the good execution of the experiment and a good understanding from participants of the exercise. It was noticed that many respondents had difficulties decontextualizing their responses and might have thought that they would still keep their motorbike – if they had one. Further modeling could be undertaken to control for the worker’s situation. © World Bank and Oxford Policy Management 52 Health Worker Survey in Timor-Leste Variable Min Mean Max sd N Has housing 0 32.37% 1 0.4679291 5,536 Has good housing 0 20.23% 1 0.40176 5,536 Has a motorbike 0 43.35% 1 0.4956063 5,536 Heavy workload 0 17.34% 1 0.3786359 5,536 Wage 590 625.03 2500 145.7719 5,536 Lacks drugs 0 84.39% 1 0.3629539 5,536 Lacks equipment 0 86.13% 1 0.345694 5,536 Lacks supervision 0 25.43% 1 0.4355262 5,536 Since the econometric framework incorporates the individual doctor’s fixed effects, any control at the health worker level cannot be included. However, the controls as interaction terms with the DCE levels can be included. The coefficient will be interpreted as the extra effect to the base for that specific group. The single control regression results for doctors are included in Annex A. Below a few features are highlighted, by attributes. Location Male doctors expressed the greatest preference for urban locations. Married health workers disregarded extremely remote locations, while non-married health workers would actually prefer to go to extremely remote locations rather than merely remote ones. Doctors owning a house or working in a different district from their family were not willing to go to an urban area. Doctors in HPs would rather not go to urban areas, while those working in a CHC had a strong dislike for extremely remote areas. As for their experience, having worked in a HP did not affect their preferences, but those who had worked in a CHC or a hospital were more inclined to prefer urban areas compared to those who had not. The letter of recommendation that some doctors received from local rural authorities was associated with a preference for remote locations and an aversion to an urban workplace. Doctors reporting themselves satisfied or highly satisfied with their job would not take urban jobs, while those who were unsatisfied showed a very strong preference for urban workplaces. Doctors with housing would prefer to go to urban areas, but those with good housing would actually prefer rural locations. Doctors who have a motorbike would prefer an urban or remote location to an extremely remote one. A heavy workload is associated with indifference between urban, remote or extremely remote workplaces. Workers reporting a lack of drugs and lack of supervision did not value urban locations; they stated remote areas as their preference. Equipment Good or medium-level equipment becomes less important with doctors’ age. Females valued it more than males. It was less valued by doctors in HPs and rural areas and those with a good level of satisfaction with their job. Doctors with higher wages did not stress the importance of equipment as much as lower paid doctors. Those who reported a lack of drugs and lack of equipment did place more value on the quality of the equipment. Training Older doctors tended to be less interested in any kind of training, while females were keener than males to opt for visits from specialists. Workers in HPs and CHCs were more interested in specialisation, but the former were more interested in visits from specialists than their colleagues in CHCs. Workshops, which were not significant in the overall regression, are significant for workers in rural areas. Workers who had ever worked in a hospital were less interested than the other © World Bank and Oxford Policy Management 53 Health Worker Survey in Timor-Leste respondents in additional training. On the contrary, workers trained by the Cuban Medical Brigade showed a significantly higher level of interest in workshops and visits from specialists in the field. Highly satisfied doctors had a lower utility for all kinds of training. Already having good housing or a motorbike increased the utility of all training types. Health Facility Type Female doctors were found to be less keen than males to be working in a CHC. Workers located in HPs and in rural areas were also are less interested in the CHC attribute than their respective base categories. Doctors who felt they had good housing were also more interested in being in a CHC than the others, as do doctors who are better paid. Respondents who worked in facilities that lack drugs and equipment were, however, less interested in going to a CHC. Housing Being female, married and living in a different district than one’s family and working in a CHC were found to be characteristics that make housing more valuable. On the other hand, working in a HP or in a rural area makes it less attractive. Doctors who have any housing value good housing less than those with no housing provided, and those who already have good housing report similar preferences to their base category (i.e. poor housing and no housing). Doctors with higher wages valued housing less, and those in facilities lacking drugs and lack equipment valued it more. Motorbike While a motorbike is not valued in the all-sample regression, it does become valuable for doctors who are married and those who have family in another district, especially if their family is in an urban district. Respondents who have a motorbike reported that they value it very much, as opposed to those who do not have one. This is the strongest relation regarding that feature, although those who evaluated the level of equipment and the level of supervision as significant, also valued motorbikes. Wage Wage was not valued in the main regression. However, the interaction terms revealed that it hides a variety of conflicting opinions: older doctors valued wage positively and the same was true for males, while female doctors’ coefficient was negative and strong (-0.003 per dollar). HP workers and rural workers saw it as a good. Those doctors that received a letter from the local authorities facilitating their entry to the programme and those with heavy workloads also valued wages positively. Doctors with a higher wage also tended to give higher weight to the wage component of jobs than their colleagues with lower wages. 5.3.3 Modelling In order to know what variables are really important, it would have been necessary to include the controls together to ascertain their respective importance. Doing so with interaction implies a regression with the base, as well as 25 controls*11 interactions. Including all these in a maximum likelihood search leads to no convergence and is beyond the scope of this report. © World Bank and Oxford Policy Management 54 Health Worker Survey in Timor-Leste 5.4 Multivariate Analysis for Nurses and Midwives 5.4.1 Basic Results The Regression Results The first basic regression had all positive coefficients, except that a good level of equipment is negative (but not different from 0) and not stronger than a medium level of equipment. Unlike for doctors, the wage had a positive coefficient of 0.0012 per dollar. The difference between the wage levels was US$ 95, which corresponds to a ‘coefficient’ of 0.12, i.e. almost the same as remote location or equipment. Location matters too: a remote area was preferred to both urban and extremely remote locations, but urban was significantly preferred to extremely remote. CHCs, good housing (over poor housing) and motorbike (over no motorbike) also had positive coefficients significantly different from zero. These results are robust to the exclusion of those who systematically picked either job A or job B. By taking into account the fact that many health workers would refuse both jobs, however, the results are slightly changed. This is discussed in the subsection on ‘ Robustness check of nurses/midwives’ in Annex C. Table 37: Logit Fixed Effect (Nurses and Midwives) Logit fixed effect Logit fixed effect All sample Without one choice dominant coef se coef se Location: remote 0.125*** 0.033 0.103*** 0.034 Location urban 0.068** 0.034 0.068* 0.035 Equipment medium 0.124*** 0.030 0.096*** 0.031 Equipment good -0.018 0.029 -0.024 0.029 Training workshops 0.236*** 0.027 0.216*** 0.028 Bachelor’s degree 0.070** 0.029 0.070** 0.029 HF type: CHC 0.102*** 0.026 0.123*** 0.027 Housing: good 0.116*** 0.027 0.131*** 0.028 Transport: motorbike 0.138*** 0.026 0.120*** 0.027 Wage 0.001*** 0.000 0.001*** 0.000 N 8,576 8,160 Note: .01 - ***; .05 - **; .1 - * Reading the MRS Table A description of how to read an MRS table is in Section 5.3. The full MRS table for nurses and midwives can be found below (based on the all-sample regression). The table contains too much information to be described in detail. However, it should be noted that since wage was positively valued by nurses and midwives, the last column indicates the value of each item (compared to their base level). This is much more intuitive than the MRS between non-monetary items. © World Bank and Oxford Policy Management 55 Health Worker Survey in Timor-Leste Table 38: MRS for Nurses and Midwives Equipment: Equipment: workshops Transport: motorbike Location: Location: Housing: Training: Training: bachelor HF type: medium remote degree urban Wage good good CHC Location: remote -1 -1.826 -1.003 6.936 -0.529 -1.783 -1.227 -1.078 -0.905 -102.725 Location urban -0.548 -1 -0.549 3.797 -0.290 -0.976 -0.672 -0.590 -0.496 -56.243 Equipment medium -0.997 -1.821 -1 6.916 -0.528 -1.778 -1.224 -1.075 -0.903 -102.428 Equipment good 0.144 0.263 0.145 -1 0.076 0.257 0.177 0.155 0.131 14.811 Training: Workshops -1.889 -3.450 -1.895 13.103 -1 -3.368 -2.319 -2.037 -1.710 -194.062 Training: Bachelor -0.561 -1.024 -0.562 3.890 -0.297 -1 -0.688 -0.605 -0.508 -57.615 degree HF type: CHC -0.815 -1.488 -0.817 5.650 -0.431 -1.453 -1 -0.878 -0.738 -83.689 Housing: good -0.927 -1.694 -0.930 6.433 -0.491 -1.654 -1.138 -1 -0.840 -95.276 Transport: motorbike -1.105 -2.018 -1.108 7.662 -0.585 -1.970 -1.356 -1.191 -1 -113.476 Wage -0.010 -0.018 -0.010 0.068 -0.005 -0.017 -0.012 -0.010 -0.009 -1 Policy Implications All variables are relevant in some way and could help to design an optimal medical staff compensation package. As the stated interest is to make remote or extremely remote work locations more attractive to health workers, this item will be discussed in more detail. However, it should be stressed that urban location seems to be less desired than remote location, although it is found to be more appreciated that an extremely remote location. The policy question then becomes how to attract nurses and midwives to extremely remote places. According to the model, there are several options in terms of encouraging nurses/midwives to leave urban locations for extremely remote places. The value of an urban location is estimated at a wage difference of US$ 56 per month. Several items have a larger value: for example, nurses and midwives would agree to give up US$ 102 per month in order to secure a medium level of equipment (instead of poor). Having a motorbike or good housing in an extremely remote location would make the extremely remote location more attractive than the urban one (if the urban job does not have a motorbike/good housing). It would be more difficult to incentivize health workers to move from remote locations to extremely remote locations, as its value is found to be much higher at US$ 103 per month, which is about the same value as medium-level equipment compared to poor equipment. Only a motorbike (US$ 113) and workshops (US$ 191) were more valued. Table 39 below gives the percentage of nurses and midwives who would pick an urban job over the other attribute listed. Only 10.49 % of nurses and midwives would pick an urban location while 89% would prefer a remote work. It doesn’t seem that there is a shortage of nurses and midwives keen to work in remote areas: it is the nurses and midwives’ preferr ed work location. Rather, the lack of interest is for extremely remote areas, which would attract only 1 health worker in a thousandth in the absence of corrective policies. The most effective policy to make extremely © World Bank and Oxford Policy Management 56 Health Worker Survey in Timor-Leste remote areas attractive would include workshops (95%), followed by providing a motorbike (78% would go), medium level equipment (73%) and provision of good housing (64%). Table 39 Urban MRS for nurses and midwives Attributes MRS % pick urban Location: remote -1.8264505 10.49% Equipment medium -1.8211667 22.04% Equipment good 0.26333827 99.76% Workshops -3.4504185 8.66% Bachelor degree -1.0243935 48.50% HF type: CHC -1.4879922 30.57% Housing: good -1.6939943 23.37% Transport: motorbike -2.0176035 18.27% Table 40: Non-Extremely Remote MRS for Nurses and Midwives Attributes MRS % pick urban Location: remote -1.3379295 27.138% Equipment medium 0.20828849 99.998% Equipment good -2.3822155 4.269% Workshops -0.66433563 84.047% Bachelor degree -0.96401299 53.109% HF type: CHC -1.1599432 36.368% Housing: good -1.445031 21.070% Transport: motorbike -1.3379295 27.138% 5.4.2 Multivariate Analysis Similar to the analysis with doctors, a few control variables related to nurses and midwives’ personal situation were selected including, studies, medical experience and their medical situation (see Table 41). Their stated preferences were avoided, as they are in the same field as the DCE and the preference effect might be diluted between the job choice and the stated preferences. Table 41: Nurses and Midwives: Control Variable Descriptive Statistics Variable Min Mean Max SD N Age 22 39.7434 67 9.170333 8480 Female 0 0.608209 1 0.4881789 8576 Married 0 0.8544776 1 0.3526473 8576 Own house 0 0.7873134 1 0.4092316 8576 Live in different district to family 0 0.1044776 1 0.305897 8576 Live in other district to urban family 0 0.0335821 1 0.1801614 8576 N years of medical practice 0 14.75 43 9.106919 8576 © World Bank and Oxford Policy Management 57 Health Worker Survey in Timor-Leste Variable Min Mean Max SD N HF type: HP 0 0.1343284 1 0.3410246 8576 HF type: CHC 0 0.608209 1 0.4881789 8576 HF in rural area 0 0.4925373 1 0.4999735 8576 Ever worked in HP 0 0.3246269 1 0.4682626 8576 Ever worked in CHC 0 0.3358209 1 0.4723042 8576 Ever worked in hospital 0 0.1156716 1 0.3198494 8576 High or very high satisfaction with job 0 0.5447761 1 0.4980201 8576 Has housing 0 0.1268657 1 0.3328418 8576 Has good housing 0 0.0671642 1 0.2503207 8576 Has a motorbike 0 0.1865672 1 0.3895864 8576 Heavy workload 0 0.1455224 1 0.3526473 8576 Wage 228 470 610 61.17584 8544 Lack drugs 0 0.8171642 1 0.3865544 8576 Lack equipment 0 0.8097015 1 0.3925595 8576 Lack supervision 0 0.3208955 1 0.4668479 8576 Midwives 0 0.4402985 1 0.4964519 8576 The single control regression results for the nurses and midwives are included in Annex A. These single control regressions can be seen as descriptive statistics reporting the mean preference by groups of health workers. The main features of the regression results are outlined below, by attribute. Location Overall, nurses and midwives preferred remote areas and urban areas to extremely remote locations. This was true especially for those who own a house or are provided with good housing, are married and are already in rural areas. Nurses and midwives that earn more money tended to avoid extremely remote locations. Being in a HP or in a CHC was rather associated with a negative preference for an urban location. Equipment A preference for good or medium-level equipment was positively related with nurses and midwives’ age and medical experience, particularly if they were female and married. CHC workers valued it much more than those assigned to HPs, even though rural health facility staff were keener to report a high value on equipment. Health workers with good personal conditions (i.e. a motorbike/good housing) placed more emphasis on the value of equipment at a job. Conversely, high earners tended to put less value on equipment. Difficult working conditions, however, made equipment more valuable: heavy workload along with a lack of drugs, equipment and supervision resulted in a higher value on the material conditions of the work place. Training Older nurses and midwives, as well as those with more medical experience, tended to be more interested in any kind of training. Females were more interested than males in attending workshops. Nurses put a higher value on training than midwives, who seemed to be indifferent regarding that aspect. Rural facility health workers were significantly more willing to finish their © World Bank and Oxford Policy Management 58 Health Worker Survey in Timor-Leste bachelor degree. Health workers working in a different district than their families did not particularly value training and workshops. High satisfaction rates were associated with a lower level of interest in training. Nurses and midwives that had higher salaries were less interested in bachelor’s degrees. This reflects the fact that they already have a degree, as wage is set based on the number of years passed at nursing school and not on experience or market power. Health Facility Type Nurses, midwives and older, married females preferred working in CHCs. Home owners preferred working in HPs or hospitals over CHCs. There seemed to be a strong effect based on where the respondent currently worked: health workers in HPs favoured the HP facility type over CHCs, while CHC workers preferred CHC facilities. Housing Midwives were less interested in good housing than nurses. Health workers with motorbikes preferred housing significantly more than those without one. Difficult working conditions (i.e. lack of drugs and supervision, heavy workload, lower satisfaction) were correlated with a greater appreciation of housing. Motorbikes Motorbikes were more appreciated by nurses (compared to midwives) that are older, more experienced, married, male, who live in the same district as their family, and are in HPs. Those who have a motorbike already had a very large and positive coefficient, while those who did not have one seemed to be indifferent to that attribute. Wage A higher wage was found to be positive. The factors associated positively with higher wage were age, experience, being male, being a nurse (compared to a midwife), being married and owning a house, being in the same district as family, working in a CHC in a rural area, not being provided with housing and having a relatively lower wage and working in a HF with a better supply of drugs and equipment. © World Bank and Oxford Policy Management 59 Health Worker Survey in Timor-Leste 6 Competence of Physicians: Findings from the Direct Clinical Observations and Vignettes This section outlines the results of the DCOs and vignettes. Medical doctors scored 92% in attitudes, 57% in history taking, 28% in physical examination, 69% in treatment accuracy and 50% in health education. The average total score was 62%. Working in urban health centres and time taken for consultation were two factors that were statistically significant as the determinants of clinical performance. The lack of knowledge of the doctors was significantly associated with non-performance. However, the lack of motivation and lack of equipment and supplies in the health facilities were not statistically significant. 6.1 Characteristics of the DCO and Vignette Cases The field team observed 635 cases, of which 255 (40.31%) were in urban and 379 (59.69%) were in rural areas. In terms of the types of health facility, the team observed 128 case-patients at hospitals, 305 cases at CHCs and 199 cases at HPs (see Table 42). Table 42: Characteristics of the DCO and Vignette Cases Characteristics DCO Vignette Overall n % n % N % Location Urban 187 42.31 69 35.75 256 40.31 Rural 255 57.69 124 64.25 379 59.69 Total 442 100 193 100 635 100.00 Type of health facility Hospital 98 22.17 30 15.54 128 20.16 CHC 217 49.1 88 45.6 305 48.03 HP 127 28.73 72 37.31 199 31.34 Missing 3 1.55 3 0.47 Total 442 100 193 100 635 100.00 Patients who were observed in DCOs were mostly visiting outpatient departments (98%). The rest of the sample was in emergency (0.7%) and in inpatient (0.9%) departments. The primary symptoms of the patients were fever (12%), cough (27%), diarrhoea (5%), and other (56%). In addition, each sampled doctor examined three vignette cases with three chief complaints – fever, cough, and diarrhoea. The team obtained 193 ‘case’ observations in vignettes. © World Bank and Oxford Policy Management 60 Health Worker Survey in Timor-Leste Table 43: Patients' Primary Symptoms Primary symptoms DCO Vignette n % n % Fever 55 12.44 65 33.68 Cough 119 26.92 64 33.16 Diarrhoea 22 4.98 64 33.16 Others 246 55.66 - - 6.2 Doctors’ Characteristics Seventy-one general doctors participated in this component of the study. They were working in three different types of healthcare facility: hospitals (19%), CHCs (42%) and HPs (34%). The health care facilities were located in both urban (38%) and rural (62%) areas of Timor-Leste. The average age of the doctors was 30 years. Table 44: Medical Doctors’ Characteristics Characteristics DCO Vignette Overall n % n % N % Gender Female 41 58.57 36 56.25 77 57.46 Male 29 41.43 27 42.19 56 41.79 Total 70 100 64 100 134 100 Location Urban 26 37.14 23 35.94 49 36.57 Rural 44 62.86 41 64.06 85 63.43 Total 70 100 64 100 134 100 6.3 Clinical Performance The scoring system was based on the criteria described in the observation tool, then analysed into two categories: DCO and vignette. The intent was to compare actual clinical performance and knowledge by using two different methods to assess clinical quality among doctors in Timor-Leste. (Leonard & Masatu, 2005). In this regard, DCOs describe what doctors do in their daily practice, while the purpose of the vignettes was to measure their competency. Figure 26 describes the average clinical performance of all observations. The total score was broken down into five components: attitude, history taking, physical examination, treatment, and health education. The overall clinical performance of the general practitioners was very good in terms of attitude and moderate in regard to history taking, health education and treatment. However, the average physical examination performance score was extremely low compared to other areas. Lack of examination tools was suspected to be the reason for the apparent underperformance in regard to physical examinations. Some health care facilities do not even have any examination beds, blood pressure monitors (sphygmomanometer) or thermometers, making it seemingly impossible to do complete physical examinations. © World Bank and Oxford Policy Management 61 Health Worker Survey in Timor-Leste Figure 26: Average Clinical Performance Score by DCO and Vignette 100% 91% 90% 80% 69% 70% 62% 60% 57% 50% 50% 40% 28% 30% 20% 10% 0% Total score Attitude History taking Physical Treatment Health examination Education A more specific description of the score, based on the method used (DCO or vignette), is below. DCOs and vignettes were compared based on the score of each component one by one. The significant statistical difference in regard to history taking, treatment, and also health education between DCO and vignettes can be seen in Table 45. Table 45: Different Scores Between DCOs and Vignettes Variable DCO Vignette p-value Total score 2.85 3.11 0.0000* Attitude 0.91 0.92 0.6874 History taking 0.54 0.65 0.0000* Physical examination 0.23 0.37 0.0000* Treatment 0.74 0.59 0.0000* Health education 0.46 0.60 0.0000* * Significant The significant difference that was found in the total score indicates that doctors did less with real patients than they did in simulated cases. This means that overall, doctors have better knowledge and skills than they actually demonstrated in their daily clinical practice (Aung, et al., 2012). Contrary to findings in relation to total score, the treatment scores were better for DCOs than for vignettes. There are some possible causes of this unusual result. First, in the vignettes the scenario, diagnosis and appropriate treatment were predetermined, based on the guidelines. In the DCOs, however, the cases varied a great deal. In those cases, the appropriateness of treatment was based on the knowledge of the medical researchers. The expectations regarding the appropriateness of treatment were higher in vignettes compared to DCOs. For example, in the vignette diarrhea case, doctors were expected to prescribe zinc and implement fluid replacement therapy for dehydration; some of the doctors did not meet those criteria. Second, the limited availability of drugs in health facilities was a factor. Doctors could only prescribe the drugs they had available, regardless of the ideal treatment for a particular disease. For example, Promethazine – a first-generation antihistamine – was often prescribed, no matter what the diagnosis was. Respondents said this was because the only drug available was Promethazine. In the vignette, on © World Bank and Oxford Policy Management 62 Health Worker Survey in Timor-Leste the other hand, they were free to prescribe any drugs they thought suitable for the case, irrespective of actual availability. For history taking and health education, better scores were achieved in the vignettes, consistent with the total score mentioned before. The primary reason was that the quantity of time spent on the vignette was longer than they had for DCOs. Vignettes were carried out after doctors had finished their work. With no real patients to see they automatically had more spare time to do the vignettes than the DCOs. Another reason was that, in the vignette, the team members sometimes interrupted and reminded doctors when they forget to do a section (mostly relating to health education). The vignette allowed the team to ascertain their knowledge level and assess their ability. The mean, standard deviation and the range of the DCO and vignettes scores are tabulated in Table 46and Table 47. Table 46: DCO Scores Variable N Mean (%) SD Min (%) Max (%) Total score 351 60.28 10.97 23.65 88.40 Attitude 442 91.03 20.39 - 100.00 History taking 442 53.53 18.00 - 100.00 Physical examination 442 23.40 20.10 - 100.00 Treatment 351 73.81 24.67 - 100.00 Health Education 442 45.64 20.57 - 100.00 Table 47: Vignette Scores Variable N Mean (%) SD Min (%) Max (%) Total score 166 65.67 12.85 31.60 100.00 Attitude 193 91.71 18.01 33.33 100.00 History taking 193 64.55 20.02 14.29 100.00 Physical examination 193 37.11 25.45 - 100.00 Treatment 166 59.08 27.75 - 100.00 Health Education 193 59.81 17.44 18.75 87.50 6.4 Attitude The attitude of doctors was generally very good. Figure 27 shows the comparative score on the attitude of the doctors towards patients. In more than 80% of cases they welcomed and greeted the patients. They were very polite to their patients, made eye contact and were empathetic. Almost all of the patients had a chair to sit on. In this respect, there was no signifficant difference between vignettes and DCOs. Thus, doctors were found to treat their patients appropriately, both in real and in simulated cases. © World Bank and Oxford Policy Management 63 Health Worker Survey in Timor-Leste Figure 27: Doctors Attitude and Communication 100% Chair to sit on 98% Rapport building (empathy, eye 92% contact) 93% 83% Welcome and greet the patient 82% 0% 20% 40% 60% 80% 100% Vignette DCO 6.5 History Taking Generally, doctors’ history taking performance was better in vignette cases, although there were some questions that they asked more frequently in DCOs. The questions in the DCOs related to the presence of a cough, sore throat and pain during swallowing, as well as social history inquiries. In order to properly understand the DCO and vignette result, it is important to know that the most febrile cases in DCO were common cold cases, which related to upper respiratory tract infection. Whereas, the fever case in the vignette was malaria. These two conditions have different signs and symptoms; in other words, they have very different clinical presentations. Thus, in the DCOs doctors frequently asked questions about upper respiratory tract symptoms (i.e. cough, sore throat, pain during swallowing) compared to the vignettes. In addition, in the DCOs they also asked about social history, especially food that the patients consumed before the onset of the illness. © World Bank and Oxford Policy Management 64 Health Worker Survey in Timor-Leste Figure 28: Fever-Related History Taking Social history (food, lifestyle, etc.) Family medical history Past medical history Past treatment history Presence of stomachache Presence of blood/ dark urine Presence of pain during urinating General malaise Presence of headache Presence of convulsions Presence of nausea and vomiting Presence of loss of appetite Presence of diarrhea Presence of cough and sore throat Presence of chills, sweats Fever pattern Duration of fever 0% 20% 40% 60% 80% 100% Vignette DCO The figure hightlights that almost all of the medical doctors asked about the duration of fever both in the DCOs and vignettes. But some of the other key questions on the history taking of a febrile case wwere not frequently asked, specially in DCO cases. © World Bank and Oxford Policy Management 65 Health Worker Survey in Timor-Leste Figure 29: Cough-Related History Taking Social history (food, lifestyle etc.) Family medical history Past medical history Past treatment history Presence of convulsions Presence of weight loss Loss of appetite Presence of night sweats Presence of fever Presence of difficulty in breathing Presence of chest pain Presence of blood in sputum Sputum production or dry cough Duration of cough 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO Doctors’ performance on cough-related history taking was better in the vignettes than the DCOs. However, the vignette case for cough was pulmonary TB, in contrast to the DCOs which were dominated by common cold symptoms. This explains why specific questions for TB were asked more frequently in vignettes than in DCOs (e.g. on weight loss and night sweats). This is similiar to the fever-related history taking. © World Bank and Oxford Policy Management 66 Health Worker Survey in Timor-Leste Figure 30: Diarrhea-Related History Taking Social history (food, lifestyle, etc.) Family medical history Past medical history Past treatment history Loss of appetite Presence of fever Presence of vomiting Presence of blood and/or mucus in stools Consistency of stools Frequency of stools Duration of stools 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO The history taking performace in regard to diarrhea was similar for DCOs and vignettes, which means doctors used their knowledge well. Nearly all of the doctors asked about the duration, frequency and consistency of the stools. The least asked questions in this section related to family medical history, past medical history, and whether the patients had already received treatment for the diarrhoea or not. 6.6 Physical and Diagnostic Tests Figure 31: Physical Examinations Check respiratory rate Checks blood pressure Count heart rate/ pulse Checks temperature with thermometer Hand washing before physical examination Perform any physical exam 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO Figure 31 illustrates that almost all doctors performed at least one physical examination. However, very few medical doctors performed all four vital sign checks (respiratory rate, blood pressure, heart rate, temperature). As was mentioned before, many facilities were not equipped with © World Bank and Oxford Policy Management 67 Health Worker Survey in Timor-Leste important diagnostic tools such as thermometers and blood pressure monitors. In the vignette, doctors could simulate any examination they felt was needed, even when there were no actual tools to use. For example, when they wanted to check the patient’s temperature but did not have a thermometer they could substitute a pencil for a thermometer. This is probably why the temperature checking rate was higher in vignettes than in DCOs. The other important finding relates to hand-washing. The hand-washing rate in DCOs was slightly above 20% and for vignettes it was below 10%. Doctors were expected to wash their hands before and after the physical examination as it is a precautionary routine procedure in clinical practice. However, while some facilities had handrub or handsoap and tap water other facilities did not, meaning doctors cannot always perform hand-washing. Figure 32: Fever-Related Physical Examinations and Diagnostic Tests Routine blood test Urine test Hemoglobin test Widal test Malaria test Look for puffy face and/ or swelling of the feet Examine for stiffness in his/ her neck Pinch skin of patient Examine the responsiveness Inspection for sunkes eyes Examine tounge Rumple leed test Abdominal examination Checks ear/throat Examine eyes for anemia and icteric 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO Figure 32 shows doctors’ physical examinations and diagnostic tests on fever cases in both DCOs and vignettes. The percentage of doctors who adviced to carry out any relavent physical examiation was below 20%. No routine diagnostic tests were advised in the DCOs. However, in about 30% of vignettes, medical doctors adviced for a malaria test. No medical doctors did a widal test in any of the DCOs or vignettes to rule out typhoid fever. © World Bank and Oxford Policy Management 68 Health Worker Survey in Timor-Leste Figure 33: Cough-Related Physical Examinations and Diagnostic Tests Chest rediograph Sputum microscopic test Examine throat Check for swelling/deformity in bones/joints Check for abdominal swelling Check for enlarged lymph nodes Auscultate the chest Chest percussion Chest palpation Chest inspection 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO Figure 33 shows doctors’ physical examinations and diagnostic tests on cases presenting a cough. In general, below 40% of cases were diagnosed using a physical examination. The diagnostic tests were more prevalent in vignettes than DCOs. As the vignette case was TB, just above 30% of doctors performed a sputum microscopic test, which is the main confirmation test for TB. Figure 34: Diarrhoea-Related Physical Examinations and Diagnostic Tests Stool test Give a drink to the baby to check thirst Took weight Pinches abdominal skin Examine for sunken front fontanel / eyes Assesses general status (alert or lethargic) 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO Figure 34 shows that in diarrhoea cases, all of the key main physical examinations and diagnostic tests were more common in vignettes as compared to DCOs. No stool samples were taken in DCOs, but just above 15% of doctors performed a stool sample to confirm diagnosis in the vignettes. © World Bank and Oxford Policy Management 69 Health Worker Survey in Timor-Leste 6.7 Health Education Figure 35 shows the health education provided by the medical doctors to the DCO and vignette cases. Figure 35: Health Education Provided to DCO and Vignette Cases Explained the result, ensure patient had understood diagnosis Ability to build a good relationship Explained whether or not to return for further treatment Gave any health education related to diagnosis (e.g. prevention) Explained the treatment being provided Informed the patient about the diagnosis and explained it 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Vignette DCO More than 80% of doctors explained the diagnosis, related appropriate health education, and built a good realtionship with their patients. Overall the vignette cases were better than the DCOs, as vignettes allowed doctors more time to communicate with their patients. 6.8 Determinant Factors of Clinical Performance When clinical performance was distinguished by type of health facility, HPs had a negative correlation with clinical performance. As discussed earlier in this report (Section 3), HPs are poorly equipped. Therefore, doctors might face difficulties reaching the standards of ideal clinical practice. Since CHCs and hospitals are much better equipped than HPs, they might allow better performance than HPs. Table 48: Scores by Type of Facility Variable Hospital CHC HP p-value Total score 3.09 2.95 2.83 0.0010 * Attitude 0.96 0.91 0.89 0.0970 History taking 0.62 0.57 0.54 0.002** Physical examination 0.24 0.30 0.26 0.0590 Treatment 0.77 0.70 0.64 0.0019 *** Health Education 0.53 0.48 0.50 0.1361 * Kruskal-Wallis test; post hoc Mann-Whitney : Hospital – CHC, p= 0.0587; hospital – HP, p = 0.0002; CHC- HP, p= 0.0203 **One way anova test; post hoc Bonferroni: : Hospital – CHC, p= 0.030; hospital – HP, p = 0.001; CHC- HP, p=0.407 *** Kruskal-Wallis test; post hoc Mann-Whitney : Hospital – CHC, p= 0.0249; hospital – HP, p = 0.0003; CHC- HP, p= 0.0433 © World Bank and Oxford Policy Management 70 Health Worker Survey in Timor-Leste Based on the regression results, the location of practice had a significant effect on doctors’ clinical performance. Those working in a rural area were less skilled than their counterparts in urban areas. It is also evident from another study (Ziller & Lenardson, 2009) that, even though the government has provided housing and transportation facilities, doctors working in rural areas face many difficulties in their daily life. Sometimes they run out of water and electricity both in their houses and clinics and it is difficult for them to travel due to ruined roads. Moreover, supplies of medical equipment, stocks of drugs and also the number of medical professionals in rural areas are limited. These factors may all contribute to doctors’ clinical performance. The time taken for clinical consultation also had a significant association. The clinical performance was better in cases where the consultation periods were longer than in others. The average consultation time per case was eight minutes. Medical doctors spent less time in DCOs (six minutes) compared to vignettes (10 minutes), which is a statistically significant difference (p= <0.05). To find lack of performance causes, a non-quality score was generated from the inverted DCO and vignette consolidated score. The causes of non-quality were then categorised by three factors: (i) physicians do not know, as can be checked with the inverse of vignettes score for each physician, (ii) physicians cannot do, because lack of equipment and supplies, as measured by the health facility survey, and (iii) physicians do not want, as measured by the lack of motivation reported in health worker survey. Table 49: Statistical Association Between Non-Quality and Performance Attributes Correlation coefficient (r) Significance (p-value) Lack of knowledge 0.34 0.00* Lack of motivation 0.07 0.13 Lack of equipment and supplies 0.06 0.19 * Significant Figure 36: Non-Quality vs. Lack of Knowledge 100 90 80 Lack of knowledge 70 60 50 40 30 20 10 0 0 20 40 60 80 100 Non-quality © World Bank and Oxford Policy Management 71 Health Worker Survey in Timor-Leste Figure 37: Non-Quality vs. Lack of Motivation 100 90 80 Lack of motivation 70 60 50 40 30 20 10 0 0 20 40 60 80 100 Non-quality Figure 38: Non-Quality vs. Lack of Equipment and Supplies 100 90 Lack of equipment and supplies 80 70 60 50 40 30 20 10 0 0 20 40 60 80 100 Non-quality As shown in Table 49 and Figure 36, lack of knowledge of the medical doctors had a significant relationship to their performance. However, lack of motivation and lack of equipment and supplies in the health facilities were not significantly associated with the non-quality (i.e. lack of performance) of the doctors (Figure 37 and Figure 38). Additional regression models of the DCO and vignettes are included in Annex B. © World Bank and Oxford Policy Management 72 Health Worker Survey in Timor-Leste 7 Conclusions This study is the first of its kind since Timor-Leste’s independence and provides useful insights into current and future challenges. The study has combined a number of research tools to produce a rich dataset on the state of health facilities, health worker characteristics, experiences and preferences, as well as competencies and performance. The findings across the reports are generally consistent with each other and indicate strong correlation on causal factors. The findings are described in the report and can be further triangulated in the future in order to generate useful policy advice for the government of Timor-Leste. This section highlights some areas with potential policy implications. 7.1 Policy Implications Functionality of Rural HPs The survey has brought to light some of the challenges of bringing Timor-Leste’ rural health infrastructure up to standard. The availability of a fixed water source and stable power supply is strongly associated with the tier of health facility: availability is lowest for HPs, followed by CHCs. Hospitals have the most stable power supplies and fixed water sources. A significant percentage of rural HPs did not have access to a permanent water source and only one-third of all HPs always have access to electricity (from a variety of sources). During the seven days preceding the survey, only 39% of all facilities had uninterrupted electricity. A considerable percentage of CHCs and HPs did not have any functioning means of communication. There is no form of privacy (private room, visual or audio privacy) in 58% of HPs. Only 43% of HPs are equipped to sterilise metal equipment. The availability of 11 supply items was consistently lower in rural facilities; pulse oximeters and oxygen cylinders were particularly scarce. The availability of a selection of 23 antibiotics and antimalarials decreased by tier of health facility (barring two exceptions); availability was lowest in HPs. The HPs also offered fewer services than CHCs and hospitals. A comparison of expected service delivery and actual service delivery at that level has not yet been undertaken. These findings highlight the urgency of equipping HPs, in particular, to mandated standards. This effort will not only improve patient care but also help to retain health staff and improve their performance. Some of the differences between the knowledge and practice of health workers, as evidenced by vignettes and observations, may well be attributed to the lack of necessary infrastructure and equipment at the HP level. Implications for Pay Policies One of the interesting findings of the study is the relative lack of importance of wages for doctors. This is in line with a number of recent DCEs in other countries, although it should be noted that most were conducted with medical students, who are likely to have different priorities to more seasoned professionals. In Ghana, a study found that a relatively high value was placed on housing and workplace attributes, as compared with increases in salary (Kruk, et al., 2010). Studies from Kenya and South Africa suggested that better educational opportunities or rural allowances would be most effective in increasing the uptake of rural posts while in Thailand better health insurance coverage would have the greatest impact (Blaauw, et al., 2010). Finally, in Tanzania, a recent study found that offering continuing education after a certain period of service is one of the most powerful recruitment instruments the authorities have available, although increased salaries and hardship allowances will also substantially increase recruitment in rural areas (Kolstad, 2011). © World Bank and Oxford Policy Management 73 Health Worker Survey in Timor-Leste In the Timor-Leste context, young doctors may see themselves as already better off (with a salary of US$ 610 per month) compared to other civil servants. In that case, their preference for professional development opportunities as opposed to increased wages may be reasonable. Although there are concerns about low salaries, very few health workers appear to be planning to change jobs in the short term, and intrinsic motivation is high. Salaries are received regularly and dual practice opportunities appear to be low. It also appears that the public policy on wage raise for health professionals is not fully functioning and the current civil service wage structure is not related to performance. It is difficult to predict whether having a performance based reward system would be effective or not in this context as it will be dependent on the progression of competing opportunities including private sector development and even overseas opportunities. Nevertheless, it is important to review salary progression over time to ensure that there are incentives to stay motivated and perform to or above standards, given the relatively flat salary progression for doctors (but not nurses and midwives) highlighted in this report. Training Policies The survey has highlighted some apparent anomalies in relation to training. Namely, that despite reporting being able to do their job and having received relatively recent training in most cases, the majority of respondents reported that they have not had enough opportunities to improve their knowledge. It is important to understand these results more deeply. It may be that the quality of the training was not satisfactory, or that there is a mismatch in terms of the type of training available (for example, although a large number of doctors are in rural areas, only 19% underwent training on community health, while 75% of doctors indicate that they require training on IMCI and 64% on EmONC). Additional training may also be valued for other attributes, such as per diems or time away from work. Training is a powerful job attribute that appears to determine job choice, including location. The fact that urban health workers enjoy more training adds to the attraction of urban jobs. Access to training should therefore be determined by both training needs and the incentive it provides (e.g. rural/urban). Younger doctors, female doctors and those in CHCs and HPs tend to value training more. Providing specialisation options may well be highly effective in retaining doctors in rural areas. Rural Attraction and Retention HPs are clearly regarded by many as a starting point for their career, as only 30% of HP staff reported a desire to continue at that level long term (compared to 60% of staff at CHCs). Although the DCE results are difficult to interpret, they show that non-financial job attributes can be traded to make rural jobs more attractive. Training (both visits from specialists and a higher probability of specialisation), as well as providing housing, better equipment and transport, would convince more doctors and other staff to take up and remain in rural service. Women, unmarried doctors, those who have worked in HPs, and those with a letter of recommendation from their local authority appear to be more receptive to working in remote or very remote areas. At present, in-kind benefits do appear to be more targeted at rural postings, but it is important to ensure that supportive supplies (e.g. fuel for motorbikes) are adequate and timely. Improved security and more consistent feedback and supervision would also address the challenges commonly raised across the workforce as a whole. © World Bank and Oxford Policy Management 74 Health Worker Survey in Timor-Leste Planning for the Workforce There are a number of issues surrounding health workforce planning that have emerged from the study. Sixty percent of the sampled health workers were based in urban locations. Planners need to consider whether this reflects the desired workforce allocation and, if not, how to change it. In particular, it is striking that nurses and midwives are more focused in urban areas than doctors, which may not be appropriate. The fact that the current cohort of doctors were largely recruited at one time (after independence) may also pose challenges in the future, as they are likely to seek advancement and retirement around the same time, leaving possible gaps. This ‘cohort’ effect can be mitigated by implementing a human resources for health planning exercise to determine how many health workers, by cadre, will be required in the future, at each facility tier. This will inform the need for further training of doctors, and identify the number of current doctors who would be able to transition to higher-level facilities. Improving Performance The areas that were low-performing in the vignettes and observed cases should be investigated further to understand how performance can be improved, beyond addressing the infrastructure and equipment challenges highlighted above. In particular, the low scores for physical examinations and the relative absence of hygiene precautions such as hand washing should be examined and addressed through training, supervision and mentoring. These efforts should be focused on, but not exclusive to rural areas and HPs. Health workers desire for additional training in the health worker survey, in-service training being a significant attribute in DCE and the narrow know-do gap in DCO and vignette – these all commonly indicate a lack of knowledge of health workers. Compliance to Clinical Protocols The survey findings suggested that doctors’ knowledge was better than their practice, and since the consultation time was one of the significant determinants, it can be assumed that the clinical performance of doctors can be improved by ensuring compliance with clinical protocols. This can be achieved by regular in-service training, monitoring and active performance management. Workload The workload of the doctors working in Timor-Leste is not too high. On average the doctors in hospitals, CHCs and HPs consult 13, 11 and 9 patients respectively which is low compared to other countries. According to the American Academy of Family Physicians survey 2013, on average family physicians consult 19 patients per day.6 In UK, GPs typically consult 30-40 patients in a day.7 A nationally representative health facility survey in Bangladesh reported a mean of 37 patients per health worker at the sub district level health facilities (FMRP, 2005). However, a significant percentage of the health workers have reported that they are over-worked. It is important to investigate the reasons behind this perception. It is possible that the health workers are often busy with administrative work. It is also important to manage the expectation of the workload. 6 http://www.washingtonpost.com/news/to-your-health/wp/2014/05/22/how-many-patients-should-your-doctor-see-each- day/ 7 http://www.medicalcareers.nhs.uk/specialty_pages/general_practice/working_life.aspx © World Bank and Oxford Policy Management 75 Health Worker Survey in Timor-Leste 7.2 Recommendations Based on the important policy implications discussed above, recommendations have been grouped into three broad themes and priority areas identified for Timor-Leste’s health workforce policy and planning. Ensure Policy Compliance at All Levels 1. Service availability should match Basic Service Package (BSP): The survey found that health facilities, particularly the rural ones often lacked services and facilities that they are mandated in the BSP. Ensuring policy compliance to BSP will not only maintain the consistency of the services across the sector, but will also improve the functionality of the rural health facilities which will eventually encourage health workers to stay motivated to work in rural areas. 2. Availability of medicines should comply with the Essential Drug List (EDL): The drug supplies were frequently inconsistent with the EDL recommendation. Complying with the list may improve health service delivery, as it would likely reduce the chance of stock-outs and misuse of drugs. However, it is possible that there is a functioning decentralised mechanism in place, which is not fully compliant to the central polity, but is effective. This needs to be investigated before implementing this recommendation. 3. Clinical guidelines should be enforced: Enforcing clinical guidelines may increase the performance of doctors. The following issues should be considered when forming policy implementation strategies. First, changing practice usually requires a lot of effort. Second, since some practitioners lack some basic knowledge, the clinical guidelines, which are mostly practice oriented, may not be fully effective. And finally, the clinical guidelines on all relevant topics might not be available or updated. Strengthen the Health Human Resource Policy 4. Ensure salary progression for doctors: The data revealed that the salary difference between the junior and senior doctors was very narrow. This may demotivate experienced health workers and cause them to leave the public sector or engage in dual practice, if opportunities arise. A clear salary progression policy will keep health workers motivated to serve in the public sector and will also encourage better performance. 5. Provide necessary trainings for health workers: The overall low score on clinical competence and the narrow know-do gap indicates lack of knowledge of the doctors in Timor-Leste, especially in rural areas. Regular in-service training and instructional visits from specialists are highly recommended for health workers so that the competence level is increased. This will also improve the confidence of young health workers and motivate them to stay in rural areas. 6. Improve the effectiveness of supervision: While the supervision visits were frequent, it was revealed that, in most cases, these visits were mostly concerned with checking official records. More efficient supervision and monitoring tools and exercises are recommended so that these visits are more meaningful and action oriented. 7. Adjust the expectations of health workers on their workload: Nearly half of the respondents reported to have high workload. However, the average number of patients consulted as observed during the survey did not reflect that statement. It might be useful © World Bank and Oxford Policy Management 76 Health Worker Survey in Timor-Leste to better understand the reason behind this statement and adjust the workload expectation by establishing clear terms of reference for the work. Improve the Functionality of Health Facilities 8. Improve the pharmaceutical chain of procurement and distribution: The survey results indicate a loosely functioning pharmaceutical chain with an incoherent distribution mechanism. Implementation of a demand driven strong pharmaceutical procurement and disbursement system in which demand will be raised in a bottom-up process and a sequential distribution mechanism is recommended. 9. Improve availability of medical supplies: The rural health facilities were frequently under-equipped and many health workers expressed this as one of the important attributes to decide workplace. Improving availability of the medical supplies in health facilities will improve the service delivery and at the same time will enhance the rural retention of health workers. 10. Improve infrastructure and the access to water, electricity and communication: Many rural facilities lacked basic infrastructure including access to water, electricity and communication. Improving the infrastructure will optimise the service delivery and rural retention. An inter-sectoral initiative, involving the other concerned ministries, is recommended to identify and prioritize potential development areas, and improve the infrastructure of health facilities, including the accommodation of health workers. 7.3 Further Studies Further research/ studies will broaden the understanding on the topic and will also answer some of the causal relationships (answering “why?”) of the study results. These are: 1. Assess the training gaps: Further studies can be carried out to understand the training gap of the doctors working in Timor-Leste. This will not only help to understand the need for training for the existing health workers but will also guide adjustments to the medical curriculum, so that the future health workforce is adequately trained. 2. Migration and exit to private sector: Migration to other countries and exit to the private sector may emerge as a problem in future. Moving forward, this potential development should be monitored using the M&E data. 3. Absenteeism and dual practice: Because unannounced visits to the health facilities were not made, the reported prevalence on absenteeism and dual practice may be underestimated. Further studies can be carried out to better understand these issues. 4. Pharmaceutical distribution: Since it was beyond the scope of the study, the nature and extent of the malfunctioning pharmaceutical chain was not fully explored. However, initial findings suggest that there are some concerns in this area. A qualitative study should be conducted to better understand the issues with pharmaceutical distribution in Timor-Leste. 5. Preference of health workers in terms of rural posting : Some health workers, especially nurses and midwives preferred rural posting over urban. A follow-up © World Bank and Oxford Policy Management 77 Health Worker Survey in Timor-Leste qualitative study, to understand the reasons for this preference, will enhance understanding and may be useful for the future scale-up of health worker deployment policy. 6. Follow-up survey to monitor the trend over years: The health sector in Timor-Leste is rapidly changing. Some of the findings and the policy implications may change in future because of this rapidly changing context. A follow-up survey using a similar methodology should be conducted in 3-5 years to monitor the trend over time and to adopt the policies accordingly. 7.4 Way Forward This survey has highlighted some of the important policy areas that can be addressed to better manage the health workforce in Timor-Leste and ensure better health service to the population, particularly in rural areas. The Ministry of Health can consider prioritising the above mentioned recommendations and can group them into short, medium and long-term initiatives. We also suggest disseminating these research findings to the appropriate audience and organising policy dialogues so that the recommendations from this survey can be taken into consideration for appropriate policy implementation. © World Bank and Oxford Policy Management 78 Health Worker Survey in Timor-Leste References Asante, A. D., Martins, N., Otim, M. E. & Dewdney, J., 2014. Retaining doctors in rural Timor- Leste: a critical appraisal of the opportunities and challenges. Bulletin of the World Health Organization, Volume 92, p. 277–282. Asante, A. et al., 2012. Analysis of Policy Implications and Challenges of the Cuban Health Assistance Program Related to human Resources for Health in the Pacific. Human Resource for Health, 10(1), p. 10. Aung, T. et al., 2012. Validation of a New Method for Testing Provider Clinical Quality in Rural Settings in Low- and Middle- Income Countries : The Observed Simulated Patient. PLoS One, 7(1), p. e30196. Blaauw, D. et al., 2010. Policy interventions that attract nurses to rural areas: a multicountry discrete choice experiment. Bull World Health Organ., 88(5), pp. 350-356. Cabral, J., 2010. Critical Apprisal: Human Resources for Health to Improve Service Delivery, Dili: Ministry of Health. Cabral, J., Dussault, G., Buchan, J. & Ferrinho, P., 2013. Scaling-up the medical workforce in Timor-Leste: Challenges of a great leap forward. Social Science & Medicine, Volume 96, pp. 285- 289. Das, J. et al., 2012. In Urban and Rural India, a Standardized Patient Study Showed Low Levels of Provider Training and Huge Quality Gap. Health Affairs, 31(12), pp. 2774-2784. Deen, J. et al., 2013. Identifying national health research priorities in Timor-Leste through a scoping review of existing health data. Health Research Policy and Systems, 11(8). FMRP, 2005. Social Sector Performance Surveys: Health and Family Planning in Bangladesh, Dhaka: Financial Management Reform Programme. Government of Timor-Leste, 2011. Timor-Leste Strategic Development Plan 2011-30, Dili: Government of Timor-Leste. Harden, R. & Gleeson, F., 1979. Assessment of clinical competence using an objective structured clinical examination (OSCE). Medical Education, 13(1), pp. 41-54. Higuchi, M. et al., 2012. Use of Medicines and Adherence to Standard Treatment Guidelines in Rural Community Health Centers, Timor-Leste. Asia Pacific Journal of Public Health, p. Epub ahead of print. ILO, 2008. Structure and preliminary correspondence with ISCO-88: International Standard Classification of Occupants, Geneva: International Labour Organization.. Kolstad, J., 2011. How to make rural jobs more attractive to health workers. Findings from a discrete choice experiment in Tanzania. Health Economics, 20(2), pp. 196-211. Kruk, M. et al., 2010. Rural practice preferences among medical students in Ghana: a discrete choice experiment. Bull World Health Organ., 88(5), pp. 333-341. Lancaster, K. J., 1966. A new approach to consumer theory. The Journal of Political Economy, 74(2), pp. 132-157. Leonard, K., 2008. Is patient satisfaction sensitive to changes in the quality of care? An exploitation of the Hawthorne effect. Journal of Health Economics, 27(2), pp. 444-459. Leonard, K. & Masatu, M., 2005. The Use of Direct Clinical Observation and Vignettes for Health services Quality Evaluation in Developing Countries. Social Science and Medicine, 61(9), pp. 1944-1951. Martins, N., Kelly, P., Grace, J. & Zwi, A., 2006. Reconstructing Tuberculosis Services after Major Conflict: Experiences and Lessons Learned in East Timor. PLoS Medicine, 3(10), pp. 1765-1775. © World Bank and Oxford Policy Management 79 Health Worker Survey in Timor-Leste Martins, N. & Trevena, L., 2014. Implementing what works: a case study of integrated primary health care revitalisation in Timor-Leste. Asia Pacific Family Medicine, 13(5). McCall, C., 2014. East Timor striving for universal access to health care. Lancet, 384(9953), p. 1491–1492. McFadden, D. L., 1976. Quantal Choice Analaysis: A Survey. Annals of Economic and Social Measurement, 5(4), pp. 363-390. MoH, 2007. Basic Services Package for Primary Health Care and Hospitals, Dili: Ministry of Health. MoH, 2012. Human Resource for Health: Country Profile, Timor-Leste, Dili: Ministry of Health. PAHO, 2012. Cuba - Health in the Americas, Washington DC: Pan America Health Organization. Quintana, F., Sarasa, N., CAnizares, O. & Huguet, Y., 2012. Assessment of Complementary Curricular Strategy for Training South African Physicians in a Cuban Medical Universities. MEDICC Review, 14(3), pp. 19-25. WHO, 2006. World Health Report 2006: Working Together for Health, Geneva: World Health Organization., Geneva: World Health Organization. Ziller, E. & Lenardson, J., 2009. Rural-Urban Differences in Health Care Access Vary Across Measures, Portland: Maine Rural Health Research Center. © World Bank and Oxford Policy Management 80 Health Worker Survey in Timor-Leste Annex A Single Control Regressions for DCE A.1 Single Control Regressions for Doctors Basic age female married Own house Diff district family base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote -0.068 0.028 -0.003 -0.091 0.034 -0.230** 0.266** 0.006 -0.143 0.040 -0.521*** location urban 0.220*** 0.614 -0.014 0.208** 0.022 0.103 0.195* 0.421*** -0.382*** 0.274*** -0.237* equipment medium 0.259*** 0.350 -0.004 0.024 0.402*** 0.328*** -0.109 0.234*** 0.049 0.303*** -0.215 equipment good 0.408*** 1.764*** -0.048*** 0.167** 0.413*** 0.214** 0.320*** 0.326*** 0.159 0.610*** -0.974*** training workshops 0.097 1.424** -0.046* 0.062 0.067 0.048 0.082 0.232** -0.253* 0.082 0.087 training visits 0.571*** 2.019*** -0.051** 0.434*** 0.238* 0.521*** 0.084 0.798*** -0.431*** 0.585*** -0.053 training specialist 0.733*** 2.518*** -0.063*** 0.618*** 0.201 0.606*** 0.211 0.929*** -0.374*** 0.798*** -0.264* HF type: CHC 0.310*** 0.843* -0.019 0.477*** -0.283*** 0.351*** -0.065 0.250*** 0.116 0.354*** -0.227* Housing: good 0.388*** 1.136*** -0.026* 0.245*** 0.249*** 0.239*** 0.243*** 0.360*** 0.056 0.338*** 0.264** Transport: moto -0.022 0.808** -0.029** -0.017 -0.008 -0.147** 0.202** 0.025 -0.088 -0.058 0.190* wage -0.000 -0.007*** 0.000*** 0.001*** -0.003*** -0.000 -0.000 -0.000 0.000 -0.000 -0.000 N 5,440 5,408 5,440 5,440 5,440 5,440 © World Bank and Oxford Policy Management 81 Health Worker Survey in Timor-Leste Diff district urban Basic Experience HF type: HP HF type: CHC rural HF family base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote -0.068 -0.066 -0.021 -0.118 0.033 -0.069 0.002 -0.265*** 0.423*** -0.186* 0.212 location urban 0.220*** 0.202*** 0.205 0.123 0.064* 0.334*** -0.548*** -0.027 0.536*** 0.157* 0.116 equipment medium 0.259*** 0.313*** -0.519*** 0.217*** 0.027 0.342*** -0.360*** 0.240*** 0.048 0.428*** -0.302*** equipment good 0.408*** 0.490*** -0.817*** 0.405*** 0.002 0.475*** -0.272** 0.339*** 0.157 0.549*** -0.255** training workshops 0.097 0.072 0.266 0.028 0.045 0.076 0.088 0.005 0.203 -0.045 0.261* training visits 0.571*** 0.569*** 0.049 0.639*** -0.045 0.481*** 0.416*** 0.606*** -0.062 0.499*** 0.130 training specialist 0.733*** 0.716*** 0.220 0.821*** -0.057 0.658*** 0.378** 0.519*** 0.476*** 0.557*** 0.322** HF type: CHC 0.310*** 0.365*** -0.533*** 0.392*** -0.053* 0.370*** -0.252** 0.285*** 0.052 0.449*** -0.248** Housing: good 0.388*** 0.373*** 0.179 0.363*** 0.017 0.547*** -0.752*** 0.280*** 0.235*** 0.497*** -0.194** Transport: moto -0.022 -0.073* 0.506*** 0.024 -0.030 0.007 -0.132 -0.094* 0.150* -0.072 0.094 wage -0.000 -0.000 -0.001* -0.001** 0.000 -0.001** 0.001** 0.000 -0.001* -0.001** 0.001* N 5,440 5,440 5,440 5,440 5,440 5,440 Ever worked in Ever worked in letter of Basic Ever worked in CHC Trained by CMB Health Post hospital recommendation base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote -0.068 -0.058 -0.158 -0.101 0.142 -0.100 1.058*** 0.186 -0.260 -0.146 0.404** location urban 0.220*** 0.218*** 0.017 0.109* 0.476*** 0.199*** 0.716** -0.119 0.347 0.354*** -0.309** equipment medium 0.259*** 0.224*** 0.547** 0.170*** 0.380*** 0.265*** -0.182 0.459 -0.204 0.219** -0.070 equipment good 0.408*** 0.388*** 0.315 0.302*** 0.452*** 0.396*** 0.394 0.480 -0.073 0.486*** -0.045 training workshops 0.097 0.080 0.266 0.092 0.024 0.130* -1.122*** -0.904* 1.023** 0.284** -0.578*** training visits 0.571*** 0.564*** 0.133 0.568*** 0.010 0.597*** -0.879** -0.182 0.769* 0.641*** -0.196 training specialist 0.733*** 0.739*** -0.070 0.743*** -0.015 0.754*** -0.713* 0.099 0.648 0.774*** 0.068 HF type: CHC 0.310*** 0.323*** -0.178 0.300*** 0.040 0.299*** 0.391 0.654** -0.351 0.269*** 0.102 Housing: good 0.388*** 0.390*** -0.017 0.310*** 0.332*** 0.403*** -0.468* 0.237 0.156 0.322*** -0.054 Transport: moto -0.022 -0.019 -0.046 -0.013 -0.037 -0.017 -0.152 -0.345 0.330 0.030 -0.076 wage -0.000 -0.000** 0.002** -0.000 -0.001 -0.000 0.000 0.001 -0.001 0.000 -0.001** N 5,440 5,440 5,440 5,440 5,440 3,104 © World Bank and Oxford Policy Management 82 Health Worker Survey in Timor-Leste high or very high Basic Has housing Has good housing Has moto Heavy workload satisfaction with job base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote -0.068 0.053 -0.225* -0.163** 0.253* -0.013 -0.255 -0.334*** 0.490*** -0.092 0.129 location urban 0.220*** 0.534*** -0.586*** 0.112 0.303** 0.201*** 0.094 -0.022 0.453*** 0.216*** 0.056 equipment medium 0.259*** 0.469*** -0.398*** 0.276*** -0.042 0.233*** 0.123 0.234*** 0.053 0.238*** 0.126 equipment good 0.408*** 0.620*** -0.400*** 0.520*** -0.304*** 0.497*** -0.404*** 0.257*** 0.287*** 0.281*** 0.733*** training workshops 0.097 0.301*** -0.382*** 0.104 -0.003 0.049 0.234 -0.136 0.445*** 0.104 -0.028 training visits 0.571*** 0.812*** -0.451*** 0.551*** 0.067 0.439*** 0.620*** 0.415*** 0.303** 0.461*** 0.638*** training specialist 0.733*** 1.061*** -0.608*** 0.693*** 0.125 0.648*** 0.409** 0.344*** 0.734*** 0.722*** 0.065 HF type: CHC 0.310*** 0.289*** 0.035 0.244*** 0.182* 0.262*** 0.231** 0.287*** 0.045 0.323*** -0.084 Housing: good 0.388*** 0.474*** -0.157* 0.475*** -0.232*** 0.376*** 0.062 0.363*** 0.057 0.420*** -0.169 Transport: moto -0.022 0.032 -0.108 -0.051 0.079 -0.057 0.168* -0.179*** 0.296*** -0.031 0.074 wage -0.000 -0.000 -0.000 -0.000* 0.000 -0.000* 0.000 -0.000 0.000 -0.000 -0.001** N 5,440 5,440 5,440 5,440 5,440 5,440 Difference btw DCE Basic Wage Lack drugs Lack equipment Lack supervision and actual wage base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote -0.068 -2.756 0.004 -3.374 0.891 0.284 -0.412** 0.040 -0.122 0.003 -0.243* location urban 0.220*** 0.485 -0.000 -0.635 0.352 0.325** -0.122 0.209 0.014 0.427*** -0.714*** equipment medium 0.259*** 5.806*** -0.009*** -3.547* 0.646 -0.066 0.379** 0.069 0.220 0.461*** -0.692*** equipment good 0.408*** 6.262*** -0.010*** -0.686 0.031 0.265* 0.167 0.547*** -0.159 0.612*** -0.676*** training workshops 0.097 2.751 -0.004 2.364 -0.644 -0.355** 0.529*** -0.055 0.175 0.178** -0.275* training visits 0.571*** 2.182 -0.003 6.032** -1.306* 0.289* 0.332* 0.131 0.509*** 0.635*** -0.229* training specialist 0.733*** 0.391 0.001 1.320 -0.205 0.596*** 0.161 0.583*** 0.174 0.730*** 0.048 HF type: CHC 0.310*** -2.957*** 0.005*** 5.765*** -1.278*** 0.524*** -0.250* 0.610*** -0.346** 0.356*** -0.132 Housing: good 0.388*** 2.544** -0.004** 2.049 -0.484 0.083 0.357*** 0.108 0.326*** 0.594*** -0.679*** Transport: moto -0.022 1.364 -0.002 -0.169 -0.022 0.139 -0.188 0.306*** -0.379*** 0.066 -0.276*** wage -0.000 -0.013** 0.000** 0.013** -0.000 0.001 -0.001* -0.000 0.000 -0.001*** 0.002*** N 5,440 5,440 4,448 5,440 5,440 5,440 © World Bank and Oxford Policy Management 83 Health Worker Survey in Timor-Leste A.2 Single control regressions for nurses and midwives Basic age female married Own house Diff district family base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote 0.103*** -0.000 0.003 0.120** -0.030 -0.169** 0.322*** -0.317*** 0.527*** 0.116*** -0.129 location urban 0.068* 0.376** -0.008** 0.096* -0.050 -0.164* 0.275*** -0.432*** 0.626*** 0.085** -0.174 equipment medium 0.096*** -0.211 0.008** 0.026 0.129** -0.134* 0.273*** 0.159** -0.078 0.099*** -0.030 equipment good -0.024 -0.362*** 0.008*** -0.004 -0.037 -0.007 -0.020 -0.092 0.085 -0.031 0.080 training workshops 0.216*** -0.146 0.009*** 0.112*** 0.193*** 0.227*** -0.013 0.096 0.149** 0.241*** -0.263*** bach degree 0.070** 0.151 -0.002 0.053 0.032 -0.100 0.201** 0.033 0.045 0.109*** -0.407*** HF type: CHC 0.123*** -0.307** 0.011*** 0.087** 0.068 0.129* -0.007 0.279*** -0.194*** 0.143*** -0.206** Housing: good 0.131*** 0.303** -0.004 0.171*** -0.074 0.155** -0.027 0.002 0.165** 0.136*** -0.043 Transport: moto 0.120*** -0.700*** 0.021*** 0.235*** -0.212*** -0.110 0.273*** 0.100* 0.028 0.155*** -0.361*** wage 0.001*** -0.003*** 0.000*** 0.001*** -0.000 -0.000 0.002*** -0.001* 0.003*** 0.001*** -0.002** N 8,160 8,064 8,160 8,160 8,160 8,160 Diff district urban Basic Experience HF type: HP HF type: CHC rural HF family base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote 0.103*** 0.098*** 0.245 -0.070 0.012*** 0.101*** 0.024 0.220*** -0.201*** 0.045 0.171** location urban 0.068* 0.077** -0.401 -0.112* 0.012*** 0.090** -0.232** 0.273*** -0.352*** 0.068 0.000 equipment medium 0.096*** 0.111*** -0.673*** -0.204*** 0.020*** 0.097*** -0.011 -0.199*** 0.505*** 0.002 0.272*** equipment good -0.024 -0.008 -0.690*** -0.300*** 0.019*** -0.014 -0.108 -0.152*** 0.220*** -0.078** 0.156** training workshops 0.216*** 0.210*** 0.256 0.030 0.013*** 0.211*** 0.053 0.181*** 0.061 0.226*** -0.026 bach degree 0.070** 0.073** -0.135 -0.171*** 0.016*** 0.059* 0.116 0.072 -0.002 -0.029 0.288*** HF type: CHC 0.123*** 0.114*** 0.447** 0.139*** -0.001 0.166*** -0.436*** -0.090** 0.367*** 0.078** 0.132** Housing: good 0.131*** 0.147*** -0.671*** 0.111** 0.001 0.127*** 0.058 0.086** 0.077 0.099*** 0.092 Transport: moto 0.120*** 0.138*** -0.735*** -0.181*** 0.021*** 0.168*** -0.486*** 0.102** 0.032 0.177*** -0.163*** wage 0.001*** 0.001*** -0.008*** -0.001** 0.000*** 0.001*** 0.001 0.000 0.002*** 0.001* 0.002*** N 8,160 8,160 8,160 8,160 8,160 8,160 © World Bank and Oxford Policy Management 84 Health Worker Survey in Timor-Leste Ever worked in Ever worked in Basic Ever worked in CHC Midwives Health Post hospital base base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote 0.103*** 0.066 0.118 0.051 0.170** 0.094*** 0.070 0.181*** -0.269*** location urban 0.068* 0.015 0.170** 0.007 0.198*** 0.045 0.162 0.129*** -0.210*** equipment medium 0.096*** 0.032 0.211*** 0.170*** -0.241*** 0.128*** -0.221** 0.086** 0.033 equipment good -0.024 -0.105*** 0.263*** -0.019 -0.014 -0.003 -0.142* 0.003 -0.094 training workshops 0.216*** 0.192*** 0.081 0.290*** -0.237*** 0.264*** -0.330*** 0.220*** -0.014 bach degree 0.070** 0.063* 0.026 0.178*** -0.350*** 0.088*** -0.119 0.067* 0.011 HF type: CHC 0.123*** 0.038 0.279*** 0.076** 0.156*** 0.131*** -0.049 0.073** 0.177*** Housing: good 0.131*** 0.095*** 0.122** 0.214*** -0.269*** 0.135*** -0.022 0.196*** -0.221*** Transport: moto 0.120*** 0.027 0.306*** 0.207*** -0.282*** 0.067** 0.371*** 0.174*** -0.183*** wage 0.001*** 0.001*** -0.001 0.002*** -0.002*** 0.001*** -0.000 0.001*** -0.000 N 8,160 8,160 8,160 8,160 8,160 high or very high Basic Has housing Has good housing Has moto Heavy workload satisfaction with job base base base*ctrl base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote 0.103*** 0.118** -0.028 0.104*** -0.009 0.086** 0.325** 0.085** 0.109 0.093** 0.070 location urban 0.068* 0.034 0.069 0.097*** -0.273** 0.069* -0.021 0.086** -0.096 0.056 0.077 equipment medium 0.096*** 0.200*** -0.207*** 0.081** 0.144 0.081** 0.284** 0.016 0.471*** 0.064* 0.205** equipment good -0.024 0.078* -0.203*** -0.011 -0.119 -0.031 0.143 -0.103*** 0.465*** -0.095*** 0.459*** training workshops 0.216*** 0.312*** -0.192*** 0.216*** -0.001 0.221*** -0.086 0.166*** 0.293*** 0.209*** 0.046 bach degree 0.070** 0.133*** -0.126** 0.081*** -0.100 0.059* 0.220* 0.059* 0.065 0.068** 0.012 HF type: CHC 0.123*** 0.192*** -0.137** 0.114*** 0.090 0.142*** -0.359*** 0.111*** 0.081 0.116*** 0.050 Housing: good 0.131*** 0.192*** -0.120** 0.130*** 0.016 0.132*** -0.009 0.054* 0.468*** 0.108*** 0.155** Transport: moto 0.120*** 0.224*** -0.205*** 0.134*** -0.128 0.126*** -0.103 0.035 0.514*** 0.103*** 0.114 wage 0.001*** 0.002*** -0.001*** 0.002*** -0.003*** 0.001*** -0.001 0.001*** 0.002*** 0.001*** 0.002** N 8,160 8,160 8,160 8,160 8,160 8,160 © World Bank and Oxford Policy Management 85 Health Worker Survey in Timor-Leste Basic Wage Lack drugs Lack equipment Lack supervision base base base*ctrl base base*ctrl base base*ctrl base base*ctrl location: remote 0.103*** -1.293*** 0.003*** -0.062 0.197** 0.224*** -0.142 0.266*** -0.534*** location urban 0.068* -1.997*** 0.005*** -0.125 0.230** 0.220** -0.179* 0.265*** -0.649*** equipment medium 0.096*** 1.169*** -0.002*** 0.458*** -0.432*** 0.265*** -0.202** 0.177*** -0.268*** equipment good -0.024 0.616** -0.001** 0.358*** -0.456*** 0.265*** -0.345*** 0.060* -0.277*** training workshops 0.216*** 0.051 0.000 0.190*** 0.031 0.388*** -0.205*** 0.174*** 0.141** bach degree 0.070** 0.930*** -0.002*** 0.235*** -0.197** 0.068 0.002 0.054 0.052 HF type: CHC 0.123*** -0.200 0.001 0.103 0.026 0.112* 0.014 -0.004 0.422*** Housing: good 0.131*** 0.338 -0.000 0.477*** -0.413*** 0.428*** -0.353*** 0.146*** -0.050 Transport: moto 0.120*** 0.383 -0.001 0.335*** -0.257*** 0.304*** -0.218*** 0.070** 0.167*** wage 0.001*** 0.016*** -0.000*** 0.003*** -0.002*** 0.003*** -0.002*** 0.001*** 0.000 N 8,160 6,444 8,160 8,160 8,160 © World Bank and Oxford Policy Management 86 Health Worker Survey in Timor-Leste Annex B Regression Models of DCO B.1 Regression Model of Total Score Clinical Performance VARIABLES Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Located at a rural facility -0.201*** -0.194*** -0.176*** -0.157*** -0.157*** -0.150*** -0.130** (0.0500) (0.0508) (0.0533) (0.0549) (0.0550) (0.0573) (0.0653) Time per patient 1.50e-07*** 1.50e-07*** 1.51e-07*** 1.44e-07*** 1.44e-07*** 1.45e-07*** 1.43e-07*** (3.91e-08) (3.91e-08) (3.91e-08) (3.93e-08) (3.94e-08) (3.94e-08) (3.97e-08) Male -0.0383 -0.0514 -0.0418 -0.0417 -0.0444 -0.0569 (0.0491) (0.0505) (0.0509) (0.0510) (0.0514) (0.0543) Age 0.00602 0.00577 0.00617 0.00605 0.00280 (0.00550) (0.00550) (0.00815) (0.00816) (0.00915) Health Post 0.0907 0.0924 0.0896 0.0986 (0.0628) (0.0677) (0.0681) (0.0704) Total years worked in the medical practice -0.000664 -0.000807 -0.00311 (0.00988) (0.00989) (0.0108) Total many health facilities have been working during career in 0.0203 0.0396 Timor-Leste (0.0455) (0.0723) Have worked previously in a Health Post -0.0502 (0.122) Have worked previously in a CHC 0.0442 (0.0973) Have worked previously in a District/Regional Hospital -0.0843 (0.147) Constant 2.998*** 3.010*** 2.822*** 2.798*** 2.786*** 2.761*** 3.009*** (0.0451) (0.0476) (0.178) (0.178) (0.245) (0.252) (0.627) Observations 511 511 511 511 511 511 511 R-squared 0.062 0.063 0.065 0.069 0.069 0.070 0.071 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 © World Bank and Oxford Policy Management 87 Health Worker Survey in Timor-Leste B.2 DCO Regression Model of Total Score Clinical Performance VARIABLES Model 1 Model 2 Model 3 Model 4 Model 5 Model 6 Model 7 Located at a rural facility -0.169*** -0.168*** -0.165*** -0.143** -0.145** -0.159** -0.119 (0.0557) (0.0567) (0.0591) (0.0610) (0.0611) (0.0654) (0.0730) Time per patient 9.22e-08* 9.22e-08* 9.23e-08* 8.35e-08* 8.24e-08* 8.18e-08* 7.66e-08 (4.81e-08) (4.81e-08) (4.82e-08) (4.86e-08) (4.86e-08) (4.87e-08) (4.89e-08) Male -0.00198 -0.00474 0.00532 0.00727 0.0129 -0.0118 (0.0554) (0.0569) (0.0573) (0.0574) (0.0582) (0.0611) Age 0.00136 0.00141 0.00609 0.00586 -0.000413 (0.00614) (0.00613) (0.00920) (0.00921) (0.0101) Health Post 0.0938 0.113 0.115 0.141* (0.0689) (0.0744) (0.0746) (0.0784) Total years worked in the medical practice -0.00762 -0.00716 -0.0118 (0.0112) (0.0112) (0.0120) Total many health facilities have been working during career in Timor- -0.0319 0.0351 Leste (0.0531) (0.0912) Have worked previously in a Health Post -0.0617 (0.128) Have worked previously in a CHC 0.128 (0.116) Have worked previously in a District/Regional Hospital -0.170 (0.154) Constant 2.924*** 2.925*** 2.883*** 2.846*** 2.715*** 2.770*** 3.077*** (0.0487) (0.0522) (0.197) (0.198) (0.276) (0.291) (0.690) Observations 346 346 346 346 346 346 346 R-squared 0.041 0.041 0.041 0.046 0.048 0.049 0.056 Standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 © World Bank and Oxford Policy Management 88 Health Worker Survey in Timor-Leste Annex C Additional Information on DCE Analysis Design of the choice set Main effects There were seven attributes and 19 levels for the DCE of the medical doctors. All the possible jobs combinations summed up to a total of 864, which is the full factorial design. However, there are 12 main effect to estimates (i.e. 12 differences within attributes) and that the choice set will be saturated with only 13 choices made: at least 13 choices will have to be observed to capture the main effect. Number of Choice Sets If we wanted to have a fully orthogonal and balanced array, we would need 72 choices, i.e. a multiple of 9 (for the uneven number of levels attributes) and 8 (to account for the even). This would guarantee an efficiency of 100%. However, handling 72 choice sets in the field for such a survey would not be practical. We decided to work with only 16 sets. This choice works well for the multiple attributes with 2 levels. However, the balance and orthogonality will not be fully enforced for the attributes with three levels: there are 18 violations of balance and orthogonality, which makes the D-efficiency fall from 100% to 96.15%. It is not dramatic. Asking a respondent to make 18 choices would decrease the number of violation to only 10 and increase the D-efficiency to 97.1%. However, this would lengthen the questionnaire and it was decided not to increase the questionnaire. Restrictions Another component to include is that some combinations of attributes are very unlikely and would make the responses incoherent. Indeed, location level “extremely remote” and facility type “Community Health Centre” will never go together and having this suggestion would add some confusion. A maximisation of the efficiency was then found on the constraint that these items are never combined. This further lowers the D-efficiency of the orthogonal array to 93.1% The reader will be interested to know that combining location and health facility into 5 levels would give a smaller factorial, but a greater number of main effect (saturation point: 14). The optimal orthogonal array would count 120 choices (against 72). In terms of efficiency: The D-efficiency for 16 choices would be 93.9, i.e. 0.8% more efficient than when we split that elements and add the restriction in the design. It was decided to split them and ask the respondent to make choices on location and facility type separately and avoid confusion. Creating the Choice Set The program SAS and it's macro %mkt* were used to perform the computations and design the choice set. First, the most orthogonal array (i.e. to account for the restrictions) was selected using the %mktEx macro (i.e. the full factorial design without the restricted cases). Using this array, the jobs were selected and associated in pair by the %choiceff. The algorythm, of the fedorov type that search many possible combinations and pick the best local maximum as overall maximum. Here, choiceff maximizes the variance of the parameters. Not necessarily by assuming a linear model to capture the main effect. © World Bank and Oxford Policy Management 89 Health Worker Survey in Timor-Leste SAS version 9.2 onward gives the D-efficiency criteria for all the proposed options. However, this task was performed using SAS 9.0.The task was repeated several times as to obtain the best efficiency. Monotone Preference Annex The table below displays the percentage of HW that choose job A over job B. It is also well balanced: most health workers alternate their choices: only 3.6% of the workers systematically picked only job A or only job B. About 90% of the HW selected between 3 and 13 job As (out of 16). It is good practice in DCE to withdraw HWs who systematically picked one job from the analysis. Table 50: Proportion of Job A Selection, by Type of Health Workers Set Doctor Nurse Midwife Total 0 1.73 0.67 1.69 1.36 1 1.16 0 0.85 0.68 2 1.73 2 0.85 1.59 3 10.4 2 2.54 5.44 4 10.4 4 7.63 7.48 5 17.92 8 10.17 12.47 6 13.87 14.67 9.32 12.93 7 16.18 12 9.32 12.93 8 7.51 7.33 12.71 8.84 9 6.94 8.67 8.47 7.94 10 4.62 8 5.93 6.12 11 2.31 8.67 5.93 5.44 12 3.47 8 9.32 6.58 13 0 5.33 3.39 2.72 14 1.16 6 4.24 3.63 15 0.58 1.33 3.39 1.59 16 0 3.33 4.24 2.27 Total 100 100 100 100 Both Jobs Refusal Analysis The same analysis was performed for the refusal rate. Twenty-seven percent of the HWs interviewed did not discard both jobs for any sets. On the upper bend, 2.04% rejected both jobs for all sets. For the robustness check, the regressions will include a specification where those who would have refused all jobs in all sets are considered as if they accepted all. Another specification will include the refusal rate of only those who refused fewer than half the sets. Table 51: Proportion of Choices That Were Rejected – by Type of Health Workers Set Doctor Nurse Midwife Total 0 37.57 24.67 15.25 27.21 1 19.08 39.33 36.44 30.61 2 2.31 4.67 4.24 3.63 3 5.78 4.67 5.93 5.44 4 6.94 3.33 8.47 6.12 5 8.09 6 3.39 6.12 6 4.05 2 9.32 4.76 7 4.62 6 4.24 4.99 8 1.73 0.67 0.85 1.13 © World Bank and Oxford Policy Management 90 Health Worker Survey in Timor-Leste 9 1.73 2.67 0.85 1.81 10 3.47 0.67 1.69 2.04 11 0 1.33 0 0.45 12 0.58 0.67 0 0.45 13 0 0 2.54 0.68 14 0 0 0.85 0.23 15 1.73 2 2.54 2.04 16 2.31 0.67 3.39 2.04 Total 100 100 100 100 Attribute Dominance Analysis It is possible that some HWs made their choices based on the level of one attribute only. Although this preference might be genuine, they must be flagged and reported. The table below shows the percentage of HW that consistently chose the level indicated (when available) over other options available. The number of occurrence of each level in a choice set where it can be chosen over another level of the same attribute are displayed in the columns “N”. Attributes’ leve ls that are often dominant (high percentage) and that have high number of appearance are considered as dominant. The MDs show a strong dominance for the higher probability of specialization, even if the number of occurrence is small (4). Visits from specialists and practice in a community health centre (CHC) also are dominant in 17% and 13% of the cases. Nurses and midwives did not show a specific pattern, dominance is either small or has too few occurrences to back it up. Table 52: Proportion of HW with Item Dominance Doctors Nurses Midwives Attributes Levels % N % N % N Urban 4% 8 5% 6 3% 6 Location Remote 0% 13 5% 10 8% 10 Very remote 14% 5 7% 4 17% 4 Poor equipment 6% 6 4% 6 8% 6 Equipment medium equipment 0% 12 4% 12 6% 12 Good equipment 8% 6 3% 6 3% 6 No training 10% 4 3% 6 4% 6 Workshops 0% 13 3% 12 4% 12 Training Visits from specialists 17% 5 Higher probability of specialization 28% 4 3% 6 7% 6 HF type CHC 13% 6 5% 7 5% 7 Housing Good housing 7% 8 1% 9 2% 9 Moto Motorbike 0% 8 5% 9 6% 9 Wage1 10% 6 5% 6 3% 6 Wage Wage2 0% 12 3% 12 4% 12 Wage3 10% 6 7% 6 6% 6 Note: columns N refer to the maximum possible occurrence of dominance within all the sets proposed. MD Robustness Checks As was highlighted in the descriptive statistics section, some HWs would have refused the jobs and maybe the choice. This can influence the interpretation of the results and deserves interest. The possibility that respondents paid less attention to these sets with two unattractive jobs, or that they randomly picked one cannot be discounted. It would not be unusual to believe that they refused both because neither has appeal and therefore, no real trade-off is to be made. Adding to the regression the crude variable for whether both jobs would be refused is not possible in the context of a fixed effect regression, since most of the effect would be captured by the fixed © World Bank and Oxford Policy Management 91 Health Worker Survey in Timor-Leste effect. To understand better how the refusal plays out, those choices could be dropped, however, this would reduce the number of appearance of some jobs or items more than other and the orthogonal array created for the DCE would lose its properties. Approximately 500 jobs were analysed (i.e. 1,000 observations). The findings are robust to the use of a less stringent definition of “both job refusal”, by not excluding those emanating from doctors who reported they would refuse all the sets (reg bis) or those who would refuse more than 50% of the sets (reg ter). The preference reported for the non-refusal jobs are markedly different: remote areas become less attractive than extremely remote, and good equipment. It was preferable to decompose the effect of each DCE attribute into the effect of the MDs that thought they wouldn’t refuse both and those who would. Concretely, the DCE attributes were interacted with the refusal variables. Those who would have picked one of the jobs now display a positive coefficient on wage, a negative coefficient on remote (and therefore would prefer extremely remote), are less interested in visits from specialists, find a positive utility in having a motorbike and are extremely satisfied to be working in a CHC (highest coefficient). The sets that would be refused were dragging up the remoteness coefficient, while pulling the CHC down. Table 53: Regression Results Base Regression with interactions regression coef Reg reg bis reg ter Location: remote -0.068 -0.446*** -0.426*** -0.420*** Location urban 0.220*** 0.175*** 0.173*** 0.186*** Equipment medium 0.259*** 0.975*** 0.904*** 0.824*** Equipment good 0.408*** 0.676*** 0.639*** 0.589*** Workshops 0.097 -0.411*** -0.351*** -0.243*** Training visits 0.571*** 0.001 0.028 0.104 Training specialist 0.733*** 0.161* 0.180** 0.158** HF type: CHC 0.310*** 0.856*** 0.804*** 0.695*** Housing: good 0.388*** 0.727*** 0.695*** 0.703*** Transport: motorbike -0.022 0.159*** 0.148*** 0.122*** Wage -0.000 0.001*** 0.001*** 0.001*** Location: remote * refusal 1.036*** 1.134*** 1.975*** Location urban * refusal 0.452*** 0.593*** 1.002*** Equipment medium * refusal -1.813*** -1.737*** -2.028*** Equipment good * refusal -0.566*** -0.389** 0.193 Workshops * refusal 1.483*** 1.392*** 0.941*** Training visits * refusal 1.470*** 1.657*** 1.925*** Training specialist * refusal 1.188*** 1.349*** 2.474*** HF type: CHC * refusal -1.681*** -1.695*** -1.669*** Housing: good * refusal -0.835*** -0.797*** -1.465*** Transport: moto * refusal -0.428*** -0.413*** -0.267* Wage * refusal 0.000 -0.000 -0.000 Number of observations 5,440 5,440 5,440 5,440 Note: p-values: 0.01 - ***; .05 - **; .1 - * © World Bank and Oxford Policy Management 92 Health Worker Survey in Timor-Leste Robustness Check of Nurses/Midwives As was highlighted in the descriptive statistics section, some HWs would have refused the jobs and maybe the choice. As for the MDs, the regressions with interaction terms for the set containing jobs that would both be refused are reported here. Preferences for jobs that would not be refused are different: location preference against extremely remote places is stronger. Medium good equipment is not different from zero anymore, while good equipment becomes very important. Housing and motorbike become more important. But the positive coefficients on workshops, bachelor degree, CHC workplace and wage are all driven by sets that would be refused. There is little robustness to the inclusion of the refusal of sets for these items. However, the direction of preferences for the location, motorbike and housing items are stable and the fact that these attributes are valued is robust. Table 54: Regression Results for Nurses and Midwives Base Regression with interactions regression coef reg reg bis reg ter Location: remote 0.103*** 0.568*** 0.548*** 0.445*** Location urban 0.068* 0.290*** 0.282*** 0.212*** Equipment medium 0.096*** 0.001 -0.007 0.006 Equipment good -0.024 0.319*** 0.309*** 0.223*** Workshops 0.216*** -0.086*** -0.047 -0.029 Bachelor degree 0.070** -0.226*** -0.194*** -0.156*** HF type: CHC 0.123*** -0.128*** -0.112*** -0.061** Housing: good 0.131*** 0.291*** 0.281*** 0.234*** Transport: motorbike 0.120*** 0.209*** 0.204*** 0.182*** Wage 0.001*** -0.000 -0.000 -0.000 Location: remote * refusal -1.498*** -1.576*** -1.437*** Location urban * refusal -0.499*** -0.510*** 0.200 Equipment medium * refusal 0.010 0.116 -0.316*** Equipment good * refusal -1.314*** -1.413*** -1.721*** Workshops * refusal 0.683*** 0.457*** 0.645*** Bachelor degree * refusal 0.516*** 0.338*** 0.099 HF type: CHC * refusal 0.648*** 0.634*** 0.347*** Housing: good * refusal -0.696*** -0.722*** -0.892*** Transport: moto * refusal -0.465*** -0.496*** -0.863*** Wage * refusal 0.003*** 0.003*** 0.004*** Number of observations 8,160 8,158 8,158 8,158 Note: p-values: 0.01 - ***; .05 - **; .1 - * © World Bank and Oxford Policy Management 93