69850 The World Bank Health Management and Accountability Study KOSOVO Report on Management Accountability in the Health Sector Consultant Gary Gaumer June 28, 2007 1 Report on Management Accountability in the Health Sector Executive Summary This report assesses the opportunities for improving the performance of the government health sector in Kosovo through better management and improved information for managers. It is based on interviews with central, regional, clinic and hospital managers in April 2007. The main findings of the study are: Management of health facilities as well as municipal, regional and central managers are supported by very little data and virtually no performance analytics or comparative benchmark information. A system of patient statistics derived from manually collected patient data (maintained by IPH) and a system of budget and expenditure tracking (maintained by MOF) are the primary management supports. Special purpose systems exist for capturing certain epidemiologic data and for collecting data from households on health expenditures. Pharmaceutical Systems are not adequate to control or monitor the distribution and inventories of essential drugs that are purchased and distributed by the MOH. A system exists for keeping inventories of pharmaceutical warehouses, though it is not installed in all warehouses and it does not link with flows of pharmaceuticals kept by customs or to retail pharmacies or clinical sites. Existing data systems are not sufficient for doing analyses of service costs or disease cost and burden. Nor, can existing data support preparation of National Health Accounts. Some HMIS development work has been piloted by MOH. This is a patient based clinical support package that is browser based and integrates registration, HR, prescribing and clinical data. This work has been done as a speculative investment by a commercial vender, though the MOH now owns the license for the system except for the pharmaceutical component. Manual medical record (historical) data is not routinely available to support patient care activities. Patient charts are kept in hospitals, but are often not able to be retrieved. A history (health card) is typically kept in clinics, but the prevalence of private sector use and 2 travel out of the country to seek care make the completeness of ambulatory records problematic. Budget development is based on budget history and monitoring is done, though managers have little flexibility to reallocate resources across budget lines (capital, salaries, goods and services). Though there is a mid year review, budgets are essentially treated as fixed for the year and cannot be overrun. Urgent needs for additional resources are sometimes financed by the hospital mangers by slowing payments to vendors. Management reports are prepared from patient statistics and budget systems but they are of little value in monitoring facility performance. Monthly budget monitoring reports and monthly reports of patient statistics are made and widely distributed. Both monitoring systems produce reports that contain information in the form of raw counts (spending by category, days, admissions, procedures, etc.). Performance measures are not used (such as LOS, occupancy rates, visits per doctor and no side by side comparisons are evident such as comparing time periods, facilities, doctors, or external benchmarks. With the exception of some hospital “midnight reports� (daily census statistics by ward) managers do not request special reports from patient statistics or the budget systems. Managers at all levels are apparently uninterested in the monitoring reports. This lack of interest reflects both the fact that the data do not reflect performance on explicit management goals and objectives (because there are not specific goals for each facility) and because the monitoring measures on the reports say little about managerial performance. The most serious issue in management performance relates to the limited scope of autonomy of health facility managers. Unlike facility managers elsewhere in Europe and in North America, managers in Kosovo are not given autonomy to perform the usual range of management functions such as planning, organizing, leading, and controlling resource utilization. Hiring, firing, and salary setting is governed by civil service law, budget rigidities, as well as other laws and policies. Accountability for the facility manager is also essentially non existent since specific goals are not explicit or written, the ownership of the results not clear, nor are there any recognition incentives (other than the negative possibility of reassignment). The facility manager is viewed by the organization as a supervisor or overseer of a rigid production process for which he is given resources to supervise. Performance improvement in health facilities requires a more modern organizational role for managers, featuring professional responsibility and autonomy. 3 The absence of point-of-service manager autonomy means that conventional policy instruments like incentive payment to improve facility performance will not be effective in this environment. Managers have insufficient flexibility over resource allocation to be able to respond to payment incentives. Recommendations. A number of short term activities to improve health care management information are identified in the report. But, this assessment does not support the idea of a major investment in health information systems at this time as a vehicle to improve performance of health facilities. Before better data and more modern approaches to performance measurement are made available, more autonomy needs to be available to managers so that they have the needed flexibility to act in response to the incentives. Important policy and legal changes will be necessary for this to happen. . 4 List of Acronyms BDMS Budget Development Managem3ent System CIDC Consulting company doing a project on primary health care payment reform CDFM Center for development of family medicine EAR European Agency for Recovery DFID Department for International Development (U.K.) DRG Diagnosis related group Epi Info Software package commonly used to analyze epidemiologic data ER Emergency room HLSP Consulting company doing a study of primary health care strengthening HMIS Health care management information system HR Human Resources KDSP Kosovo development strategic plan IPH Institute of Public Health LOS Average length of stay (days/admissions) MOH Ministry of health MOF Ministry of economics and finance MS Microsoft NHA National Health Accounts PEIR Public Expenditure and Institutional Review PEMTAG Public Expenditure Management Technical Assistance Grant PHC Primary health care SOK Statistics Office of Kosovo UNMIK United Nations Interim Administration Mission in Kosovo USAID United States Agency for International Development WB World Bank 5 1. Introduction This report assesses the opportunities for improving the performance of the government health sector in Kosovo through better management and improved information for managers. Specifically, it concerns the kinds of information used by managers in the health system, and indications of demand for additional data and performance measures. Overall, we try to assess the need for new investments in information technology in the health system as a means for improving health system performance. The health system in Kosovo is a government system of six regional hospitals, several smaller facilities, all supported by a large tertiary university hospital and associated specialists in Prishtina, and a set of municipally-managed primary health centers (and their subordinate health houses and ambulances throughout the countryside). Written referrals are required for secondary and tertiary care. Essential drugs are free in the clinics, if available. The government operates and finances this system in a traditional line budget way by paying salaries directly from the MOF, and setting strict budget limits for facility managers for goods and services and capital. Municipalities receive a capitation grant from the MOF to help to finance the clinics. Modest co payments are collected for ambulatory care. There is essentially no private insurance. Levels of funding are very limited, and physicians are moonlighting in the unregulated private sector. Anecdotal incidence suggests, patients prefer to be treated in the private sector in Kosovo or drive to clinics in Skopje. A modest national fund for financing some tertiary care services not available in Kosovo is managed by the MOH. We do not attempt to document the organizational performance of the government health facilities here. Many of the inefficiencies are noted in the PEIR. By way of a general summary, the kinds of systemic problems experienced here are commonly seen in developing countries. Secondary care facilities are underutilized but receive about half of the budget monies. Health behaviors are poor, particularly relating to reproductive health, where maternal and birth outcomes are very poor. Most telling about the service quality of the government health system, the use of the private sector and nearby out-of-country providers is growing, and may now exceed 50% of ambulatory care used by citizens. The report summarizes the data collection activities (section 2), the framework for analysis (section 3), the findings from the study (section 4), recommendations (section 5). Annexes contain data gathering forms and protocols referenced in the text. 6 2. Data Collection Activities. Between April 11 – 21, 2007 existing data systems and their use in the government health system was assessed through interviews. Central MOH officials and information system experts, regional MOH officials, and facility managers were interviewed, in about 30 meetings, including the following:  Meetings with MOH staff (including IPH) to understand data sources and the way data is used in management and monitoring  Meetings with staff from the Ministry of Economics and Finance and the Statistical Office of Kosovo (SOK)..  Meetings with PEMTAG teams (contractors HLSP and CIDC), and staff from the CDFM and DFID ---to discuss data and observations on accountability relating to municipal PHC clinics  Meetings with hospital directors, financial officers and directors of nursing in 3 facilities (University clinic, 2 regional hospitals, one MOH regional director) to understand existing management and monitoring requirements for data  Meeting with Data Guard Ltd---the commercial vendor that has been supporting the existing MOH/IPH patient statistics system, the developer of the pharmaceutical warehouse system, and the developer of a modified primary care system (registration, clinical activities, human resources, pharmacy) used now in 3 pilot sites. These meetings were guided by a framework of questions which were assembled prior to the meetings. The interview guides appear in an Annex to the report. These guides were used flexibly to structure the conversations with respondents during meetings. This report is also informed by a PEMTAG contractor’s Inception Report (Primary Care Restructuring, HSLP, March 2007) and the HMIS Draft Strategic Plan (MOH, 2006) were used as source material. Interview protocols (Annex 3) were used to guide the interviews. Information about PHC was also taken from discussions with the authors of the Inception report and other PEMTAG consultants (CIDC). Information was also taken from a previous visit to the Regional Director, the Municipal Clinic and Regional Hospital in Prizren. This report examines aspects of management, control and information systems in the MOH system relating to the capacity to improve management performance. This is not a comprehensive study of management or data in the MOH system. Nevertheless, it does 7 describe how information is used by managers, how control is maintained, and how managers are generally behaving. 3. Framework for the Analysis There may be universal measures for evaluating the performance of health systems. But, there are not any universal measures for evaluating the performance of managers of health facilities who are parts of larger organizations. Those organizations like the MOH and each municipality hire and task facility managers to accomplish certain things, which can vary widely across the system, and change from year to year. Managerial performance assessment and accountability actions by executives are enabled by performance measures coming from data systems. Therefore, the adequacy of the data systems for strengthening managerial accountability in Kosovo cannot be assessed independently from the roles and responsibilities assigned to managers. What is effective management in a large organization? How do organizations hold individual facility managers accountable for performance? How should data and performance measurement fit into this? These issues are central to this assessment of the situation in Kosovo’s health sector. In this section we outline a framework for thinking about how data is needed in organizations to manage and achieve accountability. This section begins with a brief and simple review of the organization’s need for managers and control over them, followed by some definitions. 3.1 Controls in the Organization and the Management Function Organizations (like the MOH) have several techniques for assuring that their work is done to achieve the desired result or goal. In very small, simple organizations, the highest level of decision making and goal setting is able to see the entire organization, to appraise what is working, and what is not working, and to directly supervise all aspects of planning and operations. Control is easy, because the executive can see everything personally. When organizations grow the problem arises of how to control all the pieces (or business units). Two situations are possible. When the environment of the individual business unit is well understood and not unduly influenced by external factors, the supervisor of the business unit is typically given the required resources and the responsibility for keeping them working to standard levels of productivity. The supervisor has very limited 8 discretion. No need exists for the organization to ask these persons to plan ahead, no point in giving them a budget which might allow them discretion in allocating resources. They have a stable environment and they are given a complement of resources to do the work--- and the supervisory job is to keep the resources working. When organizations have business units that operate in unstable or changing environments a much more complex control problem arises. For example, in a health clinic or hospital the nature of demand for services (and the resources to produce them) can only roughly be anticipated. Patients are different and the resources needed to deal with their presenting problems are different. The pace with which they will flow to the clinic to get service is not known. So, the clinic or the hospital does not know from day to day, or even from year to year what the specific test, drug and procedure volumes are going to need to be. They also do not know the needed requirements for specific kinds of technicians or specialists or nurses or administrative staff. At the point of service of a health care delivery organization (like the MOH) the specific resources needs cannot be known in advance. Organizations typically respond to this problem of “uncertain production requirements� by hiring a manager for the unit. This would be the job of the manager of the PHC clinic, or the hospital director. This manager has to produce the work products of the business unit, but within the complexity created by the unknown day to day requirements of production. In the MOH or Municipal settings, the point of service manager directs the deployment of resources to accommodate the flow of patients and their needs for tests, procedures, advice and all else they require. So, how does the organization control such a business unit manager? Certainly, such managers are not given a blank check to acquire whatever resources they need whenever they need them. That would compromise the organization’s own goals. Control is typically achieved by two means. (1) The manager is given goals for the business unit (that align with the entire Organization’s goals) and told to make the necessary decisions in terms of using this or that combination of resources so as to achieve the goals. (2) The manager is given a budget, or an envelope with which to purchase the needed resources, as those needs arise. 9 The budget is a critical tool of control of the organization---- it recognizes the intrinsic need to be flexible with respect to the exact level and mix of resources that will be needed. But it sets an overall limit on the level of resources over which the manager has discretion. Within a large organization there are many kinds of business units---so with situations of reasonable stability, suitable for being directed by a supervisor, while other business units will require a manager to deal with the more complex problem of an uncertain demand. 3.2 Definitions of Management and Accountability A definition of Management from Wikpedia is: Management: -- the deployment and manipulation of human resources, financial resources, technological resources for the purpose of coordinating and harmonizing that group towards accomplishing a goal The key elements here include (1) the autonomy to engage in “deployment and manipulation of resources� and (2) the goal-orientation of the management job. Managerial functions or activities help to clarify the scope of autonomy that might be part of the managerial situation: To pursue the goal, the manager typically has autonomy to engage in: • Planning: deciding what needs to happen in the future (today, next week, next month, next year, over the next five years, etc.) and generating plans for action. • Organizing: making optimum use of the resources required to enable the successful carrying out of plans. Sub units are organized, managers and supervisors selected and trained. • Leading/Motivating: exhibiting skills in these areas for getting others to play an effective part in achieving plans • Controlling: monitoring — checking progress against plans, which may need modification based on feedback. This is done for all sub units under the manager’s control Managers need discretion to command their business units. And, to the extent they are performing that job well to meet the goals they have been assigned they engage in thinking ahead and planning, organizing resources, leading the staff resources they have assembled, 10 and controlling the production process and monitoring its effectiveness. What is and effective business unit? That must depend on the goals that are set by the organization. It surely is not set externally. This is discussed below. A definition of Accountability from Wikpedia is: Accountability--A personal choice to rise above one’s circumstances and demonstrate the ownership necessary for achieving desired results; to see it, own it, solve it and do it. Essentially, this means that being accountable in the organization, the manager must take personal responsibility for the stated goal, and in return that manager is formally recognized within the organization for the result, whether favorable or not---and typically rewarded appropriately. Originally, accountability is derived from “being held to account�. Organizations apply managers and accountability processes in the following way. To do work the organization is structured into units by levels, location, and function. Managers are assigned to these units. Goals are set with these managers so that their unit’s objectives are fully aligned with the goals of the larger organization. Managers are held accountable for their performance by their superior in the organization by a simple means--- was the goal achieved? 3.3 Role of Data and Performance Measurement What kind of data and performance measurement is needed to accompany the management role? When managers are running business units in a large organization, data is needed for three critical reasons. (1) Data is needed at the end of the year to measure goal attainment (or degree of goal attainment). This is required to measure the performance of the manager, to hold them accountable for meeting their goals. (2) Monitoring Data is needed in the organization to help the manager manage. Specifically, the manager organizes the production processes in the business unit, assigns supervisors and managers, and sets goals for these units as appropriate. Data allows the manager to see how well the subordinate activities are working so as to make needed control adjustments. (3) Monitoring Data is also needed by the manager to regularly monitor how well the unit is doing to achieve overall business unit goals on which she will be ultimately held accountable. If there are going to be problems meeting goals, the manager 11 wants to know as early as possible in order to have as much time as possible to devise and evaluate options. What kind of measures are possible? What should be measured? Some types of measures have more value in practice than others. There are several dimensions of measurement that reflect the value of measures in evaluation the effectiveness of management. These considerations are: (1) Measures and monitoring indicators of the goals for a clinic or hospital manager should be reflections of the organization’s goals, not externally imposed ideas about what should be the goals should be. The goals given to a clinic or hospital manager by the organization are derivative of (but not the same as) the organization’s needs for achieving its own goals. For example, the manager of a hospital with excess capacity (due to low utilization) may be given goals that include expanding surgical volumes, and improving service quality to restore community trust. These goals may be strategically critical to the MOH. There may not be a goal relating to “efficiency� per se, or “downsizing�. This institutional goal setting is the responsibility of the organization’s executive. (2) The analytic content of measures does vary. Some measures speak more or less directly to the performance of the manager, while others require considerable additional analysis to understand the implications for managerial performance. The issue of measurement value is how well the goal indicator discriminates between an effective and an ineffective manager. The hierarchy is:  Raw counts or amounts. Examples would be numbers of patients, visits, procedures, spending on staff, or spending on utilities. These measures are of least value because they demand the user to do work to convert them into useful information about the performance of the manager of the ward or the clinic, or the hospital.  Measures that correspond to concepts of managerial performance. These measures might relate to concepts of efficiency, productivity, service quality, capacity utilizations, intensity of care, compliance with standards of care and the like These measures might include length of stay (days/discharges), visits per physician, spending per patient, occupancy rate, etc. These measures are typically ratios of raw counts and amounts.  Measures that are adjusted for factors not under the control of management of the facility. Managers are not able to control all aspects of their environment. For example, a manager of a very full hospital might be given a goal to reduce the average length of stay in order to allow the facility to accommodate more patients (in order to avoid an expensive expansion). If the LOS is not reduced does that necessarily mean the manager has not performed well? What if the hospital mix of cases had 12 changed and an unusually large number of long stay patients had been admitted? To adjust for this the LOS measure can be standardized for case mix (using DRGs or some other patient classification scheme). After standardizing LOS for the case mix, the measure will reflect how good a job the manager did in reducing LOS, given a standard mix of patients. These kinds of adjustments can be done to eliminate the influence of uncontrollable influences like volumes of patients, mix of patients, salary levels (if civil servants get a raise from the government), and prices of drugs and x-ray film.  Measures that are compared side-by-side with benchmarks. Measure or indicators by themselves are often difficult to interpret as to their meaning about managerial performance. If a manager is given a goal of “being an efficient hospital�, what does the following year end performance imply about his success: LOS 6.2 days Cost per inpatient day 456 euro Occupancy rate 73 % Staff per bed 2.1 It is difficult to form an opinion about performance here, and possible only if the person doing the interpretation applies additional data or standards. It is helpful to compare performance indicators with benchmarks of performance for (1) the same facility in prior years, (2) other similar facilities, and (3) normative standards of performance. The following display would be more helpful in assessing the manager’s performance: Hospital Last Year Other Hospitals LOS 6.2 days 6.1 5.9 Cost per 456 euro 453 euro 422 euro inpatient day Occupancy rate 73 % 73% 76% Staff per bed 2.1 2.2 2.0 Managers need to see feedback on how they compare with other managers in similar situations if they are to be motivated to perform better. And, the organization needs to evaluate the manager’s performance in context of all other managers’ performance or in comparison to last year. 3.4 Framework for Assessment of Management Performance To assist in understanding the factors on which performance improvement is codependent, we offer the following simple table (Table 1). As rows we identify the aspects of facility performance that need to be created or supported. They are the managerial motivation to 13 improve performance, the capacity to be able to be effective in doing so, and the need for the organization to hold managers accountable for goals. Three types of mechanisms or interventions for improving managerial and facility performance are shown as columns: financial incentives for facilities (and managers) to perform better, managerial autonomy for doing better resource management in response to incentives, and provision of the necessary data and measurement infrastructure. This framework is used in the findings section below to summarize our assessment and recommendations for improving facility performance. Table 1 Template of Mechanisms for Improving Health Facility Performance Means Performance- Organizational, Legal Data and Measurement Based Facility and Policy Restructuring Support Requirements Budgets or to Give Facility Managers by Level (central, Payment Flexibility to Manage municipal, facility) Performance Improvement Objective Improve motivation to improve managerial performance Improve the manager’s capacity to act on motivation to improve performance Improve Accountability of Managers in the Organization 14 4. Findings 4.1 Information ‘Systems’ in the Health Sector 4.1.1 Epidemiological data are gathered by the IPH using manual and phone reporting of certain diseases and medical events, and then computerized by using EpiInfo. Apparently, these manual and other systems are at least adequate to meet the needs of the public health officials responsible to monitoring infectious disease and other public health threats. No provisions of priorities in the KDSB relate to important systems investments for this purpose. 4.1.2 Patient data is gathered in every MOH facility (71 in total) using the EAR- Patient Statistics System, a simple MS ACCESS-based data system to report patient care statistics. It began in 2003 and was supported by Data Guard Ltd. from 2005-2007. It includes three standard forms (visits/stays, prenatal care, births, etc). The measures that are made from this form include visits, days, beds, diagnoses and procedures (ICD 10), deaths, type of specialty care, etc. It is not an electronic health information system and cannot be used to compile patient data since there is no unique patient ID in Kosovo. . The system is used in both hospitals and clinics in the same basic fashion;  The patient data is recorded in a manual ledger, maintained in the ward or clinic  Nurses complete the forms and data capture is generally managed by nurses  Computer data entry from the forms is done at a centralized location in the facility  Local (standard) reports are prepared at the facility level (usually monthly)  Data is sent monthly to the IPH on a floppy diskette.  Reports are published periodically at IPH and distributed. All utilization statistics (for hospitals and clinics) in Kosovo come from this system. Because of tardiness in keying the manual data in many facilities, the completeness of the reports is not good, though IPH staff seem aware of exactly how incomplete the data are. We identified no studies relating to verification of data quality or cross facility reliability of the data. In hospitals, sometimes daily reports are prepared (so called Midnight Report) showing the census and its characteristics. In some instances it appeared that the only summary data from the manual registries were being computerized by the facility and sent on to the IPH. 15 These data could be used to do side by side performance comparisons on hospitals and clinics. Measures of capacity utilization (using other data on numbers of beds and staff), case mix (using an adapted DRG grouper and some international weights), LOS, and some intensity measures could be made with some work. Community population data would not be applicable as denominators. The extant EAR patient statistics system in facilities is not a HMIS that could support patient care activities. While the data is gathered on encounter forms for each patient, there is no direct and reliable way to gather all of a patient’s records from the system, or even from the data base in a single facility. In municipal clinics, for instance, the EAR system provides no unique patient identifier which might be used to link patient visits so that providers could view treatment history, or to identify persons who need preventive or precautionary services (such as identifying all diabetics, or verifying that all children who are patients have had the right vaccinations). The system also does not capture any information on prescribing, does not include unique identifiers for doctors (which could be used to profile their performance), includes no linkage to budget or spending information, nor is the system networked across facilities or even with the IPH. Unfortunately, the EAR system cannot be used to track referrals, or compliance with the policy on referrals because of the problem of not having a unique identifier for patients. Patients going to regional hospitals, and to the University clinics (tertiary) must have a referral form completed by a primary care physician (emergency services are an exception). But, the EAR system cannot be used to link the same patient across facilities, in order to measure compliance with the referral policy. Since February 2007 the MOH is spending nothing on system support, and about 60 to 70,000 Euros a year to print the data entry forms used by the facilities. Data entry of the forms is as much as 6 months behind in some facilities that were interviewed. The reports produced by the system are standard, and not able to be modified at the facility level. However, according to Data Guard, the system had been sometimes modified locally, making it necessary for IPH to do custom programming in order to aggregate the data centrally.. 16 4.1.3 A Pharmaceutical System is being used to manage some of the drug warehouses. A separate Data Guard pharmaceutical system is being used in at least one (but we were told not in all) of the warehouses used by the MOH contractor to hold and distribute drugs in Kosovo. The system does track the drugs to pharmacies (which would allow tracking inventories) or to patients (which would allow tracking patient-level utilization by disease, etc). This means that this system is not adequate to control or monitor the distribution and inventories of essential drugs that are purchased and distributed by the MOH1. The new Data Guard system (discussed below) would be the sort of solution that would remedy these tracking problems. The new Data Guard system is now being used in three pilot sites (one ER and two PHC clinics). It has a pharmacy module. That module integrates (or more properly, would be able to integrate) with the pharmaceutical warehouse system, allowing (in theory) a complete in-country tracking of pharmaceuticals from procurement to distribution to prescribing. 4.1.4 Free Balance Budgeting and Treasury System. This public accounting system is used by the MEF to record and display budget monitoring reports and treasury transactions. The system is not browser based, so the reporting is done by the MEF. The budget for budgeted organizations (including hospitals and clinics) is entered in the system, and treasury reports of actual spending are recorded also. Monthly reports are generated by MEF about spending (some hospitals generate their own spending reports more often). Facility managers are very attentive to budget monitoring, and have sought real time mechanisms for accessing data. The MEF does not control day-to-day spending decisions by facilities. Once the budget is set (by budget line) the facility executive must verify that funds are available in the line before approving the spending. Typically, two persons need to sign every requisition to spend, and for certain items a procurement officer must sign as well. These two people are usually the facility director and the chief financial person. If, somehow, a requisition for excess spending would go forward, the treasury would not have funds to reimburse for it, 1 More discussion of this point can be found in a separate WB assessment of the situation in Pharmaceutical sector by Andreas Seiter. 17 and payment would be stopped. This means that overspending is not possible. A section below describes the budget process more fully. A new budget tracking system is also being installed by the MEF (called the BDMS) which allows budget formulation and tracking of treasury spending. This is not a browser based internet system, but is hardwired. Last year the municipalities were equipped with this new system. Recently the University Clinic was put on the system, and plans exist to put several of the other hospitals on the system as well. The BDMS (or even FreeBalance) could be used to support systems of performance payment to facilities, in conjunction with a system that kept track of volumes of care and any standardizing or grouping done based on diagnosis or procedure. The data in the FreeBalance system is inadequate for the breakdowns of spending for NHA. It would also not be adequate for costing studies in multi product facilities. For example, the hospital functions of inpatient and outpatient services are not treated separately by the system, so some arbitrary allocation of costs would be required in order to do a cost per inpatient day calculation. 4.1.5. Annual Household Budget Survey. A survey by the Statistics Office of Kosovo (SOK) has been done several times since the conflict, and now again in 2007. It includes some basic questions on health spending. A good review of technical methods is contained in the World Bank Report on Poverty in Kosovo, which suggests that the survey has faulty aspects in methodology which may result in underestimation of household spending on health (and most everything else). The instrument for 2007 includes an enumeration of categories of spending (where respondent households keep logs of all spending) and these are transferred monthly to interviewer records. These categories of spending are: 18  Health administration,  Medicaments  Other medical products accom, food, ambul.  Out-patient medical  Pharmaceutical  Traditional medicine services applianc.  Therapeutic equipment  Dental services  Other medical services  Medical analysis, X-ray  Hospital services  Health Insurance The basic spending questions do not match NHA requirements for out of pocket spending.2 There were also five supplemental questions asked about health matters in the most recent (2007) survey. These include opinions about health insurance and health services priorities. The survey results are being analyzed under a DFID funded contract3. 4.2 HMIS development activity to date in Kosovo has been quite limited and done only as a speculative investment by a single commercial vender, though the MOH now owns the license for the system. Data Guard Ltd., a locally owned and operated company, is owned by a former HMIS developer from the well respected British Columbia (B.C.) government HMIS group. He moved from B.C. back to Kosovo to start his own business. Data Guard signed a contract with MOH in 2005 to maintain the original EAR system, which it did through the end of the contract in February 2007. Totally on speculation, over the 2005-2007 Data Guard developed several new systems for (possible) adoption by the MOH. One of these systems is an integrated patient-based system, built around the EAR patient statistics system. The “new� system is browser based, and fully integrated across the following modules:  Patient registration (a plastic ID card is issued giving every person a unique 10 digit number)  Patient Clinical Data (roughly the same coded data as before, with provision for summary notes)  Pharmaceutical administration and prescribing (keeps track of pharmacy inventories, and patient prescriptions 2 See the Report on Preliminary Estimates for NHA by the World Bank, June 2007. 3 Gillett Steve, Vargas Veronica, et al: Kosovo: Results of Health Survey. HLSP. DFID Strengthening capacity at the Ministry of Health Kosova. June 2007. 19  Human Resources (gives a unique ID to every doctor and clinical person)  Unique IDs for all staff, facilities, and patients. Table 2 shows the comparisons with the EAR system. Table 2 EAR system and the New Pilot System from Data Guard Feature EAR /Data Guard System Modified Data Guard Number of facilities 68 3 Forms 3 (visit/stay, birth, prenatal) Same Doctor Role Complete Paper form Computer data entry Unique Patient Registration No Yes (plastic card) Integration with other No Yes (HR, Pharmacy) systems Network None Internet This new modified system is being piloted in three sites (Prizren hospital emergency unit and two municipal clinics). Doctors and nurses enter data directly into the system in the wards/exam rooms. Custom or standard reports (including comparisons) can be made by diagnosis, procedure, patient, doctor, clinic, etc. Data Guard now supports the pilot system in the three pilot locations, even though they are not being paid at all by the MOH. It is not clear if a new procurement of a system vendor will be done, or if negotiations with Data Guard will negotiate terms to continue. The license for the ‘new’ system was essentially ‘given’ to the MOH some time ago (except the pharmaceutical portion, where copyright was retained by Data Guard). A recent review of this system by the PEMTAG contractors (HLSP Inception activities) pertaining to family practice strengthening in four pilot PHC sites suggests that this modified system is very adequate for performing a full range of community practice prevention and promotion activities. The project will attempt to work out some arrangement with Data Guard Ltd. to support the system in the pilot sites. . 20 4.3 A strategic plan for HMIS in Kosovo was developed by MOH at the end of 2006. That plan was described by a MOH official. The plan is centered around the idea of rolling out of the ‘new’ pilot HMIS system developed by Data Guard to all 71 clinical sites (both hospitals and clinics). The draft strategy includes training, hardware, and software for an integrated pharmacy, human resource, clinical, and patient registration modules. An estimate of initial cost (including training) to implement in 71 facilities is about 13 Million Euros with about 240,000 Euros in annual running support costs (only about 3 times the annual cost of printing forms for the EAR system today). No MOH action has been taken on the draft strategy. As part of the strategic planning activity no evaluation was done (or planned to be done) in the three pilot HMIS sites. 4.4 Budget development and monitoring for facilities is done, though managers have little flexibility or autonomy to reallocate resources. Hospital facility budgeting is done according to law and UNMIK rules. The budget lines of (1) salaries, (2) goods and services, and (3) capital are rigid controls on resource allocation. Budget lines cannot be exceeded (indeed, salaries are paid directly by the MOF and the spending authorization is not given to the facility at all). Budget levels are set based on assumptions about inputs (rather than volumes of care). Levels are derived according to historical and political conventions, without any explicit regard to need, nor with any transparent adjustment processes to accommodate changes in circumstance. For example, the budget level of two hospital facilities the consultant visited (Ferizaj and Gjilan) were quite disparate. These hospitals had annual budgets per bed that were 14,000 Euros and 7500 Euros respectively, with no obvious reason for the high variance. An example of the rigidity of budgets to changes in circumstances was evident in one of these facilities, which had to immediately allocate monies (about 400,000 Euros) several years ago for repairs caused by an earthquake. The budget was not (and has never been) adjusted for this occurrence by the MOF, and financing of the unplanned spending was done by the hospital by postponing payments to vendors since that time. Today, the hospital budget is still about 300,000 “overrun�, and “vendor financing� is still being used to cover the problem of several years back. Like the earthquake situation, sometimes budgets do not accurately anticipate resource needs. How are these variances dealt with? There is a mid-year review of budgets with 21 MOF, and this represents the only time when adjustments might be made. However, adjustments are not very common or substantial. When unexpected variances occur in Goods and Services budget planning (where there is more director flexibility) directors often move money between planned items (supplies to drugs, drugs to utilities, etc). And, if at the year end deficits arise (use of goods and services exceeds the available budget), then this is often financed by simply not paying venders (e.g. the venders finance the deficit). This self-described “Balkan financing� has happened not only with goods and service, but also for “night shift� workers (whose pay sometimes lags behind their work). .For example, one facility reported that the recent dedication of hospital-collected revenues from co- payments to be used to pay the “nightshift� medical staff was still not sufficient. Monthly co-payment collections of around 2,800 Euros were simply inadequate to cover the bill of 8,000 Euros for night shift worker services. As a consequence, the payment to workers is being “delayed� in order to avoid a cash flow problem that will appear as a budget overrun. This form of “financing� is a consequence of the rigid budgeting process, and has essentially shifted the “financing� function of budget variances to the facilities and away from the MOF. This is unfortunate, because the facility managers do not have the necessary tools and financial autonomy to make management decisions and adjust their resource input as indicated by the circumstances and the incentives. The situation of budgeting is different in the PHC situation. Municipalities get block grants from the MEF. Anecdotally, grants are reported to be based on population size living in a municipality catchments’ area, and are subject to some negotiation. The health grant (or intergovernmental transfer) is transferred from the MEF to the municipality, with nothing going (or budgeted) to the facility itself. The Municipality health administrator, reporting to the Mayor, is responsible for transforming the block grant into a line item budget for the PHC facility, as well as for managing budget implementation. Staffing levels in some Municipal clinics is reported (anecdotally) to be quite high. Shortages of essential drugs and other supplies are also reported to be widespread. The facility managers in these clinics apparently have no authority at all over spending, so have no managerial role in resolving these problems (e.g. by cutting staff expenses in order to raise productivity and free up resources for buying essential drugs). Capital budgets for hospitals are set in negotiation with facilities and controlled centrally through the budgeting process. When slow starts or delays occur in capital investment, the budget process does not permit carry-forward monies from year to year. The new years’ budget may award funding limits that recognize the unfinished (or even un-started) 22 projects. But, often some funding is lost as a result, which needs to be internally funded. There is no long term Master-Plan based on population need forecasts to govern the direction or timing of capital investment in either municipalities or in hospitals. The implicit policy is that every existing facility is needed in the system, and renovation is a necessary and preferred way of investing in the future health system. Capital budgets for PHC sites do not exist at the health facility. The Municipality manages the setting and implementation process, with input in most cases from the director of the main clinic. 4.5 Historical data is often not available to manage patient care. Clinicians in the government hospitals have virtually no data on medical history or prior visits. In both, hospitals and specialist outpatient clinics, there is no direct mechanism for recovering medical information even when prior episodes existed, because the files are typically not returned to the ward. Repetitious data collection from the patient is required. Patients seen repeatedly as inpatients in one facility, or as ongoing patients of one specialty clinic, may have recoverable data for subsequent occasions of service, but this appears to vary. In municipal clinics there is a historical record (personal health card) which allows basic demographic, health history, and health services information to be retained at that clinic. To the extent that patients visit always the same clinic, this will be useful. But the utility diminishes when private providers are occasionally used or if care is sought internationally from time to time. Managers in health facilities do have some limited management information. Ward managers do not have budgets, and generally do not get any statistics performance reports. At the higher level, facility managers get reports on budgeted spending and typically get monthly patient statistics from the above mentioned systems. Often the mortality statistics are reviewed monthly. In some facilities, daily “midnight reports� are prepared on the census. But mainly, the information coming from patient statistics is not usually timely enough to register surprise or be used for any management purpose inside the facility. Special reports are not requested by managers. 23 4.6. The most serious issue in management accountability relates to the limited scope of staff and budget authority of health system facility managers. Facility managers have little autonomy. They have little or no authority over staffing levels, staff selection and staff performance expectations, capital strategy, and little flexibility in spending allocations. Essentially they have no conventional management tools to extract better performance in the facility. In concert with this situation, there are no expectations written down about facility performance. Nor would there likely be data to measure any expectations against benchmarks such as for cost, patient satisfaction, quality of care, etc. Control over staffing levels, salaries and employee actions is retained at the central or municipal government levels, and much of it governed by civil service law. About a third of the health facility spending in Kosovo is staff related cost. Salaries are paid directly to facility staff by the MOF. Most doctors employed in the public sector, are free to moonlight in the private sector and appear to work as little as possible in the MOH facilities, using it as a way of identifying patients with money to spend. In Municipal clinics the numbers of support staff is incredibly high, which is not surprising considering that the Mayor’s office is in charge of hiring. Facility managers have essentially no authority over staff, or their compensation because of civil service law, and other policies. Once the budget is set, nobody has management authority to make timely deviations to deal with changing circumstances. The general rigidities of budgets and staffing control barriers make monitoring and control less important throughout the hierarchy of the MOH. We looked at what managers do at various levels of the health sector (below). Monitoring of spending and patient statistics is simply not done well, nor even lamented, because there is little opportunity for corrective action in the system. There must be freedom of action to manage resources and results. It would be wrong to conclude that ‘management’ in any conventional sense of that term is being done. 4.7 Monitoring reports produced for managers are not useful for managing with an eye to improving facility performance. Discussions with persons at the different levels of the system identified the primary functions performed and related monitoring reports. These are listed in Table 3 below. 24 The reports produced to “monitor� are not very useful for management purposes. They do not focus on indicators of performance, or adjust the measures for obvious factors (like scale) that might help interpret differences. Reports never offer comparisons or benchmarks that might help to isolate performance problems or successes. Monitoring data from the patient statistics system do not focus on indicators of performance, but represent displays of numbers, which await analysis in order to provide information. The reports include compilations of raw numbers: number of patients, procedures, referrals, deaths, and counts of patients with particular diagnoses. The creation of performance indicators is usually not done (average length of stay or occupancy rate would have to be computed from the counts, procedures per patient, referrals per patient, prescriptions per patient, and the like would also have to be computed). The budget monitoring reports compute no performance measures, just spending against budget to date by budget category. In sum, there are no monitoring measures of indicators like efficiency, capacity utilization, case-mix, intensity, unit service cost, labor productivity, nor any measure of quality. Budget and patient statistics monitoring reports do not contain any indicators that standardize data for important but uncontrollable factors such as patient mix, or civil service wage increases. Thus, indicators cannot be used to make assessments of facility management performance, or to compare performance across hospitals. Budget data are not standardized for volumes of care provided (against an expected number). Nor are the patient volume statistics in any way standardized for case complexity or service mix. 25 Table 3 Management and Data for Monitoring by Level in the Health System Level Main management functions Typical Monitoring Data Central MOH policy development, selection of facility quarterly budget and statistics reports from hospital facilities Officials managers, budget formulation, and allocation and from IPH across facilities(with MOF). Procurement of monthly budget reports on spending against plan from facilities and pharmaceuticals and run central systems for from MOF data capture. flow of reports from IPH regarding patient statistics and staffing levels (not clear who uses this) Regional monitoring/ mentoring facility managers, same reports as hospitals complementary reports on patient MOH Director problem resolution, facility budget statistics for PHC clinics formulation (participation) Note: Regional MOH activities are done by an office that may have a staff of 5-6, with an experienced (former) facility director as the head. The MOH director spends time in hospitals, serving as a mentor to younger facility directors, and helping directors cope with the rigidities of the budgeting systems, etc. No special reports are prepared or sought, no particular attention to improving facility performance, other than minimizing negative impacts of unforeseen events. Hospital budget formulation and monitoring, cash flow midnight report (daily census report on patient volumes by service) Director variances resolution, staffing problem monthly statistics report resolution, capital project management and monthly spending report (the univ clinic and 2 others have on line procurement and purchasing management monitoring including approving requisitions Ward Manager patient care and data capture for EAR system manual registries and sometimes historical charts on some patients Municipal budget formulation and monitoring, cash flow quarterly statistics report on clinic volumes Government management, capital project monitoring, monthly spending report from MOF Director facility director monitoring, control over staff on spending from the Grant levels, and procurement and purchasing activities PHC Clinic budget contributions, staff management, only data available are manual registries and sometimes Manager requisitions for purchases (g&s) historical charts on some patients data capture for EAR statistics system 26 Consistent with these limitations in what is measured, there is no use of analytic benchmarks with which to make judgments about performance of a particular facility. For example, the monitoring reports seen by managers contain no side by side comparison to similar indicators for prior periods, no comparisons to external benchmarks set by the MOH, nor do they contain side by side comparisons of similar facilities (e.g. for similar sized facilities). In short, the two principle sets of monitoring reports - budget and patient statistics - are not very helpful in monitoring indicators that a performance minded manager might want to know. Also, the budget and patient statistics lack to provide the necessary information about performance to an organizational executive director. Apparently, no managers (at any level) request special, non standard reports to assist in their work. There seems no knowledge of performance differences across facilities, and no concern that such differences represent opportunities for improvement. Accountability (assessment of facility managers by executives) cannot be based on these monitoring reports in order to assess facility and managerial performance. 4.8 Conventional policy instruments like incentive payment aimed at improving performance will not work in this environment. Without managerial autonomy there is no point in trying to implement financial incentives for facility managers. Sophisticated measurement systems and better monitoring systems will not give managers the tools to manage resources differently in order to respond to the incentives. PHC managers, for example, do not receive budgets, and have no authority to move any resources around. Changing the formula for the block grant to the municipalities (to some form of performance based incentive budget) would not seem to have any effect on the clinic behavior at all (since the clinic director has no authority at all to modify resource utilization). A change in the health grant formula might encourage a change in behavior of the mayor or other municipal officials, but, would only influence clinic manager behavior indirectly, if at all. To change anything at the clinic level, the clinic manager would have to be given authority and flexibility in allocating spending, jobs, and staff performance. Hospitals could be given per case payments, for example, in order to provide incentives to promote, efficiency, lower LOS and higher capacity utilization. But, to manage input factors, the hospital director needs to be able to shift resources, resize the facility and the 27 workforce, and change the internal culture to effect change in these efficiency indicators. None of this is possible now with or without more sophisticated measurement and monitoring systems. The biggest problem in creating conditions for efficiency-enhancing incentives to work will likely be job and staff control, both of which are governed by civil service and other laws and policies. It will be very difficult to get political support to change these laws to give the authority to hire and fire and set salaries to the health facility managers. The benefit of retaining this authority by political parties and leaders is likely to be seen as huge relative to the benefits of improving the economic performance of health facilities. Without this control there is no real way to materially improve facility performance. This is what decentralization to the facility level (autonomy) is all about—making a manager responsible and accountable for performance, and giving him/her the tools to be able to do it. Authority over goods and service purchases (drugs, utilities, supplies, etc.) is too small (around 25-30% of facility budgets) to have large effects on facility performance, though it would help. The payoff of giving managers control of capital spending will not have an immediate payoff either (the payoff may take years to see). The strategic importance of facility investments should be retained by central authorities anyway, and guided by a Masterplan for the health sector. 4.9. Demand for better data and the willingness of government leaders to spend budget money on measurement will only follow expansion of management autonomy There is a demand for data and performance measurement in the MOH and in municipalities. The rather modest contract for the sole support of the EAR patient statistics system with DataGuard was allowed to lapse in February. The annual cost of that important support contract was not much more than the cost of printing the manual forms used to capture the patient data from caregivers. This lack of support for one of two data systems used for performance monitoring in the health sector may, in fact, be more rational than first appears. Today, facility managers do very little significant decision making at any level. Their hands are largely tied by law relating to the budget lines, civil service, and other policies. In this situation what are the benefits to be derived from reading monitoring reports? What actions can be taken even if we see something we don’t like? It is not at all surprising in this rigid environment for health facility managers that no one gets too worried when it is discovered that one hospital has a budget nearly double another in the same region (on a per bed basis). Such anomalies evoke no management concern or even 28 curiosity. They are simply unavoidable and unimportant consequences of the way the business of budgeting is done. No action is possible. Why puzzle over the performance implications of the sources of the problem? There is no evidence of demand for more information at any level. No patient feedback information is sought or collected by facilities. No one attempts to develop performance profiles of doctors or side-by-side performance assessments of facilities. There is no evidence of central or regional management lamenting lack of performance measurement information. Facility managers who have special computer systems staff do not seem to be asking for any special reports from their staff. At all levels, data needs and performance measurement is simply not on the priority list for managers. A supply of more and better data might change this attitude. Making facility directors accountable for facility performance would definitely change this. But, making them accountable for aspects of performance they cannot control will not solve any problem. Demand for performance information is ultimately going to have to follow policies that provide more autonomy for managers to allocate resources and manage staff. 4.10 The role of the directors of health care delivery facilities is somewhat ambiguous The organizations of MOH and Municipalities do not treat facility directors as traditional managers, but they might be seen as supervisors instead. Many, if not most of the findings noted above are the result of this ambiguity. There are budgets for use by the directors to adjust resource use to meet need, but the line item and civil service staffing controls prevent any meaningful flexibility for the director to meet community needs and the uncertainties of patient flows. The measurement systems provide “monitoring� information, yet there has been no real effort to set explicit goals for the managers, nor by the managers of their sub unit directors. The monitoring measures are unusually unhelpful in monitoring managerial performance, and probably a reflection of the lack of interest in “monitoring� since no explicit goals have been set. The directors are neither responsible managers, nor just supervisors. Because of the intrinsic uncertainties of the flows of health service needs, point of service managers (hospital and clinic directors) need flexibility in budget resource allocation. But the organization needs to have control. This control, comes from limited budget authority 29 (but ability to adjust staffing and move resources across lines) and from periodic accountability to achieve explicit goals. None of this is being done yet. 5. Recommendations. At this moment, there is no reason to believe that an investment in HMIS will improve health sector performance. The motivation and flexibility needed by managers for this to happen is simply not present. Indeed, the Director of the CDFM confirmed that a system for capturing better data on populations and patients is not, in his view, a high priority at this time. Hospital directors are more inclined to believe that having more computers and ‘systems’ are worthwhile, though it was pretty clear that this desire had nothing to do with achieving better performance. Moving quickly to invest in HMIS is not indicated here. However, there are several actions related to “preparing the ground� for HMIS investments in the future. These could be coupled together into a “HMIS: Phase 1 Project� 5.1 Evaluation of the Ambulatory HMIS Pilot Sites There needs to be an evaluation of the new Data Guard information system in the three pilot sites where it is now installed. This is not the same set of sites as the PEMTAG activities. If PEMTAG’s family medicine restructuring project (HLSP) supports the Data Guard system in those four sites, then there will be seven sites (six clinics and one emergency room) which could be evaluated. Currently, there are no plans to do such an evaluation of system interventions. An evaluation could examine: (1) Cost and resources used for training and support, (2) the way the system is actually used in patient care and monitoring of patient care (3) the way the system is used to track community needs and do outreach for preventative services (immunization, pap tests, diabetic monitoring, etc.), (4) how the clinic director and others in the reporting chain use the system’s reports, (5) data quality and timeliness issues. The evaluation should also look more closely at the costs and benefits of moving ahead with the modified Data Guard system in the health sector in Kosovo. There have been investments by Data Guard in this system, and the system is largely owned now by the MOH. But, before pursuing further investments to roll out this system, by some sort of 30 contracting vehicle, there needs to be a careful and independent analysis of benefits and costs of this modified system relative to other viable solutions that might be available. 5.2 Establish a Hospital HMIS Pilot Project. A hospital pilot has not been used in PEMTAG work, nor as a test for any system. The ER at Prizren hospital is one of the three sites now using the Data Guard system, but the inpatient wards are not included. Like the four site municipal clinic pilot where PEMTAG contractors are working, the hospital pilot would (1) couple an information system intervention (e.g. the Data Guard system, including the pharmacy module) along side interventions dealing with (2) some form of provider payment and more budget autonomy for the directors, (3) management training for the directors and the top management staff, and (4) an evaluation focusing on the same sorts of things as suggested above, as well as outcomes relating to the overall performance of the facility. 5.3 Catchment Area Pilot of Autonomy for Health Facility Managers. It would be possible to create a pilot where facility director autonomy (in budget, staff control, other ways) is stipulated and simple systems built to support the measurement needs. This could be done in a catchment area or two (a hospital and co located clinics and ambulances). Unfortunately, it is not evident now that there is any appetite for devolving the public system as a means of saving it. Unless this becomes a commonly understood diagnosis of the reason why both providers and patients are flowing to the private sector and other nearby counties to get care, it seems that putting more information resources into the system will certainly not create a magical solution. More provider autonomy and flexibility to be a manager is required. In this regard, we recommend the following: 5.4 Create a legislative provision for the MOH to occasionally waive certain laws and policies in order to conduct pilot studies that have the potential for improving government or the services it provides. Governments need to be able to experiment with programmatic matters to better understand the consequences of options on important outcomes like cost, efficiency, equity and other program performance goals. Waiver authority should exist to permit the MOH to establish pilot studies that waive existing law in order to do such studies. 31 5.5 Create a Performance Measurement Unit in the MOH. The level of awareness and skill in measuring performance is very low. Facilities do not have these capabilities, the MOH does not have them, and SOK does not have them. It is not about statistics per se, nor about data collection. It is about how to measure facility performance, how to standardize these measures for region and patient differences, and how to make meaningful standards or benchmarks of performance. The needs of the persons doing these things for the MOH health system will drive (in part) the data capture needs of any new HMIS. These staff would not be typically located in a Health Insurance Fund, though some of the same knowledge and skill would need to be located there as well. Creating, mentoring, training, and supporting a small unit in the MOH would be the nature of the proposed project. 5.6 Capital Project Strategy Recommendations A master-plan for hospital facility construction priorities should be done immediately to guide budgets for capital projects within the MOH facilities... 5.7 Summary of All Recommendations The problems outlined here point to a systemic lack of capacity and motivation for improving the performance of the government health system. There is little available data to support it, to be sure, and there is no evidence that available data are being used creatively to support the objective. There is no capacity of managers in the system to improve resource allocation in their facilities because of rigidities of budgeting and control. To assist in understanding the factors on which performance improvement is codependent, we offer the following simple table. Three types of interventions are identified here: 1) Improving provider payment, 2) creating more autonomy for resource management at the facility level, and 3) creating enabling data and measurement infrastructure. A number of recommendations pertaining to performance improvement are evident in Table 4. 32 Table 4 Summary of Mechanisms for Improving Health Service Delivery Performance Means Performance- Based Organizational, Legal Data and Measurement Facility Budgets or and Policy Support Requirements Payment Restructuring to Give by Level (central, Facility Managers municipal, facility) Flexibility to Manage Objective Improve Design and adopt More autonomy for facility Create a performance motivation to policy to pay more to managers to control measurement unit within MOH improve facilities that do more, resources (including staff) to support performance managerial or do it better measurement and performance Set explicit facility goals benchmarking needs and have subordinate units (wards, others) set goals too Improve the Develop and evaluate Skills development training Supply facility managers with manager’s capacity pilot payment for managers at all levels information (e.g. patient to act on technologies and satisfaction, quality, utilization, motivation to operational measures Provide managers with productivity etc) to identify improve on which the payment support for measuring staff internal unit performance performance technology depends performance problems and help monitor remedies Develop legal basis to Relax civil service and waive laws and policies resource mobilization in order to do pilot restrictions studies Improve Establish Managerial Promotion/retention of Conduct Performance Accountability of rewards for achieving managers based on Evaluation for all staff. Managers in the goals performance Supply managers at all levels Organization with timely and properly adjusted comparative (side-by- side) goal indicators 33 Annex 1: Interview Guides Protocol ---Care sites Name of individual Role Facility/Organization Date ____________ _________ _____________ _______ 1. How does the organization measure its own performance? What are the key indicators you rely on? Performance concept Measure Data Source (get forms) 1. _________________ ________________ ____________________ ________________ ____________________ 2. _________________ ________________ ____________________ ________________ ____________________ 3. _________________ ________________ ____________________ ________________ ____________________ 2. How are computers and surveys now used now in this measurement work? Computers-__________________________________________ Surveys __________________________________________ 3. Have there been plans to make more use of computers (get plans)? __________________________________________ __________________________________________ 4. What data do you report routinely to regional/central authorities? (get forms) Data Items Regularity To whom reported 1. ___________________________________________ _________ ______________ 2. ___________________________________________ ________ ______________ 3. ___________________________________________ _________ ______________ 5. How are the forms completed from source data? 1. ______________________________________________________________________ 2 ______________________________________________________________________ 3. ______________________________________________________________________ 34 6. What would your vision be of a more widespread availability of computers and computerized data? Point of data Capture types of data role of this site in system 1. _______ __________ ________________ ____________________ ________________ ____________________ 2. _________________ ________________ ____________________ ________________ ____________________ 3. _________________ ________________ ____________________ ________________ ____________________ E.g. patient care, provider, family order entry, utilization own system, national system Telemedicine encounters teleconference, second opinion 7. What other performance measures might be possible if there were computerized data? Can you give me examples of how this might enable better management of resources or quality? Potential Measure of value how Enable Improvements 1. __________________________________ ____________________________ 2. __________________________________ ____________________________ 3. __________________________________ ____________________________ 8. Aside from cost, what do you feel are the biggest obstacles in implementing an effective HMIS in this organization? 1.__________________________________________________________________________ 2.__________________________________________________________________________ 35 Protocol---MOH, IPH, HCAA Name of individual Role Facility/Organization Date ____________ _________ _____________ _______ I. Data 1. What are the principal data sources do you now use to manage your areas of responsibility? Function Data Sources Use of the data source of data 1.budgeting&spending _________________ ___________________ _________________ 2.facility budget monitoring___________ ___________________ _________________ 3.vendor performance_______________ ___________________ _________________ 4. public health monitoring___________ ___________________ _________________ 5. Rx inventory management ___________ _____________ ______ _________________ 6. capacity utilization_______________ __ ___________________ _________________ 7. professional staff performance_______ ___________________ __________________ 8. health status of population__________ ___________________ __________________ 9.________________________________ ___________________ __________________ 10._______________________________ ___________________ __________________ . 2. Are any of the data computerized at any point? (get forms/input screens) Name of System/Data Base Source of Data Function of computer Person Responsible 1. _____________________ _________________ __________________ _________________ 2. _____________________ _________________ ___________________ ________________ 3. _____________________ _________________ ___________________ ________________ 4. _____________________ _________________ ___________________ ________________ 3. Have there been plans to make more use of computers Or required national data sets, or new surveys (get plans)? 1. __________________________________________ 36 2. __________________________________________ 3. __________________________________________ II. System Management Activities 3. In monitoring or managing health facilities (hospitals, PH clinics, etc.) what performance measures are the key indicators you rely on? Performance concept Measure Data Source 1. budget setting/ tracking _ _______________ ____ ________________ ________________ ____________________ 2. _________________ ________________ ____________________ ________________ ____________________ 3. _________________ ________________ ____________________ ________________ ____________________ 4. How are computers used now in this measurement work? 1. __________________________________________ 2. __________________________________________ 3. __________________________________________ 5. What data do you report get from facilities or regional authorities? (get forms) Data Items Use it for What? Regularity Who Keeps 1. ______________ ______________________________________ __________ ________ 2. ______________ ______________________________________ __________ ________ 3. ______________ ______________________________________ __________ ________ 4. _______________ ______________________________________ __________ ________ 5. _______________ ______________________________________ __________ _________ 6. _______________ ______________________________________ __________ _________ 6. In the data you get from facilities, what are the most important inconsistencies in data quality and consistency? (examples) 1. ____________________________________________________________________________ 2. ____________________________________________________________________________ 37 7. What other performance measures might be possible if there were mandated and computerized data? Can you give me examples of how this might enable better management of resources or quality? Potential Measure of value How Enable Improvements 1. __________________________________ ____________________________ 2. __________________________________ ____________________________ 3. __________________________________ ____________________________ III. Vision 8. What would your vision be of a more widespread availability of computers and computerized data? What would your priorities be? Point of data Capture types of data role of this site in system 1. _______ __________ ________________ ____________________ ________________ ____________________ 2. _________________ ________________ ____________________ ________________ ____________________ 3. _________________ ________________ ____________________ ________________ ____________________ eg patient care, provider ,family order entry, utilization own system, national system telemedicine encounters teleconference, second opinion 8. Aside from cost, what do you feel are the biggest obstacles in implementing an effective HMIS in this organization? 1.__________________________________________________________________________ 2.__________________________________________________________________________ 38 Technical/Statistical Specialist Protocol Name of individual Role Facility/Organization Date ____________ _________ _____________ _______ 1. What are the principal data sources do you now use to manage your areas of responsibility? Are these sources computerized? (get forms) Function Key Uses of the data Refreshment 1.__________________ _________________________________________ _____________ Data Sources Person responsible __________________________ _______________________ 2._________________ _________________________________________ _____________ __________________________ _______________________ 3._________________ __________________________________________ _____________ __________________________ _______________________ 4. _________________. _________________________________________ _____________ ___________________________ _______________________ 5. _________________ __________________ _______________________ _____________ ___________________________ _______________________ 2. In the data you get from facilities, what are the most important inconsistencies in data quality and consistency? (examples) 1. ____________________________________________________________________________ 2. ____________________________________________________________________________ 3. Have there been plans to make more use of computers Or required national data sets or regular surveys (get \ plans)? 1. __________________________________________ 2. __________________________________________ 3. __________________________________________ 39 4. What do you believe are the most critical unmet data needs for doing your work? Unmet data need key measures or analyses permitted 1. ____________________________ ________________________________ 2. ____________________________ ________________________________ 3. ____________________________ ________________________________ 4. ____________________________ ________________________________ 5.What would your vision be of a more widespread availability of computers and routinely collected and computerized data? What would your priorities be? Point of data Capture types of data role of this site in system 1. _______ __________ ________________ ____________________ ________________ ____________________ 2. _________________ ________________ ____________________ ________________ ____________________ 3. _________________ ________________ ____________________ ________________ ____________________ eg patient care, provider ,family order entry, utilization own system, national system telemedicine encounters teleconference, second opinion 6. How would these ‘priority data needs’ lead to improved planning, efficiency and accountability in the Kosovo health system (some examples) 1.________________________________________________________ 2.________________________________________________________ 3.________________________________________________________ 7. Aside from cost, what do you feel are the biggest obstacles in implementing an effective HMIS in this organization? 1.__________________________________________________________________________ 2.__________________________________________________________________________ 8. What approach to phasing and implementation do you see as best for a HMIS in Kosovo 40 9. Will a person level health data set require a new identifier for each person in Kosovo 10. What sort of capacity now exists in Kosovo for design and implementation of HMIS? 12. Who else should we be talking to about HIS needs. _____________________________________________ 41