ICRR 11933 Report Number : ICRR11933 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/03/2004 PROJ ID : P005746 Appraisal Actual Project Name : Tn-health Sector Loan Project Costs 101 101 US$M ) (US$M) Country : Tunisia Loan /Credit (US$M) Loan/ US$M ) 50 50 Sector (s): Board: HE - Health (97%), Cofinancing 0 0 Central government US$M ) (US$M) administration (3%) L/C Number : L4294 Board Approval 98 FY ) (FY) Partners involved : Closing Date 12/31/2002 12/31/2003 Prepared by : Reviewed by : Group Manager : Group : Elaine Wee-Ling Ooi Helen Abadzi Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives Project objective was to support a program of major health reforms that aim at : a) improving the macroefficiency of public investments; b) improving the microefficiency of operations through management reforms; c ) enhancing the financial sustainability of publicly supported operations through introduction of policy measures with regard to cost controls; and d) reinforcing the quality of services through improved case management and staff training . b. Components There were 3 components: i) Facilitation of Sector Reforms and Key Actions : redefine functions of central level and regional directorates, and strengthen management autonomy of public hospitals; cost control mechanisms; a policy for greater private sector provision and financing of health services (US$5m); ii) ii) Priority Sector Investments : support essential investments in regional hospitals that would improve their referral capacity and ability to provide specialized care, rehabilitate 12 regional hospitals, strengthen national medical emergency services (NMES) and develop national health information system (NHIS) (US$91m); iii ) Institutional Strengthening in Health Policy and Planning :strengthen Health Ministry (MOH) and Directorate of iii) Studies and National Planning (DEP) in planning, policy analysis, and financial management; national health accounts (US$5m). c. Comments on Project Cost, Financing and Dates Actual project costs were US$101m and IBRD loan of US$50 m was fully disbursed. The project was extended by one year to enable full implementation of planned activities . 3. Achievement of Relevant Objectives: Overall objectives were achieved as follows : a) Macroefficiency of public investments -- The allocative efficiency (ie. right proportions of resources going to the different entities that make up the health system, eg . adequate amounts going to primary health and non urban health services) of the health system has improved . Due to the enhanced technical capacity and quality of services available at the regional hospitals, patient utilization at the regional level increased by 20-28%, thus easing the unnecessary patient load at teaching hospitals and allowing them to perform the more complex services they were designed to. b) Microefficiency of operations at the regional hospitals -- The number of afternoon consultations has doubled, and the average occupancy rates have increased by 18%. According to the ICR, while no actual (economic/efficiency) measurements were conducted, anecdotal information suggests improved administrative efficiency as a result of the management reforms, NHIS installation, and staff training . c) Financial sustainability -- The fiscal situation of regional hospitals has improved substantially, as the hospitals were able to recover expenses for services rendered to beneficiaries of the Social Insurance Funds . The amount recovered was 63.75% of costs of services rendered in 2003 compared to 2.2% in 1998. Accordingly, subsidies from the government to the operational budget of these hospitals could be reduced from 58.6% in 1998 to 7.1% in 2003. d) Quality of services -- A patient satisfaction survey (2002) reported that 50% - 80% of respondents were pleased with improvements in availability of medicines, waiting time, and respect shown to patients and families . Project supported items (medical equipment, 36 rehabilitated hospitals and medical emergency services ) likely contributed towards improved service provision . 4. Significant Outcomes/Impacts: Tunisia is among the first countries in MENA to have developed the comprehensive health information system (NHIS) that allows it to effectively monitor the National Health Strategy . The NHIS has significantly improved the Health Ministry's capability to allocate hospital budgets based on actual cost data . At the regional level, information on patient registrations and billing records has improved, helping the hospitals plan and monitor better, and recover costs . Fiscal sustainability of the Regional Health System has improved - allowing government subsidies to the operational budget of regional hospitals to be reduced to 7.1% in 2003. The Health Ministry has utilized findings from Costing Studies and Utilization Surveys to guide manpower allocation and develop new strategies and policies for the hospital network . 5. Significant Shortcomings (including non-compliance with safeguard policies): There was a lack of baselines and predetermined quantitative targets in project M&E for the many outputs expected in the project. They would have helped in determining if the achievements were reasonable or not . This is in spite of the "measurement" instruments /mechanisms available (NHIS, User Surveys and other Studies). Further, the project missed the opportunity to measure the efficiency improvements at the regional level. There was limited progress on health insurance and the private financing of health services . Frequent changes in both Bank task managers and MOH leadership adversely affected project implementation in the first half of the project but this situation was reversed after the MTR with satisfactory conclusion of the project 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Highly Likely Likely The sector has had a history of frequent leadership changes (ICR pg 4). From the evidence available, project resilience to overall risks (financial and continuity in leadership) is at a "substantial" level only. This warrants a "likely" rating in sustainability which was also the original rating given by the ICR. Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: To undertake complex reforms, adequate preparation time and initial capacity building of counterparts are required. It is not enough to develop a good M&E framework and infrastructure - - it is also necessary to develop appropriate indicators, and set predetermined quantitative targets to monitor progress . 8. Assessment Recommended? Yes No Why? This was a successful decentralization effort of health systems which achieved considerable efficiency gains in a system not accustomed to being accountable or efficient. The NHIS system and complementary studies/surveys introduced appeared to have significantly enhanced the ministry's capability in analyzing and formulating policy; and in guiding resource allocations. Good lessons can be extracted from the project. 9. Comments on Quality of ICR: ICR is satisfactory, although some statements on improvements were rather vague eg in Annex 1, pt.4. Even if projections in the last PSR were not available, data from an earlier PSR would have provided some basis for comparison. ICR also reported "anecdotal information" suggesting there was improved administrative efficiency, but did not indicate the source or details . Was this gleaned from the patient satisfaction survey? A substantial amount ($88.6m) was expended on component # 2 and the ICR could have provided the costs breakdown for the various subcomponents.