ICRR 12230 Report Number : ICRR12230 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/15/2005 PROJ ID :P035753 Appraisal Actual Project Name :Ni Health Sect II Project Costs 32.00 31.41 US$M ) (US$M) Country :Nicaragua Loan/ US$M ) Loan /Credit (US$M) 24.00 23.41 Sector (s):Central ): government US$M ) Cofinancing (US$M) 6.50 6.46 administration; Compulsory pension and unemployment insurance; Health L/C Number :C3084 FY ) Board Approval (FY) 98 Partners involved : NORAD, Nordic Closing Date 02/28/2003 12/31/2004 Development Fund Evaluator : Panel Reviewer : Group Manager : Group : Elaine Wee-Ling Ooi Roy Gilbert Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives The Health Sector Modernization Program (HSMP) of which this project is phase 1 of 2 APLs, seeks to improve the efficiency, effectiveness and equity of the Nicaraguan health system by : i) strengthening first level care and nutrition; ii) modernizing the hospital network; iii ) developing the institutional capacity of Ministry of Health (MINSA); and iv) supporting social security reform . The program is to be implemented over a 9 year period. Sector reforms and targeted investments will be implemented gradually, eventually covering the whole country . b. Components (or Key Conditions in the case of Adjustment Loans ): The project components reflect the detailed objectives of this first phase . a)First Level Care and Nutrition ($11.1m planned; $9.73 m actual): scale up integrated model of maternal and child care to all 17 health districts; provide food /micronutrient supplementation nationwide; apply successful model of NGO-operated health services to 12 women centers; and undertake program monitoring and evaluation and epidemiological surveillance; b)Hospital Modernization ($7.7m planned; $8.96 m actual): develop and introduce performance based management agreements into the public hospital network; pilot performance based hospital management systems in 5 hospitals; and execute a competitive emergency rehabilitation fund (target 30 subprojects) for peripheral hospitals; c)Modernization of MINSA ($6.8m planned; $6.24m actual): improve capacity in strategic planning, human resources, budget management, management information systems (MIS); strengthen legal/regulatory framework for health sector, ensure equitable access by the uninsured to a cost effective health care package; d)Reform of Social Security Institute (INSS) ($3 m planned; $4.5 m actual): modernize INSS, separate health and social security funds from pension accounts, expand health insurance coverage, and include occupational health in workmen's compensation; e)Project Management ($3.4m planned; $1.98 m actual): support two PCUs one each at MINSA and INSS . Amendments to activities within components : In 2001, component a) application of the integrated maternal and child care model was reduced from 17 to 6 health districts; food supplementation was discontinued, in place of which, a community based growth promotion program was to be piloted; and the number of NGO operated women centers was reduced from 12 to 10; component b)performance based incentive pay schemes were to be applied to 4 instead of 5 pilot hospitals; open competitive nature of the rehabilitation fund was discontinued and eligible for use only by the 4 pilot hospitals; component d) the expansion of insurance coverage and benefits were to include, additionally, specially targeted low income groups. Resource allocations among the components remained essentially the same . c. Comments on Project Cost, Financing, Borrower Contribution, and Dates In April 2001, the project was officially amended to i ) reflect the sector needs after Hurricane Mitch (when external donor funds poured into the country ), and ii) compensate for an overly complex project design . The latter required consolidation of different interventions, previously spread across a wider geographical area, into targeted health districts. The objectives remained unchanged but some nutrition activities were eliminated (to prevent redundancy due to outpouring of new aid), while other activities were scaled down geographically and consolidated . Issues arising from audit reports were promptly dealt with, including the irregularity in the use of government funds ($77,674) in 2004. The project was extended twice by 22 months (possibly due to the hurricane -ICR did not provide reason) and closed on 12/31/2004. 3. Relevance of Objectives & Design : The project objective of enhancing the efficacy, efficiency and equity of the health and social security systems (targeting coverage of the indigenous and poor in the rural areas ) was highly consistent with the CAS 1998 objectives. These included modernizing/decentralizing the public sector, strengthening the social safety net, and cushioning the effects of economic adjustment for the most vulnerable groups . The project was also consistent with the government's vision of health sector reform . It would appear that the objective remains relevant today, eventhough there was no mention of this by the ICR . However, (in spite of a very inclusive preparation process with stakeholder consultation including politicians and others outside the health sector ) the original project design was overly complex with too many interventions given the lack of institutional capacity in MINSA . During preparation, the project did not coordinate with a similar IDB project also being designed. In 2001, after Hurricane Mitch, the project components were scaled down geographically although the project objective remained unchanged . The PCUs of both IDA and IDB projects were "unified" resulting in greater synergy in project programming, supervision and implementation . This coordination has facilitated institutional development of MINSA, elaboration of health sector policies and preparation for operationalizing the future health sector wide program . 4. Achievement of Objectives (Efficacy) : Overall objective (improving efficiency, effectiveness and equity of the health system ) - achieved. The amended components had notably reduced the number of regions for intervention, thus reducing the project's geographical coverage (but the objectives remained unchanged ). While some of the amended key performance indicators were underachieved, others were exceeded . All the required triggers (eg. policy and regulatory changes, facility utilization rates, reduced maternal mortality), were met in this project to allow progression into phase 2 of the program. Specific objective a) strengthen first level care /nutrition - substantial . An innovative approach for growth promotion/ nutrition was implemented in 442 communities (surpassing original and amended project targets ), and was adopted as the official nutrition strategy . Management agreements with an incentive structure for primary care providers were established/implemented in all 17 districts, meeting original and exceeding amended project target of 6 districts. These improved business practices contributed to greater equity and efficiency of the health system for primary care. Specific objective b ) modernize hospital network - substantial . Management agreements tied to productivity benchmarks for hospitals are now required in all districts . Some degree of management autonomy at district and hospital levels has been achieved and non personnel budgets are no longer managed at the center by MINSA . In four pilot hospitals where these management systems have been implemented it is too early to tell if a results based culture has taken hold and if equitable, cost effective and efficient hospital care will be achieved . Initial results show the average length of stay in these hospitals was under -target (reduced by only one day ) and no data was provided for the proportion of outpatient surgery (targeted at 50% of total surgeries). However, patient satisfaction averaged 80% which surpassed the project target of 30%. Specific objective c ) develop institutional capacity of MINSA - substantial . The MIS introduced by the project enhanced strategic planning, budgeting and financial management of MINSA . Budget execution reached 99 % at project close compared to from 77% at project start. Supported by the project, a reformed legal and policy framework has been initiated and key legislation /policies were passed, reinforcing MINSA's regulatory and policy setting role while delegating service provision to lower levels . The General Health Law developed standards for accreditation of health service providers, and to certify health units . Other policy outputs: 10 Year National Health Policies,National Health Plan (2004-2015), and a detailed 5 year Implementation Plan paving the way for a health SWAP . A policy on human resources development was developed but not yet implemented to get the right skills mix for Nicaragua's health workforce. Specific objective d ) reform social security - substantial . INSS separated its health insurance functions from pension and workman's compensation, and devised /implemented a certification program for its private health providers (3 of which were shut down). With project supported MIS, a new system of reimbursement has been developed and, while not yet adopted, is expected to improve the program's transparency, operational efficiency and sustainability. Health insurance coverage increased by 36% (target was 50%) to 353,000 beneficiaries. Previously ineligible, 34,000 retirees (91% of target) were also covered. The health benefit package has been expanded to include preventive, primary and secondary services . 5. Efficiency : While the geographical scope /coverage of the project was reduced when project components were amended, the overall project efficiency was not compromised . The project accomplished most of its original objectives /targets but in a more strategic and targeted manner, eg . instead of allowing all hospitals throughout the country to be eligible for physical rehabilitation, civil works were implemented only in the four hospitals where performance based management systems were being introduced . Resource allocation for the sector has improved with adoption of a bottom-up system of budget planning . Budget execution was at 99% in 2003, compared to 77% at project start. The ERR was not estimated at appraisal but the IRR was estimated at 63% in the base case. There was no mention of ERR in the ICR. 6. M&E Design, Implementation, & Utilization: The project had a good M&E system . Appropriate indicators were selected upfront and baselines established for access, quality and efficiency (utilization rates,length of stay in hospitals, infection rates, hospital mortality rates ) in targeted areas, supplemented by special surveys and evaluation studies . The M&E system was implemented as planned albeit with occasional lapses . Tracking of project progress was enhanced by the MIS introduced in MINSA and INSS. Results of special evaluation studies (comparative analysis of 14 primary health care models being piloted, evaluation of the National Health Policy (98-02), analysis of health services offered via NGOs, evaluation of performance based management agreements introduced into hospitals, evaluation of incentive agreements with primary health care providers) and beneficiary surveys were utilized by government and donors and facilitated adoption of the most appropriate models /approaches for eventual nationwide implementation in phase 2. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): None indicated 8. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Project interventions cannot be too scattered across a wide geographical area to have meaningful impact . In this case, project supported civil works and performance based management systems were concentrated in the same hospitals to optimize patient care . Testing alternative models of health care delivery can help identify the most innovative and cost effective approach. But if too many different approaches are being tested simultaneously, it can lead to scattered strategies, competing models, and increased supervision costs . Immediate goals of human resources development and financial management need to be linked to the higher project objectives of delivering quality health services in an equitable manner . Eg. excessive attention to cost-containment can undermine the project objectives of quality and equity . It is important to coordinate the different activities of project cofinanciers in the same sector to reduce duplication, enable greater synergy, and optimize results . 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: The ICR provided a generally good discussion of the project experience . There could have been more discussion of the reasons for non implementation of the epidemiological surveillance system, and more specifically the impact of that on the program. The reporting matrix on key performance and output indicators were comprehensive despite a few gaps.