ICRR 11456 Report Number : ICRR11456 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 06/10/2003 PROJ ID : P003239 Appraisal Actual Project Name : Health Sector Support Project Costs 536.9 259.35 Project US$M ) (US$M) Country : Zambia Loan /Credit (US$M) Loan/ US$M ) 56.0 34.65 Sector (s): Board: HE - Health (97%), Cofinancing 140.9 1.6 Central government US$M ) (US$M) administration (3%) L/C Number : C2660 Board Approval 95 FY ) (FY) Partners involved : Netherlands, UNICEF, Closing Date 12/31/2000 06/30/2002 DFID, SIDA, DANIDA, EU, WHO, UNFPA Prepared by : Reviewed by : Group Manager : Group : Elaine Wee-Ling Ooi Martha Ainsworth Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The objective of the project was to improve access to and the quality of a national package of essential health services in a decentralized health care delivery system . The project aimed at improving health outcomes, particularly in terms of nutritional status, lowered fertility, and reduced transmission of HIV . This would contribute to the "social sustainability" of the ongoing adjustment program and strengthen the country's human resource base . The project supported the government's health reform program via a sector wide approach (SWAP) - one of the first of its kind in the social sectors in Africa. b. Components The IDA credit financed a "slice" of a package of integrated sector support that included three components : Policy Development - national policy development and operational research on new approaches including cost sharing mechanisms and phased testing of delivery systems ($7.0m); Investment Program and Recurrent Costs - infrastructure rehabilitation, medical equipment, training, information systems, district operations, drugs /supplies and technical assistance for improved health planning and implementation at district, regional and central levels ($528.4m); and Monitoring & Evaluation - monitor progress of Health Reforms through data on vaccination coverage, ante -natal care, independent external audits and beneficiary assessments ($1.5m). c. Comments on Project Cost, Financing and Dates SAR listed at least 5 other major donors committed to $140.9m, with the remaining $340m to be borne by the government. Country commitment to reforms wavered in 1998, and while the closing date of the IDA credit was extended by 18 months, IDA's actual disbursements totaled $ 34.65m against the government's $223.1m. Details of other donor spending were not provided /available although ICR reported their continued and significant involvement in the sector. 3. Achievement of Relevant Objectives: The project did not achieve its overall objectives but had limited gains at project closure . While immunization coverage increased, there is no clear evidence that the overall quality of, and access to, a national package of essential health services had improved . Fertility declined but infant mortality continued to deteriorate . Other health outcomes (nutritional status and HIV transmission ) remained poor but the counterfactual (ie. absence of project) could have been worse given a declining economy and drought . Gains in other reforms, ie. alternative health financing mechanisms and separation of provider -purchase of services by government, have been uneven as Borrower commitment declined in the face of political opposition and economic slowdown . Other inefficiencies and capacity constraints in the health system were inadequately addressed, but the current health leadership (six changes in Ministers of Health during project life ) has put sector wide reforms back on track . Nevertheless a decentralized system of health services has been institutionalized, and public spending has shifted from tertiary curative care to primary/preventive care in the districts . At project closure, other donor agencies have continued to be engaged in the reform process, and via an MOU have committed to channeling their assistance to the districts through a pooled funding mechanism . 4. Significant Outcomes/Impacts: As a result of Zambia’s overall health reform program, a number of significant policy and institutional developments have come to being which are likely to be sustained . (IDA's contribution may have been circumscribed as it disengaged midway during project - however at project preparation and start up, IDA was a significant player in the dialogue on policy and health reforms .) Decrease in percentage of 15-19 year olds initiating sexual activity . A functioning district level health service, with district health management teams (DHMTs) established in all 72 districts in the country. "District baskets", through which donors and government disburse directly to the districts, have streamlined reporting and ensured a predictable flow of funds for decentralized health services . A start in greater public allocations to primary and preventive care in the rural areas Legislation and implementation of the separation of provider -purchaser of services by the public sector was initiated but derailed. New MOH leadership (in 2002) has pledged to restart the process . Increased participation by NGO and private providers, in health services delivery . NGOs are eligible for support by the district baskets. Despite IDA's disengagement, good donor collaboration has resulted and appears to have sustained beyond project life. 5. Significant Shortcomings (including non-compliance with safeguard policies): (Some of the shortcomings listed may be attributable to IDA ’s own inexperience with SWAPs at project design .) Given that IDA had no prior experience in the sector in Zambia, this was a highly complex project to undertake . No formal ESW was done nor was consultation of a broader base of stakeholders carried out which could have lessened opposition to the sweeping reforms . Project was not supported by M&E system to link inputs, outputs and project outcomes (provision for M&E and HMIS did not materialize). This was all the more important as project objectives were tied to health outcomes . Health financial management information critical for the reform process was patchy and was not utilized for planning and monitoring of health sector expenditures . IDA support was an investment credit to a larger reform program using a SWAP . However, disbursement was pegged to specific investment categories, which IDA found difficult to reallocate when the reform process wavered - more than 90% of IDA funding had been committed (at SAR) to civil works, in a health sector reform project. While Borrower performance was good at the start, an overall climate of poor governance prevailed towards the later part of implementation, plagued by financial mismanagement and lack of transparency in procurements . Among others, mishandling of the contracting out of Medical Stores Ltd ., eventually led to a breakdown in IDA/Borrower relationship, undermining further, project implementation . (It was the view of the Borrower and Co-financier that IDA should have remained engaged, and participated in the "district baskets" funding mechanism. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Unsatisfactory Moderately This is in light of some, albeit limited, Unsatisfactory progress in health outcomes and policy dialogue/reforms as reflected in sections 3 and 4, achieved by the overall project . Institutional Dev .: Modest Modest Sustainability : Likely Likely Bank Performance : Unsatisfactory Unsatisfactory Borrower Perf .: Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. NOTE: 7. Lessons of Broad Applicability: For a project of such complexity and risk, politically sensitive reforms are unlikely to be achieved without concerted efforts at building broad base stakeholder support . Wavering commitment levels by Borrower can be expected and continuous engagement by Bank with Borrower and other donors is a must . Good M&E capabilities are especially critical in a SWAP to monitor progress towards goals, track sector expenditures and enable performance reporting and management . Investment intensive operations, especially where the Borrower has no prior experience with the Bank ’s procurement procedures, need sustained capacity building . 8. Assessment Recommended? Yes No Why? This is one of the first sector -wide lending programs for Health in Africa . Many lessons can be gleaned from an assessment - among them, the trade-offs between sector-wide and investment lending. 9. Comments on Quality of ICR: Quality of the ICR was satisfactory . However, the ICR focused too much on assessing the credit as opposed to the overall project. Further, too much emphasis was placed on absolute levels of health /welfare outcomes (eg. maternal mortality and nutrition) instead of the change in these variables over the life of project and the plausibility of attribution to the project. Otherwise the ICR was comprehensive and there was good analysis of the problems and achievements of SWAPs. Especially helpful were the comments by the Borrower and a Co -financier. Granted some data were hard to come by, but clearer and more comprehensive presentation of data on actual financing and costs (ie. actual donor and Borrower expenditures ) of the entire project, would have improved the ICR . The final amount cancelled by IDA for the project is unclear .