ICRR 12848 Report Number : ICRR12848 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/21/2008 PROJ ID : P073649 Appraisal Actual Project Name : Second Health Sector Project Costs (US$M): US$M ): 1,113.0 1,057.0 Program Support Project Country : Ghana Loan/ US$M): Loan /Credit (US$M): 90.0 107.0 Sector Board : HE Cofinancing (US$M ): US$M): 310.4 253.3 Sector (s): Health (100%) Theme (s): Health system performance (33% - P) Child health (17% - S) HIV/AIDS (17% - S) Population and reproductive health (17% - S) Public expenditure financial management and procurement (16% - S) L/C Number : C3731; CH019 Board Approval Date : 02/06/2003 Partners involved : DfID, DANIDA, Closing Date : 06/30/2007 06/30/2007 European Union, Netherlands, USAID, JICA, WHO Evaluator : Panel Reviewer : Group Manager : Group : Denise A. Vaillancourt Kris Hallberg Soniya Carvalho IEGSG 2. Project Objectives and Components: a. Objectives: The project's objective was to support the Borrower's Program (the second Health Programme of Work for 2002-2007 - PoW II) to improve the health of the Borrower's population, while reducing geographic, socioeconomic and gender inequalities in health and health outcomes . Over and above health outcome indicators, key indicators for measuring program performance shown in Annex 1 of the PAD are aligned with the five strategic objectives of PoW II: access, quality, efficiency, partnership and financing . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): Following on the Bank's preceding investment (Health Sector Support Program), which supported Ghana's decision to embark on a sector-wide approach, this second support to Ghana's health SWAp was designed to contribute annual tranches of financing to a common fund (Health Fund), made up of pooled donor financing that would be used and managed by the Government of Ghana (GoG), along with its own budget, internally generated funds and other (non-pooled) financing, to support the implementation of the PoW II . The eleven* components of PoW II supported under the Health Fund were : strengthening and support of priority health interventions developing human resources for health enhancing infrastructure and support services fostering partnerships for health promoting private sector participation in health service delivery improving regulation reforming organizational arrangements improving health sector financing, including the introduction of national health insurance improving management systems strengthening management information systems and performance monitoring improving links and synergies with traditional and alternative medicine . *Note: the ICR indicates only 10 components (excluding the private sector participation component ), but Government's PoW II document lists eleven, as documented in IEG's Project Performance Assessment Report , Ghana, Second Health and Population Project and Health Sector Support Project , July 31, 2007. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Total program cost was 95 percent of the initial estimate. IDA funding approved for this operation (68 million SDRs, of which 43.5 credit and 24.5 grant) was 100 percent disbursed. At the program's outset other financiers included : (a) those providing pooled funding (DFID, DANIDA, EU and the Netherlands); and (b) those supporting the PoW through earmarked funds or projects (USAID, JICA and WHO). By the project's end some of the "poolers" transitioned to budget support and a greater proportion of funds was provided in the form of earmarked funds or projects. Government's financial contribution to the program was US$ 627 million or 98 percent of initial estimates; and contributions of local communities amounted to US$ 70 million, or 93 percent of initial estimates. The project closed as scheduled on June 30, 2007, with virtually all (99 percent of) financing provided over a three year period (as opposed to the planned five -year period), with 60 percent provided in one year (2005). 3. Relevance of Objectives & Design: Overall relevance is substantial. The relevance of objectives is substantial. The main project development objective and the five strategic objectives of PoW are very relevant to country and global priorities in health and development and also fit in well with the Government's Poverty Reduction Strategy (GPRS). The GPRS aimed to eliminate hard-core poverty, with a specific strategy to redefine the role of the state to provide public goods and services and ensure equitable distribution of those benefits . It highlighted improving human service delivery as one of its five strategic pillars, with health -specific components including : (a) expanding access to health services and enhancing quality; (b) improving the efficiency and equity of health services; and (c) ensuring sustainable financing arrangements that protect the poor . The Bank's 2004 Country Assistance Strategy, in support of the GPRS, focuses on: (a) sustainable growth and jobs creation; (b) service provision for human development; and (c) governance and empowerment. The relevance of design is substantial overall, with some shortcomings. The project's SWAp design support Ghana's goal to assume leadership in the strategic management of its health sector and in the coordination of technical and financial assistance to the sector . However, the design was deficient in that the sector program results chain and causal links were not clearly laid out, nor were the indicators and M&E plan and system sufficiently developed to ensure the proper tracking and linkage of the various components of the program logframe . 4. Achievement of Objectives (Efficacy): Overall efficacy is modest. Access to services : Modest An assessment of sector-wide indicators of health service coverage shows that there has been little change in most of the general indicators and the (arbitrary) targets have not been met. Outpatient visits did not change, remaining at 0.5 visits per capita annually. Some priority services performed well against their objectives (immunization and antenatal care coverage), although it must be noted that actual baselines for immunizations were higher than initially estimated and in fact approached targets even before the project started . Other priority services fell short of targets (family planning acceptors, supervised deliveries and post natal care coverage ). There was little explicit monitoring of equity improvements in health services, but some indication of improvements in equity . Immunization coverage was highest in the most disadvantaged regions . Oral rehydration therapy use increased most rapidly among the poor. But inequities persisted (if not further deteriorated) for skilled birth attendance and family planning use . Quality : Modest Data indicate a high level of drug availability, improving over the program period, and tuberculosis cure rates steadily improved and achieved the stated target . Nevertheless, there was a lack of focus on strategies for institutionalizing quality improvement. Supervision and quality management carried out in the GHS is not systematically documented or used as a basis for changing strategies or reallocating resources, and insufficient attention was paid to disparities in quality of services across regions and wealth quintiles . This objective was not vigorously addressed or monitored, resulting in a lost opportunity . Efficiency : Negligible Bed occupancy rates (one measure of efficiency) declined, falling (at 51 percent) far short of the 80 percent target. Productivity of the health workforce decreased slightly between 2004 and 2006 (Vujicic et al 2006). Health spending has been inefficient. Increases in salaries benefited existing health workers, were not accompanied by increases in the non-salary recurrent financing, which, in fact, was underfinanced . Increases in health services outputs have been very modest, with little evidence about changes inequality . Unresolved institutional issues associated with duplication and competition between the MoH and GHS also led to gross inefficiencies . Partnership : Modest In dealing with development partners, there were some clear successes in working through common management arrangements and continuing the policy and planning dialogue of the SWAp . Yet, some development partners increasingly moved towards project management support with earmarked funding, including off -budget financing, undermining the effectiveness of the SWAp partnership . Only one Memorandum of Understanding was signed very late with one umbrella NGO group for mission hospitals (the Christian Health Association of Ghana ), resulting in very little progress in engaging the n on -governmental sector in health services provision during PoW II . Private sector facilities are still not accounted for during health sector planning or budgeting, nor are they included in monitoring progress in the sector . Financing : Modest Health sector financing increased over the program period, achieving 18 percent of GoG budget allocated to health (exceeding than the Abuja target of 15 percent and more than double the share at the outset ). The increase in the recurrent budget was instituted by the wage bill and dedicated funding for personal emoluments, and not met with a commensurate increase in the non -wage budget. GoG has continually overspent on its capital and wage budgets and underspent on its non -wage recurrent budgets -- to the detriment of needed resources for delivering health services. The funding of exemptions for the poor and other priority groups has been chronically insufficient throughout PoW II. The maternal exemptions policy was only partially implemented when funds abruptly ran out in the middle of the year. The National Health Insurance Scheme is expected to replace the exemption policy, but this will take time. There is some indication that benefit incidence of public spending at health clinics has improved to a point where it is equally distributed across income groups (Coulombe & Wodon 2007). However, the distribution of public spending at hospitals has changed little and still favors the richer segments of the population . The shift of donor financing away from the Health Fund (towards general budget support and project /earmarked support) undermines the flexibility of funding to decentralized entities and as a consequence their ability to improve health service delivery. Health Outcomes : Modest Health, fertility and nutrition outcomes will be available when the results of the planned Ghana Demographic and Health Survey 2008 are issued. Between 1998 and 2003 there were no improvements in child health outcomes, ending the long trend of improvements recorded since independence . Infant mortality was 57 per 1000 live births in 1998 and 64 in 2003, and under-five mortality was 108 in 1998 and 111 in 2003. (Neonatal mortality appeared to increase during the last survey period, and is the driving force behind the child mortality rates .) Likewise, there was no statistically significant change in total fertility (4.6 in 1998 and 4.4 in 2003). Preliminary data on key indicators for 2006 (UNICEF/MICS) show that infant and under-five mortality and fertility still have not changed substantially since 1998. GHS data on malnutrition reveal no major trends of improvement . IEG's analysis of household data on stunted and underweight children show that 2003 rates for each indicator are no different than rates in 1988, corroborating the series of GDHS survey data . According to these data, in 1988 and again in 2003, one in four children is stunted and one in four children is underweight . DHS data do provide indication of faster declines in child mortality among the poorest (who have the highest rates ), revealing possible improvements in equity of outcomes . Inequities in mortality between rural and urban groups also appear to have been reduced . 5. Efficiency (not applicable to DPLs): Project efficiency is modest. The increased financial resources for health were not used efficiently . Trends indicate a growth in investment and salaries budgets to the detriment of non -salary recurrent financing. There is still inefficiency in resource allocations across regions and distribution of human resources for health is chronically inequitable. Hospital occupancy rates reveal an under -utilization of this level of service . Furthermore, hospitals still deliver primary health care services, which are an expensive proposition . The underutilization of the non-governmental sector in the delivery of health services is also an indicator of inefficiency . During the project life the integration and decentralization of health programs and services led to inefficiencies in the delivery of these programs at the district level, with priorities accorded these programs by the districts and not always commensurate with the epidemiological justification for action . While the benefits of two Summit meetings every year and similar meetings at regional and district levels are evident (in support of the SWAp), the costs and opportunity costs of these meetings have not been calculated . Chronic overlap and competition between MoH and GHS also undermine efficiency. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Rating is based on substantial relevance and modest efficacy and efficiency . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Diligent efforts will be required to ensure that the past gains made under the SWAp are not lost . The sector currently faces a number of challenges that may jeopardize progress . First, financing of the health sector has become more complicated. Donor commitments have dropped off following the shift from pooling funds to budget support and there are signs that the Government's execution rate is retarding as processes become more complicated. The increase in earmarked funds means that Ghana's priority health programs are financed largely by unpredictable financing. There is a risk that, as the Health Fund has been replaced by funds provided through the Ministry of Finance and Economic Planning, the absence of flexible funds will undermine the capacity of peripheral budget and management centers to manage resources and deliver services . Second, it is not clear how the poor and vulnerable will fare as the NHIS is rolled out nationwide, with some concern that the scheme will not be financially viable enough to cover those in exempt categories until about 50 percent of the country is enrolled . Third, the human resources for health crisis continues to plague Ghana, with the Government's recurrent budget for health tied up in personal emoluments and salaries . Fourth, the potential decentralization of budgets and financial management may cause fragmentation in health financing and conflict over resources and implementation responsibilities between MoH, GHS, local governments and the NHIS . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Although the Bank ensured that the project was strategically relevant, the Bank failed to conduct the sufficient analysis and evaluation that could have contributed greatly to the design of the project . Such analysis might have covered institutions, equity, public expenditure . Another important design shortcoming was the weak logframe and problems associated with the indicators (appropriate choice, definition, links to objectives ). The Bank was a cooperative and supportive partner to Government and to donors and provided strong technical guidance on fiduciary matters . More proactivity in assessing and supplementing key information on program performance (equity and expenditures, in particular ) would have been appropriate. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Satisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government was strongly committed to the PoW II objectives and engaged with stakeholders and development partners in SWAp discussions . It could have expanded its consultations by including more non -governmental actors. Government's performance faced moderate shortcomings in the design, implementation and utilization of monitoring and evaluation. The performance of the implementing agencies (MoH and GHS) suffered due to their failure to be able to work together more coherently, which undermined policy and implementation of the PoW II . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Unsatisfactory c. Overall Borrower Performance :Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design . A revised set of sector-wide indicators (derived from lessons under the first SWAp ) was established for PoW II, but was still inadequate for the purposes of tracking and strategically managing program progress . And they changed over time (both in terms of the number of indicators and, in some cases, their definitions ). The M&E framework was not based on a coherent results framework, and neglected important measures of key strategic objectives (e.g., equity). Neither did it provide for the tracking of expenditures . Implementation . There are still variations in data quality and management, despite significant investment and improvements in MoH and GHS information systems . Inconsistencies in the indicators compromise the opportunity to identify trends over time and create confusion among stakeholders over which indicators are valid measures of objectives. Indicators for health financing are often subject to change, oftentimes with the definition of the denominator differing over the years and subject to interpretation . Utilization . MoH and GHS have developed systems to collect detailed health service use and outcome data at both regional and district levels, but decentralized data were hardly ever used in annual reports . Despite lack of effective use at the national level of regional and district -level data, there is evidence that some regions have made progress in this area. There was little evidence at the national level that data monitoring was used to improve resource allocation. Relevant operational research was not used to inform refinements to policy and programs . a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): The Ghana Environmental Protection Agency developed and disseminated guidelines on proper procedures for handling health care waste and the MoH agreed to use environmentally -friendly insecticides in bednets . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Moderate Significant Health sector financing has become Outcome : more complicated, with a move by some donors to budget support and by others back to earmarking and project support, thus undermining the SWAp approach and the capacity and flexibility of decentralized levels to manage and deliver services . Scale-up of the NHIS to nation-wide coverage may delay health services, particularly for the poor. Human resources for health (stability, remuneration, equitable distribution) remains a challenge. And Decentralization of budgets to district governments may cause fragmentation and potential conflict/duplication among MoH, GHS, local governments and NHIS. Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Exemplary NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: A strong analytic base is needed, even when things appear to be going well, as was the case when the project was approved. In-depth analysis and debate concerning equity of health services and outcomes, efficiency of health services and institutional analysis would have been particularly helpful in anticipating problems and furthering the agenda set by the PoW II and the GPRS; Success in establishing a process for sector -wide management and dialogue will not by itself ensure the achievement of health services performance and outcome targets . The absence of a strong results focus, the absence of a viable system and incentives for M&E and the failure to move towards a performance -based health system to ensure accountabilities can seriously undermine the achievement of sector objectives; A dialogue between Government and its health partners, which is not sufficiently rigorous or frank, can result in the neglect of critical constraints to sector performance . The institutional competition and overlap between MoH and GHS and the failure to harness the potential of the non -governmental sector for delivering health services are issues with both technical and political dimensions, which might have benefited from the technical input and leverage of outsiders; and Sector efficiency needs to be measured in order to be achieved . The failure to carry out annual public expenditure reviews and incidence analyses (to measure efficiency of resource use, and equity ), and to define fully a program results chain and monitor progress on each link has undermined Ghana's ability to measure and fine-tune the development effectiveness of the substantial investments in the health sector . 14. Assessment Recommended? Yes No Why? A review of this project's outcome would be interesting to undertake in a year or so (a) once the GDHS 2008 data are available to show trends in outcomes and service delivery; and (b) when the evolution in donor support (budget support and earmarked funding ) might be more advanced and subject to more analysis . This PPAR would build on a PPAR prepared in 2007 on the Health and Population Project and the Health Sector Support Program (Bank's support to PoW ). 15. Comments on Quality of ICR: The ICR is very candid, and well argued and substantiated . It provides fresh and insightful analysis of monitoring and evaluation weaknesses and of the Bank's performance . It is unusually outcome-oriented and draws on external resources to contribute to its extensive analysis . a.Quality of ICR Rating : Exemplary