Document of The World Bank Report No: ICR00001577 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-40500) ONA CREDIT IN THE AMOUNT OF SDR 7.2 MILLION (US$ll.O MILLION EQUNALENT) TO THE REPUBLIC OF NICARAGUA FORA HEALTH SERVICES EXTENSION AND MODERNIZATION PROJECT (APL2) December 29,2010 Human Development Sector Management Unit Central America Country Management Unit Latin America and the Caribbean Region CURRENCY EQUIVALENTS December 20, 2010 US$1 = 21.9 Cordoba FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS APL Adaptable Programmatic Loan BPHS Basic Package of Health Services (Paquete Básico de Servicios de Salud) CAS Country Assistance Strategy CM Maternal Homes (Casas Maternas) DHS Demographic and Health Survey ENDESA Nicaraguan Demographic and Health Survey FONSALUD Nicaraguan Health Sector Support Fund FYIP Five Year Implementation Plan GDP Gross Domestic Product GON Government of Nicaragua IDA International Development Association IDB Inter-American Development Bank IEC Information, Education and Communication INSS Instituto Nicaraguense de Seguro Social (Nicaraguan Social Security Institute) LAC Latin America and the Caribbean Region MAIS Modelo de Atención Integral de Salud (Health Care Model) MCH Maternal and Child Health MDG Millennium Development Goals MOF Ministry of Finance MOH Ministry of Health MOU Memorandum of Understanding NGO Non-Governmental Organization NHP National Health Plan NHDP National Human Development Plan NORAD Norwegian Development Agency PC Participating Communities PHC Primary Health Care PRSP Poverty Reduction Support Program SIAFI Integrated Financial Administrative System (Sistema Integrado Administrativo Financiero) SIGFA Administrative Fiscal Management System (Sistema de Gerencia Fiscal Administrativo) SILAIS Local Systems of Integrated Health Care (Sistema Local de Atención Integral de Salud) SWAp Sector Wide Approach RAAN North Atlantic Autonomous Region (Región Autónoma del Atlántico Norte) RAAS South Atlantic Autonomous Region (Región Autónoma del Atlántico Sur) Vice President: Pamela Cox Country Director: Laura Frigenti Sector Manager: Joana Godinho Project Team Leader: Marcelo Bortman ICR Team Leader: Sarah Elizabeth Berger ii DATA SHEET A. Basic Information Health Services Extension and Country: Nicaragua Project Name: Modernization (2nd APL) Project ID: P078991 L/C/TF Number(s): IDA-40500 ICR Date: 12/29/2010 ICR Type: Core ICR REPUBLIC OF Lending Instrument: APL Borrower: NICARAGUA Original Total XDR 7.2M Disbursed Amount: XDR 6.8M Commitment: Revised Amount: XDR 6.8M Environmental Category: B Implementing Agencies: Ministry of Health (MINSA) Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 06/28/2004 Effectiveness: 04/11/2006 04/11/2006 Appraisal: 02/14/2005 Restructuring(s): Approval: 04/05/2005 Mid-term Review: 08/15/2008 10/27/2008 Closing: 11/30/2009 06/30/2010 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: iii C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No Satisfactory at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central Government administration 5 5 Health 93 93 Sub-national Government administration 2 2 Theme Code (as % of total Bank financing) Child health 24 24 Decentralization 13 13 Health system performance 25 25 Nutrition and food security 13 13 Population and reproductive health 25 25 E. Bank Staff Positions At ICR At Approval Vice President: Pamela Cox Pamela Cox Country Director: Laura Frigenti Jane Armitage Sector Manager: Joana Godinho Cristian C. Baeza Project Team Leader: Carlos Marcelo Bortman Jesus Maria Fernandez Diaz ICR Team Leader: Sarah Elizabeth Berger ICR Primary Authors: Sarah Elizabeth Berger Jorge Barrientos F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The overall Adaptable Programmatic Loan (APL) program development objective, as defined in the 1998 Board document, was to improve health outcomes in Nicaragua, particularly among the poor, by raising the efficiency, effectiveness, equity and sustainability of the Nicaraguan health system. In this second phase of the APL, IDA financing was to contribute, alongside funds from the MOH and other SWAp partners, to improve maternal and child health in the poorest municipalities of Nicaragua, iv concentrating on roughly half of the country’s poor. By improving the health status of this segment of the population, health inequities were expected to decrease, as shown by disparities in maternal and infant mortality indicators across geographical areas and income groups. These goals would be achieved through the support of three objectives, as set up in the sector Five-Year Implementation Plan, namely: (i) the extension of coverage of a basic package of health and nutrition services to the poorest areas of the country, via the new integrated model of health care; (ii) the physical and functional strengthening of the health service facilities network, focusing on maternal-infant care, so as to provide a seamless path of essential services in the target areas; and (iii) institutional strengthening and capacity-building, concentrated specifically on developing the purchasing, monitoring and evaluation capacities of the MOH, its leadership role to enhance donors’ alignment and harmonization, and the management capacity of MOH decentralized units and Autonomous Governments of the Atlantic Coast. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Maternal Mortality Rate Value quantitative or 96 per 100,000 65 per 100,000 61 per 100,000 Qualitative) Date achieved 12/31/2001 11/30/2009 12/31/2009 Comments Achieved. Source for baseline: 2001 Nicaraguan Demographic and Health (incl. % Survey (ENDESA). Actual value source: Ministry of Health. Informe del achievement) Desempeño Institucional Preliminar. 2009. Indicator 2 : Infant Mortality Rate Value 27 per 1,000 live 23 per 1,000 live quantitative or 31 per 1,000 live births births births Qualitative) Date achieved 12/31/2001 11/30/2009 12/31/2008 Comments Achieved. Source for baseline 2001 ENDESA. Actual value source: World (incl. % Health Statistics 2010. achievement) Indicator 3 : Under-5 mortality rate per 1,000 live births Value 33 per 1,000 live 27 per 1,000 live quantitative or 41 per 1,000 live births births births Qualitative) Date achieved 12/31/2001 11/30/2009 12/31/2008 Comments Achieved. Source for baseline 2001 ENDESA respectively. Actual value source: (incl. % World Health Statistics 2010. achievement) v Indicator 4 : Percentage of population with chronic malnutrition Value quantitative or 17.8 percent 12.8 percent 16.9 percent Qualitative) Date achieved 12/31/2001 11/30/2009 12/31/2006 Not Achieved. Source: 2001 and 2006 ENDESA survey. Under the project, Comments chronic malnutrition was measured for the entire population; for the under-5 (incl. % population, chronic malnutrition decreased from 25.8 to 21.7 percent from 2001 achievement) to 2006. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Number of beneficiaries to whom the Basic Package of Health Services has been Indicator 1 : extended by any modality Value 470,000 529,652 (quantitative 129,698 beneficiaries beneficiaries beneficiaries or Qualitative) Date achieved 04/30/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 12 percent. Source: Gesaworld Report completed for MOH. achievement) Institutional deliveries in the 12 targeted Local Systems of Integrated Health Indicator 2 : Care (SILAIS) Value (quantitative 36.2 percent 37.9 percent 78.7 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 40.8 percentage points. Source: MOH administrative data. achievement) Indicator 3 : Immunization Coverage (pentavalent vaccine) in infants under 1 (three dosages) Value (quantitative 78 percent 97.5 percent 98.3 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 0.8 percentage points. Source: MOH administrative data. achievement) Indicator 4 : Early prenatal control (first trimester) in targeted municipalities Value 28,656 women 32,303 women 32,533 women vi (quantitative 34.4 percent 37.3 percent 48.1 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 10.8 percentage points. Source: MOH administrative data. achievement) Indicator 5 : Pregnancies with full (four visits) prenatal care in targeted municipalities Value 31,852 28,656 pregnancies 39,446 pregnancies (quantitative pregnancies 49.5 percent 58.4 percent or Qualitative) 55.0 percent Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 3.4 percentage points. Source: MOH administrative data. achievement) Utilization of maternal homes (Casas Maternas) by pregnant women (number of Indicator 6 : discharges) Value (quantitative 2,400 discharges 6,000 discharges 9,749 discharges or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Exceeded by 3,749 discharges. Source: MOH administrative data. achievement) Indicator 7 : Family planning methods utilization rates in fertile age women Value (quantitative 64.1 percent (ENDESA) 66.0 percent 72.0 percent or Qualitative) Date achieved 12/31/2001 11/30/2009 12/31/2006 Comments (incl. % Achieved. This target was set for the 2006 Survey. Source: 2006 ENDESA. achievement) Indicator 8 : Hospital Discharge Rates in the 12 targeted hospitals Value (quantitative 4.02 percent 5.25 percent 6.3 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Achieved. Source: MOH administrative data. achievement) Indicator 9 : Hospitals with critical path complying with MOH certification standards Value (quantitative 0 hospitals 9 hospitals 3 hospitals or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments Not achieved. Source: MOH administrative data. (incl. % achievement) vii Percentage of public health providers complying satisfactorily with management Indicator 10 : agreement targets Value (quantitative N/A 100 percent 60 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments Not achieved. Achievement fell short of target by 40 percentage points. Source: (incl. % Gesaworld Report complete for MOH. achievement) Indicator 11 : Percentage of maternal deaths audited Value (quantitative N/A 100 percent 75 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments Not achieved. Achievement fell short of target by 25 percentage points. Source: (incl. % MOH administrative data. achievement) Percentage of total budget transferred to the targeted SILAIS as purchasing of Indicator 12 : services ("provision of health services" budget item). Value (quantitative 0 percent 17 percent N/A or Qualitative) Date achieved 03/04/2005 11/30/2009 N/A Data not available for this indicator. In late 2007, when the new administration Comments took over, the practice of transferring the budget to hospitals and maternal clinics (incl. % based on services rendered was discontinued. As a result, this indicator was not achievement) monitored. Percentage of MOH budget in these regions administered by MOH budget Indicator 13 : executing units in the RAAN and RAAS Value (quantitative 0 percent 100 percent 100 percent or Qualitative) Date achieved 03/04/2005 11/30/2009 11/30/2009 Comments (incl. % Achieved. Source: MOH Budget, 2009. achievement) Indicator 14 : Per capita expenditure allocated to targeted SILAIS Value 6.73 per capita 7.18 per capita (quantitative N/A expenditure expenditure or Qualitative) Date achieved 03/04/2005 11/30/2009 N/A Comments (incl. % Data not available. This indicator was not monitored. achievement) viii G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 04/29/2005 Satisfactory Satisfactory 0.00 2 08/09/2005 Satisfactory Satisfactory 0.00 3 12/09/2005 Satisfactory Satisfactory 0.00 4 03/31/2006 Satisfactory Moderately Satisfactory 0.00 5 10/25/2006 Satisfactory Moderately Satisfactory 2.26 6 06/13/2007 Satisfactory Moderately Satisfactory 4.11 7 10/17/2007 Satisfactory Moderately Satisfactory 6.00 8 12/13/2007 Satisfactory Moderately Satisfactory 6.00 9 06/30/2008 Satisfactory Satisfactory 8.60 10 12/15/2008 Satisfactory Satisfactory 9.76 11 06/19/2009 Satisfactory Moderately Satisfactory 10.26 12 12/10/2009 Satisfactory Moderately Satisfactory 10.31 13 06/29/2010 Satisfactory Moderately Satisfactory 10.31 H. Restructuring (if any) Not Applicable I. Disbursement Profile ix NICARAGUA Health Services Extension and Modernization Project (APL 2) CONTENTS 1. Project Context, Development Objectives and Design ................ " ..................... 1 1.1 Context at Appraisal ........................................................................................... 1 1.2 Original Project Development Objectives (PDO) Key Indicators ............... 2 1.3 Revised PDO (as approved by original approving authority) Indicators, and reasons/justification ................................................................................................ 3 1.4 Main Beneficiaries ......... ,.................................................................................... 3 1.5 Original Components .................................................................... " .................... 3 1.6 Revised Components ..................................................... ,.. ,................... "............. 4 1.7 Other significant changes .................................................................................... 5 2. Key Factors Affecting Implementation and Outcomes ...................................... 5 2.1 P roJect Preparatwn, D' an d Qual'tty at. L:j[jtry ............................................... .'i · . eSlgn r' J 2.2 Implementation .................................................................................................... 7 2.2 Monitoring and Evaluation (M&E) Design, and .... 8 2.3 Safeguard and Fiduciary Compliance ....... ,......................................................... 9 2.5 Post-completion OperationiNext Phase .......................................................... 10 3. Assessment of Outcomes ............................................ ' ......................... ,........... 11 3.1 Relevance of Objectives, Design and ........ ,. ........................... 11 3.2 Achievement of Project Development Objectives ................................. " ......... 11 3.3 Efficiency .......................................................................................................... 14 3.4 Justification of Overall Outcome Rating ........................................................ 14 3.5 Overarching Themes, Other Outcomes and .................................... 15 3.6 Summary of Findings of Beneficiary Survey .16 4. Assessment of Risk to Development Outcome .................. ,.............................. 16 5. Assessment of Bank and Borrower Perfonnanee ................... ., ...................... 17 5.1 Bank Performance .............................................................................................. 17 5.2 Borrovver Performance .................................... ,,, .. ,..... ,............. ,........ ,., .............. 17 6. Lessons Learned ................................................ ,............. ,; ................................ 18 7. Comments on Issues Raised by Borrower/Implementing 19 Annex 1. Project Costs and Financing ................................. " .... " .......... ,...................... ,..... 21 Annex 2. Outputs by Component. ........................................... ,......... .,. ................... ,........... 22 Annex 3. Economic and Financial Analysis ......................................... ,........................... 28 Annex 4. Bank Lending and Implementation Support/Supervision 31 Annex 5. Beneficiary Survey Results .................................................... ".. ,...... ,. ........ ' ..... 33 Annex 7. Summary ofBon-ower's ICR and/or Comments on ............ " ........... 35 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ..... " ... ,................ 37 Annex 9. List of Supporting Documents .......................................................................... 38 Annex 10. APL I Triggers ........................................ ,................................. "..... ~ ............... 39 Annex 11. MAP ............................................................................ " ...................... ',., ......... 40 x 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. By the early 2000s, Nicaragua confronted many traditional health challenges similar to a country in the early phases of an epidemiological transition. Infectious diseases, prenatal illnesses, malnutrition, acute respiratory diseases and diarrhea dominated the public health agenda. While advances had been made in maternal and infant mortality outcomes, it seemed unlikely that Nicaragua would achieve its Millennium Development Goals (MDGs). 2. Inequality in access to health services persisted across socio-economic groups and there was wide variation in the quality of services across regions. These inequalities were compounded by relatively low coverage of health insurance under the Nicaraguan Social Security Institute (lNSS). In 2003, the IHSS covered only 9 percent of the population, but almost 70 percent of its beneficiaries belonged to the wealthiest 40 percent of the population. The poor were forced to rely overwhelmingly on services from the Ministry of Health (MOH). Out-of- pocket purchases of medicines were also high at the time of appraisal, constituting about 49 percent of all health expenditures. Many of Nicaragua's poor often relied often on self-treatment. 3. MOH fmancing was heavily dependent upon external financing, which contributed to a fragmentation in the delivery of health services. Between 1999 and 2003, external financing grew from 18 percent as a share of total MOH expenditures to 24 percent. While the increase in external funding was important for continuing services, it led to fragmentation. In 2003, the MOH reported that there were 36 distinct internationally funded projects in the health sector which supported 14 distinct primary health care models. Some of these projects were targeted to specific departments. These challenges stressed the need for the MOH to achieve a sustainable form of fmaucing, to define a sector-wide program that would be supported by all donors and multilaterals, and address the inequalities in health expenditures. 4. Recognizing these challenges, the Government agreed with the International Development Association (herein referred to as IDA) and other donors supporting the sector to coordinate official development aid by adopting a Sector Wide Approach (SWAp) model. Nicaragua had been assertively advocating for improved coordination among donors and more predictable development assistance. The country became a pilot for a number of harmonization initiatives for the Organization for Economic Development - Development Assistance Committee (OECD-D1\.C) and prepared, jointly with donor partners a National Plan for Harmonization and Alignment with a country-specific indicators and a monitoring mechanism embedded in locai processes. In line with this, development partners agreed to support a single strategic framework and set of objectives for the health sector. 5. A Five-Year Implementation Plan for the MOH's 2004-2015 National Health Plan (NHP) served as the strategic framework to align donor support. The three strategic objectives of the Implementation Plan included: (i) .the extension of coverage of a bask package of health services (BPHS) to the poorest areas of the country via the new Integrated Health Care Model; (ii) the improvement in the physical and operational capacities of the health service facilities netWork with a focus on maternal-infant care; and (iii) institutional strengthening and capacity-building at the ,central and regional level with specific emphasis on fiduciary 1 management, monitoring and evaluation, and the standardization of services. 6. Initially, development partners intended to pool funds in support of the Implementation Plan. At the time of appraisal, many of the development partners including Finland, Sweden, the Netherlands, the IDB and IDA agreed to pool their funds with counterpart funds to create a single fund to support the implementation of the Implementation Plan in defined areas of the country for a total of US$82.2 million (Table 1). Subsequently, this group constituted themselves into FONSALUD, which is described later in the ICR. Pooling of IDA funds was later found not to be legally feasible, given the difference in fiduciary procedures between the Association and other development donors. Instead, IDA and the IDB (which faced a similar situation) agreed that their projects would have identical objectives to the Implementation Plan. The Japanese and the United States Agency for International Development also agreed to participate in SWAp related activities, but decided early on not to pool their funds. The Pan-American Health Organization (PAHO) and the United Nations Population Fund (UNDP) participated as observers. T a hie 1 SWA F" . rO.lect A.ppralsaI I..p mancmgat P" Sources of financing US$ millions IDB 30.0 Sweden 20.0 IDA 11.0 Finland 7.1 Netherlands 2.0 MOH (counterpart funds) 4.1 TOTAL 82.2 7" Given that the Bank had already committed to an Adaptable Programmatic Loan (APL) for the Health Services Extension and Modernization Program, its involvement in the SWAp was through the second phase of the APL. Since 1998, IDA had been supporting the first phase of a two-phase APL. The first phase was a US$32 million health sector credit that was co-financed by the Norwegian Agency for Development and Cooperation. The first phase of the APL aimed to improve the efficiency, effectiveness and equity of the health system by: (i) strengthening first level care and nutrition; (ii) modernizing the hospital network; (iii) developing the institutional capacity of the ministry of health; and (iv) supporting social security reform. Through APL I, the MOH developed mechanisms and regulations to improve the Ministry's regulatory and management tools. For the implementation and execution of the funds of APL 2, APL 1 achieved specific triggers which are included in Annex 11. 8. The objectives the second phase of the APL were in line with Nicaragua's 2001 Poverty Reduction Strategy Paper (PRSP) (Report No. 22627-NI) and the sector reforms and actions ofthe World Bank's Country Assistance Strategy (CAS) (Report No. 25043-NI) discussed by the Executive Directors on December 18, 2002. The 2002 CAS outlined a framework for assistance during FY03-FY05 around the objectives and targets set out in the 2001 PRSP; the second phase of the APL was included in support of two of its pillars (investment in human capital and better protection of vulnerable groups). 1.2 Original Project Development Objectives (PDO) and Key Indicators 2 9. The overall APL supported improvements in health outcomes in Nicaragua, particularly among the poor, by raising the efficiency, effectiveness, equity and sustainability of the Nicaraguan health system. The objective of the Project, the second phase of the APL, was to improve maternal and child health in Nicaragua's poorest municipalities by: (a) ensuring that Participating Communities (PCs) have access to a package of basic health and nutrition services; (b) strengthening the Government's health services facilities network; and (c) providing institutional strengthening and capacity-building to the MOH. 10. The Project's Development Objective (PDO) Indicators were based on the common set of indicators proposed under the FYIP and agreed on by all development partners supporting the SWAp. These consisted of a set of five indicators aimed at capturing the longer- term program impact on key MDGs. Additionally, fourteen Intermediate Outcome Indicators were defmed to measure performance on: (i) improved access to maternal and child services, (ii) strengthening of health services networks, and (iii) improved stewardship and institutional strengthening. These indicators are described in greater detail under S~ction 3 of the main text and Annex 2. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 11. The PDO and Key Indicators were not revised during project implementation. 1.4 Main Beneficiaries 12. The Project's main beneficiaries were vulnerable rural and indigenous communities living in underserved areas of the country. The target population comprised poor and extremely poor families, in particular mothers, children and infants, in 79 municipalities in 10 departments and two indigenous territories. The Project supported various forms of training, improvements in technology and,health facilities, and changes in management structures, which benefitted health providers, administrators and hospital management staff. 1.5 Original Components 13. Original components were designed in line with strategic objectives set for the health sector as a whole. Within this context, the Project components, as stated in the Credit Agreement, were set up in terms of the ~ame strategic objectives as the FYIP. The three components I are described below: 14. Component 1: Delivery of the Basic Package of Health Services (BPHS). The objective of this component was to (i) pay capitation transfers to SILAIS administrative. units to deliver the basic package of health services to participating communities and (ii) provide technical assistance to develop institutional capacity in the SILAIS and municipios that will implement the delivery of the basic package of health services. The basic package of services had a strong focus on maternal and child health care. It included monitoring and promotion of growth and 1 In the Project Appraisal Document these are stated as strategic objectives. 3 development of children under-5 years of age, integrated care for women of reproductive age and detection and care of prevalent diseases (See Box 1 in Annex 2). For the delivery of services, the MOH entered into management agreements with the Local Systems of Integrated-Health Care (Sistema Local de AtenCion Integral de Salud- SILAIS) Administrative Unit. The MOH transferred the capitation payments to the SILAIS for the purchase of services from public, local providers (muiCipios) and/or qualified, private providers (mainly NGOs). 15. Component 2. Strengthening the Health Services Facilities- Network. The objective of this Component was to strengthen the MOH's health services facilities network through reinforcing physically and functionally the primary and secondary care facilities of the MOH to enable them to deliver the basic package of services through (i) rehabilitating and providing medical and hospital support equipment for the operation of the hospitals and health care centers in the municipalities covered by SILAIS and strengthening their managerial capacity; and (ii) expanding the Casas Maternas (Maternal Homes) network by (a) building or rehabilitating approximately 30 Casas Maternas and acquiring eqUipment-and furniture for all Casas Maternas in the network and funding their recurrent costs; and (b) strengthening the provision of family planning services. The rehabilitation of primary health care (PHC) centers and hospitals and the purchase of medical equipment were focused in the municipalities~d SILAIS targeted under the Project. All minor works and goods purchased under this component were aimed at increasing the functionality of the PHC centers and hospitals. Furthermore, for the hospitals, it aimed to improve their overall management including information, procurement and fmancial management systems. 16. Component 3. Stewardship, Institutional Development and Decentralization. To carry out the National Health Plan and effectively lead the SWAp, the MOH needed to undertake a variety of institutional reforms. This Component supported the MOH to (i) develop its leadership capacity and accountability by: (a) developing capacity to evaluate the performance of the national health system; (b) providing technical assistance to coordinate and monitor implementation of the population policr; (c) providing capacity building and awareness raising activities to encourage municipal Governments to take on their responsibilities regarding public health; and (d) strengthening its ability to coordinate and harmonize foreign -aid in the health sector by, inter alia, fmancing the operational costs of the entity that will be coordinating the FYIP with the Government, the Ass()ciationand the donors; (ii) strengthen the institutional capacity to improve health service delivery and equity by: (a) developing the MOH's purchasing function; (b) providing technical assistance, training, and information technology systems development to improve its fiduciary capacity; and (iii) further the MOH's decentralization agenda by: (a) transferring human and fmancial resources to SILAIS Administrative Units and local health care service providers; and (b) providing technical assistance, training and information technology systems for the two autonomous regions of the Atlantic Coast. 1.6 Revised Components 17. The project components were not revised during implementation. 2 This refers to the Government's policy on reducing the fertility rate. 4 1.7 Other significant changes 18. The closing date was extended by seven months from November 30, 2009 untH June 30, 2010 to enable the completion of minor rehabilitation works for health facilities, finalize procurement of medical equipment for indigenous communities and to complete the financial management assessment of the capitation transfers. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 19. The Project design benefitted from a well-defined Five-Year Implementation Plan, which drew on successful models of primary health care delivery and the financing and management of service delivery. The Integrated Health Care Model (MAIS) was developed using inputs from a comprehensive evaluation of the 14 primary health care delivery projectsthat were being implemented in the public sector nationwide in 2004.' The main strategy of the MAIS was to establish one basiC package of health services that would be adjusted to the needs of the target population. The model also dermed the technical framework or key instruments for the organization and delivery of quality health services (health team's organization and functions, clinical protocols and service standards, contracting mechanisms and performance-based payments). The Project design benefitted from some of the best practices in the country and the region including the use of performance agreements with service providers and the use of capitation payments. 20. The design of the Project also took into account the lessons'learned from the first phase of the APL. This included the successful experience of the Maternal Homes (Casas Maternas), introduced as a pilot intervention under APL I. These Maternal Homes are low-cost, assisted-living homes where women, mainly from remote, rural areas, can stay in the days leading up to giving birth and shortly thereafter, to ensure that women have ready access to an institutional facility when they go into labor. Evaluations showed that both the community and the women found the services important. Another important lesson was that the decentralization of administrative responsibilities to the SILAIS and the provision of services by local providers proved to be an effective way to expand coverage of services. However, the Implementation Completion Report for APL I also noted that it was necessary to improve the MOH's regional capacity and ability to monitor and evaluate ,performance. 21. Another innovative feature of, the Project was the use of the sector wide approach, leading to coordinated donor support for a standard package of health services and f"mancing and delivery model. This reduced the duplication and fragmentation of donor support in the health sector. 22. Nonetheless, although the SWAp aligned donor support around common sector objectives and strategies, efforts to pool all donor and counterpart funds did not succeed. During appraisal, many of the development partners, including IDA, agreed to pool their funds in support of the Government's Implementation Plan. The MOH created a group, the Nicaraguan Health Sector Support Fund (FONSALUD), which would pool all the funds and follow identical financial and procurement processes. At appraisal, the IDA project preparation team included 5 this as part oftne Project's design and a Memorandum of Understanding was signed between the , donors to this effect. It was only after the Project became effective that it became clear that IDA wOj.il.ld not be able to pool its funds with the other donors due to differences in fiduciary rules and processes. As a result, the IDAteam continued to participate in the meetings of FONSALUD, butthe Project continued to use a dedicated account, dedicated accounting, reporting and audits. 23. Anothe:r limitation the Project design was the selection and measurement of the PDO Indicators. These indicators were agreed upon collectively by the Government and the development partners for the Government's overall program. As part of the SWAp, the IDA team used these indicators to measure progress towards the PDO, although it was unlikely that the Project alone could have led to an improvement in these indicators (the maternal mortality rate, infant mortality rate, under-5 mortality rate, and childhood malnutrition rate). Furthermore, the indicators were measured at the national level, although the Project operated only in the poorest municipalities. The adoption of these PDO Indicators was in line with the intended objective of the development partners to measure national progress towards the MDGs. In order to do this, the donors agreed on using the Nicaraguan Demographic and Bealth Survey (ENDESA), which is carried out every five years. This created a problem of defining both an appropriate baseline and targets. In effect, the baseline values of the indicator were drawn from the 2001 ENDESA, though in fact, it would have been more appropriate to use the 2006 ENDESA indicators as baseline values,as the Project became effective in 2006. There was no means by which to measure the progress at the end of the Project (originally scheduled in 2009), as the next survey was scheduled for 2011. In the absence of a strong vital statistics information system, it was not possible to use other MOB administrative sources of information to measure changes in these development indicators. The lack of fmancing to strengthen the information system was a shortcoming of the design of the Project. 24; Despite issues with the PD~ Indicators, most of the Project's Intermediate Outcome Indicators were appropriate and aimed at capturing measuring progress towards the project development objectives. The Project's Intermediate Outcome Indicators were primarily targeted at monitoring changes in access and use of PBe services, compliance with performance agreements, and the mitigation of key risk factQrs may contribute to increases in mortality (prenatal controls, postpartum visit, immunization coverage, and institutional deliveries). They also aimed to measure improvements in the MOB's capacity through improvements in the managing of capitation payments and compliance with management agreement targets. 25. The MOB was responsible for the overall strategy while local SILAIS were accountable fo:r managing health service contracts with private and public providers and supervising performance agreements with public providers. Overall, the MoB Board of Directors, composed of the Minister, Vice-Minister and Directors Generals was responsible for planning and implementing the sector's overall strategy. Within that group, the General Division of Planning and Development (DGPD) was responsible for coordinating the Project. TheMoH also created a formal Purchasing Unit, which was responsible for the administration and execution of the capitation transfers. For the delivery of services, the MoB entered into management agreements (convenios de gesti6n) with the SILAIS for the transfer of per capita transfers and the delineation of management roles. Under these agreements, the SILAIS were responsible for contracting the primary care health services, qualifying service providers, supervising service contracts between the SILAIS and service providers and providing assistance 6 in the monitoring and evaluation of performance. Service contracts between the SILAIS and private service providers (NGOs, among others) ensured that the SILAIS, using the capitation transfers, could purchase the basic package of health services from the providers. SILAIS also established performance agreements with institutional MOH public providers (municipios and hospitals) to enforce the delivery of the basic package of services. 26. Placing project coordination and implementation under the DGPD was a notable change from the arrangements under API. I. When designing the second phase of the APL, the Government decided to abolish the Project Implementation Unit (PIU) that had been used under the fIrst phaseofthe APL and place the coordinator at the Director of the DGPD. The PIU was dissolved and its staff, who were already familiar with the Bank's policies and processes, were absorbed under the various technical divisions ofthe MOH. This proved to important for purposes of continuing the momentum from APL I and for promoting sustainable capacity in the MOH. While Project coordination and implementation responsibilities remained under the DGPD for the remainder of the Project,one of the challenges with this change was retaining highly qualified fiduciary staff at lower salaries. 27. Finally, the Project's risk matrix accurately assessed Project risks and related mitigation actions. The main risks identifIed included: (0 lack of continuity commitment with the project's policies and strategies; (ii) lack of capacity manage the complexity of a multi-donor program; (iii) mismanagement and comlption issues; (iv) lack of understanding and agreement among donor agencies and Government in planning, monitoring and evaluating the program; and (v) insufficient funds. Risks (Ii), (v) were mitigated through the development of the SWAp over the five years, ensured sufficient funds for the Implementation Plan. Furthermore, even with a change in administration in 2007 and the changes that took place in the health sector (see paragraphs 34~36), the Government, in general, remained committed to the Project's overall strategies of supporting a single model of health care (although the model changed), .a standard set of basic health and nutrition services (with greater emphasis on prevention) and the decentralization of budget management to the SILAIS .. 28. Readiness conditions. The Project had two effectiveness conditions, which were met by April 2006. 29. No formal QAG review was undertaken for this project. 2.2 Implementation 30. The Project was implemented over five and a half years. The US$11 million credit (SDR 7.21p.illion) for phase two of the APL was approved by the Bank's Board on AprilS, 2005 and signed on June 7, 2005. The Project became effective on April 11,2006 and closed on June 30,2010, after one extension of the Closing Date. 31. Although the members ofFONSALUD could not pool their funds or use common operational features, such as single accounts and financial reporting and audits, the group became a highly effective mechanism for donor coordination. In the fIrst two years of Project implementation, both the Bank and IDB were strongly criticized for not using country financial 7 and procurement procedures, as other donors had agreed to use under the Memorandum of Understanding. Over time, though, members of FONSALUD' w,eIcomed IDA's. strong commitment to meet regularly with donors, to coordinate actions,' to discuss and receive feedback on activities directly related to aspects financed under the Credit, and to.follow·up on agreed common indicators. ; , , 32. The change in administration, one year after the Project's' effectiveness, also led to some changes in the health sector and service delivery; however,' Dlany of these changes built on the policies and strategies in place and therefore there: was no impact on the Project's objectives. In 2008, the MOH presented a new National Health Plan which was based on the previous model (MAIS) and aimed to address the equity, efficiency, and access gaps in the Nicaraguan health sector through the implementation of a new Community and Family Health Model (Modelo de Salud Familiar y Comunitario, or MOSAFC). The MOSAFC was heavily based on the MAIS, with a focus on providing universal and :free access to basic health and nutrition services and hospital services; it varied from the MAIS in that the MOSAFC package of services placed more emphasis on health promotion and: disease prevention and on collaboration with local community organizations. It also sought to phange the past health care paradigm in which the MOH's institutions passively waited for patients to demand medical attention before providing services, and involve the family and the c<;lmmunity as key partners in health promotion and prevention. 33. The new national health plan also had a greater emphasis on payment for results, rather than services. Under the first years of the Project, the practiceoftransferring a portion of the MOH budget to hospitals and maternal clinics based on a system of services rendered was still common. By early 2008, though, with the change in administrations, this practice was discontinued. This was replaced with budget transfers linked to perfhnnance agreements between the SILAIS and services providers. With this change, one of the Project's Intennediate Outcome Indicators, percentage of total budget transferred to the targeted SILAIS as purchasing of services, was no longer monitored. ' ' 34. Finally, the MOB decided that only MOB public health facilities would provide basic health and nutrition services, rather than NGOs or other private providers. The Project design anticipated service provision would be done either thrqugh MOH public institutions and/or private institutions (primarily NGOs). With the change in administration however, the Government moved completely away from the delivery of serylc~sthrough NGOs to service provision only through MOH public institutions (hospitals and health centers). Where facilities did not exist, they sought to build them. This change was made: in an attempt to strengthen the MOH's institutional capacity and access of services at the local level, which was in line with the Project's objectives. 35. The establishment of the Project coordination under DGPD proved to be important for promoting the Project's sustain ability. Throughout the. lifetime of the Project, there was only one change in the Director of the DGPD (project coordinator). This occurred at the time of the change in administrations. 2.2 Monitoring and Evaluation (M&E) Design, Impleme~tation and Utilization 8 36. The overall responsibility for managing the M&E system was given to the DGPD at the central level. The DGPD was responsible for coordinating M&E efforts with the SILAIS teams at the regional level. The SILAIS were responsible for monitoring progress at the local level, which was then reported to the central level. Monthly and quarterly reports were designed to keep adequate control on progress towards meeting results established under the Project. In general, these reports were received in a timely manner. 37. In 2008, due to the shortcomings in the PDO Indicators and the data mentioned earlier, the Project and the IDB began financing the implementation of an annual technical audit to validate and verify information collected. During 2008, an external auditing firm was contracted to assess the technical outcomes of the SWAp. The technical auditor conducted a thorough review of raw data generated at the SILAIS and municipal level, the processes used for validating and consolidating the indicators produced and verification of reported results. The quality of results was evaluated based on a sample of municipalities. Overall, the last auditor's report found that infonnation related to the Intennediate Outcome Indicators provided an acceptable basis for detennining achievement of results. However, it was not able to audit the PD~ indicators because the'systems to monitor these indicators were not in place. Furthennore, improvements were needed at the primary health care centers regarding accuracy and completeness of basic data gathered by them. 38. Overall, the M&E system was implemented well. Although it took some time to produce reports on a regular basis, by the end of the second year of implementation the system was working satisfactorily. 2.3 Safeguard and Fiduciary Compliance 39. Given the presence of Indigenous Peoples in the Project area, an Indigenous Peoples Plan (IPP) was prepared; however, achievements on complying with the IPP were modest. The IPP provided recommendations on how to provide the basic package of health services considering traditional medicine, strengthen local networks of midwives, and provide social control mechanisms. An assessment (2008) of the plan's implementation; however, found that little progress was made on reaching remote areas, establishing new sub-regional health centers, training of additional doctors and midwives, and strengthening social control mechanisms and that funds had primarily gone toward purchasing medicines and medical equipment and expanding the number of Casas Maternas in the Atlantic Regions. This was largely due to the exclusion of key stakeholders from the central MOH during supervision in the dialogue with representatives from the two autonomous regions, which lead to tension between the two parties. As a result, the IPP rating was reduced to moderately unsatisfactory for nearly two years during implementation. Based on these findings, the GON prepared and presented an action plan in mid- 2009 to address the deficiencies and implement corrective actions. Subsequently, the MOH has adapted the new MOSAFC and the delivery of services to meet the needs and different demands of indigenous communities and ethnic populations. By the end of implementation, the final safeguard rating was changed to moderately satisfactory. 40. The Project complied satisfactorily with the environmental safeguards. In general, practices for handling hospital waste management were improved. The environmental assessment conducted during Project appraisal was focused on setting up a program for 9 completing preparation of guidelines on hospital waste management and established mechanisms to enforce its application. During implementation, most hospital rehabilitation programs included the redesign of waste management facilities and revised practices were established. 41. The Project's rmancial management performance was moderately satisfactory. The main issues during implementation were related to overdue Interim Financial Reports, fixed assets controls, overdue audits, and the MOH's weak follow-up of the audit and supervision recommendations. In 2008, an audit identified some questionable costs related to the capitation , transfers· to the SILAIS and health centers. As a result, a fmancial management assessment was completed shortly thereafter and concluded that for the period under review, the correct accounting and internal controls were in place and the capitation transfers were well managed. It also confirmed that these funds (vis-a-vis the capitation payments) provided the flexibility and immediate availability needed for the operation of the SILAIS and the health units. Overall, the MOH complied with the financial covenants of the Credit, and presented the Project's annual fmancial and external audit reports with no significant delays. 42. Likewise, Project procurement was moderately satisfactory through much of the second half of the implementation period, and at one point, it was rated unsatisfactory. First, there was frequent turnover of procurement staff. The MOH had a difficult time keeping highly qualified procurement staff in its Procurement Division that were knowledgeable of the Bank's .procurement guidelines and procedures. This was in large part because the salaries of individuals in the unit were not high enough to keep them there. Toward the end of the Project, the Bank agreed to finance two procurement staff under the Procurement Division of the MOH to avoid this type of turnover in the future. The second reason is because of the declaration of misprocurement for two contracts. In· 2009, the Bank;s Guidelines were not followed in two contracts. In one case, national bidding procedures were used instead of international. And in the other, only two proposals were requested under local shopping procedure, rather than of the standard three, as stated in the Association's guidelines. Consequently, in August 2010, the Bank declared misprocurement and cancelled US$714,762.08 of the health credit which corresponded to disbursements made in two contracts in 2006 and 2008. 3 In response, the Government communicated to the Bank, in a letter on August 6, 2009, that these amounts would be repaid after the National Budget Adjustment was approved by the National Assembly in October 2009. On November 10, 2009, the Bank received the full amount related to the cancellation. 2.5 Post-completion Operation/Next Phase 43. The Bank is supporting the implementation of the MOSAFC Model that the Government implemented in 2008 through a follow-up operation, the Improving Community and Family Health Care Services Project (Report No: 56262-NI, approved by the Board on December 7, 2010), which draws on lessons from the SWAp and the APL. The objectives of the Project are to: (i) improve the access to, and the quality of, preventive and promotion health and nutrition services among poor and vulnerable populations in Nicaragua; (ii) strengthen the operational capacity of the MOH through the rehabilitation of health centers; and 3 The referred contracts are the following: (i) No CP-MlNSA-BM-7-2006 (PROCOSAM), US$2I,2I5; and (ii) PN- BM-25-4-2008 (works for a Health Center, San Juan de Rio Coco, phase-I), US$682,977.85. 10 (iii) ensure fmancial support in case of a public health emergency. The new project will finance capitation transfers for the purchase of a portion of the set of basic health services (conjuntode prestaciones) defmedunder the MOSAFC in the 32 municipal health networks currently lacking resources for implementation of the complete set of services. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 44. The Project supported the standardization and effective delivery of the basic package of health and nutrition services in poor and vulnerable communities, which continues to resonate today. In an effort to reduce the cost of health care expenditures and to rapidly expand access to services, the. Government focused establishing and expanding a standard package of health and nutrition services vis-a-vie capitation transfers. This proved to be an effective way to introduce incentives for providers to enroll beneficiaries and deliver services. The MOH still uses this mechanism to deliver health promotion and prevention services today, as evident by the design under the new Bank financed operation (Report No: 56262-NI). 45. The Memorandum of Understanding with FONSALUD remains important for Government-donor relations. While the Bank quickly learned that it was not going to be able to pool its funds with the other donors because of the different fiduciary rules and regulations, FONSALUD became an important tool for harmonizing donors' support for policies and programs within the sector. This prevented overlap and duplication of services in the same municipalities and localities .. Most recently, the current donors that make up FONSALUD, including the World Bank, agreed to support a common set of indicators and goals proposed by the MOH. 46. The Project objectives continue to be in line with the Government's current health strategy, the National Human Development Plan (NHDP) 2009-2011 and the MDGs. The GON has acknowledged that the Project's objectives remain relevant to the current NHDP 2009- 20ll. The updated NHDP takes into account Nicaragua's domestic environment in a recovering global economy. Priorities of the health sector emphasize preventive health care and the availability of comprehensive services,· particularly for vulnerable populations. The Project's objectives support several policy objectives articulated in the NHDP, including health promotion, fighting against maternal and child mortality, increasing services to the poor and closing access gaps, adapting appropriate care for indigenous groups, and making better use of existing resources. 4 Finally, while Nicaragua has only achieved one of eight MDGs to date, the Projects' objectives continue to be relevant in achieving these goals including MDGs 4 (Reduce Child Mortality), 5 (Improve Maternal Health, currently categorized as "off track"), and 6 (Combat HIV/AIDS, Malaria, and Other Diseases). 3.2 Achievement of Project Development Objectives 4 Nicaragua: Updated National Human Development Plan 2009-2011: Technical Summary (Moving Forward Despite the International Economic Crisis). IMF, May 2010. II 47. Significant progress was made in maternal and infant care. During the project life, the maternal, infant and under five mortality rates decreased significantly nationwide. Furthermore, in the targeted municipalities, the MOH successfully extended the basic package of health and nutrition services from 129,700 individuals to 529,300 individuals. The Project also supported increases in maternal and infant care as evidenced by an increase in institutional deliveries from 36.2 percent to 64.4 percent and a rise in the percentage of pregnancies with full (four controls) prenatal care from 49.5 percent to 5S.4 percent.· The Project also increased the use of services, particularly maternal homes (Casas Maternas), and increased hospital discharge rates in targeted hospitals. 4S. Despite the change in Government in 2007 and modifications in the MOH's service delivery model, important progress was also made in terms of strengthening health service networks and improving the institutional capacity of the MOH. The Project supported improvements in infrastructure and renovated existing primary health care centers allowing for the MOH to extend access of a standard package of basic health and nutrition services to rural and remote communities. Important progress was also made in the decentralization of budget management and monitoring to the SILAIS. After the misprocurement, the implementation of an action plan contributed to improve procurement capacity. However, the management and institutional capacity of the SILAIS at the departmental level still needs much improvement. Toward the conclusion of the Project, the Government began to evaluate the capacity of the SILAIS and identify areas for improvement. 49. Progress towards achievement of Project Development Objectives is mixed. As shown below, while some of the indicators' targets were exceeded or met, others were only partially met or not met at all. Data for two of the Intermediate Outcome Indicators, under the institutional strengthening objective, are not available. One of the indicators, "percentage of total budget transferred to the targeted SILAIS as purchasing of services" was dropped halfway through the Project and another indicator, "per capita expenditure allocated to target SILAIS" was never a part of the M&E system and therefore was never monitored. 50. Below is an evaluation of each of the key PDO Indicators and Intermediate Outcome Indicators. PD~ Indicators were aimed at capturing, the longer-term program impact on key MDGs and therefore data represent national averages. Measured impact cannot be directly attributed to the Project. Data on Intermediate Outcome Indicators was collected for all 79 municipalities that were supported with IDA funds and funds from other donors. Annex 2 provides a more in-depth description of these indicators. PDO Indicators 51. Of the four PDO Indicator targets, three were achieved and one was not achieved. During the lifetime of the Project, the maternal mortality rate nationwide fell from 96 per 100,000 live births registered in 2005 to 61 per 100,000 in 2009, exceeding the target of 65 for 2009. In the period 2001-200S, the infant mortality rate fell from 31 per 1,000 live births to 23 per 1,000 live births and the under-5 mortality rate fell from 41 per 1,000 live births to 27 per 1,000 live births nationwide. The percentage of population with chronic malnutrition fell from 17.S percent in 2001 to 16.9 percent in 2006, which falls short of the 2009 target of 12.8 percent. Under the project, chronic malnutrition was measured for the entire population; for the under-5 population, 12 chronic malnutrition decreased from 25.8 to 21.7 percent from 2001 to 2006. Intermediate Outcome Indicators 52. Improved coverage and access to the basic package of health and nutrition services for targeted population. Overall, the use of the capitation transfers for the extension of the basic package of health and nutrition services proved to be an important mechanism for improving maternal and infant health care. The six targets related to improved access and coverage of the basic package of he~lth and nutrition services were all achieved. The Basic Package of Health Services was extended to 529,652 beneficiaries in 2009, exceeding its target of 470,000 beneficiaries. MOH administrative data also showed notable results in maternal and infant health care between 2005 and 2009, including an increase in institutional deliveries in the targeted SILAIS from 36.2 percent to 78.7 percent; an increase in immunization coverage in infants under 1 from 78 percent) to 98.3 percent; an increase in early prenatal controls from 34.4 percent to 48.1 percent; and ~ increase in pregnancies with full prenatal care (four visits) from 49.5 percent to 58.4 percent. There was also an increase in the family planning methods utilization rates in fertile women between 64.1 percent in 2001 and 72 percent in2006. 53. Strengthening Health Services networks in targeted area. Of the five targets proposed under this second strategic objective, two targets were fully achieved and three targets were not achieved. The two targets that were achieved were an increase in utilization of Maternal Homes (Casas MaternasJas measured by the number of discharges, with 2,400 in 2005 and 9,749 in 2009 and an increase in the hospital discharge rate from 4.02 in 2005 to 6.3 in 2009. The three indicators not met include the number of hospitals with a critical path complying with MOH certification standards, the percentage of public health provider complying satisfactorily with management agreement targets, and the percentage of maternal deaths audited. There was some progress made on the last two indicators. Qualitative evidence also shows that the management agreements proved to be important instruments for holding the SILAIS accountable for purchasing the services. SILAIS were required to meet the requirements set out in the agreement, particularly the adequate management of capitation transfers and the monitoring of the provision of health services. During implementation, these agreements also served as a tool to increase the operational and administrative capacity of the staff in the SILAIS. 54. ImprovedSector Stewardship and Institutional Strengthening. Of the three results aimed at capturing th~ impact of this strategic objective, only one was achieved, while the other two indicators wetenot monitored throughout much of the Project. For this third strategic objective, only the percentage of MOH budget in Indigenous territories administered by the RAAN and RAAS Governments was monitored and achieved, with 100 percent of the MOH budget for Indigenous temtories going to indigenous territories. Of the other two indicators, the one measuring the percentage of total MOH budget transferred to target SILAIS that is going toward the purchase of services was dropped when the new administration decided to discontinue the fee-for-servic~ practice; the other indicator, per capita public health expenditure allocated to targeted BILAIS, was not included in the M&E system nor was· it monitored throughout the lifetime of th~ Project. Project restructuring to add new indicators was never considered given the lack of interest of donors to change the common set. of indicators and a greater emphasis placed on other Intermediate Outcome Indicators. 13 3.3 Efficiency 55. The cost benefit analysis of the SWAp revealed considerable net benefits. A cost- benefit analysis was carried out at appraisal, for the sectoral program as a whole, which included contributions from members of FONSALUD other bilateral donors including USAID and Japan. No estimates were made of net benefits accruing only from the IDA project. It was expectedthat the Program would produce the following direct benefits: (i) reductions in hospital admissions of children under-5 years of age;· (ii) a reduction in hospital bed-days due to efficiency improvements; and (iii) improvements in the efficiency of the MOH oWing to advances in the process of decentralization. It was further assessed thatthe Program would also produce indirect benefits in the form of potential years of life that would be saved as a result. of nutrition interventions, together with the lives saved from reducing child and maternal mortality rates in the targeted areas. The present value of net benefits, over a 10...year plaflning horizon, was estimated. at US$66.7 million, with an internal rate of return of 54 percent. . 56. Due to the scope of the sector-wide program, it was difficult to attribute the net benefits of the program to anyone donor's investment. During implementation, some of the investments covered sector needs beyond the original scope set for the five-year plan in the 79 municipalities in the 12 targeted SILAIS. Additionally, other sector investments not channeled through FONSALUD also reached the target areas and therefore the impact of single· donor- supported intervention was difficult to capture. The consequence was a difilassociation between overall investment costs financed by MOH, through FONSALUD, IDA the IDB,and other donors and the benefits foreseen under the original· scope of the SWAp scheme. For this reason, it Was not possible to replicate the methodology used in the PAD to assess net benefits. 57. After the closing of the Project, the net benefits were re-assessed only considering the return on the investment of the Project and found to be marginal (Annex 3) Conservatively, it was assumed that benefits directly associated with the Project's investment could be in a range between 30 and 40 percent of the total benefits. The assessment concluded that when benefits were estimated to be closer to 30 percent, the rate of return was slightly negative. In tum, when benefits were calculate to be closer to 40 percent, the rate of return can be estimated at close to 6 percent. Benefits were associated with the population in the 79 municipalities and estimated as follows: (i) a reduction of maternal and infant mortality, (ii) a reduction in chronic malnutrition, (iii) institutional and private cost gains resulting from improved practices,and (iv) savings generated due to improved efficiency of MOH management processes from decentralization scheme. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 58. The Project's Overall Outcome Rating is rated Moderately Satisfactory. Overall, the Project had satisfactory results and represented a good experience of harmonization between the Government and several development donors. The main achievements include: (i) supporting the achievement of 3 of the 4 PD~ Indicators and achievement of 10 of the 14 Intermediate Outcome Indicators, and; (ii) an increase in prenatal care and institutional deliveries; (iii) the defmition and provision of a basic package of care through primary health care services; (iv) the use of an innovative mechanism (capitation payments) and performance agreements to improve 14 results; (v) improvements in the Ministry of Health's managerial and operational capacity; (vi) an agreement among development donors to support the same primary health care model; and (vii) donor harmonization. 59. Nonetheless, there were implementation challenges. Some of the main weaknesses and challenges include: (i) the dropping of two of the PD~ Indicators; (ii) the challenge of measuring some of the PD~ Indicators with a national focus, when the Project was targeted to specific areas; (iii) the need to invest in surveillance and monitoring and evaluation systems; (iv) the need to fme-tune the institutional and fiduciary arrangements; (v) the inability to pool donor funds, as subscribed under a Memorandum of Understanding; (vi) the dropping of all NGO contracting; and (vii) the need to continue to strengthen the MOH's capacity, partiCUlarly at the regional level, to manage the capitation payments and monitor service delivery. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 60. In general, the Project did have a direct impact on many of the communities living in extreme poverty. The Project's interventions reached municipalities with the highest vulnerability, with an explicit focus on maternal and infant care. The Project's activities targeted improvements in access to health services in rural areas where the concentration of poverty is extremely high. At the same time, it focused on indigenous populations living in the same rural areas. Efforts were also made to adjust he~lth services procedures to the characteristics and the socio-cultural needs of the different districts and communities with indigenous population. In particular, the Project supported training of traditional midwives in order to provide institutional deliveries in indigenous communities. (b) Institutional· Change/Strengthening 61. The role of the MOH as the leader of the SWAp 'proved to be important in the harmonization of donor support. The SWAp not only proved to be an important instrument to pool some of the donors' funds (through FONSALUD), but it also became an instrument for the MOH to bring together all donors and to harmonize technical and financial support. It also provided an avenue for the MOR to support a single integrated model of health services and to provide a standard basic package of health and nutrition services. This proved to be an important institutional change for the MOR who had formally had to qeal with fragmented and sometimes duplicative support from donors. 62. The decentralization of budget allocations and monitoring to the SILAIS also proved to be an important for the delivery of services. While the MOR was responsible for overall coordination of the health strategy, supervision, and management /administration of capitation transfers, the SILAIS received capitation transfers and monitored progress at the local levels. These agreements held the SILAIS accountable for achieving results framework in the Government's FYIP. Today, their role remains critical in the provision of health services at the departmental and municipal levels. While there is still a need to strengthen the SILAIS' institutional and operational capacity, the MOR continues to support this institutional change and has found these agreements to be valuable management tools. 15 (c) Other Unintended Outcomes and Impacts (positive or negative) 63. No other unintended outcomes and impacts are worth noting. 3.6 Summary of Findings of BeJleficiary Survey andlor Stakeholder Workshops 64. Two beneficiary surveys were carried out in 2009a\nd 2010 aiming at obtaining information about the perception that beneficiaries had. about the quality of services provided in primary care health .centers. "Beneficiaries were women seeking service attention for themselves or their children less than 5 years. The specific objective was to evaluate access to health services and users satisfaction. A sample of 549 beneficiaries was selected for interviews that took place as they were exiting a health service facility. Nearly 80 percent of beneficiaries interviewed were women living in rural areas. Overall, 77 percent of the beneficiaries had a positive perception of the quality of the services. The vast majority of beneficiaries, however, complained that they were not treated well and that there needed to be considerable improvement in the cleanliness ofhealth facilities. Additional findings are summarized in Annex 5. 4. Assessment of Risk to Development Outcome Rating: Moderate 65. The Risk to Development Outcome is considered Moderate. The Project's main contributions were in expanding coverage of a standard basic package of health and nutrition services as part of a single integrated health care model and improving maternal and infant health care services. Changes in the institutional management of the provision of health services were also notable. That said, .there are three risks that· could potentially threaten development outcomes. 66. The first risk is that Nicaragua is highly prone to external shocks, which could. affect important gains made in the health sector. Over the past couple of years, Nicaragua has endured multiple crises, including hurricanes (Felix and Ida), recurrent droughts ("el Niiio" induced) and floods and the epidemiological emergencies such as the AlHINI flu pandemic and the dengue epidemic. In June 2009, the first case of the AlHINl human influenza virus was confirmed in Nicaragua. As of November 2009, the virus had spread to 101 ofthe country's 153 municipalities, requiring the health sector to provide nearly 16,000 treatments for AlHINI patients and their contacts. The combination of these shocks considerably strained the Government of Nicaragua's (GoN) financial and institutional resources. If these shocks continue to occur, they are likely to threaten existing gains in poverty reduction and other social indicators including those in the health sector. 67. The second risk is the potential shut down of the health sector, as was the case in 2006. In 2006, the entire Nicaraguan health sector went on strike, demanding an increase in salary (Nicaraguan doctors and nurses were receiving the lowest paying salary.in the region). Hospitals and health clinics shut down for more than six months, contributing to increases in maternal and infant mortality, missed appointments· and a decrease in pre-natal controls. If a strike of this magnitude were to occur again, achievements supported by the Project could easily be reversed. 68. The third risk is that external funding for the health sector is reduced having a direct impact on access and services coverage. Recently, some development partners have reduced or 16 withdrawn their support to the Nicaraguan health sector. Given the sector's traditionally high inflows of assistance, a significant reduction could affect the number of individuals, families, and communities receiving basic health and nutrition services throughout the country that had previously been supported under the Project. While the GON is currently reaching out to other donors to maintain the level of funding, fewer resources would require the MOH to be more efficient with the management and use of funds. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 69. Bank Performance in Ensuring Quality at Entry is rated Moderately Satisfactory based on the following factors: In general, the technical design of the Project at entry was both relevant for the health sector and considered the lessons learned under the first phase of the APL. Nonetheless, the signing of the Memorandum of Understanding without a detailed assessment of the Bank's fiduciary restrictions proved to be a challenge for project implementation in the beginning. Finally, achievement of the PD~ indicators was difficult to follow given that data from the ENDESA survey were only collected every 5 years. (b) Quality of Supervision Rating: Moderately Satisfactory 70. The Quality of Supervision is rated Moderately Satisfactory. Despite challenges at entry, the supervision of the Project was moderately satisfactory. While the Bank was not able to pool its funds with the rest of the donors from FONSALUD, it played a key role in helping to coordinate donor support througholl;t the lifespan of the Project. Bank supervision also relied heavily on the local Bank office procurement and financial management staff and the human development operations officer for frequent monitoring and follow-up with the MOH. The addition of annual technical audits proved to be an effective tool to hold the Government accountable for results. However, there were shortcomings in the monitoring of key PD~ Indicators, primarily due to the absence of a strong monitoring and surveillance system and partly because some of the intermediate indicators became irrelevant during the Project. The Project was never restructured, though, in part because the donors were not keen on re-defining the indicators halfway through the SWAp and because there was greater emphasis placed on the results from other Intermediate Outcome Indicators. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 71. Overall Bank Performance is rated Moderately Satisfactory. This is based on a Moderately Satisfactory rating for Quality at Entry and a Moderately Satisfactory rating for Supervision. 5.2 Borrower Performance 17 (a) Government Performance Rating: Satisfactory 72. The Government Performance is rated Satisfactory. Government performance is interpreted as the broader Government, i.e., the MOF, the Ministry of the Presidency and other central Government institutions. The GON showed ownership and commitment for the timely implementation of the five-year implementation plan and provided strong leadership and commitment through the change in administrations. Project objectives were closely aligned with both administrations' health sector agendas and therefore notable outcomes in the provision of maternal and infant health services were achieved. Finally, the Government was extremely supportive in the design and development of the recently approved health sector operation (Improving Community and Family Health Services Project; Report No: 56262-NI, which builds on the successes ofthis Project). (b) Implementing Agency Performance Rating: Moderately Satisfactory 73. Overall, the Implementing Agency's performance is rated Moderately Satisfactory in view of the differing levels of capacity and frequent turnover of critical fiduciary staff. The MOR, as the implementing agency, was highly committed to achieving the results established under the Project. It supported implementation of a decentralized scheme, strengthening SILAIS and establishing new budgetary processes more in line with modem management schemes. Furthermore, the MOR gained important ground as the leader of the sector-wide program and the FONSALUD group. Nonetheless, frequent turnover of key fiduciary staff led to problems in procurement and financial management. The MOR also struggled to comply with the poorly defined IPP covenants. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 74. Overall, the Recipient's Performance is rated Moderately Satisfactory given that more emphasis is placed on the performance of the implementation agency in palticular. 6. Lessons Learned 75. The importance of surveillance and. monitoring and evaluation should not be underestimated. Given the challenges associated with availability of data to monitor PD~ indicators and Intermediate Outcome Indicators, the Project could have had a stronger focus, including dedicated financial resources, to strengthen surveillance, monitoring and evaluation. Faced with these challenges, a project restructuring may have been helpful in addressing these issues. 76. The use of national outcome indicators as PDO indicators was problematic given that the Project had a targeted approach. While the use of common national targets for all donors was helpful in terms of donor coordination and harmonization, it was not appropriate to measure the success of this project which featured interventions in targeted areas. This affected the economic analysis of the Project, and raised additional issues of attribution. 18 77. Notwithstanding the issues in monitoring and evaluation, the Project also demonstrated that the expansion of Maternal Homes (Casas Maternas) made a critical contribution to improving institutional deliveries. It is estimated that 12 percent of institutional deliveries during 2009, in municipalities covered by the Project, can be explained by the increase in attendance to Women's Homes, with respect to the base year. In addition, the number of pregnant women who came from rural areas to Women's Homes in urban areas increased fourfold between 2005 and 2009. Coordinated action by these homes and community health networks has also increased involvement in family planning programs. Building on this success, further areas for improvement were identified during project implementation, including the importance of: (i) harmonizing reproductive health training programs run by different Government agencies, donors and NGOs; (ii) setting service standards and creating a Federation of Women's Homes; and (iii) building capacity in additional communities. 78. More generally, the Project demonstrated that a Primary Health Care model based on a basic package of services works in Nicaragua. The Bank will continue to support this approach in the subsequent health project. 79. Performance Agreements became a very useful tool to establish a results-oriented culture. This was a powerful tool that had a positive impact for managing decentralized health programs. However, the system still needs further improvements for evaluating achievements on a regular basis. Auditing the M&E system also helped to monitor the implementation of performance agreements using a consistent methodology to enable corrective actions to be taken in a timely manner. This was a highly positive contribution introduced by the Project that enhanced the use of existing systems to produce reliable information on the PDO and Intermediate Outcome Indicators used to assess achievements of results. Overall, it confirmed that the information produced by the M&E system was reliable. 80. The Project also proved that it is possible to move from a situation of many models and sources of financing to a coordinated sector approach. In fact, the sector wide approach in the health sector proved to be a highly effective mechanism for coordinating donors' support, and it may be possible to replicate this model in other sectors. 81. One final lesson, however, is that pooling funds is very difficult. The necessary conditions to allow for pooling of funds are challenging, and in hindsight it may have been overly optimistic given the country's capacity constraints and fiduciary systems to expect that this could have been possible from the start. However, despite the fact IDA and the IDB were not able to channel their financial resources through FONSALUD, their involvement helped to strengthen coordination on the health sector investment programs and ensure that key priorities were shared. Similarly, other donors who opted not to channel resources through this mechanism, notably USAID and Japan, maintained active participation at critical meetings and a coordinated approach in the sector. 7. Comments on Issues Raised by BorrowerlImplementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers 19 (c) Other partners and stakeholders 20 Annex 1. Project Costs and Financing usn Million 7.5 5.2 69 the network of services areas to support the implementation 2.8 3.9 139 ofhealth services Stewardship, 0.7 1.2 171 94 US$714,000 cancelled due to misprocurement. 21 Anne~ 2. Outputs by Component 1. Component 1 - Extension of the Basic Package of Health Services. The objective of this component was to (i) pay capitation transfers to SILAIS administrative units to deliver the basic package of health services to participating communities and (ii) provide technical assistance to develop institutional capacity in the SILAIS and municipios that will implement the delivery of the basic package of health services. In general, the results from this component were positive. The basic package of services had a strong focus on maternal and child health care. It included monitoring and promotion of growth and development of children under-5 years of age, integrated care for women of reproductive age and detection and care of prevalent diseases (See Box 1). Box 1. Package of Basic Health Services offered under Integrated Health Care Model 1. Monitoring and Promotion of Growth and Development, for children under 5 years a) Promotional and Preventive Actions • Promotion of healthy diet, personal hygiene, oral health, breastfeeding, distribution of information and education material. • Monitoring of infant and prenatal mortality. • Training, early evaluation and classification of newborn infants based on their level of risk. • Parental education about basic care of newborn infants, and infant and adolescent growth and development. .. Periodic care and monitoring of infant growth and development, including evaluation of vision and hearing, monitoring of height and weight and development of psychometric and motor skills. .. Promotion of a complete immunization plan during the first year of the infant's life. • Implementation of the compulsory immunization program. • Supply of vitamin A. iron, iodine, fluoride, folic acid and ferrous sulfate. b) Early detection and care • Health Care of newborns and treatment of complications. • Outpatient care for infant health and children under 5 years • Referrals and cross-referrals for complicated patients. • Treatment of communicable diseases. • Detection of high-risk cases (such as victims of childhood abuse, mistreatment and violence). c) Development of community health and nutrition interventions • Education and development of healthy diet for children, personal, family and household hygiene. • Distribution oflEC material. • Prevention, early diagnosis and treatment of malnutrition. • Supply of vitamin A, iron, iodine, fluoride, folic acid and ferrous sulfate. • Implementation of the Community Program of Health and Nutrition. • Treatment of nutritional problems and associated diseases. • Case referral. 2. Integrated care for women of reproductive age a) Pre, post natal and obstetric care • Promotion of healthy diet, oral health, newborn hygiene and care, breastfeeding, institutional births, family planning, distribution of IEC material. • Care of normal low-risk pregnancies and deliveries. • Referral and follow-up of high-risk cases. • Early detection and care of pregnancies. • Prenatal care. • Evaluation of nutritional status of pregnant women. 22 • Provision of nutritional supplements to pregnant and post-natal women. • Anti-tetanus immunization of pregnant women. • Family planning education. • Diagnosis, treatment and counseling for sexual and reproductive diseases. • Emergency care for pre and post natal complications and complications .during delivery. • Diagnosis and treatment of diseases in pre and post natal women. • Incentives for referral to Casas /vfaternas. b) Early detection of cervical and breast cancer • Promotion of self-examination and PAP smears. • Distribution ofIEC material. • PAP smears. • Breast examination and self-examination .. Case diagnosis and referral. c) Sexual education and family planning • Sexual health and education with emphasis on abstinence in adolescence. .. Training and care of adolescents with risk factors. .. Detection, care and referral of cases of violence. • Promotion of spacing between births. .. Promotion of correct usage of traditional and modem methods. .. Distribution of lEC material. • Training and delivery of family planning methods. • Distribution of contraceptives in the communitL-_ _ _ _ _ _.___ _ _ 3. Care of prevalent diseases .. Treatment ofsexuaUy transmitted infections (including HIV/AIDS). .. Early detection, control and treatment of malaria, dengue and tuberculosis. ~~ j .. Early detection and treatment ofh ertension and diabetes mellitus. 2. For the delivery of services, the MoB entered into management agreements (convenios de gestiOn) with the SILAIS for the transfer of per capita transfers and the delineation of management roles. Unde:· these agreements, the SILAIS were responsible for contracting the primary care health services, qualifying service providers, supervising service contracts between the SILAIS and service providers and providing assistance in the monitoring and evaluation of perfonnance. Service contracts between the SILAIS and private service providers ensured that the SILAIS, using the capitation transfers, could purchase the basic package of health services from the providers. SILAIS also established performance agreements with institutional public providers (municipios and hospitals) to enforce the delivery of the basic package of health services. 3. Capitation transfers were analogous to health insurance premiums for purchasing coverage and were calculated on the basis of the average unit costs of ensuring access to the defined package of health services to the target population. For the purchase of services, the SILAIS entered into perfonnance agreements with public providers and services contracts with private providers to ensure that the services were delivered in a timely manner. The transfers were made monthly and adjusted annually to account for changes in population. 4. This component contributed to the financing of capitation transfers to selected SILAIS. The basic package of health services aimed at (i) monitoring promotion of growth and development for children under-5 years, (ii) providing integrated care for women of reproductive age, and (iii) providing care for common diseases, including sexually transmitted diseases, 23 malaria, dengue, Tuberculosis and the treatment of hypertension and diabetes~ From the Project, a total ofUS$5.2 million in ca PI Ion trans £ers reached the 12 SILAIS: ·tar Amount ~ SILAIS million) Boaeo 0;13 Chinandega 0.2 Chontales L09 Esteli 0.48 Jinotega 0.32 Leon 0.31 Madriz 0.2 Matagalpa 0.29 RAAN 0.36 RAAS 0.69 Rio Sanjuan 0.84 Segovia 0.54 5. Improved coverage and access to the basic package of health and nutrition services for targeted population. Of the six Intermediate Outcome Indicators set up to monitor· improved access and coverage of the basic package of health and nutrition services, all six targets were fully met. • Beneficiaries to whom the package of Basic Health Services has been extended. The Project set a target of adding 470,000 new beneficiaries with access to the basic package of health services. At the time the Project was appraised, 129,698 individuals were benefitting from the basic package. By 2009,529,652 beneficiaries were benefitting from the basic package of health and nutrition services, exceeding the baseline by 12 percent. • Institutional Deliveries. In 2005, only 36.2 percent of all women in the targeted municipalities had an institutional birth. Only a very modest increase was expected: the target was 37.9 percent. By Project closing, institutional deliveries had increased by 40.8 percentage points, to 78.7 percent, exceeding the target. . • Immunization coverage (pentavalent vaccine) in infants under 1 year of age. At the time of appraisal, immunization coverage was at 78 percent for infants under 1 year of age. The Project set the target at 97.5 of all infants under the age of 1. The Project target was exceeded, with 98.3 percent of children under-l receiving their immunization. • Early prenatal control (fIrst trimester) in targeted municipalities. Notable improvements were made in terms of increasing prenatal controls for pregnant women. In 2005, 34.4 percent of pregnant women were receiving prenatal controls. The percentage of women attending their prenatal control increased to 48.1 percent by the end of 2009, 24 exceeding the target 37.3 percent. This was an important advance for the Project. • Pregnancies with full (four visits) prenatal care in targeted municipalities. The baseline was 49.5 percent of pregnant women in the targeted municipalities had pregnancies with ibll prenatal care and the target was 55 percent. By the Project closing, 58.4 percent of pregnant women in the 79 targeted municipalities had received full prenatal care. Many of the pregnant women not completing their controls reside in the indigenous territories or communities. For many of them, the examination performed by a male doctor presented numerous cultural barriers. • Utilization of family planning methods among fertile-aged women. In 2001, the ENDESA survey reported that 64.1 percent of fertile-aged women were using some sort of family planning method. Under the Project, a target value was set for 2006 at 66 percent, and in 2006 the ENDESA survey reported that 72 percent of all fertile-aged women had used some form of family planning. Although this indicator was not measured again at the conclusion of the Project, the outcome value did meet the target value. 6. Component 2 - Strengthening Health Services Networks. The objective of this Component was to strengthen the MOH's health services facilities network through reinforcing physically and functionally the primary and secondary care facilities of the MOH to enable them to deliver the basic package of health services through (i) rehabilitating and providing medical and hospital support equipment for the operation of the hospitals and health care centers in the municipalities covered by SILAIS and strengthening their managerial capacity; and (ii) expanding the Casas Maternas (Maternal Homes) networks in targeted areas. 7. This component included the rebabilitation of two health posts and three health centers with a total cost of US$190,OOO. The largest· rehabilitation project fmanced was the rehabilitation of the critical path in the San Juan de Dios Hospital in Estell with a total cost of US$2.4 million. It also financed supply and installation of medical equipment and emergency units in 12 hospitals for US$1.01 million, and the supply of US$158,000 in ambulances and equipment in the Atlantic regions to support implementation of the indigenous people plan. 8. Strengthening Health Services networks in targeted areas. Of the five Intermediate Outcome Indicators proposed under this second strategic objective, two targets were fully achieved and three targets were not achieved. • Utilization of Maternal Homes (Casas Maternas) by pregnant women. One of the lessons learned from APL I was the importance of the maternal homes. As a result, the Project promoted the use of these ~ervices to reduce mortality and improve maternal and infant care. In 2005, the MOH had registered about 2,400 discharges of pregnant women from the maternal homes. The Project had projected that by 2009 there would be 6,000 discharges from increased use. This target was exceeded by 3,749 discharges, reaching a total of 9,749 discharges in 2009. • Rate of hospital discharges in 12 targeted hospitals. This efficiency indicator measures 25 the rate at which hospitals discharged patients, or the number of annual hospital discharges per 100 individuals in the areas who had access to or used the services in the targeted hospitals. By appraisal, the MOH was reporting an average hospital discharge rate of 4.02 percent. By Project closing, the hospital discharge rate had increased to 6.3 percent, exceeding the target. .• Number. of hospitals whose critical path is in compliance with MOB certification standards. By appraisal, none of the hospitals in the targeted departments were certified; the target of 9 hospitals. Supervision missions reported that three of the 12 targeted hospitals had been visited and that their critical paths were in full operation. It was never confirmed, though, that the MOH had certification standards. As a result, this indicator was not met. • Percentage of public health providers complying satisfactorily with management agreement targets. This indicator aimed to capture if the targets set out in ,the management agreements between the MOH (through the SILAIS) and the institutional providers (hospitals and municipalities; other agreements were used for NGOs and private providers) were satisfactorily met. Some of the elements set out in the management agreements included the number of individuals covered, institutional capacity improvements, and management of capitation transfers (as defmed in the results framework in the GON's Five-Year Implementation Plan. While the target was set at 100 percent, by Project closing, it was reported that only 60 percent of the public health providers had satisfactorily complied with the management agreement targets. As a result, this indicator was partially met. • Percentage of maternal deaths audited. A target of 100 percent of maternal deaths audited had been set at appraisal. By 2010, only 75 percent of the deaths had been audited. As a result, this indicator was only partially met. 9. Component 3 - Stewardship, Institutional Strengthening and Decentralization. In order to carry·out the NHP and effectively leading the SWAp, the MOH needed to undertake a variety of institutional reforms. To do so, this Component supported the MOH to (i) develop its leadership capacity and accountability; strengthen the institutional capacity to improve health service delivery and equity; and (iii) further the MOH's decentralization agenda. Some of the activities included the carrying out of the 2006 ENDESA survey, the strengthening of procurement and financial activities in MOH, and technical and financial aUditing. It also included support for improving budget processes in targeted SILAIS, budgetary decentralization achieved in the Atlantic regions and capitation allocations. It should be noted that improvement of budgetary processes for purchasing of health services was not implemented, since the Ortega administration revised the policy of contracting out these services. Consequently, this indicator became irrelevant. 10. Improved Sector Stewardship and Institutional Strengthening. Of the three Intermediate Outcome Indicators aimed at capturing the impact of this strategic objective, only one of the indicators was achieved, while the other two indicators were not monitored throughout much of the Project. 26 • Percentage of total MOB budget transferred to target SILAIS that is going toward the purchase of services. During appraisal, the practice of transferring a portIon of the MOR budget to hospitals and maternal clinics based on the system of services rendered was still common. In late 2007, with the change in administrations, this practice was discontinued. As a result, this indicator become irrelevant and was excluded from the Monitoring and Evaluation system. • Percentage of MOB budget in Indigenous territories administered by the RAAN and RAAS Governments. Given the importance placed on extending the basic package of health and nutrition services in the two autonomous indigenous territories (RAAN and RAAS), this indicator aimed to measure how much of the funds the two territories were managing at the conclusion of the Project. The target was set at 100 percent and the Project met the target. • Per capita public health expenditure allocated to targeted SILAIS. No information regarding this indicator was included in the M&E system. 27 Annex 3. Economic and Financial Analysis 1. This annex presents the summary results of the cost-benefit analysis of the second phase of the APL. A cost-benefit analysis was carried out at appraisal, for the sectoral program as a whole, which included contributions from members of FONSALUD other bilateral donors including USAID and Japan. No estimates were made of net benefits accruing only from the IDA project. It was expected that the Program would produce the following direct benefits: (i) reductions in hospital admissions of children under-5 years of age; (ii) a reduction in hospital bed-days due to efficiency improvements; and (iii) improvements in the efficiency of the MOH owing to advances in the process of decentralization. It was further assessed that the Program would also produce indirect benefits in the form of potential years of life that would be saved as a result of nutrition interventions, together with the lives saved from reducing child and maternal mortality rates in the targeted areas. The present value of net benefits, over a 10-year planning horizon, was estimated at US$66.7 million, with an internal rate of return of 54 percent. 2. Due to the scope of the sector-wide program, it was difficult to attribute the net benefits of the program to anyone donor's investment. During implementation, some of the investments covered sector needs beyond the original scope set for the five-year plan in the 79 municipalities in the 12 targeted SILAIS. Additionally, other sector investments not channeled through FONSALUD also reached the target areas and therefore the impact of single donor- supported intervention was difficult to capture. The consequence was a disassociation between overall investment costs financed by MOH, through FONSALUD, IDA the IDB, and other donors and the benefits foreseen under the original scope of the SWAp scheme. For this reason, it was not possible to replicate the methodology used in the PAD to assess net benefits. 3. This cost-benefit analysis of the Second phase of the APL is based on several key assumptions. First, the population covered by the 79 municipalities supported by the Project is estimated to be about 2 million. The project extended coverage, by the end of 2009, and reached about 530,000 beneficiaries. Using a discount rate of ten percent, this assessment takes into the benefits detailed under the original Project Appraisal Document. However, unlike annex 9 of the original analysis, the benefits deriving from a reduction in patient-days were not taken into consideration, since no significant gains were observed during project implementation. Key benefits were estimated as detailed in the paragraphs below. 4. There was a significant reduction of maternal and infant deaths and chronic malnutrition. The Project's implementation of prevention and promotion activities during the pre- and postnatal stages of pregnancy aimed at reducing the number of maternal and infant deaths. For the purpose of this analysis, it has been assumed that saved lives would produce economic benefits that can be roughly quantified assuming the individual's future participation in the labor force. It was conservatively assumed that child mortality (under five years of age) would continue to be reduced from a level of 35 per thousand live births in 2009 to 28 by 2019. Similarly, maternal mortality would be further reduced from the present level of 110 per one hundred thousand live births to 95 by 2019. It was assumed that chronic malnutrition would be further reduced from the current level of 13 percent to 10 percent by 2019. It was further assumed that income per capita of an individual in the rural area would remain relatively the 28 same over the evaluation period. Benefits were estimated based on future income flows received by persons whose lives were saved as they become part of the economic active population and on the increased income-earning capacity of persons whose nutrition status is improved. 5. Other benefits accrue from cost gains and private savings resulting from improved practices. These were estimated based on: (i) the creation of regional mobile maintenance centers that reduce expenditures in new medical and nonmedical equipment and (ii) the increase achieved in prevention and promotion health care services imply a reduction in the demand for medicines and medical supplies and a change in the type of medicines purchased, and (iii) beneficiaries' savings since now they are purchasing less rounds of expensive antibiotics and more generic medicines. These savings were estimated at US$1.9 million yearly during the next five years, representing about 0.8 percent ofMOH annual budget. 6. The decentralization of management resulted in savings in the MOH budget. This process resulted in· improved monitoring of performance and supervision of health care services by the SILAIS. A recent study commissioned under the Project, estimated that savings from implementation of decentralized management and administration of health care services were equivalent to nearly 1 percent ofMoH's current budget. 7. Project costs and benefits associated with the APL II Project resulted in a marginal rate of return. Although the costs stream can be easily identified from investment costs and future operating costs, benefits are considerably harder to link with investments financed under IDA's project. Overall benefits were estimated based on the assumptions mentioned above. But only a portion of these benefits are associated with direct investments financed under the Project that, conservatively, can be estimated in the 30 to 40 percent range. When benefits are in the lower end of the range, the rate ofretum becomes slightly negative, while in the.upper end of the range the rate of return can be estimated at 6 percent. The following table summarizes total estimated benefits and costs. Estimated Costs and Benefits Streams US$ millions Year Project Costs Benefits 2006 4.1 0 2007 3.2 1.4 2008 2.4 2.1 2009 1.3 2.9 2010 0.6 3.5 2011 0.6 5.0 2012 0.6 5.4 2013 0.6 5.9 2014 0.6 7.0 2015 0.6 7.8 2016 0.6 8.0 8. Fiscal impact of the Project is marginal to the recurrent costs of the health sector. For fiscal year' 2011, the public health budget for the MoH is currently estimated at about 29 US$252 million of which about 59 percent are recurrent expenditures, It should be noted that the bulk of the investment budget is fmanced with support from multilateral agencies and bilateral donors. Overall, it represents about 3.4 percent of GDP and 11.5 percent of GON budget. Assuming that the MoH's budget and spending remain roughly the same relative to GDP over the few, years, the Government could easily absorb not only recurrent costs associated with the Project but with the larger program supported by the medium-term three-year program. 9. Overall, the Consolidated Public Sector deficit is estimated to have increased from 1.5 percent of GDP in 2008 to 3.9 percent in 2009 (well below the program target of 4.6 percent of GDP). The authorities are currently planning to start reducing the fiscal deficit. To this end, they have recently adopted the following measures: (i) a revenue-enhancing tax reform; (ii) an austere budget (that keeps nominal spending unchanged relative to budgeted 2009 levels); and (iii) a one percentage-point increase in the pension contribution rate. These measures are not expected to affect MOH's recurrent budget, although some reduction.in the investment level can be expected, because some donors have announced a decrease in their contributions for the next few years. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes J'!2II:l~]( Team members i~~~r()gram Assistant LCSHH Assistant 31 ,.........................................................,.. ..................•......................................................... ·····························T'·······················.........................•.............._ ..•...................••..•..•......... .'!~.£.:. ~y~~ ...... ~._...._.... _ ..... _ J!:.~~~£~.!>.!~~!~l!.~~t~~~!._ .. ____ ._.. _ .J.............._L.C_S_.... . .D~.............~., .... ... . . .... H 32 Annex 5. Beneficiary Survey Results 1. Two beneficiary surveys were carried out in ZOO9 and 2010 aiming .at obtaining informatIoo about the perception that beneficiaries have about the quality of services provided in health centers. Beneficiaries were women seeking service attention for themselves or their chilw-en less than 5 years. The specific objective was to evaluate access to health services and users satisfaction. A sample of 549 beneficiaries was selected for interviews that took place as they were exiting a health service facility. Nearly 80 percent of beneficiaries interviewed were women living in rural areas. Main fmdings are summarized below. 2. Over 50 percent of beneficiaries walked to health centers and about 80% said that they had obtained service. However, nearly all participants highlighted that waiting times were too long, which may be due to an increased demand for services combined with lack of availability of doctors and nurses in certah;l areas. 3. About 77 percent of beneficiaries had a positivepereeption that the time used by staffto provide services was adequate. This assessment was fairly constant in both surveys. Health advice provided, laboratory tests carried out in health facilities and medicines delivered to patients were other quality factors that also. had' a favorable perception by users. Over 90% of persons who were prescribed medicines reported that these were supplied to them on a timely., manner and 75 percent of these patients received proper instructions from pharmacists.... Cleanliness of health facilities was also mentioned as an aspect that needed considerable improvement. 4. In summary, cleanliness of facilities and waiting times were singled out as the most importaqt issues that needed improvement. Over 50' percent of beneficiaries interviewed concurred on this matter. Availability of me ArGhIVQ1$Qg 36 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable. 37 Annex 9. List of Supporting Documen~ World Bank Documents • Health Services Extension and Modernization Project. Project Appraisal Document. March 2005. • Amilisis del Gasto Publico en Salud en Nicaragua. 2007 • Country Partnership Strategy. October 2007 • Las Casas Maternas en Nicaragua. Cuadernos de Genero Series. Joint Publication by the World Bank: and the Inter-American Development Bank:. Washington DC. 2008 • Nicaragua - Improving Community and Family Health Care Services Project. Project Appraisal Document. July 2010 • Joint IDA-IMF Staff Advisory Note on the Poverty Reduction Strategy Paper. April 14, 2010 Borrower Documents • Informe Tecnico de Monitoreo. Gesaworld. Managua. Febrero 2009 • Informe de Gesti6n Institucional Enero - Agosto 2008. MINSA. 2009 • Encuesta de Satisfacci6n de U suarios. Gesaworld. Managua. Marzo 2009 • Ministerio de Salud (MINSA) - Evaluaci6n del Plan Indfgena - Proyecto 4050 BM. Julio 2009 • Informe Tecnico de Monitoreo. Gesaworld. Managua. Febrero 2010 • Encuesta de Satisfacci6n de Usuarios. Gesaworld. Managua. Marzo 2010 • Ministerio de Salud. Informe del Desempefio Institucional, 2009. April 13, 2010. • Borrador.. Nicaragua 2010: Objetivo de Desarrollo del Mileno. August 20,2010. • Informe de Cierre del Prestatario - Proyecto 4050. MINSA. Octubre 2010. Other Documents • ASDI- Evaluaci6n de los Procesos de Apropiaci6n, Alineamiento y Armonizaci6n de la Cooperaci6n Internacional en el Ministerio de Salud. Mayo 2008 • ASDI - 30 afios de Cooperaci6n Sueca con el Sector Salud de Nicaragua. Managua. Diciembre 2009 / 38 Annex 10. APL I Triggers 1. For the implementation and execution of the funds of APL 2, APLI achieved specific triggers indicated below. The design of APL II and the SWAp in general took into account the successes from the APL 1. 3. Major changes introduced in resource allocation, with activity and performance-based management . with SlLAfS and . directors. 4. Separation of core business in lNSS (including accounts and management), and number of policy holders increased 20 to Maternal and Child care Model and nutrition interventions 6. National to reduce maternal and 7. in five 8. At least 30 rehabilitation .......,.......t., 9. Information in Mon in 8 SILAlS. 10. Collective affiliation substitutes individual affiliation in INSS and new financial management system in 39 Annex 11. MAP 87'W 86'W 85'W 84'W 83'W 15°N -I)-"C;oyos Miski/os 14°N 13°N 13°N Caribbean Sea PACIFIC OCEAN ............. This mop was prodv ill r= DEPARTMENT BOUNDARIES ) TO Son Jos6 COS TA I 0 I 10 I 20 I 30 I 40 I 50 Miles O w w ~ _ INTERNATIONAL BOUNDARIES 86'W ~ if) ° 85'Wt'~ 83'W t; 0- o ~ ;;0