Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00003474 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-80760 ON A LOAN IN THE AMOUNT OF US$80 MILLION TO THE REPUBLIC OF EL SALVADOR FOR AN STRENGTHENING PUBLIC HEALTH CARE SYSTEM ( P117157 ) November 29, 2018 Health, Nutrition and Population Global Practice Latin America and Caribbean Region CURRENCY EQUIVALENTS Currency Unit = US Dollar (US$) FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS CPHC Comprehensive Primary Health Care CPS Country Partnership Strategy ECOS Family Health Community Teams (Equipos Comunitarios de Salud Familiar) EDs Environmental Diagnostics GDP Gross Domestic Product GOS Government of El Salvador FM Financial Management IRI Intermediate Outcome Result Indicator IMR Infant Mortality Rate MARN Ministry of the Environment and Natural Resources (Ministerio de Medio Ambiente y Recursos Naturales) MDGs Millennium Development Goals MICS Middle Income Countries MINSAL Ministry of Health (Ministerio de Salud) MMR Maternal Mortality Rate MTR Mid-term Review NCDs Non- Communicable Diseases NHI National Health Institute NRC National Radiotherapy Center NRL National Reference Laboratory OM Operational Manual OOP Out of Pocket Spending PAD Project Appraisal Document PCU Project Coordinating Unit PDOs Project Development Objectives RF Results Framework RIISS Integral and Integrated Health Care Service Network (Redes Integradas y Integrales de Servicios de Salud) SEM National Medical Emergencies System (Sistema de Emergencias Médicas) SUIS Single Unified Health Information System UACI Institutional Procurement and Contracting Unit (Unidad de Adquisiciones y Contrataciones Institucional) UCSF Family Health Community Units (Unidades Comunitarias de Salud Familiar) UFI Institutional Financing Unit UHC Universal Health Coverage Regional Vice President: Jorge Familiar Country Director: Yaye Seynabou Sakho Senior Global Practice Director: Timothy Grant Evans Practice Manager: Daniel Dulitzky Task Team Leader(s): Amparo Gordillo-Tobar ICR Main Contributor: Claudia Macias Angeles TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 10 II. OUTCOME .................................................................................................................... 11 A. RELEVANCE OF PDOs ............................................................................................................ 11 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 12 C. EFFICIENCY ........................................................................................................................... 16 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 18 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 19 A. A. KEY FACTORS DURING PREPARATION ............................................................................ 19 B. B. KEY FACTORS DURING IMPLEMENTATION ...................................................................... 20 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 22 A. A. QUALITY OF MONITORING AND EVALUATION (M&E) ..................................................... 22 B. B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE....... 24 C. C. BANK PERFORMANCE .................................................................................................... 25 D. D. RISK TO DEVELOPMENT OUTCOME ................................................................................ 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 27 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 35 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 37 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 38 ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS .. 46 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 57 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P117157 STRENGTHENING PUBLIC HEALTH CARE SYSTEM Country Financing Instrument El Salvador Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Republic of El Salvador Ministry of Health Project Development Objective (PDO) Original PDO The objectives of the Project are to: (i) expand the coverage, quality, and equity in utilization of priority health services provided under the RIISS; and (ii) strengthen MINSAL stewardship capacity to manage essential public health functions. Page 1 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 80,000,000 79,853,295 79,853,295 IBRD-80760 Total 80,000,000 79,853,295 79,853,295 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 80,000,000 79,853,295 79,853,295 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 21-Jul-2011 11-Dec-2012 19-Oct-2015 30-Jun-2016 30-May-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 31-Dec-2012 0 Reallocation between Disbursement Categories 20-Feb-2015 4.88 Change in Results Framework Reallocation between Disbursement Categories 07-Mar-2016 19.12 Change in Results Framework Change in Loan Closing Date(s) 06-Apr-2017 35.11 Change in Components and Cost Reallocation between Disbursement Categories Change in Disbursements Arrangements 08-Aug-2017 46.61 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial Page 2 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 03-Dec-2011 Satisfactory Satisfactory 0 02 06-Jun-2012 Satisfactory Satisfactory 0 03 07-Jan-2013 Satisfactory Satisfactory 0 04 03-Sep-2013 Satisfactory Moderately Satisfactory 0 05 03-May-2014 Moderately Satisfactory Moderately Satisfactory 2.64 06 01-Dec-2014 Moderately Satisfactory Moderately Satisfactory 4.07 07 05-Jun-2015 Moderately Satisfactory Moderately Unsatisfactory 5.88 08 22-Dec-2015 Satisfactory Moderately Satisfactory 13.76 09 21-Jun-2016 Satisfactory Satisfactory 22.85 10 21-Dec-2016 Satisfactory Moderately Unsatisfactory 31.18 11 15-Feb-2017 Satisfactory Moderately Satisfactory 32.18 12 16-Aug-2017 Satisfactory Satisfactory 46.61 13 01-Mar-2018 Satisfactory Satisfactory 68.11 14 31-May-2018 Moderately Satisfactory Satisfactory 78.04 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Health 100 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Human Development and Gender 100 Disease Control 10 Health Systems and Policies 88 Health System Strengthening 60 Reproductive and Maternal Health 18 Child Health 10 Nutrition and Food Security 2 Nutrition 1 Food Security 1 ADM STAFF Role At Approval At ICR Regional Vice President: Pamela Cox Jorge Familiar Calderon Country Director: Carlos Felipe Jaramillo Yaye Seynabou Sakho Senior Global Practice Director: Keith E. Hansen Timothy Grant Evans Practice Manager: Joana Godinho Daniel Dulitzky Task Team Leader(s): Rafael A. Cortez Amparo Elena Gordillo-Tobar ICR Contributing Author: Claudia Macias Page 4 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Following more than a decade of civil war (1980-1992), El Salvador was able to make considerable social and economic progress, undertaking significant social sector reforms that contributed to the improvement of social indicators. Starting in the 1990s, the country’s health outcomes improved, as reflected in an increase in life expectancy, decrease in child and infant mortality and improvement in maternal and reproductive health outcomes. El Salvador‘s average life expectancy at birth had increased from 67.1 years in 1992 to 71.7 years in 2009.1 Under-five child mortality was reduced from 92 deaths per 1,000 live births in 1990 to 18 deaths per 1,000 live births in 2008, nearly reaching the Millennium Development Goal (MDG) of 17 deaths per 1,000 live births2. Similarly, infant mortality was reduced from 48 deaths per 1,000 infants (under the age of one) in 1990 to 16 deaths in 2008.3 Despite a decline in the maternal mortality ratio, from 157 deaths per 100,000 live births per year in 1990, it still remained high at 64.5 deaths per 100,000 live births per year in 2009.4 2. Despite gradual progress in the country’s health demographics and toward the Millennium Development Goals (MDGs), El Salvador faced important challenges regarding inequalities in health care access and quality. Challenges in health care access had a disproportionate impact on the poor and those living in rural areas. Furthermore, high out-of pocket (OOP) spending was an obstacle to health care demand and a reflection of insufficient public health care spending. OOP expenditures on health represented 37 percent of total health expenditures (or 2.4 percent of GDP in 2007). Similar to global trends, rising non-communicable diseases in El Salvador were placing additional pressure on an already tight fiscal envelope. 3. In addition to inequalities in health care access and quality, the country’s health system faced the challenge of enhancing its response capacity to effectively address the increase in morbidity, mortality, and disability from communicable diseases and chronic noncommunicable diseases. The incidence of arterial hypertension for adults 30-59 years is high, affecting 1,746 persons per 100,000 people. The mortality rate from cirrhosis and other chronic liver diseases had increased from 11 deaths per 100,000 in 2000 to 14 deaths per 100,000 in 2008; while estimated deaths from Ischemic heart disease had also risen from 44 per 100,000 in 2003 to 55.5 per 100,000 in 2008.5 From 1990 to 2015, chronic noncommunicable diseases steadily increased and now represent a major burden for the health system. 1 PAHO Basic Health Indicators, 2009. 2 WHO Global Health Observatory and child and maternal health data from United Nations Development Program ―Without Excuses... Let us Achieve the Objectives of the Millennium in 2015: A Plan for Action‖ 2008. 3 WHO Global Health Observatory: Earliest data available (1990) and latest data available (2008). 4 PAHO Basic Health Indicators, 2009. 5 PAHO Basic Health Indicators, 2009. Page 5 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 4. In 2009, the Government of El Salvador (GoS) launched a health system reform. The reform established the general guidelines for advancing firmly toward a unified National Health System, with universal coverage and access, based on the strategy of comprehensive primary health care within an Integral and Integrated Public Health Care Service Network (RIISS) and the development of human resources for a universal and equitable health system. The National Health Strategy (2009-2014) aimed to reinforce the Ministry of Health (Ministerio de Salud – MINSAL) plan to expand health service coverage and reduce inequalities in access, improve health service quality, and strengthen information system management to enhance the monitoring and oversight capacity of the ministry. Health care funding constraints encouraged the establishment of a comprehensive multi agency plan for the achievement of the health care reform objectives.6 5. At the time of appraisal, the Governement of El Salvador was focused on prioritizing a comprehensive health sector reform. The reform was key to strengthening the sector, as well as contributing to poverty reduction and boosting shared prosperity. The reform sought to strengthen the country’s health system, that was suffering from fragmentation, poor quality, obsolete equipment, poor infrastructure and human resource shortages. In addition to strengthening the quality of health service delivery, the reform also aimed to expand health care coverage throughout the country in an effort to narrow health care inequities. The Rationale for Bank Involvement 6. The Project was consistent with the priorities outlined in the World Bank Group’s Country Partnership Strategy (CPS) for El Salvador FY 2010-2012 (Report No. 50642-SV) discussed by the Executive Directors on November 24, 2009, with the Government’s National Development Plan 2010- 20147 and the MINSAL’s National Health Strategy (2009-2014) The Project was closely aligned with the key strategic development challenge identified in the CPS: strengthening the delivery of social services, which included improving the quality and coverage of primary health care services. Furthermore, the Project Development Objectives (PDOs), as well as the overarching goals of the reform, such as universal coverage and the improvement in health outcomes, were in line with global trends recognized in the Millennium Development Goals efforts to reduce inequalities through the provision of health care for all. Theory of Change (Results Chain) 7. The Project was fully aligned with the overall National Health Strategy’s goal of improving access to and utilization of MINSAL quality services, especially for the poor in target areas, as well as on strengthening the overall health system. Project activities were focused on building a solid system sufficiently equipped with human resources, infrastructure, technology, and medicines. Activities were also directed on strengthening MINSAL’s capacity for managing resources, implementing models for the provision of integral services, and decision making. Chart 1 below demonstrates the relationship between activities, outputs, the project development objectives and outcomes, and the long-term outcomes. 6 The GoS expected to finance the reform, among other measures, through progressively increasing the percentage of public spending on health to 5% of GDP during the 2009-2014 administration and the integration of external cooperation funding into a financial plan that covers the activities and resources of the National Health Plan. 7 Plan Quinquenal de Desarrollo 2010-2014, April 14, 2010. Page 6 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Chart 1. Theory of Change (Results Chain) LONG TERM ACTIVITIES OUTPUTS PDO / OUTCOMES OUTCOMES Component 1- Expansion of Priority Health -Strengthened services and priority health programs Services and Programs -Improved user satisfaction PDO1 Expand the -Purchase of equipment for health centers and second and third level -Improved MINSAL health service delivery network coverage, quality hospitals -Significant progress in comprehensive care for cancer and equity un patients utilization of priority -Rehabilitation and maintenance of infrastructure -Strengthened National medical emergency system Equitable -Increased availability of child and reproductive health health services access to -Construction and equipping of the Radiotherapy Unit and nutrition services provided under the and National Biomedical Laboratory -Increased availability of supplies and equipment in quality RISS -Purchase of ambulances and emergency equipment referral centers -Percentage of children less health -Increase in the number of vaccinated infants and in - Training for medical emergency attention the percentage of women of reproductive health than 1 year old with the third services doe of pentavalent in the 92 receiving an ANC visit in target municipalities selected municipalities for the -Purchase of vaccines and medical supplies -Improved response to medical emergencies at the -Percentage of institutional Salvadora -Technical assistance to improve hospital management and help community level delivery by the public implement the accreditation of health centers and hospitals network in the 92 selected n municipalities population -Training and updating of health promoters - Improved coordination between the national hospital Improved Component 2: Institutional Strengthening PDO2 Strengthen -National Health Survey (MICS), National Survey of chronic kidney system and other entities health -Improved data quality and information flow regarding MINSAL stewardship disease, diabetes mellitus, hypertension and risk factors in the adult the health of the population outcomes population (EFRAES) capacity to manage -Patient information system strengthened essential public (including -Health systems challenges identified -Acquisition of a building to house the National Institute of Public Health -National drug policy strengthened health functions reduction and the National Reference Laboratory -Laboratories and warehouses network improved -Number of eligible in -Increase in access to cancer care hospitals (24) that comply - Setting up of the National Radiotherapy Center -Managerial staff competencies improved maternal with standard bio-hazard -Purchase of supplies, furniture and equipment, including information -Greater efficiency and reliability in the management waste and infant and analysis of health statistics for decision making equipment -Improved monitoring of adverse drug reactions in mortality) -Training of staff to improve managerial competencies, health facilities pharmacotherapy -The country has well defined lines of action and the protocols to follow in case of disasters due to public -Development of environmental diagnostics health emergencies -Development of the country’s Emergency, Disaster Preparedness and Response Plan Component 3: Project Management and Monitoring -Compliance with Project’s objectives and commitments -Project execution plan (PEP), Annual operating plan (POA) project stablished in the Project’s loan agreement financial forecasting tool -Relevant information on health coverage users ‘satisfaction -Development of user’s satisfaction surveys -Relevant Information on overall impact of the project on the -Project’s impact evaluation health status of the beneficiary population -Mid-Term and Impact Evaluations -Publication of project achievements Page 7 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 8. The investments made under Project components lead to increased access and utilization of MINSAL priority services. In the long-term, this was expected to contribute to better health outcomes and reduction of the maternal mortality rate (MMR) and the infant mortality rate (IMRs) in target municipalities. Investments under Component 2 helped strengthen MINSAL stewardship capacity to manage essential public health functions, implement the Project’s prioritized activities and follow up investment plans. Annex 1.B details outputs by components vis-à-vis achievement of objectives. Project Development Objectives (PDOs) 9. The PDOs of the Project were to: (i) expand the coverage, quality, and equity in utilization of priority health services provided under the Integral and Integrated Health Care Service Networks (RIISS); and, (ii) strengthen the Ministry of Health‘s (MINSAL) stewardship capacity to manage essential public health functions. The PDOs were not revised during project implementation. Key Expected Outcomes and Outcome Indicators 10. The key expected outcomes and PDO indicators used to assess the achievement of the Project’s development objectives are: Outcome 1: “Expand the coverage, quality and equity in utilization of priority health services�. • PDO indicator 1: Coverage of children less than 1 year old with the third dose of pentavalent in the 92 selected municipalities. • PDO indicator 2: Coverage of institutional delivery by the public network in the 92 selected municipalities. Outcome 2: “Strengthen MINSAL’s stewardship capacity to manage essential public health functions�. • PDO indicator 3: Number of eligible hospitals that comply with standard bio-hazard waste management system. 11. In addition, there were five intermediate outcome indicators to measure expansion of priority health services and programs, and the strengthening of MINSAL stewardship capacity and administrative management of the Project. The results framework (Annex 1) also includes the intermediate outcome indicators directly linked to the expected outcomes under individual subcomponents. a. Number of Family Health Community Teams (ECOS) and hospitals in priority municipalities equipped according to technical guidelines defined by MINSAL (Outcome 1) b. Number of Hospitals integrated into the National Medical Emergencies System (Outcome 1) c. Number of Health personnel receiving training (Outcome 2) d. Number of Facilities in priority Departments with an operating information system (Oucome 2) e. Number of user satisfaction surveys carried out (Outcome2) Page 8 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Components The Strengthening Public Health Care System Project included three reinforcing and interrelated components: 12. Component 1: Expansion of Priority Health Services and Programs. Under this Component, the Project sought to strengthen the quality and delivery capacity of priority health services under the Integral and Integrated Health Care Service Network8 through: (i) the gradual implementation of the Comprehensive Primary Health Care (CPHC) strategy, which included the maintenance and minor rehabilitation of health facilities; and, (ii) the improved delivery of selected priority public health programs, including those focusing on maternal health, reproductive and sexual health, teenage pregnancy, child health, nutrition, dengue control, and chronic kidney disease. The Component also supported the implementation of the National Medical Emergencies System (SEM). 13. Component 2: Institutional Strengthening. Under this component, the Project sought to strengthen MINSAL‘s institutional and management capacity to perform essential public health functions through: (i) the improvement of the Single Unified Health Information System (SUIS); (ii) the implementation of a national pharmaceutical policy aimed to enhance quality, efficiency and security in the access of the rational use of medicines as well as procurement transparency; and (iii) support of the National Health Institute and the Central Laboratory activities. The component included a contingency fund to support immediate emergency response for natural disasters9 officially declared by the Government within the first three years of the Project.10 The component also aimed to develop an emergency preparedness plan for Public Health Emergencies implemented through the financing of Eligible Public Health Expenditures. 14. Component 3: Project Management and Monitoring. Under this component, the Project aimed to support: (i) the monitoring of the Project activities through semi-annual, external technical reviews; (ii) annual health facility surveys to assess the achievement of health coverage and results targets; and (iii) an impact evaluation. The Component also financed the Project Coordinating Unit’s (PCU) management, monitoring and supervision of activities of the Project, including training and financial audits. 15. At approval, the total project cost was estimated at US$80 million financed by the Bank. Although the objective of Project components remained unchanged, their financing was revised as described in Section B. 8 Integral and Integrated Health Networks, include all levels of health system care: Community Health Community Teams (ECOS), Community Health Units and Hospitals. 9 Including earthquakes, tsunamis, volcanic eruptions, hurricanes, landslides, droughts, floods, dengue, A/H1N1 and other health outbreaks. 10 The contingency fund, however, was not triggered during the life of the Project since there was no declared emergency and therefore these funds were allocated to other Project activities, as allowed by Section IV.B.2. of the Project’s Loan Agreement. Page 9 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 16. The original PDO remained valid and unchanged throughout Project implementation. The Project’s results framework was first updated on February 20, 2015 through a level two restructuring and later on March 7, 2016. The first level two restructuring improved the PDO measurement, and reviewed Project targets according to evidence and feasibility. During this restructuring, the original four Project Development Objective Indicators were removed and replaced by three more specific indicators. The March 7, 2016 level two restructuring updated the end of Project targets for one PDO indicator and two intermediate indicators and redefined one intermediate indicator to improve the accuracy of indicator definitions and data (See Annex 1 for Project’s actual Results Framework and Key outputs). Revised Components 17. The objective of the Project’s components remained unchanged throughout implementation, budget allocation for the components was adjusted and some component activities were added. A significant portion of the loan was reprogrammed toward MINSAL revised priorities with a focus on NCDs, in particular cancer. Although the challenge of NCDs had been identified as an important element of the health reform and included in project design (2011), these became more relevant during the implementation of the first two years of the 2015-2019 Health Strategic Plan.11 The first restructuring (December 6, 2012) and second restructuring (February 20, 2015) reallocated US$9 million to Component 1. These resources were directed toward: (i) the purchase of medical equipment; (ii) the design of the National Strategy for Chronic Diseases and the National Plan for prevention, diagnosis and treatment of cancer; (iii) the creation of the National Radiotherapy Center (NRC); (iv) the purchase of a cobalt pump to strengthen the Cancer Institute and; (v) the preparation of an emergency response action plan to allow for immediate response in the case of a public health alert or emergency. The fourth restructuring (April 6, 2017) added the purchase of a building for the National Reference Laboratory (NRL) under a new subcomponent 212 and reallocated funds between categories to finance the building purchase. (See Annex 3 for estimated, adjusted and actual costs by component). Other Changes 18. The Project went through several restructuring process with a view of adjusting to the changing conditions and achieving the desired results. The actual implementation of the Project lasted five years and six months, a few months longer than originally planned, if one takes into account the 18 month delay in Project effectiveness. The Project was restructured five times. The Project reallocated loan proceeds on three occasions (December 2012, February 2015, and April 2017 restructurings) and extended the closing date twice (March 2016 and July 2017 restructurings). The closing date extensions responded to the need to: (i) compensate for the 18 month delay in Project effectiveness; and, (ii) grant additional time 11 The Project was implemented under the same political party and two different Health Ministers. 12 A new Project subcomponent 2.1(d) was be added under Part B.1 of Schedule 1 to the Loan Agreement. Page 10 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) to ensure for the timely completion of additional activities that were carried out with Project savings. The restructurings were the following: a. December 6, 2012: reallocated funds across loan categories given that some planned activities related to the SUIS were financed with Government funds (IDB project) while additional infrastructure and equipment needs for strengthening the health care networks were identified, including the purchase of a cobalt pump for the Cancer Institute. b. February 2015: (i) revised and updated the Results Framework, ensuring results were possible to measure using MINSAL’s administrative systems; and (ii) reallocated funds across disbursement categories to reinforce Component 1 with US$8 million transferred from Component 2 for the start-up of the NRC. The Results Framework, including indicators, baseline, and targets, was revised to increase clarity, improve the accuracy of indicator definitions and data, and review the Project targets according to evidence and feasibility. c. March 7, 2016, after the Mid Term Review: revised the Results Framework to: (i) update end of Project targets for one PDO indicator and two intermediate indicators; (ii) redefine one intermediate indicator to improve the accuracy of indicator definitions and data; and (iii) extend the Closing Date by 18 months (June 2016 to December 2017) to compensate for the months lost due to the delay in effectiveness. d. April 6, 2017: (i) changed Project Component costs to include the purchase of a building for the NRL under a new subcomponent; (ii) included a new definition of the National Reference Laboratory in the Loan Agreement; and, (iii) created a new disbursement category and reallocated funds across categories to reinforce Component 2 and finance the purchase of the new building for the National Reference Laboratory (NRL) and the Nationa Health Institute (NHI) (US$9 million). e. August 2017: extended the closing date by five months (from December 2017 to May 2018) to allow enough time for the delivery of goods and completion of the user satisfaction survey. Rationale for Changes and Their Implication on the Original Theory of Change 19. As indicated above, the changes mentioned did not affect the PDOs or the reforms to be implemented under the components. Thus, the underlying theory of change remained valid. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 20. The Project’s PDO was and remains highly relevant to the health needs of El Salvador. The Project supported the GoS health strategy (2009-2014) and the Government’s National Development Plan 2010-2014. These emphasized the need to improve the overall health and well-being of the Salvadoran population by extending a set of priority primary health care services both to the poor and in remote areas, and strengthening the institutional capacity of the public health care system. The objectives of the Page 11 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Project fit firmly within the National Health Strategy (2015-2019), the Government’s five year Development Plan 2014-2019 and the Social Development Plan for 2018. All aim to consolidate and strengthen the health sector and generate the basic conditions to guarantee that the Salvadoran population, in particular the most vulnerable, has equal access to quality health care services. The Project effectively supported policy goals articulated in the Government’s plans, including the reduction of maternal and child mortality and the progress toward the MDGs, expanding priority health services and programs; implementing strategies and interventions for the prevention and control of cancer; promoting investments at the central regional and local level and improving the structure, organization and operation of Integrated and Integrated Health Networks (RIIS). 21. Project objectives were fully consistent with the current WBG Country Partnership Framework (CPF) for the period FY2016-FY2019 (Report No. 95185, dated May 26, 2015). The World Bank Group’s Country Partnership Strategy for El Salvador FY 2010-2012 (Report No. 50642-SV dated November 24, 2009) identified the need to strengthen the delivery of social services and to improve the quality and coverage of primary health care services as priorities. The PDOs continued to be relevant throughout Project implementation, as evidenced by the continued prioritization of improving social services in the WBG Progress Report to the CPS FY2010-2012 (Report No. 61113-SV, dated August 3, 2011) and of building foundations for promoting inclusive growth, improving the efficiency of public spending, and taking an integrated approach to provide health services, among other priorities highlighted in the current WBG Country Partnership Framework. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome Rating: Substantial 22. Project efficacy is rated as Substantial The overall PDO was evaluated on the achievement of the two PDO parts: (i) expand the coverage, quality, and equity of priority health services under the RIISS; and (ii) strengthen MINSAL’s stewardship capacity to manage essential public health functions. 23. PDO 1 achievement: Substantial. One of the PDO indicators surpassed its end of Project targets and another was partially achieved. The cumulative accomplishment of this PDO was the foundation for preventing maternal mortality through improved coverage and quality of maternal and child care provision and better control of NCDs. The Project improved access to pre-natal checks, surpassed targets related to institutional births and made progress in child immunization with the pentavalent vaccine. In addition, the Project engaged in: (i) the design, preparation, and implementation of a well-rounded Non- Communicable Disease (NCD) strategy to respond to the increasing rate of diabetes, cancer, kidney disease, and other non-communicable diseases; (ii) strengthening the country’s ability to respond to medical emergencies and epidemiological alerts, with the support to the National Emergency Network and to the National Reference Laboratory; and, (iii) contributed to public policy discussions regarding the country’s long term public health agenda, including NCDs and Communicable Diseases. Page 12 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 24. PDO 1, indicator 1: Coverage of children less than 1 year old with the third dose of pentavalent in the 92 selected municipalities. Rating-Substantial. The indicator was achieved in three out of the five years measured (2013: 95.0/92.5 percent; 2014: 94.4/92.5 percent; 2015: 92.4/92.5 percent; 2016: 95.0/92.6 percent; 2017: 80.6/92.8 percent). The challenges regarding PDO indicator 1 are largely explained by: (i) an increase in marginal urban and rural violence which, affected health personnel’s ability to reach target populations; 13 and, (ii) the fact that population projections made based on the 2007 CENSUS for the establishment of targets do not reflect real population growth. It is important to note that there was a significant improvement in the indicator just after the closing of the Project. The indicator reached 86.12 percent by end of June 2018 which represents an improvement of 5.52 percentage points with regard the previous period (80.6 in December 2017), though it did not reach the end of Project target of 92.8 percent (see Table 1). Table 1 Achievement of PDO indicator 1 year 1 Year 2 Year 3 Year 4 Year 5 (2013) (2014) (2015) (2016) (2017) Baseline/target 92.5 92.5 92.5 92.6 92.8 Achievement 95.0 94.4 92.4 95.0 80.6 Source: World Bank internal documents 25. PDO 1, indicator 2: Coverage of institutional delivery by the public network in the 92 selected municipalities. Rating-High. The indicator was surpassed in each Project implementation year, exceeding the Project target of 95 percent, at 99.8 percent by end of 2017. The increase in institutional deliveries is associated with a decrease in maternal mortality from 65.4 deaths per 100,000 live births in 2006 to 31 in 2017, reaching the target for MDG 5. The expansion in institutional deliveries and improvement in neonatal equipment also had an impact on infant mortality. Infant mortality fell from 26.9 in 2000 to 12.5 in 2017, while adolescent pregnancy fell from 119.5 in 1980-1985 to 76.7 between 2005-2010 and 66.8 in 2010-2015. 26. Two Intermediate Outcomes Results Indicators (IRIs) supported the achievement of PDO 1: • Intermediate Indicator 1 Community Health Facilities (UCSF) and hospitals in priority municipalities equipped according to technical guidelines defined by MINSAL achieved its target of 30 hospitals and surpassed its target of 169 ECOS, equipping 269 ECOS. • Intermediate indicator 2 Hospitals integrated into the National Medical Emergencies System achieved its target of 8 hospitals. 13 Ninety out of the 92 municipalities targeted in the Project are considered highly dangerous by the National Civil Police according to a list produced by the National Civil Police “Municipios con 0 homicidios del 1 de Enero al 31 de Agosto del 2017�. Page 13 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 27. The Project improved the access and quality of priority health services and programs among the poor and vulnerable population. The investment made in equipment, supplies and furniture in the 269 ECOS, has improved the quality of attention at the first level of care. The installed capacity of the country’s 30 hospitals for delivering specialized services improved substantially due to the purchase of high tech medical equipment, medical clinical furniture, and non-medical equipment. MINSAL RISS’s response capacity for chronic disease patients was improved through the provision of kidney dialysis machines, supply kits and medicines. The establishment of the national biomedical laboratory and implementation of a system based equipment maintenance system with Project funds will ensure the long-term maintenance of medical equipment and meet the demand for maintenance services throughout the RISS and the continuity of care to the beneficiary population. The physical condition of the 51 health facilities and various central MINSAL facilities improved due to the financing of their rehabilitation and maintenance work. 28. Project investments in a National Medical Emergency System strengthened MINSAL’s capacity to provide emergency services. MINSAL’s ability to provide emergency services for the population was boosted through the purchase of emergency service equipment, in particular 44 ambulances and first response equipment. The Project supported the improvement of coordination between national hospitals and other entities, and implementation of a national system to deal with emergencies, improved and expanded the equipment for reanimation in ambulances, and trained and strengthened medical personnel in the management of medical emergencies. Strengthened emergency services coordination results in quicker and better attention and is associated with an improvement in the probability of saving lives. 29. The Project effectively reinforced priority health programs and interventions. Fourteen thousand three hundred and eighty five community health promoters and committees received training in comprehensive community health care, facilitating the establishment of health committees for health promotion and prevention. The vaccination program was strengthened with the purchase of anti-rabies, MMR, polio, yellow fever, TD-Adult, 13-valent pediatric pneumococcus and adult TDPA vaccines. To control dengue, the Project purchased fumigation equipment, vehicles and fuel benefiting 1,200 residents in the most affected areas. The reproductive sexual health program was strengthened with the purchase of medical and instrumental equipment, equipment for human milk banks and human resources training. 30. Project funds also supported activities prioritized by MINSAL to face the challenge of chronic diseases. Among these are the design, preparation, and implementation of a well-rounded Non- Communicable Disease strategy to respond to the increasing rate of diabetes, cancer, kidney disease, and other non-communicable diseases; the creation of the NRC; the purchase of a Cobalt bomb to strengthen the Cancer Institute. Investments made with Project funds are providing services that were previously not available in El Salvador. The newly established NRC (2017) will provide first class treatment for cancer, free of charge, for the population with specialized equipment. The functioning of the NRC will likely have a measurable impact on the reduction of morbidity and mortality associated with cancer, as well as an impact on families OOP spending. The center is projected to attend approximately 2,200 patients annually. Page 14 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 31. PDO 2 achievement: High. The Project was effective in strengthening MINSAL’s institutional capacity to manage essential public health functions by supporting: (i) the implementation of the Unified Health Information System (SUIS); (ii) the preparation of the National Policy for Pharmaceuticals; (iii) the strengthening of the National Health Institute and National Reference Laboratory; and (iv) the design of the National Certification of Quality of Care process. It contributed to strengthening the MINSAL through the provision of technological and telecommunications equipment to enable better communication among levels of reporting, data collection, data analysis and the monitoring of the national statistical reporting system. Overall, the Project contributed to a more efficient managerial model in the areas of resource management and strategic decision making for health policies. 32. PDO 2, indicator 1: Number of eligible hospitals that comply with standard bio-hazard waste management system. Rating-High. This PDO indicator exceeded the original target of 14 and the adjusted target of 26, reaching 30 hospitals. Thirty hospitals now comply with a standard bio-hazard waste management system. The Project supported the establishment of environmental regulations that now provide biohazard diagnostics and norms that all the institutions must follow. The country will soon count with an information system that generates data on biohazard waste generation, which will facilitate the identification of actions and measures, and the budget required to ensure adequate management of hazardous hospital waste. Three Intermediate Outcomes Results Indicators contributed to the achievement of PDO 2: • Intermediate indicator 3- Health personnel receiving training. The Project financed the training of health personnel, which increased from a baseline of zero in December 2012 to 4,660 in December 2017, far surpassing the end target of 1,200. The Project also financed the training of 35 epidemiologist for the national laboratory and 3,000 health promoters. • Intermediate indicator 4- Number of facilities in priority Departments with operating information system achieved its end of Project target of 30 hospitals and 105 UCSF; Improvements in the generation, monitoring and evaluation of health information support more informed decisions regarding the population’s health and strengthen MINSAL’s stewardship capacity. • Intermediate indicator 5- Number of user satisfaction surveys carried out achieved its end of Project target of implementing two satisfaction surveys. The Project financed implementation of two of the National Health Surveys: MICS (2014) and the NCD survey (2014-2015). The surveys generated information on the health demographics of the population needed to provide better data on where and how public health funding should be earmarked. The results from the NCD survey “Primera Encuesta Nacional de Enfermedades Crónicas en poblacion adulta en El Salvador�, ENECA- ELS 2014-2015� supported the development of a multisectoral strategic program aimed at preventing and managing NCD’s.14 14 “Plan estratégico nacional multisectorial para el abordaje integral de las enfermedades no transmisibles�. Page 15 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Justification of Overall Efficacy Rating 33. Overall, the efficacy of the Project is rated as Substantial. The Project successfully achieved two of the three PDO indicators and made substantive progress in the third one. The Project had an important impact in expanding and improving the resources needed to provide health care quality services. It addressed supply side constraints that would have restricted service provision to targeted beneficiaries had they not been in place. Progress on the institutional strengthening side was equally important and directly attributable to the project’s interventions. C. EFFICIENCY 34. The economic analysis prepared at Appraisal estimated that the Project would yield significant benefits. It would avoid between 2,953 and 3,938 deaths, which translated into monetary benefits between US$129 million and US$172 million. The Project's benefit-cost ratio was greater than one and its net present value was positive. Assessment of Efficiency and Rating 35. The efficiency of the project is rated as High. The results of the Project, over the medium term, will far out weight the resources invested. Immediate results stem from improved coverage, quality and equity of health services through capital investments and medical personnel training. Longer term results stem from reductions in morbidity and mortality, which have macroeconomic implications. 36. The Project demonstrated positive efficiency gains from reductions in mortality and inequality indicators and improvements in health indicators. Project investments contributed to sustaining and improving social gains in life expectancy, decreasing child and infant mortality and improving maternal and reproductive health outcomes, while improving equity through the expansion and utilization of health services for those in the poorest municipalities. Nevertheless, attribution is complicated by the ongoing implementation of a comprehensive health care reform in El Salvador initiated in 2009. 37. The Project experienced an efficiency impact from the delays in the implementation of Project activities. The Project experienced a two-year delay largely due to procurement related obstacles, which impacted disbursements and Project activities. Delays in project implementation and in particular procurement suggest Project inefficiencies given the opportunity cost of capital and the extension on the service fee paid. 38. The ICR sought to mimic the benefit cost analysis used in the 2011 Project Appraisal Document (PAD) economic analysis. The ICR sought to quantify the impact of the expansion of priority health services and programs by obtaining a value for the avoided deaths due to the Project’s activities, as well as quantifying the benefits of other Project activities. Avoided deaths were calculated using the reduction in infant and maternal mortality rates, while a life expectancy of 73 was used to measure the remaining years of life.15 If we assume that approximately 25 percent of child deaths are averted due to Project activities then we obtain 126 infant saved lives in 2016 and 660 between 2013-2017. If we use an average 15 The ICR uses a life expectancy of 73, while the PAD used one of 72. Page 16 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) salary of US$300 per month for the 53 working years per infant saved, we obtain an economic benefit of US$126 million. 39. The Component actions aim to improve health care for infants under one year. The World Health Organization (2017) has estimated that at least 30% - 40% of child deaths result from poor antenatal care during pregnancy and delivery, while the international literature suggests babies whose mothers had adequate antenatal care are 30% less likely to die in their first six months. If we assume that approximately 25% of child deaths are averted due to Project activities, then we obtain 126 infant saved lives in 2016 and 660 between 2013-2017. If we use an average salary of US$300 per month for the 53 working years per infant saved, we obtain an economic benefit of US$126 million. 40. The Project not only focused on education to decrease adolescent pregnancy but also supported maternal health, through prevention and promotion actions regarding antenatal and postnatal care. Though the maternal mortality rate experienced a significant fluctuation, it fell from 59 in 2010 to 27 deaths per 100,000 live births per year in 2017.16 This resulted in 10 maternal deaths in 2016 for the targeted population. International literature suggests that at least 20 percent of maternal deaths can be prevented through the implementation of community-based prenatal interventions. Therefore, it is assumed that approximately 2.5-3 maternal deaths could be averted per year and approximately 18 over the targeted period. If we use an average salary of US$300 per month (2015) and an average employment rate of 50 percent, the economic benefit amounts to US$1.8 million. 41. Evaluating the Project’s efficiency by using the cost benefit analysis of the PAD likely underestimates the benefits of the Project since it only captures the benefits that result from reductions in maternal and infant mortality. The model excludes all other deaths avoided in part due to new medical equipment, improved health service quality and the setup of the National Medical Emergency System. The benefits resulting from the new equipment will likely be measurable over the medium to longer term, in particular those associated with the new Cancer Center and fleet of ambulances. In addition, it does not consider benefits from reductions in morbidity and out-of-pocket expenditures. D. JUSTIFICATION OF OVERALL OUTCOME RATING 42. Overall outcome rating is Satisfactory with high relevance of PDO and Substantial efficacy and efficiency. Outcome dimensions Ratings Relevance High Efficacy Substantial Efficiency High Outcome Rating Satisfactory 16 WB indicators. Page 17 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) E. OTHER OUTCOMES AND IMPACTS Gender 43. The Project had a direct impact on gender through its focus on priority programs, in particular, maternal, reproductive and sexual health, and teenage pregnancy. The Project specially targeted vulnerable communities, achieving important results on the reduction of the maternal mortality rate and improving coverage of the services associated with the health MDGs. The project addressed issues related to pre and post-maternal and neonatal health. To complement these efforts, the Bank- Trust Fund (TF) Safeguarding Human Capital of Urban Poor Children in the context of recurring food crisis (P125760) supported MINSAL in the implementation of strategies for the prevention of adolescent pregnancy, with an even greater focus on the prevention of a second pregnancy.17 Among the actions implemented by the TF are the training of young promoters and the creation of friendly spaces within health facilities for easy access to preventive care. These strategies helped to protect the family nucleus. Institutional Strengthening 44. The Project had a significant impact on MINSAL’s stewardship role. The Project contributed overall to a more efficient managerial model in the areas of resource management and strategic decision making for health policies. The project strengthened MINSAL’s institutional capacities in the administrative, technical and operational areas. The Project contributed to the development of policies, updated regulations, human resource training at different levels, planning and management. MINSAL strengthened capabilities combined with an improved information system not only improved the quality of patient care but is also facilitating the making of evidence-based decisions. The expansion of the functionality of the SUIS increased the access capacity of users of the different SUIS modules especially in the implementation of the Patient Information System. The Project contributed to the creation of protocols and preparedness mechanisms to respond to sanitary emergencies. MINSAL's pharmacovigilance capacity was increased. The Directorate of Health Technologies (DIRTECS) capacity to analyze and make decision regarding the incorporation of new essential drugs, considering their efficacy and safety, has been strengthened. Mobilizing Private Sector Financing Not applicable. Poverty Reduction and Shared Prosperity 45. The Project’s activities and investments had a direct impact on the poor. First, El Salvador’s public health program, while not limited to serving the poor, is in practice the only health system available to lower-income populations. Though the project will benefit the general population, the Project particularly benefitted 2.1 million individuals who were among the poorest living in the 92 municipalities in both rural and urban areas. Out of pocket (OOP) spending fell from 50 percent of current health 17In 2016, MINSAL reported 21,477 pregnancies of girls, adolescents and young people between 10 and 19 years old (30% of all pregnancies) (Estrategia_Nacional Intersectorial de Prevención del Embarazo en Ninas y Adolescentes 2017-2027, MINSAL 2016). Page 18 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) expenditure in 2011 to 28 percent in 2015.18 High OOP spending, in particular OOP catastrophic spending, is an obstacle to health care demand, is usually the most inequitable type of financing, and generates problems of financial protection. Other Unintended Outcomes and Impacts 46. The complementary Safeguarding Human Capital of Urban Poor Children in the context of recurring food crisis in El Salvador (P125760) Grant provided a series of successful experiences which demonstrated specific results in nutrition in targeted communities and at the national level. The grant successfully identified, among others, innovative strategies to work with communities under critical conditions; and, provided micro-nutrients, fortified food, medical equipment, as well as office equipment, information systems and audio-visual equipment and training to USSF and to ECOS-F who work with families living under critical conditions. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. A. KEY FACTORS DURING PREPARATION Design 47. Assessment of Project design. The PDO, the balance of the activities among the project components and project implementation arrangements were formulated realistically and in line with the Government priorities and experiences. The Project was consistent with international good practice which included making strategic capital investments in the health system to complement recurrent state expenditures and ensure improvement in quality of service delivery. It was also in line with the emphasis placed by the National Health Strategy not only in treatment but also disease prevention and health promotion. The Project was fully articulated around the eight strategic axes of health reform.19 It had three well-defined and structured components with clear operational logic and presented a simple results chain connecting the project’s components to its objectives and contributing to the higher-level outcome of improving health outcomes for the Salvadoran population. Yet, the original results framework had some shortcomings that were addressed by the February 2015 Level 2 restructuring (see Section B). 48. Design alternatives were considered in Project design. The Project considered the introduction of a results-based payment mechanism using capitation transfers from the central level to the Regional Health Directorates to finance the gap required to expand health care services. This approach was not included in the Project design, among other reasons, because the Project sought to support the health system through the improvement of the coverage and quality of services through capital investments and the acquisition and replacement of critical equipment rather than the financing of recurrent expenditures. 18WHO Global Health Expenditure Database. 19 (1) Construction of Integral and Integrated Health Services Networks; (2) Development of human resources in health as the cornerstone of the system; (3) Response to the demand for medications and vaccines; (4) Establishment of the National Health Forum; (5) Construction of a National Emergency Medical System; (6) Development of a Single System of Strategic Information; (7) Inter-sectoral work with public health providers; (8) Establishment of the National Institute of Health and strengthening of the Laboratory Network. The project in its design included activities related to environmental health that would be included as one of the two additional axes (the other was health and violence) due to the importance of the issues in the country. Page 19 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 49. The background analysis, based on relevant assessment and previous similar projects’ experience was satisfactory. The project team accessed extensive and relevant background information about the health conditions, Government initiatives, and constraints in the national context and in the project intervention areas. The Project was built on the experience of using mobile brigades from public health facilities and contracting out of NGO’s for expanding primary health care services in the poorest municipalities in the decade prior to Project preparation. Technical design of the Project, in particular Component 2, was informed by the Bank-financed Essential Public Health Functions Project (P110599) in Argentina and the Improving Family and Community Health Project (P106870) in Nicaragua (related to the Government’s ability to rapidly respond to public health emergencies). The Project also built on lessons learned from other experiences in the country. These included the importance of (i) strengthening the fiduciary capacity of the PCU from the start of the Project to help avoid delays in procurement processes; and (ii) involve all stakeholders to ensure a successful Project implementation. Risks 50. Assessment of Project risks. The PAD described the overall Project risk as Medium20 which was overall a partial assessment of the environment in which the Project would develop. The main risk identified (High) to achieve the PDO that materialized was related to MINSAL’s knowledge of the Bank’s processes and procedures, and on its ability to coordinate the different elements of the Project and carry out fiduciary tasks. Weaknesses in the procurement capacity to manage large and complex contracts, identified as Medium-Low risk materialized and impacted implementation during most of the Project. The risk of initial Project implementation delays due to loan effectiveness delays, was identified during Project preparation (see Aide Memoire June 10-17, 2010), however, it was not included in the Operational Risks Assessment Framework (ORAF). The risk of gang violence as an obstacle to the achievement of PDO 1 (Coverage of children less than 1 year old with the third dose of pentavalent in the 92 selected municipalities) and a threat to public health was not considered despite the fact that the PAD recognized in its Sectoral and Institutional Context (Section B) that sexual and physical violence were key challenges faced by the Salvadoran society and therefore effective responses were required. The different risk mitigation measures considered in the PAD were pragmatic and clearly identified although probably not enough to offset Project risks. 51. Delays in Project effectiveness. Project effectiveness was delayed by 18 months. The Project was approved on July 21, 2011, signed on April 30, 2012 and declared effective on December 11, 2012. As a result, the Bank had to approve two Project effectiveness extensions and take further measures to ensure that the Project became effective within the time frame of the second extension. The Project was ratified by the Legislative Assembly on November 28, 2012; the Project’s first reallocation took place on December 6, 2012 and a few days later the Project was declared effective (December 11, 2012). B. B. KEY FACTORS DURING IMPLEMENTATION 52. Implementation issues related to delays in effectiveness. During the 18-month delay period, some of the activities included in the original procurement plan were carried out with funds from an IDB- financed project under implementation with similar objectives while targeting different municipalities. Consequently, by the time of Project effectiveness, the implementation plans were outdated and a 20 Medium I= a high impact if the anticipated risk is not mitigated, with a low likelihood of occurrence. Page 20 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) reassessment of activities to be funded by the Bank was necessary (see Section B). It took some time for MINSAL to offset the time lost during the initial phase of the Project. 53. Procurement. The Project faced numerous procurement challenges that were largely overcome from 2015 onwards. MINSAL´s limited procurement capacity and difficulty in handling large procurement processes in a timely manner, particularly at the start of Project implementation was a key implementation obstacle.21 Other difficulties stemmed from a steep learning curve regarding World Bank procurement policies and regulations; the weaknesses in the formulation of technical specifications; the poor quality of the market studies needed to prepare bids; and MINSAL´s bid evaluation committees perception that Bank Procurement Guidelines were not in tune with national procurement guidelines. In addition, the MOH’s occasional protracted internal processes and the Bank's protracted review of non- objections often slowed Project progress, while some executing units or loan resource beneficiaries did not take the necessary steps to initiate the procurement and contracting processes as scheduled in the project execution plan (PEP). Procurement issues were successfully resolved through a joint MINSAL- World Bank effort. The productive dialogue between the World Bank and the "Corte de Cuentas" in the country served to clarify that Bank Procurement Guidelines were aligned with national procurement laws. The implementation of a procurement mitigation strategy developed by the PIU and agreed with the Bank, the training of MINSAL’s technical units and suppliers on purchasing Bank guidelines as well as very intensive and permanent Bank technical guidance and procurement support and supervision improved the procurement process and ultimately allowed the Project to achieve the desired results. 54. Government commitment and leadership. The Government demonstrated, at all times, unwavering commitment and ownership to the Project and its objectives. The health sector has been prioritized in the implementation of the poverty alleviation strategy; as well as, for the implementation of the Health Sector Reform process, in the annual increase in health spending over the last 5 years, and the maintenance of free services for health service users. Public spending on health as a percentage of GDP increased from 3.7% to 4.5% in 2007-2016.22 Project activities were financed through earmarked budget allocations, while activities not originally planned, such as the National Center for Radiotherapy were also fully financed. In addition, the Bank’s strong determination to find a way to support the Project and Salvadorean population made this possible. 55. Disbursements. Project disbursements were affected by the 18-month effectiveness delay and consequently the outdated implementation plans; delays in the approval of the budget for the Project once it became effective, and the procurement bottlenecks, especially at the beginning of the project. By June 2015, the Project disbursements were low (at 7.6%). The Project was included on the list of outliers due to the high undisbursed balance. In December 2016, the Project overall implementation progress rating was downgraded from S to MU (see ISR dated 21 Dec 2016) because the Project reached 5.3 years since Bank approval (July 11, 2011) with a 38 percent disbursement ratio (US$30.1 million) in spite that the actual implementation time since effectiveness (December 11, 2012) had only been 3.1 years. Thanks to the measures taken and deep commitment from the PCU, UFI and the World Bank team to accelerate procurement processes, Project disbursements started to accelerate starting in 2015. Disbursements 21 There were a few deserted processes, particularly ICB bidding related to the purchase of medical equipment and ambulances. Among the factors that explained this are: lack of quality in the presentation of bids, due to weak understanding of the bidding documents by the bidders, mainly in aspects related to the terms and the filling of price-related forms. 22 MINSAL.Work report ( Informe de Labores) 2016-2017, El Salvador. Page 21 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) increased notably from 7.6% of total loan amount in FY14 to 17.6% in FY15 28.2% in FY16, 56.8% in FY17, and 99.9% by in FY18 when the Project closed. 56. Complementary World Bank-financed Operations. The Project’s objectives were also supported by a US$60 million Inter-American Development Bank (IDB) investment loan (Integrated Health Program, PRIDES (ES-L1027), initiated in 2011. The objectives of the IDB-financed operation were basically the same as the Project’s but targeted different municipalities. In addition, close collaboration with other development partners including the Pan American Health Organization (PAHO), and the Spanish Agency for International Development Cooperation; complemented financial support and enhanced activities and interventions. 57. Project’s PCU commitment and capacity. The Project Coordination Unit (PCU), established by negotiations, was always committed to improve the Project performance and achieve its goals. The PCU was in charge of coordinating with MINSAL’s technical units that were directly in charge of implementing Project activities; with MINSAL’s existing Institutional Procurement and Contracting Unit (UACI) and the Institutional Financing Unit (UFI), in charge of procurement and financial management activities, respectively. The PCU, through the life of the Project, revealed significant Project management strengths necessary to solve implementation obstacles. Overall there was a very low turnover of staff within MINSAL executing units and the PCU, which allowed for Project continuity. Some fiduciary Bank staff turnover affected Project implementation, particularly at the beginning. However, the fact that there was only one TTL during the implementation period and practically no fiduciary and operational staff changes from the Mid-term review mission onwards contributed to the successful completion of Project activities. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design Rating: Modest 58. Overall the original M&E framework presented monitoring challenges. The PDO was well articulated and could be measured objectively. The PDO indicators were overly ambitious and were difficult to measure. These shortcomings were addressed early in implementation by the first restructuring (February 20, 2015). The restructuring was an improvement to the M&E framework and allowed to better monitor progress toward the PDO's. Nevertheless, it did not have the necessary indicators to track progress on several NCDs related activities (particularly cancer) which received a significant portion of loan proceeds. This is explained by the difficulty of including relevant indicators since the Plan of the National Policy for the prevention and control of cancer was published in 2017, that is, towards the end of the project. Also, the NRC was put into operation a few days before the project's closing date. Additionally, MINSAL does not have yet statistics on patients completing a cancer treatment once the initial diagnosis is completed because until recently there was no a specialized resolution center. 59. The new RF offered a good proxy for provision expanding the coverage, quality and equity in utilization of priority health services. PDO indicator 1 Coverage of children less than one year old with the third dose of pentavalent in the 92 selected municipalities was directly associated with a decrease in Page 22 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) maternal mortality and it is also is an essential component for controlling the spread of communicable diseases. It also represents the primary health care system’s ability to reach and serve children multiple times during the first year of life. PDO indicator 2 Coverage of institutional delivery by the public network in the 92 selected municipalities is associated with a decrease in maternal mortality. It is important to note that according the World Bank’s PHCPI initiative, indicators such as these are crucial to measure, but are not sufficient. Performance data about the important processes required to convert inputs to high-quality care delivery and coverage outputs are critical to producing strong outcomes. Comprehensive data to pinpoint specific weaknesses, understand their causes, and strategically direct resources to address them is still lacking in El Salvador, as in many other countries in the region. For instance, good data on how often health workers are present and how accurately they diagnose and treat patients is not available in the country yet. Although it should be noted that the patient’s surveys carried out as part of project activities provided data on the experience of patients who receive care. 60. PDO indicator 3 Number of eligible hospitals that comply with standard bio -hazard waste management system was able to monitor an important aspect of MINSAL strengthened capacity to manage essential public health functions. Medical waste generated in hospitals was an issue of great concern and importance in view of potential public health risks associated with such wastes. The Project supported waste management practices and the development of policies and guidelines formulations. Original PDO indicator “Eligible secondary and tertiary hospitals certified by MINSAL“ was replaced since conditions did not exist to measure it. However, the Project financed activities aimed at developing the necessary conditions for the certification of hospitals in the near future. M&E Implementation Rating: High 61. MINSAL and the Bank systematically reported on key indicators and provided regular updates to the Bank. Project outcomes and results were analyzed using the MINSAL’s institutional statistics and monitoring system to track progress on the indicators identified in the results framework. The team monitored progress on the Project and prepared semi-annual reports that were shared with World Bank staff upon completion. The reports included a summary of the progress accomplished for each component, a summary of the obtained achievements, a summary of the main obstacles encountered, and the challenges for the next period. The report also provided updated information on the M&E indicators, the state of acquisitions and contracting and financial performance. The MINSAL and the Bank also jointly completed a mid-term evaluation report. M&E Utilization Rating: High 62. Good data availability was a Project strength, which enabled MINSAL to adjust the course during implementation. Information obtained via the M&E framework and from project surveys were used to inform project management and decision-making. The M&E data and performance and results progress periodically reported were used and incorporated in the Implementation Status and Results report (ISR). In addition, an aide-memoire was prepared and shared with MINSAL after WB implementation support and supervision missions. Finally, Project indicators were monitored against Page 23 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) their target levels to measure the accomplishment of objectives. In addition to monitoring progress on implementation and towards the development objectives, MINSAL supported the implementation of a number of additional surveys. Justification of Overall Rating of Quality of M&E 63. The Overall rating of the quality of M&E is Moderate. Despite significant shortcomings with the design, which were partially addressed through the Project restructurings; the M&E system was sufficient to assess the achievement of the objectives. M&E finding were disseminated and used to inform the Project direction and evolution. B. B. ENVIRONMENTAL SAFEGUARD, SOCIAL SAFEGUARD, AND FIDUCIARY COMPLIANCE 64. Environmental. Although the Project triggered the environmental safeguard policy (OP/BP 4.01), it was categorized as Environmental Category C given that it was not expected to generate adverse environmental impacts. During preparation, an environmental assessment of the potential impacts of the Project and an Environmental Management Plan were developed, disseminated and disclosed. 23 Environmental Diagnostics (EDs) developed in the 30 National Hospitals made it possible to comply with the Environmental Assessment Policy (OP / BP 4.01) and with the environmental legal provisions issued by the Ministry of the Environment and Natural Resources (MARN), thereby contributing to compliance with the environmental legal framework of the country. The Directorate of Environmental Health facilitated the identification of actions and measures, and the budget required both for investment and for operation, to ensure adequate management of hazardous hospital waste. In addition; 100% of hospitals are complying with the management and proper treatment of solid waste and with properly functioning environmental committees. Overall, there has been a recognition on the relevance of the environmental safeguards in MINSAL’s Directorate of Environmental Health and in its hospital network and an improvement in the handling of dangerous chemical substances and control of atmospheric emissions. The NRC financed by the project was considered viable from the environmental point of view and complied with the national environmental legislation and achieved the certification of the International Institute of Atomic Energy. 65. Social. The Project triggered the Indigenous peoples safeguard policy OP/BP 4.10 (Indigenous peoples) as it was expected to have a positive social impact on the population with attention on the urban and rural poor including indigenous peoples. An Indigenous Peoples' Planning Framework (IPPF) was prepared to guide Project implementation. This was discussed and approved by MINSAL through consultations with the indigenous community and associations and disclosed afterwards.24 Nevertheless, despite the existence of the IPPF, the Project did not produce indigenous plans because within the 92 municipalities targeted by the Project, there were no indigenous population that met the four criteria of OP4.10.25 However, some actions implemented under the Project have indirectly benefited indigenous peoples. These include the expansion of the coverage of primary health services to remote rural areas; 23 The environmental assessment was disclosed in-country on the MINSAL‘s website on March 10, 2011 and in the Bank‘s website on March 22, 2011. 24 The IPPF was disclosed in-country on the MINSAL‘s website on March 10, 2011 and in the Bank‘s website on March 22, 2011 25 See Tracking Social Performance – projects Under Supervision. (March 20, 2018). Page 24 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) the provision of quality health care services; and the training of health personnel to ensure adequate response to the population needs. Fiduciary 66. Financial management. All Financial Management (FM) supervision missions were rated Satisfactory. FM arrangements in terms of accounting, budgeting, flow of funds, internal control and financial reporting were satisfactory throughout the Project’s life. FM risk rating was rated as Moderate since control mechanisms to minimize risks were in place since the start of the Project. FM staff had experience managing the FM aspects of internationally financed projects and had been trained on Bank’s policies and procedures. The Project’s Interim Financial Reports (IFRs) were prepared and delivered in a timely manner and deemed acceptable. Likewise, the three financial audit reports were submitted on time and considered acceptable for the World Bank’s purpose. The financial execution of the project was efficient, complying with the contracting and disbursement of the budget assigned to the components, subcomponents and programmed. 67. Procurement. As already discussed in paragraph 53, procurement issues were carefully supervised by the World Bank team. Yet, Bank support during the first two years of implementation was probably not enough as reflected in the low effectiveness of procurement processes during this period. A high turnover of procurement specialists at this key stage impacted support. However, several MINSAL and Bank initiatives to build a strong procurement team and speed up the pace of procurement processes26 were critical. Among the measures are: (i) the strengthening of UACI by the hiring of qualified additional staff; (ii) training of UACI’s procurement specialists on Bank procurement policies and procedures and staff from MINSAL units in the development of technical specifications; (iii) continuous guidance and review of procurement process to identify bottlenecks. Three Procurement post reviews were carried out, one in June 2017 and two in February 2018. The February 2017 Procurement Post Review findings were: (i) procurement plan: all processes were consigned within an agreed version with the Bank; (ii) some internal delays were delaying the processes (i.e.: bidding document elaboration and contract signing took longer than expected); (iii) files were found incomplete, with duplicated information and lack of order, a more efficient filling system needed to be in place so that auditing and/or post reviews could be done efficiently; (iv) before awarding any contracts, all firms/individuals needed to be checked if not included in the debarred list; and, (v) NCB document needed to be agreed given a no objection by the Bank. These deficiencies were mostly addressed as revealed during the subsequent February 2018 Procurement Post reviews. C. C. BANK PERFORMANCE Quality at Entry Rating: Moderately Satisfactory 68. The World Bank team was efficient in responding to the demand of the GoS for completing the Project preparation. The Project was prepared and approved in a 12 month-period, as agreed with the 26The procurement performance was rated Moderately Satisfactory over the life of the Project mainly due to the slow pace of procurement processes, particularly at the beginning of the Project. Page 25 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Government, which required a fast-paced design and a selection of indicators available at that time. In retrospect some of the main risks that affected project implementation were identified during preparation but were underestimated while others were identified but measures for these were not implemented. Weaknesses in the M&E framework affected the rating although they did not prevent the measurement of Project achievement at the time of the ICR. Quality at Supervision Rating: Satisfactory 69. The task team conducted seventeen supervision missions and videoconferences and weekly follow up meetings to closely monitoring implementation and identify bottlenecks, in particular related to procurement processes. The Bank team made sustained efforts throughout the life of the Project, timely restructuring when necessary. The stability and technical skills of the the Bank’s team in providing intensive supervision support, with frequent communication with the Project team, developed mutual confidence and promoted efficient implementation of Project activities. During Project implementation, the Bank demonstrated well-balanced and effective teamwork, with strong cooperation between, legal, procurement, and disbursement staff, both at headquarters and in the country, as well as specialized consultants. Environmental safeguards were well monitored, resulting in the elaboration of bio-hazard waste management plans in 30 hospitals. 70. The World Bank’s team provided technical assistance and brought international experience, particularly to ensure technical and institutional sustainability of Project activities in the medium and long term. Overall, the Bank team was recognized by the Government for its professionalism, pro- activeness and responsiveness as well as for its understanding and respect of El Salvador’s programs. The Bank team mobilized global knowledge on various topics related to the better management of chronic diseases. Justification of Overall Rating of Bank Performance 71. The Overall rating for Bank performance is Moderately Satisfactory, considering that despite moderate shortcomings with Quality at Entry, the supervision effort succeeded in supporting MINSAL to address implementation challenges. D. D. RISK TO DEVELOPMENT OUTCOME 72. The Project does not pose a substantial risk to the development outcome. The Project was developed in line with the Government’s Health Reform. It supported the country’s need to respond to the burden of disease and was implemented through the National Health Directorates (without a PIU), therefore strengthening national capabilities in the process. The cascade effect of working with the regional levels through the Ministry empowered and promoted commitment at all levels of care (central, regional, departmental and municipal). Sustainability is ensured through the achievement of several key milestones, such as: (i) the creation of the National Directorate of NCD, which was established in 2017 and which will continue the implementation of the National Strategy for NCDs in the country; (ii) the qualification of the National Reference Laboratory as Level 2 for biosecurity that will provide timely response for the identification and treatment of diseases to the public system; (iii) the integration of Page 26 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) national health information systems, including the automatization of the clinical and family charts for optimized case follow-up; (iv) the increased capacity to maintain and repair medical equipment at all levels of care; and (v) the increased capacity to respond to medical emergencies with specialized equipment and ambulances. The Project improved the quality of the services and developed a qualified health workforce. The health facilities will continue to satisfy the local people’s needs, based on sustainable mechanisms for service provision at the local and community levels. V. LESSONS AND RECOMMENDATIONS 73. World Bank financial support can be effective in sustaining and expanding access to a country’s health services, thereby protecting the poor from the adverse impact of the crisis. The World Bank financed Project was approved shortly after the 2009 global financial crisis that negatively impacted El Salvador. Because of the crisis, poverty rose once again, especially in rural areas, partially reversing the social sector gains achieved in the decade following the civil war. The Government designed and began implementing a robust social policy and health strategy to increase access to health services and to expand coverage and quality of attention of the prioritized population. However, a tight fiscal envelope hindered the implementation of many of the initiatives and objectives laid forth in the reform World Bank resources helped establish the fiscal space necessary to implement the strategy. 74. Political commitment contributed to a successful outcome. The country was politically committed to raising the health status of the poorest and socially excluded (rural, urban-marginal and indigenous peoples; evidenced in the survey of users of health services), to improve health services delivery and to modernize its operation to face the double burden of disease. Project performance and progress were reviewed and discussed in weekly meetings at the Health Minister’s Office and monthly meetings at the President’s Office. The Government of El Salvador earmarked significant resources to finance activities that increased in access to health services and that achieved health system results. 75. The Project’s components were mutually supportive. It is unlikely that Component 1 would have been as successful without similar progress in the implementation of Component 2. The Government’s full commitment to the decentralization of health services and to the redefinition and improvement of MINSAL’s policy making and stewardship role were fundamental in the achievement of project results. 76. Realistic assessments at the design stage can ensure that adequate mitigation measures are incorporated into project design. Two important bottlenecks faced by this project were the procurement capacity of the Borrower, and the impact of gang violence in targeted areas. If properly identified at the design stage, both challenges could be been anticipated and taken into account early on, with adequate mitigation measures in place. 77. The use of a small but properly staffed Project Coordinating Unit can be essential for Project success. The PCU facilitated the coordination of Project activities implemented by the different executing units within MINSAL. Although the different units had a common institutional purpose - implementing the health sector reform- which was an important element for performance and ownership, the PCU was key in speeding up project activities and creating an environment that fostered interactions among the various units with a minimum amount of disruptions, overlaps and conflict. . Page 27 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Expansion of Priority Health Services and Programs Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 1. Coverage of children less Percentage 92.50 92.80 92.80 80.60 than 1 year old with the third dose of pentavalent in the 92 12-Dec-2012 30-Apr-2018 30-Apr-2018 31-Dec-2017 selected municipalities. Comments (achievements against targets): Source: MINSAL information system The indicator was achieved in three out of the five years measured (2013: 95.0/92.5 percent; 2014: 94.4/92.5 percent; 2015: 92.4/92.5 percent; 2016: 95.0/92.6 percent; 2017: 80.6/92.8 percent. The lack of achievement of PDO indicator 1 is largely explained by an increase in urban marginal and rural violence which affected health personnel’s ability to reach target populations and the fact that population projections based on the 2007 CENSUS for the est ablishment of targets, do not reflect real population growth. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 2. Coverage of institutional Percentage 81.00 84.00 84.00 99.80 delivery by the public network in the 92 selected 12-Dec-2012 30-Apr-2018 30-Apr-2018 31-Dec-2017 municipalities. Page 28 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Comments (achievements against targets): Source: MINSAL information system Target surpassed. % of achievement: 118.8% =99.8/84 Objective/Outcome: Institutional Strenghtening Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 3. Number of eligible Number 0.00 20.00 26.00 30.00 hospitals that comply with standard bio-hazard waste 12-Dec-2012 30-Jun-2016 30-Apr-2018 30-Apr-2018 management system. Comments (achievements against targets): Source: Environmental Health Unit, MINSAL Target surpassed. % of achievement: 115.3% =30/26 A.2 Intermediate Results Indicators Component: Component 1: Expansion of Priority Health Services and Programs Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion ECOS and hospitals in priority Text ECOS 0 ECOS708 Hospitals30 ECOS104 Hospitals 30 ECOS 269 Hospitals 30 municipalities equipped Hospitals 0 according technical guidelines defined by 12-Dec-2012 30-Jun-2016 30-Apr-2018 30-Apr-2018 MINSAL Comments (achievements against targets): Source: MINSAL information system Target surpassed. % of achievement: ECOS: 258.6% = 269/104; Hospitals 100% =30/30 Indicator was added in the February 20, 2015 restructuring and target revised in the February 12,2016 restructuring Page 29 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Hospitals integrated into the Number 0.00 8.00 8.00 8.00 National Medical Emergencies System 12-Dec-2012 30-Sep-2015 31-May-2018 30-Mar-2018 Comments (achievements against targets): Source: MINSAL information system Target achieved. Date of target completion was revised during the February 12, 2016 restructuring Component: Institutional Strengthening Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health personnel receiving Number 0.00 1200.00 1200.00 4660.00 training (number) 12-Dec-2012 30-Jun-2016 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Source: MINSAL information system Target surpassed % of achievement: 388.33%= 4,660/1,200 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Facilities in Number 0.00 12.00 134.00 158.00 priority Departments with operating information 12-Dec-2012 30-Jun-2016 30-Apr-2018 30-Mar-2018 system. Page 30 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Comments (achievements against targets): Source: MINSAL information system Target surpassed % of achievement: 117.9% =158/134 Indicator description was modified during the February 20,2015 restructuring. Indicator target and date of completion was revised in the February 12, 2016 restructuring Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of user satisfaction Number 1.00 2.00 2.00 2.00 surveys carried out. 30-Sep-2014 30-Jun-2016 30-Apr-2018 31-May-2018 Comments (achievements against targets): Source: MINSAL Target achieved. Indicator was added in the February 20, 2015 restructuring. The date of indicator completion was revised during the February 12,2016 restructuring. Page 31 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1- Expand the coverage, quality and equity in the utilization of priority health services. 1. Coverage of children less than 1 year old with the third dose of pentavalent in the 92 selected municipalities. Outcome Indicators 2. Coverage of institutional delivery by the public network in the 92 selected municipalities. 1. Fammily Health Community Teams (ECOS) and hospitals in priority municipalities equipped according to technical guidelines defined by MINSAL. Intermediate Results Indicators 2. Health personnel receiving training. 3. Hospitals integrated into the National Medical Emergencies System (SEM). Component 1. Expansion of Priority Health Services and Programs 1. 269 ECOS equipped in 92 priority municipalities (purchase of basic equipment). 2. 30 hospitals equipped (highly specialized medical equipment, emergency medical equipment and 44 ambulances). 3. 8 hospitals integrated into the SEM. Key Outputs by Component 4. 52 minor rehabilitation works, and maintenance of the Family (linked to the achievement of the Objective/Outcome 1) Health Community Units (UCSF) executed. 5. Establishment of the Biomedical Laboratory to support the national plan for effective maintenance and repair of medical and non-medical equipment. 6. Establishment of the National Radiotherapy Center (NRC) for the treatment of patients with cancer. 7. Training of health promoters in integral general health care. Page 32 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 8. Purchase of specialized equipment for emergency areas in hospitals of the second and third levels of care and training of 662 SEM staff to deal with medical emergencies efficiently. 9. Establishment of a Non-Communicable Diseases (NCD) department and strengthened tools for improving the health sector’s ability to address the steady rise in NCDs. 10. Training of MINSAL staff in the accreditation model of health facilities. 11. Purchase of medications for the treatment of chronic kidney disease. 12. Purchase of a cobalt pump to improve the Cancer Institute Objective/Outcome 2- Strengthen MINSAL’s stewardship capacity to manage essential public health functions. 1. Number of eligible hospitals that comply with standard bio-hazard Outcome Indicators waste management system. 1. Number of Facilities in priority departments with operating Intermediate Results Indicators information system. 2. Number of user satisfaction surveys carried out. Component 2. Institutional Strengthening 1. 30 hospitals comply with standard bio-hazard waste management practices. 2. 158 facilities in priority Departments with operating information systems. Key Outputs by Component 3. Preparation and Execution of 2 surveys (MICS and EFRAES) for (linked to the achievement of the Objective/Outcome 2) obtaining the trends of the health indicators and their health determinants and the development of a program with a systemic, epidemiological and clinical approach for the integral approach in the prevention and treatment of non-communicable chronic diseases. Page 33 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 4. 2 users satisfaction surveys carried out for MINSAL’s services quality improvement. 5. Purchase of the building to house the National Health Institute (NHI) and the National Reference Laboratory (NRL) for their reinforcement. 6. Purchase of equipment for improving MINSAL’s editorial unit. 7. 1429 staff trained in the rational use of medicines, pharmacovigilance and evaluation of health technologies. Page 34 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Rafael Cortez Task Team Leader Maryanne Sharp Senior Operations Officer Eleonora Cavagnero Economist Rocio Schmunis Consultant Jimena Luna Consultant Alvaro Larrea Procurement Specialist Gunars Platais Environmental Safeguards Specialist Mary Lisbeth Gonzales Environmental Safeguards Specialist Joana Godinho Task Team Leader Chririne Alameddine Analyst Ivan Gonzalez Health Specialist Jimena Garrote Counsel Christina Novinskey Health Policy Specialist Fernando Montenegro Health Economist Kristine M. Ivarsdotter Social Development Specialist Marcos Miranda Engineer Rory Narvaez Economist Supervision/ICR Amparo Elena Gordillo-Tobar Task Team Leader Monica Lehnhoff Procurement Specialist Sandra Lisette Flores De Mixco Financial Management Specialist Maria E. Colchao Team Member Claudia Macias Team Member Page 35 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Marco Antonio Zambrano Chavez Environmental Safeguards Specialist Maria Gabriela Moreno Zevallos Team Member Carlos Tomas Perez-Brito Social Safeguards Specialist Gabriela Grinsteins Counsel B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 18.230 86,505.25 FY11 48.255 271,776.45 FY12 0 3,016.80 FY17 0 4,860.00 Total 66.49 366,158.50 Supervision/ICR FY12 38.631 399,226.73 FY13 24.048 216,956.86 FY14 18.197 96,678.75 FY15 35.361 140,140.09 FY16 31.750 156,366.72 FY17 45.651 263,698.62 FY18 20.421 246,781.19 FY19 13.050 173,438.61 Total 227.11 1,693,287.57 Page 36 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Expansion of Priority Health Services and 45.0 60.6 134.6 Programs. Component 2: Institutional 31.4 16.9 53.82 Strengthening Component 3: Project 3.4 2.2 64.7 Management and Monitoring Front end fee 0.2 0.2 100 Total 80.00 79.80 0.00 Table 1. Estimated, Adjusted and Actual Costs by Component (US$ million) Component Estimated First Second Third Actual Component Name in PAD restructuring restructuring restructuring (June 30, 2018) Expansion of Priority Health 1 45.0 54.41 63.41 60.064 60.6 Services and Programs Institutional 2 31.4 22.74 13.34 17.085 16.9 Strengthening Project 3 Management and 3.4 2.650 3.050 2.65 2.2 Monitoring Front-end fee 0.2 0.2 0.2 0.2 0.2 Total 80.0 80.0 80.0 80.0 79.9 Page 37 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 4. EFFICIENCY ANALYSIS Project context 1. The Strengthening Public Health Care System Project supported the Government of El Salvador National Health Strategy 2009-2014, which sought to guarantee greater access and universal coverage of health services, with high-resolution capacity through the implementation of a Health Reform and the Comprehensive Health Primary Care strategy. Though the Project focused on maternal-child health through some of its components it also sought to address comprehensive care through the expansion of priority health services and programs. The Project implemented a wide-ranging set of essential health interventions and improved delivery of priority programs, while strengthening the stewardship capacity of MINSAL ‘s essential public health functions for service delivery. Economic and Financial Analysis at the PAD stage 2. The economic analysis in the PAD undertook a sustainability analysis to determine necessary costs, both recurrent and capital expenditure budgeted and projected against the recurrent costs needed to implement the Government ‘s Comprehensive Primary Health Care strategy. It also considered a series of Project assumptions regarding Project strategy, interventions, and goals, such as the population covered, discount rate and direct and indirect benefits. 3. The analysis concluded that the Project was expected to yield substantial benefits. It would avoid between 2,953 and 3,938 deaths, which would translate into monetary benefits of between US$129 million and US$172 million. For all scenarios, the Project's benefit-cost ratio was greater than one (between 1.61 and 2.15) and its present monetary value was positive (between US$49 and US$92 million). Table 1: Results of the Economic and Financial Analysis Scenario 1: Scenario 2: Scenario 3: High Basic Low Cost of intervention 80 80 80 (US$ million) Deaths averted 3,938 3,150 2,953 Avoided years of life 201,943 161,554 151,457 lost PMV of deaths averted 172 137 129 (US$ million) Net benefit (US$ 92 57 49 million) Benefit-cost ratio 2.15 1.72 1.61 Abbreviations: Present Monetary Value (PMV). Page 38 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Economic and Financial Analysis at the ICR stage: 4. The Project revealed high overall efficiency since the resources assigned to finance activities and equipment yielded and will continue to yield significant benefits for the country. The expansion of priority health care services and programs providing comprehensive, essential health interventions and the improvement of priority program delivery and health service infrastructure, in addition to the strengthened stewardship capacity of MINSAL‘s all resulted in the efficient use of Project resources. 5. The Project’s benefits and impact is difficult to quantify in this analysis. The Project demonstrated positive efficiency gains from reductions in mortality and inequality indicators and improvements in health indicators. Project investments contributed to sustaining and improving social gains in life expectancy, decreasing child and infant mortality and improving maternal and reproductive health outcomes, while improving equity through the expansion and utilization of health services for those in the poorest municipalities. Nevertheless, attribution is complicated by the ongoing implementation of a comprehensive health care reform in El Salvador initiated in 2009, while a monetary valuation of the benefits difficult to quantify. 6. Efficiency was boosted through improved health provider performance, the purchase and use of adequate and reliable medical equipment, and reductions in cost interventions and medicine. Through the financing of new capital investments such as medical equipment and training of health personnel working in priority health programs, Components 1 and 2 lessen the demand for curative services with better provision of preventive care. Savings from the capture of disease at an earlier stage, for example through the focus on non-communicable diseases, support efficiency and allow for the reallocation of funds towards increased health care coverage of the population. 7. The Project provided additional resources to fill demand-side and supply-side gaps for the delivery of health care services under MINSAL. The expansion in coverage and focus on quality strengthened health service delivery, while improved targeting of underserved and vulnerable populations fostered greater equity in the utilization of health services. Equity in health service delivery may indirectly increase demand for health services, especially for the underserved, and can directly impact health outcomes positively. Without this intervention, underserved populations would continue to experience difficulty in utilizing health services, especially at the first level of care, and health providers would have to work with inadequate and inefficient medical equipment, and lack of trainings. This would be costlier for MINSAL as outpatient and curative services are more expensive. 8. The Government’s commitment towards increased coverage and better stewardship and monitoring capacity accompanied more efficient and effective use of budget funds and personnel. Since the end of the civil war in 1992, El Salvador has almost doubled its percentage of tax revenue to gross domestic product (tax-to-GDP), from 8 percent in 1992 to almost 16 percent in 2015, and in turn its Government’s health expenditures-to-GDP ratio from 2.4 to 4.6 percent in 2013, while individuals’ out- of-pocket health expenses steadily decreased. Page 39 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 9. Increases in public and external health funding financed the policies set forth in El Salvador’s health reform and supported a reduction in out of pocket (OOP) spending. OOP spending fell from 50 percent of current health expenditure in 2011 to 28 percent in 2015.27 This is due to an increase in general Government health expenditure and an expansion in health coverage. OOP payments are generally the most inequitable type of financing because they tend to impact the poor the hardest by creating a barrier to health care access and/or by denying individuals financial protection from catastrophic illnesses. 10. From a macroeconomic perspective reduction in mortality and in morbidity raise productivity and overall GDP levels. Studies support evidence of causal effects from both GDP to health and health to GDP when examining child and maternal mortality.28 11. From a fiscal perspective, investments in the project though beneficial could impact the country’s already weak fiscal position. The level of indebtedness has increased rapidly, driven by the rise of pension- related debt linked to the 1998 pension reform and subsequent changes in legislation that led to higher Government obligations, relatively small but persistent primary deficits and a weak growth environment. Between 2008 and 2017, debt of the non-financial public sector rose from 48 to 71 percent of GDP. 12. Though the Project investments imply higher fiscal liabilities, these are largely related to high startup costs and not operating costs. The expenses related to project investments have been incorporated into the Government budget, which is prepared annually, according to Ministry of Finance guidelines. For example, the National Radiotherapy Center (NRC) implied a high start-up cost (infrastructure and equipment), but low operating cost since most of the costs are related to workers payroll, maintenance of equipment and training requirements. The estimated budget for the NRC for 2018 amounts to around US$700,000 and US$1.5 million for 2019, all annually earmarked in the Government budget. The health personnel working there (5 radiation oncologists, 10 radiotherapy technicians, 4 medical physicists, 5 nurses, 1 anesthetist and 5 administrative staff are all included in the Ministry of Health payroll.) Patient care is free. The operating expenses of the Medical Emergency System (SEM) Coordinating Center are covered by the Government budget, amounting to approximately US$1.5 million. 13. The Project experienced an efficiency impact from the delays in the implementation of Project activities. The Project experienced a two-year delay largely due to procurement related obstacles, which impacted disbursements and Project activities. Delays in project implementation and in particular procurement suggest Project inefficiencies given the opportunity cost of capital and the extension on the service fee paid. The total financing of the Project of US$80 million was originally to be disbursed over a period of five years (2011-2015). However, implementation was delayed, and disbursements did not begin until 2014, with Project investments completing by May 2018 (four and a half years). 14. The Project benefited 2.1 million people who were among the poorest living in 92 eligible municipalities in both rural and urban areas of El Salvador. Component 1 expanded coverage of the target population with a comprehensive set of essential health interventions and improved delivery of priority programs focused on health promotion, disease prevention, treatment, and rehabilitation. There are several direct and indirect benefits associated with the Project intervention, namely: (i) a reduction of 27 WHO Global Health Expenditure Database. 28Amiri, Arshia and Ulf-G Gerdtham, Impact of Maternal and Child Health on Economic Growth New Evidence Based Granger Casuality and DEA Analysis. March 2013. Page 40 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) maternal and infant mortality; (ii) greater efficiency of health services through capital investments and training of health provider; and (iii) an increase in coverage expanded through the RIISS to strengthen delivery of services. 15. The Project‘s implementation of prevention and promotion activities during the prenatal stages of pregnancy aimed to reduce the number of maternal and infant deaths. The focus on priority health programs, such as maternal and infant health and increased coverage/access to rural and often lower income deciles supported a reduction in mortality rates. Reductions in mortality rates, in particular for the young, produce economic benefits through the savings of lives that in turn produce economic benefits through an individual‘s future labor force participation.29 16. Benefits: Maternal and Child Mortality Rates 17. The ICR sought to mimic the benefit cost analysis used in the 2011 Project Appraisal Document (PAD) economic analysis despite a lack of access to the data from the benefit cost analysis. From what we can gleam from the PAD, the analysis included only maternal and infant deaths in the analysis. The ICR sought to quantify the impact of the expansion of priority health services and programs by obtaining a value for the avoided deaths due to the Project’s activities. Avoided deaths were calculated using the reduction in maternal and infant mortality rates, while a life expectancy of 73 was used to measure the remaining years of life.30 Maternal deaths were assumed to occur at 20 years of age though there is a preponderance of adolescent maternal mortality, so the years of life saved is probably greater than that used in the analysis. The ICR used actual data for 2013-201631 (World Bank databank) and assumed similar lives avoided for 2017 as in 2016. 18. The Component actions aim to improve health care for infants under one year. The World Health Organization (2017) has estimated that at least 30% - 40% of child deaths result from poor antenatal care during pregnancy and delivery, while the international literature suggests babies whose mothers had adequate antenatal care are 30% less likely to die in their first six months. If we assume that approximately 25% of child deaths are averted due to Project activities, then we obtain 126 infant saved lives in 2016 and 660 between 2013-2017. If we use an average salary of US$300 per month for the 53 working years per infant saved, we obtain an economic benefit of US$126 million. 19. If we calculate the change in infant deaths from one year to the other, the number drops to 117 infant deaths avoided during 2013-2017. Though the infant mortality rate in El Salvador dropped from 16 deaths per 1,000 live births in 2010 to 13 in 2016, live births per 1000 also decreased so that the amount of lives saved due to the reduction of infant mortality in the targeted population did not increase significantly. If we use this methodology (this assumes that all the infants work 53 years of their lives) we obtain the economic benefit of US$22 million. 20. Similarly, we can obtain a similar value by quantifying the reduction in maternal mortality. The Project not only focused on education to decrease adolescent pregnancy but also supported maternal health, through prevention and promotion actions regarding antenatal and postnatal care. Though the maternal mortality rate experienced significant fluctuation it fell from 50 in 2010 to 27 in 2017. This 29 We assume that income per capita remains constant for an individual living in rural areas. 30 The ICR uses a life expectancy of 73, while the PAD used one of 72. 31 Except live births which used the same 2015 data for 2016. Page 41 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) resulted in 10 maternal deaths in 2016 for the targeted population. International literature suggests that at least 20 percent of maternal deaths can be prevented through the implementation of community-based prenatal interventions. Therefore, we assume that approximately 2.5-3 maternal deaths could be averted and approximately 18 over the targeted period. If we use an average salary of US$300 per month (2015) and an average employment rate of 50 percent, the economic benefit only amounts to US$1.8 million. 21. The Healthcare Access and Quality (HAQ) Index provides a summary measure of healthcare access and quality for a given location. This measure is based on risk-standardized mortality rates or mortality- to-incidence ratios from causes that, in the presence of quality healthcare, should not result in death – also known as amenable mortality. How does personal health care access and quality measure up? 1990 2000 2016 HAQ Index HAQ index 63.2 HAQ index 52.1 38.1 Source: bit.ly/HAQ-GBD2016 22. This approach is likely to underestimate the benefits of the Project for three main reasons. First, the model captures only benefits that result from reductions in maternal and infant mortality. Second, the model excludes all other deaths avoided in part due to new medical equipment, improved health service quality and the setup of the National Medical Emergency System. The benefits resulting from the new equipment will likely be measurable over the medium to longer term, in particular those associated with the new Cancer center and fleet of ambulances. Thirdly, it does not consider benefits from reductions in morbidity and out-of-pocket expenditures. 23. As a result we tried to evaluate some additional benefits. 24. Benefits Oncology: Equipment investments considered by the Project include high-cost cancer treatment medical equipment. First, a cobalt 60 radiation machine for cancer treatment (US$0.9 million) was procured and installed in the "Instituto del Cáncer" premises (Cancer Institute), a specialized non- profit institution that has been serving patients from the public sector for decades for therapeutic and palliative cancer treatments. Currently, the equipment treats approximately 800 patients per year. In 2017, MINSAL built and equipped the NRC for Cancer Treatment (MM US$8). Investment in related infrastructure and equipment will annually deliver radiotherapy and brachytherapy services to more than 1,300 patients. The center opened in 2018. 25. As a public health epidemic (chronic non-communicable disease), cancer, is on the rise and over the upcoming decade will become the top cause of death among the adult population worldwide. To Page 42 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) estimate demand for oncological curative and palliative care we projected the cancer incidence for future periods. By applying the modelled cancer incidence to "El Salvador"32 population, based on estimates made by GLOBOCAN,33 a joint initiative led by the International Agency for Research on Cancer (IARC) and the World Health Organization (WHO), aimed to provide contemporary estimates of the incidence of, mortality and prevalence from major types of cancer, the results projected 8,956 new cases by 2016 and 10,944 by 2027. 26. Cancer treatment can involve surgical interventions, chemotherapy, radiotherapy and brachytherapy, either alone or their combinations, with the estimates of service delivery are made depending on the proportions required from each therapy, according to care treatment standards. These estimates are presented below: Table 2. Activity Projections by 2017 and 2026 – SV % Rise Surgical PROJECTION PATIENTS 2017- Interventions Chemotherapy Radiotherapy Brachytherapy ACTIVITY 2017 2026 2026 2017 2026 2017 2026 2017 2026 2017 2026 SVD 8.956 10.944 22,20% 3.290 4.011 4.761 5.785 4.541 5.534 545 664 TOTAL 8.956 10.944 22,20% 3.290 4.011 4.761 5.785 4.541 5.534 545 664 Source: Author’s estimates based on IARC/WHO incidence data. 27. The patients estimated for 2017 and 2026 are obtained by applying cancer incidence rates to the beneficiary population for the respective years. This shows that in general terms, the number of patients has grown by 22.2%. 28. To determine the number of surgical interventions, and chemotherapy, radiotherapy and brachytherapy treatments required for these cases, the percentages included in Table 2 were applied. 29. With these figures in hand, we now appraise physical and equipment resource requirements. 30. RT Treatment: Based on the cancer incidence method, the number of radiotherapy procedures projected for 2017 and 2026 are 4,543 and 5,534 respectively. Considering the duration of RT procedures (0.25 hr. per session), 20 sessions per patient, average, 250 working days per year and 9 hour of equipment availability per day, the number of devices needed for the project horizon would be 12 Linear Accelerators (LINAC). 31. Brachytherapy: According to our estimates 12% of RT patients may require Brachytherapy (664 patients, 12% of 5,534 patients by year 2026). They have estimated equipment usage time (9 hours a day), duration of procedure (1 hour) and 4 sessions per patient multiplied by 250 working days. As a result, the requirement is equivalent to 1.18 devices. By expanding the number of working hours, the demand is met by a single device. 32 El Salvador is part of GLOBOCAN initiative through population-based cancer registries accredited by the International Association of Cancer Registries (IACR). 33 GLOBOCAN 2012, Cancer Incidence and Mortality Worldwide: IARC Cancer Base No. 10 [Internet]. Lyon, France: International Agency for Research on Cancer; 2017. Available from: http://globocan.iarc.fr. Page 43 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 32. Chemotherapy Treatments: To determine the number of chairs required, we assigned the total number of cases that will require chemotherapy treatments (4,761 in 2017 and 5,785 in 2026), 6 chemotherapy cycles, average, chemotherapy session duration (4.5 hrs. average), 250 working days per year, 9 working hours a day. This results in 57 and 69 chemotherapy chairs for 2017 and 2026 respectively. These facilities are in base hospitals of San Salvador. Table 3: Activity Projections by 2017 and 2026 – SV Chemotherapy chairs LINAC Brachytherapy Year 2017 2026 2017 2026 2017 2026 Number 57 69 10 12 1 1 devices Source: Author’s estimates 33. Based on the above, demand exceeds both the domestic public and private supply and therefore, lead time applied to NRC healthcare will depend on the needs for addressing oncological pathologies. 34. To estimate the benefits of this intervention, the information sourced by El Salvador National Cancer Institute has been utilized for determining the number of patients receiving Cobalt therapy treatment and average procedure costs (US$1,000 per patient by private health providers). In case of Radiotherapy (RT) and Brachytherapy (BT), the increasing usage of current facilities is projected to reach 100% of rated capacity by 2021. Daily equipment usage has been set at 9 hours a day, though other countries employ longer working hours, depending on the demand for healthcare services. In the case of Radiotherapy (RT) and Brachytherapy (BT) average costs of US$8,000 for RT and US$1,200 for BT were considered. Usually, these costs are charged by private health providers. 35. Chemotherapy (CT) has been excluded because it is not part of the project scope. 36. Surgical Interventions: Operating room equipment refurbishment included operating tables, surgical light heads, anesthesia machines, monitors, instruments, etc. were part of equipment investments. New equipment increased surgical outputs. The number of surgical procedures increased annually by 9,940 from a baseline of 108,094. (To compute this benefit, we utilize the ISSS surgical discharge cost, whose DC value is US$324.6834 and average stay of 4.09 days. Therefore, the financial analysis is based on US$1,327.94 minus VAT (13%). 37. Imaging: As part of equipment investments, refurbishment also covered traditional imaging equipment (fixed and portable), including arch angiography, mammogram and ultrasound devices. While equipment upgrade translates into increased marginal service delivery versus baseline year 2010, the ultrasound and mammogram outputs stand out by far. Taking as baseline 181,667 tests in 2010, between 2014 and 2017, average additional tests amounted to 98,608 ultrasound scans annually and 12,957 additional mammogram tests annually versus 10,742 tests performed in the baseline year. To give them a monetary value, an average cost of US$40 minus VAT has been considered for ultrasound tests and US$20 minus VAT for mammogram tests. 34 Actuarial valuation of ISSS Health Scheme and Occupational Health Risk Scheme, November 2016. Page 44 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) 38. Physical Medicine: PHC equipment investments included physical medicine and rehabilitation equipment. When considering the number of PM activities in baseline year 2010, on average additional 423,253 physical medicine activities are reported for 2014-2017 period versus 2010 (357,249). To monetize this benefit, we took the average cost of physical medicine services delivered by MINSAL (Cost Analysis Study of Morazán Health Services Network 2013 PAHO), which yields US$2.5 per activity. When updated and adjusted to an ISSS service value coefficient (62% higher than average MINSAL), the resulting value amounts to US$5 per service minus VAT. 39. As noted above, the number of services versus baseline values soared, which obviously implied additional benefits to those already considered, not included by the analysis (i.e. Dialysis, conventional imaging, increased outpatient-based preventive care, decreased emergency care, curative medicine, etc.). All these facts add greater social profitability to the Project. Page 45 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS Unofficial translation REPUBLIC OF EL SALVADOR MINISTRY OF HEALTH Project Coordinating Unit STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT Agreement IBRD 8076- SV FINAL SUMMARY OF PROJECT ACHIEVEMENTS AND OUTCOMES This Executive Summary is drafted under the Implementation Completion and Results report to consolidate key achievements and outcomes under the “Strengthening Public Health Care System� Project. The Government of El Salvador and IBRD signed the US$80-million Loan Agreement Nº 8076-SV to implement this Project, which officially became effective on November 28, 2012 and the first disbursement took place on September 2013. The effective implementation period spanned between 2014 and 2018. The Project had the following PDOs: (i) Expand the coverage, quality, and equity in utilization of priority health services by selected populations; and (ii) strengthen the Ministry of Health‘s (MINSAL) stewardship capacity to manage essential public health functions. To achieve them, the Project defined three Components: Component 1: Expansion of Priority Health Services and Programs. Component 2: Institutional Strengthening. Component 3: Project Management and Monitoring. The Project benefited 2.1 million people who were among the poorest living in 90 eligible municipalities in El Salvador, through services delivered by 30 national hospitals. I. PROJECT OVERVIEW • Loan approved by WB: Jul-21-2011 • Loan agreement signed by and between World Bank and Ministry of Finance: Apr-30-2012 • Ratified by Legislative Assembly: Nov-28- 2012 • Eligibility clearance by World Bank: Dec-11-2012 • First loan disbursement: Sep-5-2013. Amount US$500,000.00 • Closing date: May-30-2018 Page 46 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) II. KEY ACHIEVEMENTS OF PDOs PER COMPONENT COMPONENT 1: Expansion of Priority Health Services and Programs. a) Strengthening Primary Health Care Investments focused on medical and non-medical equipment, including: dental equipment and instruments for PHC facilities (dental units, ultrasound machines and dental curing lights and different instruments), Physical Therapy Equipment (electrical muscle stimulator machines (EMS), gym equipment, including stationary bikes, adult and children's parallel bars, shoulder wheels, treadmills, vertical stander, training stairs, etc.; hospital rehab tanks, tubs and heat and cold packs; UCSF office furniture, warehouses and administration offices (lockers, shelves, racks, chairs, tables, desks, etc.); transportation equipment for central and regional storage facilities and environmental sanitation; central warehouse cold room equipment, occupational safety equipment, closed circuit surveillance system, rack and shelving and pallet handlers in MINSAL warehouse facilities; aedes arbovirus vector control spraying equipment and spare parts (Dengue, Zika and Chikungunya), IT equipment and mobile devices (tablets) for geo- referenced family information collection. PHC interventions included minor rehabilitation and maintenance works for 56 health facilities, as well as several works across different central MINSAL facilities. No. PROJECT DEPARTMENT ACTUAL COST* Mejora de UCSF Intermedia de San Francisco Menéndez 1 Ahuachapán $118,868.33 (urbano). Ahuachapán. 2 Mejora de UCSF Intermedia de Atiquizaya, Ahuachapán Ahuachapán $50,640.13 Mejora de UCSF Intermedia de Candelaria de la Frontera, 3 Santa Ana $147,196.03 Santa Ana. Rehabilitación Almacén Regional, Región Occidental de 4 Santa Ana $126,308.03 Salud, Santa Ana. Mantenimiento de UCSF Básica de Santa Rosa. Municipio 5 Santa Ana $53,394.06 de San Sebastián Salitrillo, Santa Ana. Mejora de UCSF Básica San Miguel Texistepeque, 6 Santa Ana $163,976.38 Municipio de Texistepeque, Santa Ana. 7 Mejora de UCSF Intermedia de Texistepeque, Santa Ana. Santa Ana $102,709.68 Mejora de UCSF Intermedia Dr. Adalberto Aguilar Rivas, 8 La Libertad $160,211.17 Municipio de Santa Tecla, La Libertad Mejora de UCSF Intermedia de San José Villanueva, La 9 La Libertad $133,267.82 Libertad. Rehabilitación de Almacén Regional, Región Central de 10 La Libertad $71,579.99 Salud, La Libertad. Page 47 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) No. PROJECT DEPARTMENT ACTUAL COST* Mejora de UCSF Intermedia de San Juan Opico, La 11 La Libertad $114,457.89 Libertad. Mejora de UCSF Intermedia de Lourdes, Colón, La 12 La Libertad $109,211.36 Libertad. Rehabilitación de Almacén Regional, Región 13 San Salvador $120,744.89 Metropolitana de Salud, San Salvador. Mantenimiento de Obra de UCSF Intermedia de 14 San Salvador $60,874.77 Aguilares, San Salvador Adecuación de local para montaje de la unidad de 15 San Salvador $448,657.97 atención al cáncer Adecuación de local para montaje de laboratorio de 16 San Salvador $503,350.72 biomédica Readecuación de las instalaciones del 4° nivel del edificio 17 B para adecuar las oficinas administrativas del LNR/INS San Salvador $269,008.35 MINSAL 18 Reparación de cisterna del plantel El Matazano San Salvador $39,107.53 19 Mantenimiento de techo de bodega de mantenimiento San Salvador $67,831.14 Mejora de Obra de UCSF Intermedia de San Rafael 20 Cuscatlán $104,582.44 Cedros, Cuscatlán. Mantenimiento de Obra de UCSF Intermedia de Paraíso 21 Cuscatlán $63,082.26 de Osorio, Cuscatlán Mejora de Obra de UCSF Intermedia Santa Lucía, 22 Ilobasco, Depto Cabañas Cabañas $89,434.61 23 Mejora de Obra de UCSF Intermedia de Victoria, Cabañas Cabañas $138,480.62 Mantenimiento de Obra de UCSF Intermedia de Dolores, 24 Cabañas $76,905.15 Cabañas. Mantenimiento de Obra de UCSF Intermedia de San 25 Cabañas $69,045.73 Isidro, Cabañas. Mejora de Obra de UCSF Intermedia de San Sebastián, 26 San Vicente $139,133.60 San Vicente. Mantenimiento de Obra de UCSF Intermedia de San 27 San Vicente $72,031.98 Cayetano Istepeque, San Vicente. Mejoras de Obra de UCSF Intermedia Periférica de San 28 San Vicente $135,750.55 Vicente. Mejoras de Obra de UCSF Intermedia de San Lorenzo, San 29 San Vicente $117,092.44 Vicente. Page 48 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) No. PROJECT DEPARTMENT ACTUAL COST* Mejoras de Obra de UCSF Intermedia de Santo Domingo, 30 San Vicente $97,047.52 San Vicente Mejora de Obra de UCSF Intermedia San Carlos Lempa, 31 San Vicente $119,783.29 Municipio Tecoluca, San Vicente Obras de mejoras y remodelación Almacén de insumos y 32 medicamentos de Almacén de Dirección Regional de San Vicente $142,478.94 Salud Zona Paracentral, San Vicente. Mejora de Obra de UCSF Intermedia de Zacatecoluca, La 33 La Paz $121,183.31 Paz. Mejora de Obra de UCSF Intermedia de Santa María 34 La Paz $136,376.12 Ostuma, La Paz. Mejora de Obra de UCSF Intermedia de San Pedro 35 La Paz $130,908.47 Masahuat, La Paz. Mantenimiento de Obra de UCSF Intermedia El Achiotal, 36 La Paz $46,181.41 Municipio de San Pedro Masahuat, La Paz. Mejora de Obra de UCSF Especializada de Olocuilta, La 37 La Paz $134,424.10 Paz. Mantenimiento de Obra de UCSF Intermedia El Rosario, 38 La Paz $73,564.48 La Paz. Mantenimiento de Obra de UCSF Intermedia San Pedro 39 La Paz $46,289.40 Nonualco, La Paz. Mejora de Obra de UCSF Intermedia de Santiago 40 La Paz $103,992.15 Nonualco, La Paz Mantenimiento de Obra de UCSF Intermedia de Puerto El 41 Triunfo, Usulután Usulután $74,503.16 42 Mejora de Obra de UCSF Intermedia de Jucuapa, Usulután Usulután $131,433.22 Mejora de Obra de UCSF Intermedia de Ereguayquín, 43 Usulután $78,941.18 Usulután. Mejora de Obra de UCSF Intermedia de El Tránsito, 44 Usulután $110,503.37 Usulután. Mantenimiento de Obra de UCSF Intermedia de Santa 45 Usulután $57,514.57 María. Usulután Rehabilitación Almacén Regional, Región Oriental de 46 San Miguel $121,582.21 Salud, San Miguel. Mejora de Obra de UCSF Intermedia de Meanguera, 47 Morazán $96,853.56 Morazán. Page 49 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) No. PROJECT DEPARTMENT ACTUAL COST* Mantenimiento de Obra de UCSF Intermedia de 48 Morazán $57,998.99 Quebrachos, Municipio de Jocoaitique Mantenimiento de Obra de UCSF Intermedia de San 49 Morazán $81,320.02 Carlos, Morazán Mejora de Obra de UCSF Intermedia de Isletas, Municipio 50 Morazán $95,032.46 de Yamabal, Morazán 51 Mejora de Obra de UCSF Intermedia de Polorós, La Unión La Unión $115,522.98 Mejora de Obra de UCSF Intermedia de Nueva Esparta, La 52 La Unión $136,003.55 Unión Mantenimiento de Obra de UCSF Intermedia de El Sauce, 53 La Unión $67,714.20 La Unión. Mantenimiento de Obra de UCSF Intermedia de 54 La Unión $55,897.50 Conchagua, La Unión. Mantenimiento de Obra UCSF Intermedia Meanguera del 55 La Unión $58,700.07 Golfo, La Unión Mejora de Obra de UCSF Concepción de Oriente, dpto La 56 Unión La Unión $99,607.84 TOTAL $6,448,103.39 * Includes the cost of the work plus supervision b) Investments in hospital medical equipment for secondary and tertiary health care Installed capacity of 30 national hospitals for delivering specialized services improved substantially thanks to the procurement of high tech medical equipment, medical clinic furniture, and non-medical equipment, including central sterile services departments (CSSD) equipped with thermal washer disinfectors, arch angiography with a monoplane system for surgery, defibrillator crash cart equipment, shadow less LED surgical lighting, ICU electric beds, three-gas anesthesia machine with capnography, vital signs monitors including ICP monitoring, operating room table for major surgeries, adult ventilator, defibrillator crash cart equipment, stationary incubator, neonatal monitor, ultrasound machines, EGD and colonoscopy equipment, digital X-ray system, medical imaging management and processing system, digital radiology unit with breast imagining post processing station, high frequency oscillatory ventilator and neonatal ventilator; sterilizer steam generator and power facilities. c) Establishment of National Radiotherapy Center (NRC) for Cancer Treatment In 2015, MINSAL created the Cancer Care Unit and that same year, conducted the Situational Analysis of Cancer in El Salvador, and prioritized cancer prevention and control, focusing efforts on building a comprehensive oncology care model. In September 2015, MINSAL approved the “National Policy for Cancer Prevention and Control� to implement cancer prevention and control interventions, reduce the burden of disease and improve patients’ quality of life. All these efforts were brought to fruition with the construction of the NRC. Page 50 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Investments on this achievement consisted namely of procuring specialized equipment, including two linear accelerators (LINAC), one brachytherapy device, one X-ray unit, CAT scanner, basic and ENT equipment. Investment included retrofitting and adapting the infrastructure to accommodate the unit, procuring medical clinic furniture and training medical staff in how to use equipment: training medical physicists and biomedical equipment technicians. The NRC opened in December 2017, is going to deliver outpatient care to those users referred from MINSAL specialized tertiary hospitals. Service packages include outpatient care, radiation therapy treatment, and preparation and recovery areas. The CNTR is the first hi-tech radio therapy service delivered by the Ministry of Health, with investments around 8.0 million dollars. The expected impact from the CNTR operation is an increase in access to quality health care for around 2,000 – 2,200 cancer patients every year and a reduction in related morbidity and mortality, as well as a reduction of family out-of-pocket spending. Other key elements considered for the integral treatment of Cancer was the cobalt bomb for strengthening the Cancer Institute. The Project purchased a cobalt 60 radiation machine (for US$0.9) for cancer treatment that was given to the “Liga Contra El Cáncer de El Salvador�, and is being used for cancer treatment. d) Hospital medical equipment preventive and corrective maintenance program A National Biomedical Laboratory was created to strengthen and sustain the medical equipment investment across different MINSAL health facilities benefitted by the Project, which inter alia, was tasked to verify procured medical equipment and devices have adequate preventive and corrective maintenance, as planned, to meet their expected service life, as determined by the manufacturer. The National Biomedical Laboratory consists of: • Metrology Laboratory: By procuring standard weights, the program would be carried out to verify devices measuring weight, namely weighing devices and scales are accurate. • Quality control testing of radiology equipment used in medical diagnostic: Through the ongoing International Atomic Energy Agency (IAEA) technical cooperation project and equipment procured under the Project, a QC program for different MINSAL analogic and digital X-ray devices may be in place for testing radiology diagnosis equipment accuracy, ensuring doses are kept in conformance with parameters set out for each user. • Electricity/electronic Laboratory: Created to carry out and train BMET staff in clinical engineering practices. • Biomedical Laboratory: Provides technical and metrological support for measuring biomedical parameters across different types of medical equipment installed in the public hospital network for accuracy, and train BMETs in clinical engineering practices for servicing medical equipment adequately, including their proper functioning. • Maintenance: Hospitals will be provided with TA for high-tech and medium-tech medical Page 51 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) equipment maintenance and servicing. Metrology and calibration equipment was purchased, as well as vehicles, tools, training equipment and spare parts and consumables and, also conducted biomedical maintenance training (30 empirical staff and 25 BMETs), for overall amount of US$3’497,000.00 Retrofitting and adjustment of infrastructure to accommodate the national biomedical lab (0.5 million) was successfully completed. An electronic medical equipment maintenance management system is in place and users responsible for health facility maintenance nationwide have been trained. e) Strengthening Community-based Health Work. 14,385 community-based health promoters and committees went through capacity building actions. Communities are empowered with local leaders who pull social communication triggers, facilitating social organization and participation processes. 2,437 health committees have been established, which developed local situational assessments and action plans for promoting health and preventing diseases. f) Priority Health Programs Priority health programs have been reinforced by 2.6 million investments, including procurement of vaccines. These strengthened arbovirus control capabilities, including dengue fever, Zika and chikungunya, translated in mosquito spraying truck, spraying equipment and parts, while youth health promoters across the five health zones were trained in comprehensive community health care. g) Strengthening CKD interventions Interventions have focused on strengthening Integral and Integrated Health Care Networks (RIIS) to upgrade their response addressing chronic kidney disease patients, especially hospitals in the eastern zone, by providing kidney dialysis machines, supply kits and medicine for CKD patients. h) Quality Assurance MINSAL stewardship capacity was strengthened by implementing a QA system for health facility certification by meeting systematized certification program evaluation criteria in terms of characteristics, levels of complexity, social and health realities, attributes, size, technologic level and qualitative/quantitative capabilities of health suppliers. To achieve this, MINSAL developed the health facility accreditation regulatory framework, created and standardized administrative e-tools for self-evaluation, efficient and timely reporting, and trained technical staff in accreditation self-evaluation. i) Medical Emergency System (SEM) The Project strengthened SEM by procuring a digital radio communication system, radio base stations, portable, two-way radio transceivers to link the four system base stations, ambulances and emergency units in the 8 hospitals across the greater San Salvador metropolitan region. Based on the foregoing, hospital doctors, and Emergency Care Coordination Center and ambulance medical staff may have greater communication effectiveness, reporting the patient status and his Page 52 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) evolution onsite and during his transportation to the relevant hospital emergency room. The Project supported the procurement of 29 type “B� ambulances for hospitals and 15 types “C� ambulances for the same number of primary UCSF, totaling US$3.7 million. Medical and paramedical personnel nationwide were trained in community-level emergency medical management. Ambulances will be run by the fleet management system, a software financed by the project proceeds. It consists of implementing an electronic ambulance system for filling out the pre-hospital emergency care and ambulance control sheets. COMPONENT II: Institutional Strengthening a) Single Unified Health Information System (SUIS) The Single Unified Health Information System (SUIS) has strengthened MINSAL processing data capabilities to retrieve information, indicators and vital statistics for informed decision-making. The Project invested US$2.8 million in telecommunication and WAN intranet ownership infrastructure, completing successfully MINSAL Radio Frequency Project Phase One. This has enhanced communications among the Secretary of State, all metropolitan area hospitals, key administration buildings and the alternate site located in El Coyolito, Tejutla, Chalatenango. Furthermore, the Project purchased equipment to set up local area networks connecting 11 hospitals and two regional warehouses and more than 500 new network points were installed. They switched from telephone to free voice over IP by installing telephone exchange servers in 20 hospitals, two regional departments, El Matazano warehouse facility and the National Reference Laboratory. Investments also covered hardware to back up the electronic medical record (EMR) and other SUIS modules. Expanded information capture and user service (family record and clinic record) solutions are available in 28 hospitals and 124 UCSFs. b) National Medicine Policy Through an US$0.9 million investment, the Project supported the Drug Quality Control Laboratory and strengthened microbiological and physical – chemical testing methodologies by procuring reagents and lab equipment, so RIIS medicinal product procurement meets international standards of quality. Specialized hospital pharmacies were also included for the implementation of Good Practices and Rational Use of Medicines (RUM), Health Technology Assessment (HTA) and Pharmacovigilance (PV), providing them with equipment for adequate preservation, storage and transport of pharmaceuticals. Furthermore, 1429 people were trained in RUM, PV and HTA. As a result, all (100%) health zones, SIBASIs and hospitals implement PV activities and monitor adverse drug reactions (ADRs). c) Strengthening National Health Institute (INS). Supported by the Project-funded US$1.6 million investment, the National Health Institute conducted different research works, the following being the most significant: National Health Survey, with a 14,160- household sample size, helped find out trends in health indicators and determinants, with emphasis on childhood and sexual and reproductive health; National Survey of Chronic Non-Communicable Diseases in Adults in El Salvador-ENECA-ELS 2015. This turned El Salvador, in Central America, in a trendsetter for this type of research. Research findings have led to a systemic epidemiological clinical approach program for comprehensive prevention, care and rehabilitation of chronic diseases (NCDs.) Page 53 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) In addition, other key research works include: • Detection of carbapenems in gram-negative bacteria with epidemiological clinical relevance, detected in the National Reference Laboratory in 2017. • Microbiological, and chemical quality of well water used in schools in municipalities adjacent to Rio Lempa watershed, El Salvador 2017. • Identification of arbovirus co-infection cases: Chikungunya, dengue fever and Zika, detected in saline solutions from arbovirus surveillance patients, El Salvador 2017. • External quality assurance assessment of cervical cancer screening conducted by the Cytology Lab, National Reference Laboratory, MINSAL. 2014-2017. • Noroviruses of GI and GII genogroups identified in children aged between 2 and 5 suffering acute gastroenteritis seen in rotavirus surveillance sentinel sites, El Salvador, 2017. • Seroprevalence of antibodies to dengue and chikungunya viruses in dengue surveillance blood samples in 2016. • Prevalence of Salmonella spp in fresh chicken meat sold in authorized retail outlets in four health zones in El Salvador, 2017. By supporting the procurement of lab equipment, reagents and HVAC equipment (US$2.6 million), as well as infrastructure investment (US$8.2 million), the Project fostered the establishment of the clinic laboratory network and provided the National Reference Laboratory with a new operations center. This is now one of the most modern laboratories in Central America, enhancing supply and responsiveness. III. FSSP PROJECT CONTRIBUTIONS TO HEALTH REFORM PROCESS The Strengthening Public Health Care System Project (FSSP by its acronym in Spanish) was implemented under the Health Reform and the Integrated Primary Health Care Strategy. For this, a new health care model was established, implemented through Integral and Integrated Health Care Networks across all healthcare levels. The health policy and reform aimed to expand healthcare access and coverage among populations, prioritizing groups at risk. It called for a health system with strengthened stewardship capacity, health infrastructure, equipment and technology; it also requires human resources with technical and administrative skills needed to develop health programs successfully. Thus, the health reform defined 10 core themes, and the Project contribution is detailed as follows: Axe 1 – Integral Health Care Networks (RIIS): Strengthening of first level units (medical equipment, office furniture) and substantial improvement of the installed capacity in the 30 hospitals in the country that provide services in different areas of care (medical and non-medical equipment and furniture). Health facilities construction. Creation of the NRC for cancer care. Implementation of preventive and corrective maintenance program for hospital medical equipment. Development of an accreditation process for health facilities. Axe 2 – National Medical Emergencies System (SEM): Purchase of 44 ambulances for hospitals and Community health units. Creation of a digital radio communication system. Strengthening of staff in the handling of medical emergencies care and the transfer of critically ill newborns (emergency equipment and training to health personnel of the 30 hospitals in the country. Page 54 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Axe 3 – Medications and Vaccines: Purchase of vaccines for maternal and childcare. Strengthening of microbiological and physicochemical analysis methodologies for quality control at MINSAL laboratory to comply with international quality standards in the acquisition of medicines for the RIIS. Support to the hospitals specialized pharmacies for the implementation of good practices related to the rational use of medicines, evaluation of health technologies and pharmacovigilance. Axes 4 and 6 – Intersectoral Work in Health: Establishment of focal points to help activate social communication, strengthen interventions in communities and facilitate organization processes and social participation. A total of 2,437 health committees were established which generated a community diagnostics and action plan to promote and prevent diseases at rural level. Axe 5 – National Institute of Health (INS): Strengthening of research capacity in different health topics. Increase in the response capacity of the national reference laboratory and the laboratory network with the acquisition of new infrastructure, equipment and reagents. Axe 7 – Human Resources Development in Health: Training of 14,385 human resources among promoters and health committees in capacity for care, through the development of a health-training plan. Strengthening of the capacities of 1429 human resources in the rational use of medication, pharmacovigilance and evaluation of health technologies. Also, the national modernization of human resource knowledge in the area of biomedical maintenance. Axe 8 – Integrated Health Information System (SUIS): Establishment of an in-house WAN intranet telecommunications infrastructure, equipment for building the LAN network, transition from telephony to free VoIP, investment in infrastructure hardware to support the single file and other SUIS modules. Axe 9 – Violence and Health: The clinics of 8 national hospitals were strengthened to provide care for victims of sexual violence with an emphasis on adolescents, through the acquisition of medical equipment and instrumental and clinical furniture. Axe 10 – Environmental Health: Support for 100% coverage of UCSF and hospitals with management, treatment and final disposal of bio infectious waste. The 30 hospitals comply with environmental diagnoses required by the Ministry of Environment and Natural Resources. Page 55 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) IV. US$80 MILLION INVESTMENT OVERVIEW Investment item U$S million % PNA medical equipment, furniture and supplies 3,773,571.87 4.7% Strengthening regional and central warehouses 471,278.24 0.6% Maintenance and minor rehabilitation of PHC facilities 5,673,142.49 7.1% Hospital Medical Equipment, Furniture and Supplies 28,151,813.83 35.2% Cobalt 60 bomb for Cancer Institute 908,103.00 1.1% National Radiotherapy Center 7,209,879.55 9.0% Training of promoters and health committees in capacity for care 178,150.94 0.2% Implementation of preventive and corrective maintenance program for hospital medical equipment 4,356,058.53 5.4% Strengthened priority health programs 2,578,795.16 3.2% Improved chronic kidney disease (CKD) care 1,828,496.80 2.3% Strengthened National Medical Emergency System 5,391,001.41 6.7% Hospital certification and quality assurance 242,415.06 0.3% Improved Single Unified Health Information System (SUIS) 2,834,614.24 3.5% Strengthened National Health Institute 2,473,111.48 3.1% Strengthening of MINSAL Laboratory Network 10,092,313.32 12.6% Support pharmacovigilance system 932,963.18 1.2% Institutional Strengthening 554,234.89 0.7% Project management and monitoring 2,203,351.15 2.8% Total executed 79,853,295.14 99.8% Loan canceled amount 146,704.86 0.2% Total loan amount 80,000,000.00 100.0% Page 56 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) ANNEX 6. SUPPORTING DOCUMENTS REPUBLIC OF EL SALVADOR: Strengthening Public Health Care System Actuarial valuation of ISSS Health Scheme and Occupational Health Risk Scheme, November 2016. AENOR Centroamérica 2018. Informe Consolidado “Encuesta de Satisfacción del Usuario de los Hospitales de la Red de Servicios MINSAL�. Presentación al Ministerio de Salud, Gobierno de El Salvador. Abril 2018. AENOR Centroamérica 2018. Informe Consolidado “Encuesta de Satisfacción del Usuario sobre los Servicios Recibidos en los Establecimientos del Primer Nivel de Atención�. Presentación al Ministerio de Salud, Gobierno de El Salvador. Abril 2018. Amiri, Arshia and Ulf-G Gerdtham, Impact of Maternal and Child Health on Economic Growth New Evidence Based Granger Casuality and DEA Analysis. March 2013. Estrategia Nacional Intersectorial de Prevención del Embarazo en Ninas y Adolescentes 2017-2027, MINSAL 2016. GLOBOCAN 2012, Cancer Incidence and Mortality Worldwide: IARC Cancer Base No. 10 [Internet]. Lyon, France: International Agency for Research on Cancer; 2017. Available from: http://globocan.iarc.fr. Gobierno de El Salvador, 2017. Policía Nacional Civil “Municipios con 0 homicidios del 1 de enero al 31 de agosto del 2017�. Gobierno de El Salvador. Plan Quinquenal de Desarrollo 2010-2014, April 14, 2010. Gobierno del El Salvador, 2017. Informe de Gestión: Gabinete de Gestión Económica y Subgabinete Financiero, 2017. Ministerio de Salud, Gobierno del El Salvador 2015. Dificultades y Estrategias de Mejora en la Ejecución del Préstamo: Proyecto “Fortalecimiento del Sistema de Salud Pública� Convenio de préstamo BIRF 8076-SV, Febrero 2015. Ministerio de Salud, Gobierno del El Salvador, 2018. Logros del Proyecto Fortalecimiento del Sistema de Salud Pública: Viceministerio de Salud. Unidad de Atención Integral a la Mujer, Ninez y Adolescencia. Enero 2018. Ministerio de Salud, Gobierno del El Salvador, 2017. Informe sobre Proyecto Banco Mundial “Fortalecimiento del Sistema de Salud Pública – FSSP, Convenio de préstamo BIRF 8076-SV. Periodo del 2014 al 2017�. Ministerio de Salud, Gobierno del El Salvador, 2017. Plan Estratégico Nacional Multisectorial para el Abordaje Integral de las Enfermedades No Transmisibles. Marzo 2017. Page 57 of 58 The World Bank STRENGTHENING PUBLIC HEALTH CARE SYSTEM PROJECT (P117157) Ministerio de Salud, Gobierno del El Salvador, 2018. Logros del Proyecto Fortalecimiento del Sistema de Salud Pública: Préstamo BIRF 8076-SV, Enero 2018. Ministerio de Salud, Gobierno del El Salvador. Informe de Labores 2016-2017. Ministerio de Salud, Gobierno del El Salvador. Sistema único de Información en Salud (SUIS): BIRF 8076. Ministerio de Salud, República de El Salvador, 2018. Evaluación de Impacto del Proyecto Fortalecimiento del Sistema de Salud Pública. Contrato de Préstamo BIRF 8076-SV, Consultoría No. FSSPSV-428-3CV-CI. Abril 2, 2018. MINSAL 2016. Estrategia Nacional Intersectorial de Prevención del Embarazo en Niñas y Adolescentes 2017-2027, MINSAL 2016. MINSAL. Work Report (Informe de Labores), El Salvador. 2016-2017. PAHO Basic Health Indicators, 2009. Sistema de Emergencias Médicas, SEM. “Fortalecimiento del Sistema de Salud Pública – FSSP� Convenio de préstamo BIRF 8076-SV. WHO Global Health Expenditure Database, 2009. WHO Global Health Observatory and child and maternal health data from United Nations Development Program ―Without Excuses. Let us Achieve the Objectives of the Millennium in 2015: A Plan for Action‖ 2008. WHO Global Health Observatory: Earliest data available (1990) and latest data available (2008). Page 58 of 58