Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004520 IMPLEMENTATION COMPLETION AND RESULTS REPORT P129663 ON GRANTS IN THE AMOUNT OF US$623.9 MILLION TO THE THE ISLAMIC REPUBLIC OF AFGHANISTAN FOR THE AFGHANISTAN - SYSTEM ENHANCEMENT FOR HEALTH ACTION IN TRANSITION (SEHAT) PROJECT January 29, 2019 Health, Nutrition & Population Global Practice South Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 30, 2018) Currency Unit = Afghanis (AFN) AFN 73.28 = US$1 US$0.014 = AFN 1 FISCAL YEAR December 21 – December 20 Regional Vice President: Hartwig Schafer Country Director: Shubham Chaudhuri Senior Global Practice Director: Timothy Evans Practice Manager: Rekha Menon Task Team Leader(s): Ghulam Dastagir Sayed, Mohammad Tawab Hashemi ICR Main Contributor: Joy de Beyer ABBREVIATIONS AND ACRONYMS AF Additional Financing AHS Afghanistan Health Survey AIDS Acquired Immunodeficiency Syndrome AMS Afghanistan Mortality Survey ANC Antenatal Care ANPDF Afghanistan National Peace and Development Framework ARTF Afghanistan Reconstruction Trust Fund BHC Basic Health Center BPET Budget Planning and Expenditure Tracking BPHS Basic Package of Health Services BSC Balanced Score Card CHC Comprehensive Health Center CHW Community Health Worker CIDA Canadian International Development Agency CPF Country Partnership Framework CPR (Modern) Contraception Prevalence Rate CSO Central Statistics Office DA Designated Account DHS Demographic Health Survey EHSRDP Emergency Health Sector Rehabilitation and Development Project EPHS Essential Package of Hospital Services ESMF Environmental and Social Management Framework EU European Union FA Financing Agreement FCV Fragility, Conflict and Violence FM Financial Management FP Family Planning GAVI Global Alliance for Vaccines and Immunization GDP Gross Domestic Product GFATM Global Fund to Fight AIDS, Tuberculosis and Malaria GNI Gross National Income GRM Grievance Redress Mechanism GOIRA Government of Islamic Republic of Afghanistan HCWM Healthcare Waste Management HCWMP Healthcare Waste Management Plan HF Health Facility HIS Health Information System HIV Human Immunodeficiency Virus HMIS Health Management Information System HNP Health, Nutrition and Population HNP GP Health, Nutrition and Population Global Practice HR Human Resources HRCDP Human Resources Capacity Development Project HRITF Health Results Innovation Trust Fund IBRD International Bank for Rehabilitation and Development ICRR Implementation and Completion Results Report IDA International Development Association IDU Injecting Drug User IFR Interim Financial Report IMAM Integrated Management of Acute Malnutrition IMCI Integrated Management of Childhood Illnesses IR Intermediate Result IRI Intermediate Result/s Indicator ISN Interim Strategy Note IYCF Infant and Young Child Feeding M&E Monitoring and Evaluation MDR-TB Multi-Drug Resistant Tuberculosis MICS Multi-Indicator Cluster Survey MOF Ministry of Finance MOPH Ministry of Public Health NGO Non-Governmental Organization NPC National Procurement Committee NRVA National Risk and Vulnerability Assessment PDO Project Development Objective PHD Provincial Health Directorate PHO Provincial Health Office PPA Performance-Based Partnership Agreements PPU Procurement Policy Unit PWID People Who Inject Drugs RF Results Framework RBF Results-Based Financing SBA Skilled Birth Attendance SC Sub-Center SEHAT System Enhancement for Health Action in Transition SHARP Strengthening Health Activities for the Rural Poor TA Technical Assistance TP Third Party THE Total Health Expenditure UNDP United Nations Development Programme UNESCO United Nations Educational, Scientific and Cultural Organization UNFPA United Nations Population Fund UNICEF United Nations International Children's Emergency Fund USAID United States Agency for International Development WB The World Bank WHO World Health Organization TABLE OF CONTENTS DATA SHEET ................................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES............................................................. 5 A. CONTEXT AT APPRAISAL ...................................................................................................... 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ............................. 12 II. OUTCOME .............................................................................................................................. 15 A. RELEVANCE OF PDOs ......................................................................................................... 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ................................................................................. 15 C. EFFICIENCY......................................................................................................................... 28 D. JUSTIFICATION OF OVERALL OUTCOME RATING .............................................................. 29 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ...................................................................... 30 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME..................................... 31 A. KEY FACTORS DURING PREPARATION............................................................................... 31 B. KEY FACTORS DURING IMPLEMENTATION ....................................................................... 33 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ..... 37 A. QUALITY OF MONITORING AND EVALUATION (M&E) ...................................................... 37 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .............................................. 41 C. BANK PERFORMANCE........................................................................................................ 44 D. RISK TO DEVELOPMENT OUTCOME .................................................................................. 45 V. LESSONS AND RECOMMENDATIONS..................................................................................... 46 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ................................................................. 48 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION .............................. 61 ANNEX 3. PROJECT COST BY COMPONENT................................................................................... 65 ANNEX 4. EFFICIENCY ANALYSIS ................................................................................................... 66 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ....... 68 ANNEX 6. SUPPORTING DOCUMENTS (SOME WITH LINKS) ......................................................... 72 ANNEX 7. ADDITIONAL INFORMATION ......................................................................................... 76 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Afghanistan: System Enhancement for Health Action in P129663 Transition Project Country Financing Instrument Afghanistan Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Supplement P152122-SEHAT 25-May-2015 Recipient Executed Activities Additional Financing Organizations Borrower Implementing Agency Ministry of Finance, Afghanistan Ministry of Public Health Project Development Objective (PDO) Original PDO The project development objectives are to expand the scope, quality and coverage of health services provided to the population,particularly for the poor, in the project areas, and to enhance the stewardship functions of the Ministry of Public Health. Page 1 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing P129663 TF-95691 12,000,000 12,000,000 10,556,143 P129663 IDA-H8290 100,000,000 100,000,000 88,071,402 P129663 TF-15005 480,000,000 480,000,000 459,798,325 Total 592,000,000 592,000,000 558,425,870 Non-World Bank Financing Borrower/Recipient 30,000,000 30,000 0 Total 30,000,000 30,000 0 Total Project Cost 622,000,000 592,030,000 558,425,870 KEY DATES Project Approval Effectiveness MTR Review Original Closing Actual Closing P129663 28-Feb-2013 20-Dec-2009 17-May-2016 30-Jun-2018 30-Jun-2018 P152122 25-May-2015 30-Jun-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 29-May-2013 0 Change in Loan Closing Date(s) 25-May-2015 59.94 Additional Financing Change in Results Framework Change in Legal Covenants KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory High Page 2 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 16-May-2013 Satisfactory Satisfactory 3.97 02 26-Nov-2013 Satisfactory Satisfactory 25.26 03 31-May-2014 Satisfactory Moderately Satisfactory 46.64 04 09-Nov-2014 Moderately Satisfactory Moderately Satisfactory 99.53 05 05-May-2015 Moderately Satisfactory Moderately Satisfactory 138.63 06 05-Nov-2015 Moderately Satisfactory Moderately Satisfactory 183.97 07 23-Jan-2016 Moderately Satisfactory Moderately Satisfactory 255.42 08 05-Jul-2016 Moderately Satisfactory Moderately Satisfactory 308.64 09 10-Jan-2017 Moderately Satisfactory Moderately Satisfactory 381.52 10 19-Sep-2017 Satisfactory Moderately Satisfactory 427.00 11 30-Jun-2018 Satisfactory Moderately Satisfactory 524.16 SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Public Administration - Health 25 Health 75 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Human Development and Gender 100 Disease Control 15 Health Systems and Policies 65 Health System Strengthening 40 Reproductive and Maternal Health 15 Child Health 10 Nutrition and Food Security 20 Nutrition 10 Food Security 10 ADM STAFF Role At Approval At ICR Regional Vice President: Isabel M. Guerrero Hartwig Schafer Country Director: Robert J. Saum Shubham Chaudhuri Senior Global Practice Director: Jesko S. Hentschel Timothy Grant Evans Practice Manager: Julie McLaughlin Rekha Menon Ghulam Dastagir Sayed, Task Team Leader(s): Ghulam Dastagir Sayed Mohammad Tawab Hashemi ICR Contributing Author: Joy Antoinette De Beyer Errors in the Data Sheet in FINANCING numbers: (1) SEHAT financing from TF95691 was US$7 million (not $12 million). An HRITF grant (TF 95691) of US$12 million was approved in 2009 as part of the SHARP project; US$7 million was transferred to SEHAT so that the pilot Results- Based Funding project financed by the HRITF grant could be completed and evaluated. The TF95691 closing date was extended to align with SEHAT’s closing date, (incorrectly) recorded as a restructuring of SEHAT on p.2 above. (2) The original amount of financing from the Afghanistan Reconstruction Trust Fund (TF15005) was US$270 million. Total original financing was (7+100+270+30) US$407 million, not US$622 million as in the system-generated table above). Additional financing of US$246.9 million was approved in May 2015, bringing the revised total financing (7+100+270+246.9+30) to US$653.9 million (not US$592.03 million). As paragraph 25 below explains, US$480 million of the US$516.9 approved under the ARTF was released, and Government did not provide the $30 million committed, so total actual available project funding was 7+100+480 = US$587. (3) Final disbursement data in the system also differ from project records (Government and WB) which are as follow: TF 95691 US$6.38 million, TF 15005 US$452,149.068, and IDA: US$90,477.872. Page 4 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Country Background: economy, security, human development 1. More than 30 years of conflict had severely affected Afghanistan’s development, causing severe human losses and displacement, physical and institutional destruction. Ethnic divisions had been sharpened, government and the rule of law weakened. In 2013, per capita income at US$680 (current US$) was well below every other country in the region (Annex 7), and even below the low-income country average (Table 1). The data show a similar picture for other key indicators of health and population well-being. About 36% of Afghanistan’s population of 32 million lived below the national poverty line. Population growth was high at 3%; child malnutrition was a serious development challenge. Life expectancy at birth was only 62.5 years. Despite impressive improvements since 2002, maternal, infant and child mortality remained high. Literacy was amongst the lowest in the world, especially among women. Table 1: Population and Health Indicators 2013, Afghanistan and Comparators Afghanistan Pakistan Bangladesh South Asia Low Income GNI per capita Atlas Method, current US$ 680 1360 1010 1,176 747 Population (million) 32 182 158 1,633 796 Poverty (% pop below national poverty line) 36 30 27 .. .. Population growth rate (% per year) 3.3 2.1 1.2 1.4 2.6 Child (%U5) stunting 41 .. 39 39 38 Urban population % 25 35 32 32 31 Life Expectancy at birth 62 66 72 68 62 Maternal Mortality Ratio (per 100,000 live births) 327 211 (UN) 242 (UN) 228 (UN) 519 (UN) Infant Mortality Rate (per 1000 live births) 79 68 (UN) 33 (UN) 52 70 Under 5 Mortality Rate (per 1000 live births) 103 84 41 54 81 Literacy Rate: Men 15+ 45 69 64 78 67 Women15+ 18 42 58 59 50 Total health spending per capita (US$) 56 32 26 51 36 Health spending as % GDP 8.9 2.6 2.7 3.6 5.7 Source: World Development Indicators; UNICEF (indicated as UN); UNESCO for literacy data for Afghanistan and Bangladesh. Stunting is defined as weight for height below two standard deviations from the norm. Poverty data for Bangladesh interpolates from data for 2010 and 2016. 2. Economic growth had been strong from 2003 to 2012 (9.4% on average), but very dependent on foreign aid, and on agriculture, which accounted for over 50% of GDP and 60% of employment and is vulnerable to adverse weather. Despite several years of strong government revenue growth (around 20%), public spending remained very highly dependent on donor funding. Although it had not yet become apparent, 2013 was a turning point in Afghanistan’s economic and security situation: GDP per capita began to fall (a trend that has persisted), and violence began to escalate, starting a sharp increase in conflict-related deaths and numbers of internally displaced people from 2013 onwards. At appraisal in 2012, these trends were not yet evident; despite continued Taliban insurgency and uncertainty about the impending Presidential election and US intentions to withdraw troops in 2014, there was cautious optimism. However, even then, the level of insecurity was far worse than any other country afflicted by conflict, fragility and violence (CFV) except Syria, as Figure 1 shows. Page 5 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Figure 1: Number of Battlefield Deaths by Year, 2011-2016, Afghanistan, Iraq, Yemen, Somalia, South Sudan Source: Uppsala Data Base. Health sector context 3. Afghanistan had achieved great improvements in the health sector. In 2002, most of the scant health services that existed were provided by Non-Government Organizations (NGOs) with (often inadequate) humanitarian funding. There was little coordination, no standards, and large areas of the country had no modern health services at all. The newly formed government had little capacity to deliver services, and decided that the best strategy was to contract out most service delivery to NGOs, and focus on stewardship: policy and standard setting, planning, coordination and oversight, monitoring and evaluation. This was consistent with World Bank (WB) advice. 4. In the decade between 2002 and appraisal, MOPH leadership and well-coordinated donor support, sound public health policies, innovative service delivery approaches, and careful program monitoring and evaluation had changed a somewhat chaotic situation to one in which NGOs were competitively selected and contracted to provide a well-defined Basic Package of Health Services (BPHS) in lower-level facilities and an Essential Package of Hospital Services (EPHS) in hospitals, with each province covered by a single NGO (or NGO consortium). The Ministry of Public Health (MOPH) delivered BPHS/EPHS services in three provinces including Kabul, supported by a team of individual consultants, called the “contracted in” or “strengthening mechanism” (SM) approach. 5. From the start, the government gave clear and consistent priority to reducing maternal and child mortality, especially in rural areas, and to equitable delivery of health services. The BPHS addressed Afghanistan’s most pressing health challenges, using proven cost-effective interventions that were feasible, affordable, able to be scaled up nationally, in an equitable way, and likely to benefit a large part of the population. The BPHS comprised: (1) maternal and newborn health – antenatal, delivery post-partum, and newborn care, and family planning; (2) immunization and integrated management of childhood illnesses (IMCI); (3) nutrition – treatment of clinical malnutrition, and micronutrient supplementation; (4) control of TB and malaria, and (5) essential drugs required for basic services. Mental health and disability (physiotherapy and orthopedic services) were included, but initially implemented only where financial and human resources permitted, and then became fully part of the BPHS in 2005. The BPHS also defined four types of health facilities making up the primary care system – health post, basic health center, comprehensive health center and district hospital, and the catchment population for each. It clearly defined specific health services to be provided by each type of facility and the types and numbers of staff, equipment, and essential drugs needed to do so. The EPHS was developed in 2005. It specified in-patient clinical and ancillary services, and defined staffing, equipment and drugs needed to deliver the services for district, provincial, and national hospitals. (See Annex 7 for a description of the health system.) 6. Coverage of reproductive and child health services had increased greatly: from just 496 functioning health facilities in 2002 to over 2,000 in 2013; 80 percent of facilities had at least one female staff (MOPH data). Rapidly expanded access to a well-selected basic package of high impact maternal and child preventive and promotive services had resulted in dramatic gains in related outcomes. Between 2001 and 2013, infant mortality fell from Page 6 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 165 to 77 per 1,000 live births, child (under 5) mortality fell from 257 to 97 per 1,000 live births, and maternal mortality fell from an estimated 1,600 to 327 per 100,000. Life expectancy at birth rose from 43 in 2001 to 62 in 2013. (Data from the Afghanistan Mortality Survey 2010 and other household surveys.) 7. Health spending. At appraisal, health spending was 10% of GDP, more than twice the norm in the region. However, Government revenue funded only 6% of total health expenditure (THE), and health’s share of government spending from its own revenues was 4%, similar to allocations in other South Asian countries. Health spending per capita was relatively high at US$56 in 2013 (see Table 1), but 76% of it was private spending, mostly (73% of THE) out-of-pocket spending at the point of service (Afghanistan National Health Accounts 2011/2012). Only 24% of THE -- US$14 per capita - was public spending on health, 75% of which from external sources and 25% from government revenues. 8. Donor support for the health sector. The project built on a decade of substantial international assistance. The WB had provided US$300 million mainly through two health projects: The Health Sector Emergency Rehabilitation and Development Project, 2003-2009, and Strengthening Health Activities for the Rural Poor (SHARP), 2009-2013, and a small HIV/AIDS Prevention project (2007-2012). The United States Agency for International Development (USAID), and European Union (EU) were the other major donors, funding health projects off-budget or bilaterally. The Global Fund to Fight AIDS, TB and Malaria (GFATM) and Global Alliance for Vaccines and Immunization (GAVI) also provided substantial funding. UNICEF and WHO were among the most active of UN technical agencies. Health sector challenges 9. Despite the dramatic gains noted above, much more remained to be achieved. There had been much less progress in family planning and immunization, and vitamin and mineral deficiencies were high among women and children. Child malnutrition was still amongst the worst in the world; about 55% of Afghan children suffered from chronic malnutrition. The MOPH and development partners were also concerned by data from the 2010 Multi- Indicator Cluster Survey (MICS) showing worsened inequality in use of most health services – generally because higher income groups had increased their use of services more rapidly than the poor. Although services in the BPHS and EPHS were (and still are) supposed to be provided free to all, evidence had also emerged of very high out-of-pocket spending, even among the lowest income groups (National Health Accounts, 2011-2012). This reflected use of private providers inside and beyond Afghanistan. There also were rising expectations and demand for quality hospital services and for access to basic health services for those not yet reached. 10. Health care in Afghanistan was hampered by many factors. Within the sector, there were staff shortages especially of trained midwives and female providers, weak capacity in MOPH, and low operating budgets. Exogenous barriers added challenges: the low rate of urbanization (25%) and low population density in much of the country, many mountainous areas, some highly prone to earthquakes, floods, and impassable in winter snow, and limited transportation networks still needing rebuilding and development. High levels of poverty, restrictive cultural gender norms, and very low levels of education among women also inhibited use of health services. 11. Worsening insecurity was a severe challenge. The Implementation, Completion and Results Report (ICR, February 2014) for the SHARP health project implemented between March 2009 and Sept 2013, lists some of the difficulties caused by insecurity: (a) in districts with frequent insurgencies, it was difficult to maintain personnel (especially female workers), and health facility hours of operation were often reduced; (b) monitoring of service providers was restricted in insecure localities; (c) outreach services, especially immunization, were often constrained; (d) disruptions occurred in the distribution of medicines and supplies; and (e) the population, especially women, feared to seek services in times of higher insecurity. Page 7 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) The System Enhancement for Health Action in Transition (SEHAT) Project 12. The “Systems Enhancement for Health Action in Transition” (SEHAT) Project was developed as SHARP approached completion. A three-month overlap in implementation provided good continuity. SEHAT’s design aimed to expand and improve on SHARP, give renewed emphasis to serving the poor and a new focus on chronic malnutrition as an important development challenge, and continue moving from the emergency response of the initial projects to a system-based approach focused on the medium/longer term development of the sector. It also incorporated the elements of the free-standing HIV and AIDS Prevention project “demonstrably most likely to efficiently deliver HIV services” (HIV/AIDS Project ICR p.13) that closed towards the end of 2012. 13. The new project continued the progress SHARP had made towards a sector-wide approach. Under SHARP, each of the three major development partners in health had financed BPHS/EPHS delivery in specific provinces: USAID funded 13 provinces, the EU 10, and the WB 11. There were some differences in the contractual and oversight mechanisms among the donors, but overall, each supported geographic parts of a single system, well- coordinated under clear national policies. Central MOPH functions had been supported by the three partners in a largely complementary way. The new SEHAT project took the important step of establishing the framework for a single financing mechanism for the sector. Instead of the previous arrangement of parallel financing, under SEHAT, the EU would channel its funding through the Afghanistan Reconstruction Trust Fund (ARTF) administered by the Bank, a multi-billion-dollar trust fund with more than 30 contributors, through which most SEHAT funding would be provided. The project thus covered health services delivery in 21 of Afghanistan’s 34 provinces. In the remaining 13 provinces, USAID would continue to finance BPHS/EPHS using parallel financing, but SEHAT’s design made it possible to expand the number of provinces in the project if USAID were to decide to channel its support through the ARTF. 14. Another important way in which SEHAT built on and improved on SHARP was by moving from full lump-sum NGO contracts to contracts in which 20% of payment were conditional on performance. HMIS and survey data during SHARP’s first years found that increases in access and use of services had slowed and even stagnated. With funding from the Health Results Innovation Trust Fund (HRITF), a results-based financing (RBF) pilot in 14 districts began under SHARP (continued and evaluated under SEHAT). When SEHAT was being appraised two years into the pilot, it was showing some promising results. There was accumulating evidence from other countries of potential impact of RBF. SEHAT incorporated and mainstreamed RBF into its design hoping it might help motivate NGOs to regain the stalled momentum in expanding coverage and improving quality of health services. 15. In 2010, the MOPH had revised the BPHS, adding more services at all levels. The new contracts under SEHAT would require delivery of the new expanded BPHS, and thus expand the scope of services provided. The project document noted the need for expanded access to mental health and disability services and especially for nutrition services for mothers, infants and young children. Analytic work led by the WB between 2009 and 2014 on malnutrition as a chronic, broad development challenge, whose causes and impact go well beyond the health sector helped inform the national Nutrition Action Framework 2012-2016 and motivate the increased emphasis on nutrition interventions. Analytic findings and recommendations were discussed with development partners and the government, including the Finance Minister, and supplemented by a Technical Assistance program on implementing interventions to address the causes of chronic malnutrition, starting with a review of the experience of NGOs in delivering BPHS nutrition interventions, to identify bottlenecks and recommend actions to accelerate scaling-up and improve the quality of BPHS nutrition interventions (Afghanistan Nutrition Solutions Series, P122781). 16. The earlier projects had allocated modest budgets (US$10 million and US$16.5 million) for strengthening the MOPH central and Provincial levels. These budgets paid for Technical Assistance (TA), contractual staff to fill key Page 8 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) gaps, transport and communications for the MOPH, and training, workshops and conferences, as well as Monitoring and Evaluation costs. SEHAT planned a significantly increased focus on developing the capacity of MOPH’s regular structures in Kabul and in the provinces, and improving MOPH systems to better perform its health sector stewardship role. Stewardship encompasses many areas: budgeting and planning, financial management, procurement, policy and other decision-making, standards setting, supervision and support, pharmaceutical regulation, private sector engagement, quality assurance, human resources, health promotion, health information systems, surveys and surveillance, and accountability – including handling patient complaints. This was a daunting agenda, much of it complex, and clearly could not all be taken on at once. MOPH did not yet have a roadmap or sequenced plan for this system strengthening. Knowing that MOPH top leadership might well change after the 2014 election and given the pressing urgency of ensuring continuity in BPHS/EPHS delivery, detailed decisions on the systems strengthening agenda were deferred, and the design of that part of the project was left flexible. Theory of Change (Results Chain) Page 9 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Project Development Objectives (PDOs) 17. The PDO is consistently stated in all documents: “The objectives of the Project are to expand the scope, quality and coverage of health services provided to the population, particularly to the poor, in the Project areas, and to enhance the stewardship functions of the Ministry of Public Health.” (Financing Agreement p4, PAD p.4). Key Expected Outcomes and Outcome Indicators There are five expected outcomes in the PDO, as follows: (1) whether the scope of health services expanded; (2) whether health services coverage expanded (numbers or percentages reached); (3) whether the quality of health services improved; (4) whether the poor benefited from improvements in service provision; and (5) whether the project enhanced the stewardship functions of the Ministry of Public Health (MOPH). Page 10 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 18. Project areas: The project initially covered BPHS/EPHS delivery in 21 of the country’s 34 provinces, and provided for the possibility of adding more provinces (as noted above), which it did, covering provision of these services in the entire country. Other activities were country-wide throughout the project. The key indicators to be used to assess each part of the PDO are as follows: Outcome Indicators 1. Scope • Proportion of children under age five years with severe acute malnutrition who are treated (range of services • Percentage of pregnant and lactating women who receive counselling on feeding infants provided) and young children 2. Coverage • Number of deliveries attended by skilled health personnel • Contraceptive prevalence rate (any modern method) (number or • Health facility utilization rate: consultation per person per year percentage of • Number of pregnant women receiving antenatal care during a visit to a health provider people reached • Injecting drug users reached with Needle/Syringe program by services) • Immunization rates for children under 2 years (not in Results Framework) 3. Quality • Score on balanced scorecard items examining quality of care items in SCs, BHCs and CHCs (lower level facilities) • Scores on the hospital balanced scorecard items that examine quality of care in public hospitals delivering EPHS • TB treatment success rate 4. Gains by Poor • Births attended by skilled health personnel among lowest income quintile (up to 2015) • PENTA3 coverage among children aged between 12 -23 months in lowest income quintile • Antenatal care coverage - at least one visit among lowest income quintile (up to 2015) 5. Stewardship • Accreditation for procurement of goods and works achieved and maintained (by MOPH) • NGO contracts for BPHS/EPHS services delivery signed and properly managed per agreed (Please see Annex timeline 7 for WHO • Health Information Statistics annual report prepared and disseminated definition of • Number of national hospitals with full budgetary autonomy stewardship) • Proportion of budget from the Provincial Budgeting Initiative executed • Capacity of drug quality control laboratory developed • Proportion of MOPH core development budget executed Note: Although the PDO lists scope, quality and then coverage, there is considerable overlap between scope and coverage so they are discussed before turning to quality, which applies to all services. Components Component 1: Delivering and Improving the Basic Package of Health Services and Essential Package of Hospital Services (Original estimated cost: US$307 million, Revised cost: US$593.4 million, Actual Cost: US$520.5 million) 19. (i). Delivery of a Basic Package of Health Services (BPHS) and Essential Package of Hospital Services (EPHS) in project area provinces, by NGOs contracted by MOPH under Performance-based Service Contracts. (ii). Support for direct service delivery of BPHS and EPHS in three provinces including urban Kabul, by the MOPH, “contracting in” management services in the provinces. (iii). Activities designed to improve access to, and quality of, the BPHS and EPHS, including training of community midwives and community nurses; funding contracted services specifically for marginalized populations such as prisoners and nomads; and HIV/AIDS prevention and harm reduction services for Injecting Drug Users (IDU). (Instead of IDU, “People Who Inject Drugs” is now the preferred term, per UNAIDS advice, rather than IDU.) (iv). Completing the results-based financing pilot interventions in 14 provinces and pilot evaluation that began under the predecessor project and which was showing promising Page 11 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) results; refine and mainstream RBF through the performance-based contracts with the NGOs funded under part (i) of Component 1; conduct an in-depth evaluation of RBF to inform its future direction under SEHAT. 20. The Financing Agreement (FA) description of component 1 (iv) introduces a little confusion by including “testing innovations in the hospital sector aimed at strengthening performance” (p.4) as if it is a distinct sub- activity. In fact, it is clear from a close reading of the PAD (and other project documents), the RBF scheme and its mainstreaming applied to all contracts to deliver health services both at lower levels of the system (BPHS) and in hospitals (EPHS). The PAD and FA are fully consistent in all other respects. Component 2. Building the stewardship capacity of MOPH and system development (Original estimated cost: US$90 million, Revised cost: US$38.5 million, Actual cost: US$14.920 million) 21. This component intended to fund activities to strengthen the capacity of MOPH to fulfill its stewardship functions; such as: (a) strengthening sub-national government capacity (notably provincial health departments); (b) health care financing analysis, and testing new financing models; (c) reinforcing pharmaceutical regulatory mechanisms and quality assurance systems; (d) analysis of and engagement with the private sector; (e) designing and piloting a provider payment mechanism to enhance accountability and performance of public hospitals; (f) strengthening MOPH human resources; (g) strengthening MOPH's governance and social accountability framework; (h) strengthening health information systems, use of information technology, and MOPH's capacity to carry out the monitoring and evaluation of BPHS and EPHS through contracting a third party (TP) evaluator, specific activities to include establishing demographic and health surveillance, integrated disease surveillance; and setting up a data warehouse; (i) strengthening MOPH's health promotion unit and supporting its program and behavior change campaigns; (j) improving MOPH's fiduciary systems, including upgrading the financial management system, simplifying payment procedures, training finance and internal audit staff, accreditation of the procurement department, piloting electronic government procurement; and strengthening procurement capacity in provinces. The component was loosely defined at appraisal, deferring decisions until the MOPH could discuss the plans of other development partners (to avoid duplication and ensure coherent, consistent activities) and then submit detailed proposals to the Bank during implementation for consideration for funding under SEHAT. Component 3. Strengthening Program Management (Original estimated cost: US$10 million, Revised cost: US$22 million, Actual cost: US$13.604 million) 22. This component funded incremental operating costs for MOPH at central and provincial levels; selective short- term technical assistance (TA) to fill pressing capacity gaps in MOPH and to transfer knowledge; and a Gender Assessment. These activities were to be coordinated with the “Capacity Building for Results Facility” (P123845) project that was funding staff positions and training for MOPH and other Ministries. Although the distinction between Components 2 and 3 is not very clear from the PAD summary description or from the FA description of component 3, the differences are clear from the list of activities to be funded under each component. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Additional Financing (AF) 23. AF of US$246.9 million was approved by the Bank Country Director and ARTF Management Committee on May 23, 2015, to fund service delivery in the 13 provinces financed by USAID, and USAID’s “on-budget” health system development support. This required a change in allocation: most (US$223.9 million) of the AF was allocated to Component 1, US$17 million to Component 2, and the remaining US$6 million to Component 3. 24. As noted above, health services delivery under SEHAT initially covered 21 provinces where about half of the population lived. It had been anticipated during project preparation that this might change, and project documents provided for the possibility of covering additional provinces. In November 2013, USAID decided to change from Page 12 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) directly financing health services in 13 provinces in parallel with SEHAT, to instead channel much of their funding for health through the multi-donor Afghanistan Reconstruction Trust Fund (ARTF) administered by the Bank. USAID continued to manage the existing contracts in the 13 provinces until June 2015, but SEHAT began supporting MOPH in the onerous work of procurement for 21 new contracts during the first year of the project. Amendment of the ARTF Grant No. TF015005 Agreement 25. The original ARTF Grant Agreement for SEHAT (TF015005) was signed on August 27, 2013, with three expenditure categories (Table 2). A first tranche of US$100 million was released in October 2013, a second tranche of US$100 million was released in April 2015. The TF015005 Grant Agreement was Amended and Restated on June 13, 2015, noting that AF of US$246.9 had been approved for the SEHAT project in May 2015, and releasing a third tranche of US$100 million (Table 2). Another amendment on April 25, 2017 increased the grant funds by US$120 million, bringing the total released to US$420 million (final column of Table 2). A final tranche release of US$60 million brought the total grant released from the ARTF to US$480 million (of US$516.9 approved under the ARTF). Table 2: Amendments to eligible expenditure categories (Section IV.A.2 Schedule 2 of the Agreement) Grant Allocation Amounts (in US$) Category Original June 2015 April 2017 August 2013 (1) Goods, works, non-consulting services, consultants' services, Training 98,000,000 55,692,000 55,692,000 and Incremental Operating Costs for the Project (excluding Parts 1(d); (2)(c)(ii) and (iii); and 2(h)(ii) of the Project) (2) Goods, non-consulting services, consultants' services and works for Part 1,250,000 -0- -0- 2(c)(ii) and (iii) of the Project (3) Goods, non-consulting services, consultants' services and works for Part 750,000 -0- -0- 2(h)(ii) of the Project (4) Goods, works, non-consulting services, consultants' services, Training (not in 244,308,000 364,308,000 and Incremental Operating Costs for the Project (excluding Part 1(d) of the Original) Project) TOTAL AMOUNT 100,000,000 300,000,000 420,000,000 Note: In all categories, 100% of expenditures were financed. In 2015, categories (2) and (3) were dropped (unneeded, see Annex 7), remaining funds in category (1) were moved to (4) which replaced categories (1), (2) and (3). Revisions to Results Framework (RF) 26. No changes were made to the PDO. The AF in May 2015 made the following changes to the RF: (i) One PDO indicator was changed from the percent of births to women in the lowest income quintile attended by skilled personnel, to the total number of births (deliveries) (in all quintiles) attended by skilled personnel (see below for the new target). This Core HNP Sector Indicator used routinely collected HMIS data, rather than requiring a household survey to identify women from the lowest quintile. (ii) Similarly, the Intermediate Result indicator for antenatal care coverage and its target were changed to also make it a Core Sector Indicator (see table below for details of the original and revised versions, baselines and targets). (iii) At appraisal, no baseline data were available for the proportion of children under five years with acute malnutrition who were treated. A baseline of 24% and targets for each year of the project (30% for year 1, increasing by 5% each year to 50% in year 5) were added in the Revised Results Framework presented in the AF paper (p.19). (iv) The target for the Intermediate Indicator: “NGO contracts for BPHS/EPHS service delivery signed and properly managed as per agreed timeline” was increased from 27 to 48 to reflect the additional contracts to be financed under the AF. Since the project was designed in the Page 13 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) context of a sector-wide approach from the beginning, baselines and targets for all other indicators were based on national data and did not need to be changed under the AF. Original Indicator Revised Indicator Baseline Target Baseline Target Percentage of births among lowest income quintile Number of births (deliveries) attended by skilled health attended by skilled health personnel personnel (Number, presumably per year) 15.6% (2010/11) 35% 429,305 (HMIS 2012) 566,683 (+32%) Antenatal care coverage – at least one visit among Number of pregnant women receiving antenatal care during a lowest income quintile visit to a health provider 26% (2010/11) 40% 723,614 (HMIS 2012, adjusted)* 926,226 (+28%) Note: * The 2012 HMIS data were adjusted for missing data and over-reporting after verification. Revised Components No components were formally revised. Other revisions Budget Reallocation Across Components 27. During the Mid-Term Review (2016), it was agreed to reallocate funds unlikely to be used for Component 2 to Component 1. This did not require a formal amendment as there was only one expenditure category. Dated Implementation Covenant Deleted 28. A dated implementation covenant that required MOPH to upgrade its financial database within the first six months of the project was deleted in the AF Amendment letter signed on June 13, 2015 by the Minister of Finance and the Country Director, because of a change in plans, and because it was not important for achieving the PDO. The covenant had been stated as follows in Schedule 2, Section 1.B.3 in the Original FA: “The Recipient shall, by not later than December 31, 2013 and in accordance with terms of reference acceptable to the Association, upgrade the MOPH's budget planning and expenditure tracking database to facilitate maintenance of subsidiary books of accounts and generation of periodic reports.” (p.7) 29. The plan at appraisal was a simple upgrade to the existing Budget Planning and Expenditure Tracking (BPET) database in MOPH to meet the needs of the finance unit. Instead, MOPH proposed a web-based system connecting multiple locations as part of a broader system development. The BPET service provider was unable to carry out an upgrade of such scope and complexity. It was agreed that MOPH would procure off-the-shelf accounting software (Quick Books) to meet the immediate needs of the finance department. Quick Books was in place and being used to maintain basic accounting records by 2016. Change in Project Coordinator 30. WB and MOPH agreed that the size and nation-wide scope of the SEHAT project and its provision of a platform for coordinating donor financing to the health sector made it appropriate that the project coordinator be the Minister of Public Health or designee instead of the Deputy Minister for Policy and Planning (AF paper, p. 11). No other changes were made to institutional arrangements. Rationale for Changes and Their Implication for the Original Theory of Change 31. The rationale for each change is explained above. The changes had no implications for the Theory of Change. Page 14 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) II. OUTCOME 32. This assessment of outcome does not use a split rating methodology (per ICR guidelines) as the PDO was not changed. The quality and consistency of data from various surveys used in assessing outcomes are discussed below. A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 33. The project PDO is completely aligned with the current World Bank Country Partnership Framework (CPF) for FY17-FY20 (October 2, 2016), with WBG corporate priorities, and with Afghanistan’s national health and development strategies. It is fully relevant to the most pressing health needs of the country. 34. SEHAT directly supports pillars 1 and 3 of the current CPF and clearly aligns with one or more of the specific objectives of each. Pillar 1 is to “Build strong and accountable institutions to support the government’s state- building objectives and enable the state to fulfil its core mandate to deliver basic services to its citizens, and create an enabling environment for the private sector.” The last part of the PDO – to enhance the stewardship functions of the MOPH – aligns with both Objective 1.1: Improve public financial management… and Objective 1.2: Improve performance of key government ministries. The CPF Pillar 3 aims to “Deepen social inclusion through improved human development outcomes and reduced vulnerability amongst the poorest sections of society.” SEHAT is discussed as an important part of Bank efforts towards Objective 3.1: Improved human development (p. 26). Enhanced scope, coverage and quality of health services, especially for the poor, also aligns with Objective 3.3: Improved government and community capacity to manage and respond to natural disasters, which requires responsive, good quality, readily accessible health services. 35. Also, the purpose of enhanced health services is to help improve health outcomes as a healthier population can better contribute to Pillar 2: support inclusive growth. The project’s particular focus on services for the poor aligns directly with the emphasis on inclusion in the CPF. In the CPF discussion of the possible need for selectivity and prioritization of development interventions, access to basic social services is listed among those of highest priority (CPF p. 10). 36. Given that WBG priorities for country support take careful account of national priorities, and that the WB health team has a decade-long and well-respected engagement in health policy-making in Afghanistan, it is not surprising that the PDO also aligns well with national priorities as outlined in government documents such as the National Peace and Development Framework (ANPDF), September 2016; and the National Health Policy 2015- 2020. Investing in human capital development – notably health and education systems, and improving access to health care, particularly for women and children’s health – is high on the Government of the Islamic Republic of Afghanistan’s (GOIRA) agenda. 37. The PDO also aligns well with the Health, Nutrition and Population Global Practice (HNP GP) Priority Directions for 2016-2020, especially the focus on “Service Coverage – ensuring equitable access to affordable, quality HNP services”, as one of three key paths for assisting countries to accelerate progress towards Universal Health Coverage. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 38. This assessment evaluates achievements in each of the five parts of the PDO, per the original design, which included five expected outcomes. Page 15 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) PDO Part 1: Expand the scope of health services provided – Rating: High 39. As seen in the results chain, the project expanded the scope of services in two ways: (1) by including additional services in the BPHS that NGOs were contracted to provide, with particular focus on nutrition services; (2) by upgrading facilities, for example by procuring diagnostic or other equipment and additional staff, so that they were able to provide additional services, as well as by adding a nutrition counselor to the required staff list for all health facilities in 18 provinces with the most severe child malnutrition. 40. New services. The BPHS had been revised in 2010, adding interventions to address Afghanistan’s high rates of malnutrition; introducing primary eye care as a part of the BPHS to be gradually implemented through training, primary eye care services and referral services; and giving new emphasis to mental health and disability rehabilitation services, defining staffing, training, services, supplies and equipment, to be implemented as funding became available. The additional requirements included a physiotherapist in each district hospital, and a psychosocial counselor (nurse) at comprehensive health centers with funding for mental health interventions. 41. The services delivery contracts under SEHAT required the NGOs to provide the expanded BPHS package. The PDO indicator: “Proportion of children under age five years with severe acute malnutrition who are treated” reflects the priority given to addressing malnutrition in the expanded scope of services under SEHAT. This indicator greatly surpassed its target (baseline 24%, target 55%, actual value 77% in 2018). The IRI for nutrition services: “Percentage of pregnant and lactating women who received counselling on infant and young child feeding” also exceeded its target (baseline 0, target 50%, actual 58% in 2018). (Annex 1 shows percentage calculations comparing target and actual changes.) 42. The gains in nutrition services are not solely attributable to SEHAT. The complementary program of analysis and technical assistance led by the WB was mentioned above (P122781, financed by the South Asia Food and Nutrition Security Initiative Trust Fund). A USAID-funded project supported curriculum development and training-of-trainers for Nutrition Counselors, and, in the final months of the project, community-level training. The EU provided 17 million euros for innovations to improve nutrition interventions at health facilities and in communities. UNICEF played an important role, working with MOPH to assess the impact of the Nutrition counselors, support community- level monitoring and behavioral interventions, and develop national guidelines on detecting and treating acute malnutrition, known as Integrated Management of Acute Malnutrition (IMAM). UNICEF, WHO, World Food Programme (WFP) and other Nutrition Cluster partners helped develop annual strategic plans to operationalize and implement the IMAM guidelines, train community health workers, evaluate the program, and address bottlenecks. UNICEF noted that SEHAT support for the BPHS and EPHS provided the platform that enabled smooth and rapid roll- out when the IMAM approach was adopted. National IMAM guidelines were endorsed in early 2014; by 2017 the program was in place in all 34 provinces, and 78% of districts (313/399) had at least one component of the IMAM program. Outpatient management of severe acute malnutrition was operational in 1,028 sites in 34 provinces, including 50% of all health centers and 63% of all regional, provincial and district hospitals. Program performance was above the global standard of at least 75% cure rate in 2017, which was met in all but 6 provinces. Performance monitoring showed significant improvements in coverage and quality of services between 2015 and 2017 (Qarizada et al 2018, Figure 5, see para. 64 below). 43. New and Upgraded Facilities. No indicator was defined for this aspect of expanded scope, but much was done. In 2016 MOPH created 292 new Sub-Centers and 48 new Family Health Houses. New facility sites were guided by a systematic analysis to identify “white areas” without health facilities. The project upgraded 259 health facilities to enable them to provide additional services. In many cases, the upgrades enabled facilities to meet the criteria for a higher category of facility and they were reclassified from, say from a Basic Health Center (BHC) to a Comprehensive Health Center (CHC) (See Annex 7 for a description of the health system and the catchment area, staffing, and Page 16 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) services defined for each facility level.) Many of the upgrades added a laboratory. Equipment procured under the project was in line with the increased list of essential drugs and equipment required under the 2010 BPHS revision at all categories of health facilities up through district hospitals. PDO Part 2: Expand the coverage of health services – Rating: Substantial 44. Coverage of services was tracked by two PDO indicators -- births attended by skilled personnel, and prevalence of modern contraception use (contraception prevalence rate - CPR); and three IRIs – the number of women who received antenatal care during a visit to a health provider; the number of health facility visits per person per year; and IDUs reached by a needle/syringe program. Child immunization is also a good coverage indicator and is considered here as well as in PDO part 4 (as defined in the RF, it focuses on the lowest income quintile). 45. The number of births attended by skilled health personnel more than doubled over the five years of the project from 429,305 in 2012 to 890,240 in 2017/18, greatly surpassing the target of 566,683. The target was set too low – it was surpassed in 2015/16. However, the accomplishment was substantial: in addition to the number of attended deliveries more than doubling, the standard measure of coverage --the percentage of all deliveries attended by skilled personnel (SBA)-- increased from 47% in 2012 (AHS) to over 58% in 2015 (AHS), a 23% increase. Although coverage was a little lower in 2018 at below 56%, much of the gain was maintained (Figure 2). (Although successive AHS surveys are generally comparable, it should be noted that AHS 2015 data were collected by Silk Route, the TP sub-contractor, whereas KIT conducted the 2018 AHS itself, so there may be some systematic differences between the surveys.) Figure 2: National trends in maternal health care indicators, 2003-2018 SEHAT Project began Source: AHS 2018, Figure 3.3-1, p.81. 46. The number of women who received antenatal care (ANC) during a visit to a health provider (the revised form of the indicator that originally focused on ANC among the lowest quintile) surpassed its target, reaching 1,461,781 in 2018, more than double the 2012 baseline of 723,614, and far above the target of 926,226, which aimed for a 28% rise. As with attended deliveries, although the target was too low, performance was strong. As seen in Figure 2 above, data from successive AHSs show that ANC coverage rose steadily throughout the project: from 54% of all pregnant women in 2012, to 61% in 2015 and approaching 64% in 2018. Figure 3, taken from the 2016/17 ALCS, shows that the increase in ANC coverage (at least one examination) accelerated during SEHAT (CSO 2018, p.194). Figure 3: Numbers of women reporting at least one ANC examination by a skilled provider, 2013-2017 Project start Source: CSO 2018, Figure 9.8, p.194. Page 17 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 47. The contraceptive prevalence rate (CPR) – the percentage of married women aged 15 to 49 using any modern method fell short of the target of 30%, and may have declined, although different data sources are inconsistent (use of FP is considered a private matter; MOPH has concerns about the accuracy of survey data). The MICS 2010/11, used as the baseline, reported 18% use of modern methods (the PAD lists 19.5% but that included traditional methods). Other sources during the project are as follows: MOPH 2014 HIS Factsheet 2014-- 13.8%, AHS 2015/16 - - 18.2% (recomputed, the original estimate was 20%, AHS 2015, p.103), and AHS 2018 -- 16.3%. All household surveys in Afghanistan have found considerable unmet need for contraception; that is, married women who want to prevent or delay pregnancy. The 2015 AHS estimated unmet need at 25%. Women said that 89% of all births and pregnancies in the past 5 years had been wanted, 6% were mistimed and 4% were unwanted. Of all currently married women, 24% had wanted to wait at least 2 years before becoming pregnant, and 26% said they wanted no more children, a total of 50%, compared to the CPR of 18% (AHS 2015 p.91). More than 80% of women had not talked about contraception with a community health worker (CHW) or health provider in the previous year, suggesting numerous missed opportunities. The AHS 2018 asked fewer questions about family planning (FP), and asked about current pregnancies rather than future desires, so comparisons with 2015 cannot be made. In 2018, 17% of currently pregnant respondents would have preferred to wait (a mean of almost 4 years) and 8% had not wanted more children, a total of 26% (AHS 2018, p.79). 48. Low use of modern contraception is deeply entrenched in cultural norms in Afghanistan, reinforced by (still) high infant and child mortality, poor knowledge of contraception among men and young women, and women’s low levels of education, agency, and labor force participation. Uptake of modern family planning is a complex behavior change and takes more than information and supply of services. It is usually accompanied by social change, urbanization, rising incomes and falling infant and child mortality, not all of which are present in Afghanistan. On the other hand, decreases in CPR are unusual, and tend to indicate problems in FP services provision (such as decreased access, quality, product supply, or choice of methods) or weak policy and/or provider focus on FP (e.g. Haiti and Egypt, Allman et al 1987, Radovich et al 2018). A comprehensive Family Planning Assessment in 2015 for the MOPH (technical input from UNFPA, funded by USAID) found a mixed picture – positive policies and commitment by MOPH leadership, but little focus from provincial offices, some health system gaps particularly the need for more female providers, occasional stock-outs of FP commodities, uneven quality of FP services, and little effort to create demand. Since then, MOPH has begun working with the FP2020 initiative towards the national goals of 30% CPR and reducing unmet need to 10%. MOPH sees the main issues as social barriers to use, poor privacy in many health facilities, and too little provider time and confidence to offer FP services. In 2016, contraceptive implants were added to the Essential Drugs List, and a program of provider training and community mobilization began to be implemented. The BSC shows good availability of FP supplies in BPHS facilities, with some decline in 2018 (Table 3). Table 3: SC data (2015-2018) National median Availability of Family Planning Supplies in BPHS facilities Supply Item 2015 2016 2017 2018 Condoms 92 92 91 87 Oral contraceptive tablets 95 95 94 88 DMPA (Injectible) 93 93 91 85 IUD 86 86 91 85 Overall average 92 92 92 86 Source: BSC data provided by KIT. DMPA injections protect against pregnancy for 12 weeks. 49. The number of health facility visits per person per year surpassed the target according to the HMIS data, increasing from a baseline of 1.6 to 2.2 (target of 2.0). The KIT verification team were unable to trace and verify all of their (small) sample of patients listed in facility registers, and suggested a corrected HMIS figure of 1.9, which would be 75% of the target increase. However, the 2015 data were also above the 2.0 target, and patient Page 18 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) tracing was made increasingly difficult in the later years of the project by insecurity and population displacement as violence escalated in large parts of the country. Given the many other indicators of increased use of health services – notably the doubling of the number of attended births noted above, it seems likely that the utilization increase was achieved. 50. The project did not report data for the indicator percentage of Injecting drug users (IDUs) reached by needle/syringe program, because, despite assiduous efforts, no willing qualified bidder could be found to conduct the Integrated Bio-behavioral survey (IBBS) needed to estimate the denominator (total number of IDUs). (The term People Who Inject Drugs - PWID - is preferred to IDU.) Data on PWID reached by needle/syringe programs indicate good performance in serving this particularly vulnerable and underserved group (Table 4), but not enough to meet the target of almost doubling coverage from 27% to 50%. The number of PWIDs was estimated in 2009 at around 19,000, and in the 2012 IBBS, at around 16,700 in 4 cities (Kabul, Herat, Mazar-i- Sharif, Jalalabad). In the 2012 IBBS, 64% of the 1,163 PWID interviewed had heard of HIV risk reduction services and 58% had ever used them. This varied greatly across the 5 cities in the survey: use was not reported for Jalalabad, it was 85-99% in the 3 other large cities, but only one PWID in the small town of Charikar had heard of or used HR services. The MOPH’s 2017 GFATM grant continuation request stated that only 13% of PWID were covered by HR services. Table 4: Coverage of Harm Reduction Services for People Who Inject Drugs Service Period of Service Provision Jan-June July-Dec May-Oct Nov 2016- July-Dec 2015 2015 2016 June 2017 2017 PWIDs provided with regular HR services 3,431 3,526 3,654 4,187 3,951 Clean needles/syringes distributed (millions) 0.69 1.44 1.30 1.9 0.83 Used needles/syringes collected for safe 0.63 1.29 1.18 1.7 0.75 disposal Source: MOPH Semi-Annual Reports to WB on SEHAT (2015-2017). Note: all reporting periods were for 6 months, except Nov 2016-June 2017 which was 8 months. 51. NGOs were contracted under the project to provide harm reduction (HR) services to PWID in 5 cities, including in prisons; a comprehensive HR package for IDUs was implemented in targeted provinces, and providers trained. The HR package followed best practice. It included HIV and other STD testing and treatment, opioid substitution therapy, clean needle/syringe provision and exchange for used ones, counselling, condom distribution, and basic medical services. SEHAT provided 3,400-3,950 PWIDs with regular HR services throughout the project (Table 4), more than the total number of PWID reached by the dedicated HIV/AIDS Prevention project over five years (2008-2015, as reported in the ICR). The MOPH 2017 grant request to the GFATM reported an increase in the number of people on Opioid Substitution Therapy from 70 to 207, an achievement of SEHAT. 52. Nutrition services were successfully scaled up country-wide by MOPH and BPHS/EPHS implementers through SEHAT, with support also from UNICEF, WHO and other partners, as noted above. The number of acutely malnourished children provided with life-saving treatment almost doubled between 2014 and 2017, increasing from 236,121 In 2014, to 315,890 in 2015, 400,488 in 2016 and about 457,000 in 2017 (data reported by UNICEF). 53. Immunizations all show a consistent trend – a very strong increase in coverage during the first years of the project (2012 to 2015), some apparent loss of ground between the 2015 and 2018 AHSs, but still ending the project with markedly higher coverage than at the start of the project (Figure 4). The rise in the percentage of fully immunized children is especially notable, because it requires repeated visits to a health facility. (The immunization indicator focused on the lowest income quintile and is discussed under PDO Part 4 performance.) Page 19 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Figure 4: National trends in child immunizations, 2003-2018 Source: Household surveys. 54. Taken together, the project indicators and other data show substantial expansion in the coverage of health services under the project, and in many cases, an acceleration in the rate of increase compared to the years immediately before SEHAT. This is consistent with the results chain – the project supported activities to increase the number of facilities and numbers of female staff, ensured continuity of contracts with NGOs to provide services and built performance incentives into the contracts, as well as activities and incentives to increase the quality of care which are documented below. These activities were expected (and intended) to increase coverage. PDO Part 3: Enhanced Quality of Health Services is rated Modest 55. One PDO indicator and two IR indicators in the RF measure quality. Two of these use the composite score on the Balanced Score Cards (BSC). Afghanistan is one of only a few low income countries regularly measuring healthcare quality. Data collected by the independent Third Party (TP) summarize performance of health facilities in each province in delivering the BPHS, and of hospitals delivering the EPHS. The BSC data were collected from random samples of at least 25 facilities in each of the 34 provinces, a total of between 725 and 807 facilities (generally increasing over the years), including observations of patient-provider interactions and exit interviews with 7,000-8,000 patients and interviews with 2,400-3,000 health workers and 489-1150 CHWs. The reports provide average scores by province for each of 26 BSC domains/areas, and the median score across all provinces. These median scores are used for the RF indicators. The BSC provides policymakers, health managers and staff with data on trends, strengths and weakness relative to other provinces, as a basis for efforts to improve quality and services. This section draws on the 2018 BSC reports compiled by KIT, which include data from prior years. Enhanced quality of services in health centers and health posts – modest to substantial 56. The PDO indicator, Score on the balanced scorecard examining quality of care in SCs, BHCs and CHCs, improved steadily from the 2011 baseline of 55% to reach 64% in 2017, and then fell to 59% in 2018, below the target of 70% (partially achieved). However, this median composite average score across provinces masks a great deal of variation among and within provinces, and in trends in scores on different items and areas within the scorecard. The paragraphs that follow look more closely at the components of the BSC. The general pattern is of steady improvement during most of the project, beginning to fall off a little in some areas in 2017, and a clear decrease in scores on many areas/items in 2018, but usually still remaining well above the scores at the start and in the early years of the project. Although the summary indicator at its highest achieved 60% of the target increase and ended the project at 27% of the target increase, a disaggregated look at the BSC data provide evidence of some substantial improvements in the quality of BPHS services during the project, as described below. Page 20 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 57. Client satisfaction and perceived quality of care (cleanliness of facility, toilets, waiting time, opening hours, cost, privacy, respect from health workers, explanation of illness and treatments, time spent with patient, convenience of getting prescribed medicine) rose from 75% in 2012/13 to around 80% in 2015, 2016 and 2017, and then dropped to 70% in 2018. In no year were there differences in client satisfaction across wealth quintiles in any province. 58. The BSC showed improved performance on staffing in the health centers, which is crucial to the quality of care. The indicator measuring the extent to which Sub-, Basic- and Comprehensive Health Centers met the staffing guidelines improved significantly. In 2011/12 and 2012/13, the median average score was only 24-25%, and fewer than 20% of provinces met the upper benchmark. In 2015, 2016, and 2018, the median rose to 32-35%, and was 29% in 2017. Between 40 and 60% of provinces met the upper benchmark, and 90-100% of provinces scored better than the lower benchmark in all 4 project years when these data were collected. (The upper and lower benchmarks are cutoff points for the top and bottom quintiles of provincial scores for the item in 2011). 59. The median average scores on the 36 questions to health workers on their work satisfaction remained steady around 65-66%. Similarly, 20 questions about worker motivation consistently scored 71-72%, although the percent of provinces scoring above the upper benchmark fell from above 40% in in the initial project years to below 30% in 2018. The 2018 fall probably reflects delays in salary payment: the percentage of provinces where workers’ payment was current rose from 72% in 2012/13 to 90% in 2016 and 87% in 2017, but was only 50% in 2018. 60. The indicator on whether staff had received job-related training in the past 12 months rose from 9% in 2012/13 to a steady 15-17% in all 4 BSC rounds, including 2018. The questions testing health worker knowledge of how to manage common health conditions and illnesses had a consistent median average score of 62-70%, but dropped substantially in 2018 to 53%. 61. Interaction with patients. Providers were observed interacting with patients, and scored on whether they greeted patients and asked age, reason for visit, nature and duration of complaint and previous treatment, perform a physical examination and ensure privacy. The score on this indicator rose from 80% in 2012/13, to 83% in 2015 and 87-89% in all later years, with steady increase in the percentage of provinces meeting the upper benchmark to almost 90%, but a fall to 60% in 2018. An indicator that observes 8 items on how well providers communicate important information to patients also improved steadily from 33% in 2012/13 to 63% in 2017, with some falling- off (to 54%) in 2018. The indicator of whether or not providers spent at least 9 minutes with each client had very low and declining levels during the project (from 12% falling to 3%, and rising to 18% in 2018 – the only area in the BSC that improved in 2018). 62. Equipment, drugs and supplies availability, laboratory functionality, clinical guidelines, and status of infrastructure. The indicator for availability and functionality of 23 items of equipment in SHCs and BHCs and 26 items in CHCs, rose steadily from 75% in 2011/12 and 81% in 2012/13 to above 90% in 2017 and 2018. All provinces met lower benchmarks in all years from 2015 on, and percentages of provinces that met upper benchmarks rose to above 80% in 2017, with a slight fall in 2018 to just below 80%. Availability of pharmaceuticals and vaccines also improved from just below 80% at the start of the project to 80-85% in all BSC rounds from 2015-2018. CHC laboratories are scored on the ability to perform 16 tests (pregnancy, HIV, TB etc); the score rose from 71% in 2012/13 to 79-82% in all other years. Availability of various clinical guidelines also improved steadily from 79% in 2012/13 to reach 84% in 2017, with a slight falloff to 86% in 2018. While still leaving much room for improvement, the score on the condition of 10 items of infrastructure (water, heat, electricity, toilets, walls, doors, roof, windows and grounds) also showed improvement from 62-61% in 2012/13 and 2015, to 64-69% in 2016, 2017 and 2018. 63. A new BSC item added in 2015 to assess health post staffing, repairs, supervision and HMIS reporting, as well as CHW satisfaction, motivation, activeness, and having a kit, equipment, supplies, protocols and guidelines, had a Page 21 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) very stable median score of 50% in all years, but a steady fall in the percentage of provinces scoring above a low- end benchmark from over 90% in 2015, to 85% in 2016 and 2017, and only 60% in 2018. 64. Improvement in the quality of nutrition services was noted above: performance monitoring of the outpatient IMAM program showed significant improvements in coverage and quality of services between 2015 and 2017 (Qarizada et al 2018, Figure 5, reproduced below). These improvements are attributed to country-wide training (by UNICEF and WHO) of NGO health workers (funded by SEHAT) responsible for delivering nutrition services, including CHWs, hiring of more nutrition counselors, and successful integration and strengthening of nutrition services in the BPHS/EPHS. Figure 5: Improved Quality of Nutrition Services, 2015 to 2017 Source: Qarizada et al. 2018. Improvement in quality of hospital services - High 65. The indicator for Score on the hospital balanced scorecard that examines quality of care in public hospitals delivering EPHS surpassed its target, rising from a 69% baseline to 78% (target was 77%). Of the 34 sub-domains in the hospital BSC, more than three quarters improved (21 showed improvement during all or most of the project and 5 improved for at least part of the project), and just under one quarter (8 of 34) showed disappointing results, with gains that were not sustained and greater falls in score or little/no change. These 8 areas were client satisfaction, staff motivation and provider knowledge scores, communications and transport, infrastructure, and HMIS, hospital autonomy, and security. The areas that showed the strongest improvement were: moving from 50% in 2011/12 to 100% in all other years for hospitals not charging user fees, meeting staffing norms, staff management and staff satisfaction, having women care providers, and salaries being up-to-date except in 2018, equipment functionality, laboratory and X-ray, the record system, hotel services, safety precautions, and “female friendly facilities”, functioning of standing committees, client counseling, and following biohazard precautions, management team, equipment management, and local financial management, gender equity among patients, and compliance with MOPH policies and local laws. (Annex 7 includes a table of scores for all years of the project for all 34 sub-domains, and a description of the items included in each.) 66. The TB treatment success rate infers quality of care from an outcome. The goal was to increase this from 89% to 90% and maintain 90%. This ICR’s judgement is that the goal was not met. HMIS data and ISRs report no improvement in 2015, a decrease in 2016, and 90% in 2018. This is not consistent with the MOPH six-monthly reports to the WB, which report treatment success as consistently below the target: 2014: 78%, 2015: 88% (AHS), 2016: 80%, and 2017:74%. SEHAT’s role in TB was limited to funding the BPHS/EPHS which included TB treatment, while several other development partners supported projects specifically to improve TB detection and treatment. Page 22 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) In 2017, WHO reported that TB cases and deaths were increasing, and that multidrug-resistant TB (MDR-TB) had emerged as a serious challenge. (http://www.emro.who.int/afg/afghanistan-news/tuberculosis-burden-increases- in-afghanistan-with-over-60-000-new-cases-every-year.html) This helps explain the fall in treatment success rate. GFATM and UNDP funding and support from Médecins Sans Frontières are expanding capacity to treat MDR-TB from one hospital in Kabul to specialized facilities in 4 other cities as well. 67. Overall, with the TB indicator not met, the hospital quality indicator surpassed, and the BPHS quality indicator partially met and with many areas of substantial quality improvements, the part of the PDO that aimed to improve the quality of health services is rated as modest, although an argument could be made for a rating of substantial. PDO Part 4: “Particularly to the Poor” – Rating is Substantial 68. The PDO included the phrase “particularly to the poor”. A literal interpretation would be that the poor should benefit at least as much, if not more than others from the expanded services. However, the realities and correlates of poverty make this an unreasonable standard. Many poor people face formidable barriers to accessing health care, including cost, distance, transport, and education/information. For example, 33% of the lowest income quintile but 70% of the highest quintile in Afghanistan live within 30 minutes travel time of a health facility; 17% of the bottom two quintiles but only 3% of the top quintile live more than 2 hours travel from a health facility. The result is that when services expand, the poor tend to benefit less than other groups. This often results in large and increasing disparities in use of health services and in health outcomes as services expand (well documented by Gwatkin et al 2005, Yazbeck 2009 and others). Only when coverage reaches high levels, or if aggressive efforts are made to target and reach the poor, do inequalities tend to narrow (Houweling 2010, Victora et al, Gwatkin, Yazbeck). In light of this, it seems more reasonable to assess the extent to which the project expanded health services “particularly to the poor” using a broader lens than whether the lowest quintile gained at least as much as others from expanded services. 69. Two PDO indicators (SBA and PENATA3 coverage) and one IR indicator for ANC focused specifically on the poorest quintile, although the SBA PDO indicator was revised in 2015 to refer to the whole population. 70. The original PDO indicator Percentage of births among women in the lowest income quintile that were attended by skilled health personnel almost fully achieved its target for 2015, rising from the baseline of 16% (2010/11 MICS) to 34.8% (AHS 2015) very close to the target of 35% (Figure 6, blue line). Performance fell to 30% in the AHS 2018 but remained well above the baseline, but this indicator was revised in 2015 to cover the whole population instead of only the lowest quintile. Although SBA among the poorest quintile remained fairly low, it was far better in 2015 than in Nigeria in 2013, and the disparity between the lowest and highest quintiles was much smaller in Afghanistan (Figure 7). In 2015 Afghanistan had almost “caught up” to Pakistan’s 2013 poorest quintile SBA coverage. The disparity in SBA across quintiles in Afghanistan narrowed substantially over the project, with large increases among the lowest three quintiles, a significant achievement (see Equity Analysis table in Annex 4). 71. The intermediate indicator At least one antenatal care visit among women in the lowest wealth quintile almost doubled from a baseline of 26% to 48% in 2015 and dropped just a little to 46% in 2018 (Figure 6), well above the project-end target of 40% (target surpassed). Page 23 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Figure 6: Trends in selected maternal health services, 2003-2018, Lowest wealth quintile Source: AHS 2018, Figure 2.3-2, p.88. Figure 7: Percentage of births among women in the lowest and highest income quintiles attended by skilled health personnel, Afghanistan 2015, Nigeria and Pakistan 2013. Source: Afghanistan AHS 2015, Nigeria DHS 2013, Pakistan DHS 2013. 72. The indicators focus on the lowest quintile, but that does not include all of the poor. The percent of the population living below the national poverty line rose from 36% at the start of the project to 55% in 2016/17 (Table 1 and CSO 2017). The rural population is perhaps a better measure of the situation of the poor even though it excludes the urban poor and includes the rural non-poor. Among rural women, ANC and SBA improved throughout the project (Figure 8). Between 2010/11 and 2018, ANC coverage improved by 46% and SBA coverage improved by 63% to levels that were well above the targets set for the SBA and ANC indicators for the poor. Figure 8: Trends in selected maternal health services, 2003-2018, Rural Source: AHS 2018, Figure 2.3-3, p.88. 73. The PDO Indicator PENTA3 vaccine coverage among children aged 12-23 months in the lowest quintile rose from the baseline of 29% (2010/11 MICS) to almost double in 2015/16 (58%), well above the year 3 target, but then fell back to 45% in 2018 (AHS), 52% of the target increase (of more than doubling to reach 60%) (partially Page 24 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) achieved), but still 45% above the baseline. Rates for other immunizations were higher for the lowest quintile, and had much smaller disparities across wealth groups than the PENTA3 vaccine (AHS 2018, p.34, see Annex 7, Figure A7.1). Compared with Pakistan and Nigeria in 2013 (FCV countries with much higher incomes), in 2015 Afghanistan had far smaller gaps between the highest and lowest wealth quintiles in PENTA3 coverage and under 5 mortality (Figures 9 and 10). Figure 9: Figure 10: Source: Compiled by B. Loevinsohn, using DHS data for 2013 for Nigeria and Pakistan, and DHS 2015 for Afghanistan. 74. In summary, the PDO for SBA (dropped in 2015) met its 2015 target, the PDO for Penta3 coverage among the poorest quintile partially achieved its target, and the IRI for ANC among the poorest quintile surpassed its target. Beyond the indicators, five factors show that the project was highly successful in providing enhanced health services particularly to the poor: (1) The disparities in access, use and coverage across quintiles are much smaller than in other countries affected by fragility, conflict and violence and with low levels of female literacy (even with much higher incomes), and tended to improve during the project, avoiding the worsening so often seen as services expand. (2) It is unusual and admirable that patients in the lowest quintiles were just as satisfied with health services as those in the top quintiles throughout the project (as seen in the BSC data). (3) The dramatic increase in the percent of the population living below the national poverty line during the project makes the rural population a better proxy for “the poor” than the poorest quintile, and services improved particularly for the rural population. (4) The project established many new health centers and used mobile services to deliver care in so-called “white areas” where there had been no service delivery, and to disadvantaged groups including displaced populations. (5) The bulk of the project funded the BPHS and EPHS which were specifically chosen to address the health needs of the poor and to be delivered in an equitable way, and resisted pressures to allocate more of the limited health budget to tertiary services, which tend to be far less “pro-poor” than primary care. These factors together justify the rating of Substantial for this part of the PDO. Are the enhanced scope, coverage and quality results attributable to SEHAT? 75. SEHAT’s national funding of health services at all levels up to and including provincial hospitals means that most or much of the documented increases in the scope, coverage and quality of services are largely attributable to the project. There is clear evidence of project support for the activities undertaken to achieve these results (as shown in the results chain), and also of the results themselves. Of course, SEHAT was not the only funder or source of support in the health sector. Financial and/or technical contributions (some substantial) also came from the GFATM, Global Vaccine Alliance, UNICEF, USAID, the EU, WHO, UNDP, UNFPA, Japan International Cooperation Agency, the Canadian International Development Agency, and the Agha Kahn Foundation. Many interventions were fairly small scale or of too short a duration to have made a significant difference in national data. Some of the Page 25 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) NGOs contracted to deliver the BPHS received additional grants and technical support that helped them achieve reported increases in scope, coverage and quality. SEHAT provided a unified platform for country-wide delivery of health services, well guided by clear, strategic MOPH policies. It greatly facilitated implementation of other projects and initiatives, and supported the important verification, monitoring and evaluation activities of the Third Party (discussed in the M&E section) that improved accountability, transparency and the reliability of national data. The counterfactual without SEHAT would have been a severely underfunded and far less coherent health sector. PDO Part 5: Enhance Stewardship Functions of MOPH – Rating is Modest 76. One PDO indicator and six IR indicators relate to enhanced stewardship of the MOPH. The PDO indicator was that the MOPH receive and maintain accreditation for procurement of goods and works (as distinct from procurement of services, which accounted for most project expenditures). This was not met, and one of the last Aide Memoires notes that MOPH capacity to procure goods needed to be improved. However, procurement of goods and works were a small part of the project, and the much more important procurement of services was performed well. The MOPH has long been accredited to procure services with no threshold limit through its Grants and Service Contracts Management Unit (GCMU), and exceeded the all-important IRI that 48 (revised up from 27) “NGO contracts for BPHS/EPHS service delivery [be] signed and properly managed as per agreed timeline”. This included one year when new contracts had to be issued unexpectedly, because the National Procurement Committee (NPC) required new contracts to be issued instead of allowing the contracts with NGOs that were performing satisfactorily to be extended, as the project had planned. This NGO contracting was essential to health services delivery in 31 of the 34 provinces, and, in the later years of the project, was accomplished within five months (less than half the time of earlier contracting rounds). 77. Five other IR indicators also related to Stewardship. (1) The proportion of the MOPH core development budget that was executed surpassed annual targets in every year of the project, rising from a baseline of 54% in 2013 to 87% in 2017, well above the target of 75%. This is particularly strong performance given that the overall government development budget execution was only 54% in 2016 and 67% in 2017 (Noori 2017, p.170, and Fiscal Performance Improvement Support Project P159655 ISR Nov 2018. The budget process was reformed substantially in 2018, aiming at more realistic allocations and much greater execution, so 2018 is not comparable with earlier project years.) The indicators (2,3) “Proportion of budget from the Provincial Budgeting Initiative executed” and “Number of national hospitals with full budgetary autonomy” became inapplicable because the National Unity Government that took office after the 2014 elections decided not to move ahead with the decentralization policies that had been planned. (4) Annual Health Information reports, weekly surveillance reports, and periodic national survey reports were produced and disseminated including on the MOPH website. The final IR indicator for the stewardship part of the PDO, (5) “Capacity of drug quality control laboratory developed” (at least 70% of an annual plan implemented) was met, with almost all of the activities in the annual plan completed. 78. In summary, the PDO indicator for Stewardship was not met, two of the six IR indicators were surpassed, two were met and two became inapplicable. Implementation of Component 2 was lackluster, although MOPH notes that of 65 agreed deliverables, 30 (46%) were accomplished, 15 (23%) were partially achieved, 11 (17%) overlapped interventions in off-budget projects and/or projects managed by other government sectors, 7 (11%) were dropped or failed to be accomplished, and 2 (3%) deliverables were carried forward to the Sehatmandi project (details are in Annex 7). Moreover, reviewing MOPH’s stewardship performance over the past five years against WHO criteria (see Annex 7) provides a positive assessment. MOPH policies are sound and based firmly on global evidence, and respond well to the most pressing national needs. Health policies and programs are consistent and coherent, and there is strong MOPH leadership, with no indication of development partners pursuing their own agendas, or exerting undue influence. National policy documents articulate well the importance of health and health services for all. The consistent and sustained focus on basic and essential services is evidence of MOPH’s ability to manage Page 26 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) health sector resources in ways that contribute to the achievement of agreed policy goals. Moreover, unusual effort and resources are put into monitoring and evaluation, including verification and publication of data. There are of course gaps in MOPH’s stewardship performance, but it is better than in many countries with comparable and higher levels of income, including countries not affected by fragility, conflict and violence. Justification of Overall Efficacy Rating of Substantial 79. The overall achievement of the PDO is rated Substantial, considering that part 1 of the PDO is rated High, parts 2 and 4 are rated Substantial, and parts 3 and 5 are rated Modest. Higher ratings could be justified for parts 3, 4 and 5 if more weight were given to the full range of available evidence on the performance of the project and less to the number of indicators whose targets were met. The rating for enhanced quality of health services is a good example. One of three indicators was surpassed, one was partially met and one decreased from the baseline. However, rigorous measurement of quality through the BSC and other evaluations show strong evidence of substantial improvement in the quality of services during most of the project – an achievement that is considerably better than modest. 80. Many indicators in the household and facility surveys declined in 2018. This is judged to be partly a measurement methodological issue and partly a decline in performance. In late 2017, KIT terminated the contract with their sub-contractor, Silk Route, and took on all data collection themselves. KIT hired surveyors with more education, and trained them to apply the criteria strictly. This may explain some of the decrease, but two factors are likely to have caused a fall in use of services. As noted above, the BSC showed a large fall in the percent of health workers whose pay was current (explained below in Section III), and there was a substantial increase in violence in many parts of the country, especially in 2018 around the Parliamentary election and in anticipation of the 2019 Presidential election and the possibility of negotiations with the Taliban. It is not possible to disaggregate or quantify the effects of these factors on health services. However, despite the reversal of some of the project gains towards its end, there is robust evidence that the project achieved its PDO and expanded the scope, coverage and quality of health services to the population, especially the poor, and that the MOPH continued to exercise good stewardship, under extraordinarily difficult circumstances. SEHAT enabled the government to keep its strong focus on rural areas and cost-effective basic services that address Afghanistan’s most prevalent causes of death and illness – especially among the poor. 81. An additional indicator of the success of the project is the impact on infant and child mortality: estimates using the 2018 AHS data show continued improvement during SEHAT. (The 2018 AHS collected information on too few maternal deaths to enable a reliable estimate of maternal mortality.) Table 5 shows that post-neonatal mortality (deaths after the first month but within in first year of life), child mortality (deaths of children aged 1 to 5) and under 5 mortality (death before the fifth birthday) all fell during SEHAT (AHS 2018, p.96) (although less than in the previous five years). Table 5: Early childhood mortality rates, five-year periods, 2004 – 2018 Mortality Rates Neonatal Post-neonatal Infant (before 1st Child Under-5 (first month) (Days 31-364) birthday) (Ages 1-4) years During SEHAT 2014-2018 22.98 18.25 41.23 8.74 49.61 2009-2013 22.44 22.64 45.08 10.08 54.69 2004-2008 26.53 26.63 53.16 17.09 69.34 Source: AHS 2018, p.96. Rates are deaths per 1,000 live births except for child mortality which is deaths per 1,000 children surviving to age 1. Page 27 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) C. EFFICIENCY Rating: Substantial 82. The gold-standard measure of efficiency of health interventions is the cost-effectiveness ratio (CER). Benefits are expressed in natural units such as deaths and years of disability averted (these are added in the most commonly used health benefit metric – disability-adjusted life years, or DALYs). An intervention is typically considered highly cost effective if the cost per DALY averted is less than the country’s per capital GDP, and cost-effective at less than three times the per capita GDP. Annex 4 shows cost effectiveness estimates for core BPHS/EPHS interventions: pregnancy care and skilled birth attendance, modern contraception, treatment of severe acute malnutrition, immunization, and HIV/AIDS prevention (harm reduction) for PWID, ranging from just US$26 per DALY to US$143 per DALY – less than one third of Afghanistan’s 2016 GDP per capita, far above the “highly cost-effective” threshold. 83. As part of SEHAT project preparation (and to assess the preceding project), the MOPH carried out a cost analysis of the BPHS in 2012. The average per capita cost of the BPHS was US$2.57, with a range of US$1.44 (district hospitals) to US$4.56 (health sub-centers). This was well below then-current cost estimates of US$5-10 per capita to provide a basic package of services for the health-related Millennium Development Goals. Analysis for the new Sehatmandi project estimated the cost of delivering the BPHS/EPHS at about US$200 million per year. Although we do not have an updated per capita cost estimate, dividing this total cost by the current population of 36 million provides a rough cost estimate of US$5.7 per person per year, still at the bottom end of the (dated) Millenium Development Goals for health cost estimate. 84. There are several reasons that BPHS and EPHS services delivery continues to be cost-effective. The set of services in the packages was selected on the basis of global evidence on the most cost-effective, affordable, equitable, and feasible interventions for the conditions that contribute most to Afghanistan’s burden of disease and preventable deaths. These criteria also apply to the nutrition, mental health and disability services added to the BPHS financed under SEHAT. With regard to technical efficacy, the services included in the BPHS and EPHS are consistent with international best-practice based on available evidence, and clinical guidelines have been developed to help health workers deliver care in cost-effective, efficacious ways. 85. Competitive bidding for contracts helped manage costs, while performance incentives and regular monitoring helped ensure that cost-consciousness was not at the expense of the quality and reach of services. An earlier comparison had found that province-wide contracts were more efficient than contracts for smaller service areas, so SEHAT continued to issue contracts for provinces, and to limit any contractor to no more than two provinces in any round of contracting, to preclude being stretched too thin. (NGO contracts were done in two phases, so each NGO could hold contracts for up to four provinces.) The provisions and intention of the contracts gave substantial managerial autonomy and decentralized decision-making to managers on the ground, who are best placed to react to local needs and constraints. Contracting out allowed the MOPH to focus on roles they are uniquely able to undertake, such as planning, standard setting, regulation, and public health functions. Contracting an independent Third Party to undertake rigorous M&E enabled a strong focus on measurable results. However, it is recognized that MOPH technical units and provincial health departments have not done enough to help manage the contracts and boost performance of the NGOs. 86. SEHAT also increased efficiency by including HIV/AIDS prevention services, which had previously been supported through a separate project. This saved all the administrative cost (for both the MOPH and WB) of an additional project. Moreover, only the elements from the HIV/AIDS project that were “demonstrably most likely to efficiently deliver HIV outcomes – harm reduction services for PWID and regular surveillance of HIV in key populations ... support for the pioneering and unique work providing health services for prison inmates” were retained for support under SEHAT (HIV/AIDS Prevention Project ICRR, p.13). A sensible decision was made to close Page 28 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) the one dedicated site that had provided opioid substitution therapy (OST) under the stand-alone HIV project because of very high cost, low utilization, and continued strong political opposition. Instead, this service was included in the contracts of the four NGOs hired to provide HIV/AIDS services to PWIDs as part of a package of harm reduction services. The NGOs were able to triple coverage of OST without drawing attention to the service. 87. The benefits to the population of increased access and use of health services during the project include productivity gains from better health, reduced loss of productivity and costs of care when ill, and reduced mortality. A good additional indicator of efficiency that also supports the logic of the results chain by linking project activities to coverage increases, is the finding from the Afghanistan Living Conditions Surveys that the cost of travelling to the nearest health facility fell substantially over the project (see Figure 9.2 from the ACLS 2016/17 report, CSO 2018 p.183, Annex 7). “Over time, the costs for transport to obtain health services has come down. For instance, while in 2011-12 the average one-way trip to a district or provincial hospital was 368 Afghani, it was 277 Afghani in the 2013-14 survey and only 146 Afghani in the ALCS 2016-17. The reductions in costs were in the same order of magnitude for the other types of health services. The most likely reason is that with the expansion of the health system, distances to the nearest health facility have come down which consequently reduced cost.” (CSO 2018, p. 183). Improvements in roads and a fall in the price of oil also may have contributed. 88. Implementation efficiency was strong. Although Component 2 was considerably delayed and smaller scale than originally planned, the project completed most planned activities and disbursed 94% of the total available funding, with no extension of the closing date. Disbursement was higher than expected during most of the project, which is unusual in the health sector. This efficient implementation was maintained despite the disruptions of the 2014 elections and delays in establishing the new government; the turnover in MOPH leadership in the initial years; the large increase in geographic scope and funding of the project in 2015; and the very difficult challenges posed by the escalating violence. Contracting of NGOs was somewhat slow initially but became more efficient over the project. The one major procurement that could have been done more expeditiously was the contracting of the Third Party for M&E, which began in August 2013, but did not result in a signed contract until January 2015. However, this delay had little impact on project implementation apart from delaying one payment of the performance-related part (20%) of the NGO contracts in 2015. The contractor, the KIT Royal Tropical Institute of the Netherlands (hereafter KIT) was able to complete all the required deliverables, with one exception. The plan had been for least three rounds of quality testing of samples of drugs from facilities as part of the quality assessment. However, the first round took much longer than anticipated because of the need to send samples out of the country for testing, and so it was agreed to suspend the tests until a more efficient process could be arranged. The very small cost of project management – US$13 million for a project of US$623.9 million, or just 2%, is another indicator of SEHAT’s efficiency. D. JUSTIFICATION OF OVERALL OUTCOME RATING 89. The overall outcome rating is Satisfactory, with High Relevance, Substantial Efficacy and Efficiency. The project achieved substantial gains in expanding the scope, quality and coverage of health services, especially for the poor, with better equity than in many countries. Outcome dimension Rating Relevance High Efficacy Substantial Efficiency Substantial Outcome Rating Satisfactory Page 29 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 90. The project had a strong emphasis on maternal and child health services, as noted, resulting in substantial benefits particularly for women, as seen in the data on increased coverage and improved quality for pregnancy- related services cited above, and the new nutrition services which improved nutrition outcomes and health. 91. The status of Afghan women, low literacy levels, and cultural restrictions are major barriers to women’s access to health care. The enhanced coverage, scope and quality of services, training of midwives and community health workers, training and deployment of about 1,800 female nutrition counsellors, and hiring of more women health workers under SEHAT all had particular benefits for women. The goal of ensuring at least one female health worker in every health facility was largely achieved, although the extreme difficulty of retaining female health workers in many areas resulted in high turnover and periodic vacancies, especially in facilities in insecure, remote, and rural areas. Being able to see a female health provider is essential for many Afghan women to be able to receive care. SEHAT appears to have been able to close any remaining gender gap in access to care: the AHS 2018 found no significant differences in treatment-seeking between female and male respondents, and well below 1% of respondents who had been sick but not sought care said that the reason was the lack of a female provider. Nor were there any gender differences apparent in data on illness and treatment among young children. 92. Within the MOPH, SEHAT funded three additional staff for the Gender Department, and four months of support from an international consultant, to help strengthen the Department. (Other partners/projects have also funded staff on fixed term contracts.) As a full Department, Gender has higher status in MOPH than most other ministries, which tend to have only a Gender Unit. However, the staff lacked confidence and/or capacity to overcome the dauntingly difficult environment, and the modest progress accomplished during the project relied heavily on work done by consultants. Six specific achievements are worth noting: (1) the following Gender Policy Statement in the Afghanistan National Health Policy 2015-2020: “It is the policy of the Ministry of Public Health to ensure that all planning, budgeting, implementation and evaluation processes at each level of the health system and all projects and programmes of the Ministry are rights based and sound from a gender perspective. The prevention of, and response to, gender-based violence features particularly highly in the work of the Ministry. … It is also the policy of the Ministry to see more women in senior positions in the Ministry and as programme and project managers.” (2) Standard Operating Procedures (SOPs) for dealing with sexual harassment, for reviewing all programs and operations with a “gender lens”, and for assessing hospitals from a gender perspective (such as whether there is a secure and private room where women can change and rest while at work); (3) Training and awareness sessions in MOPH central and provincial offices; (4) a second officer was hired to work in the MOPH gender-based violence (GBV) Grievance Redress System; (5) efforts to improve GBV case management and referral were rolled out in 16 provinces (this work is continuing); (6) with help from UNFPA, a GBV treatment protocol was developed and introduced in the main provincial hospitals. 93. The new Sehatmandi project (currently under implementation) is building on this progress through a proposed additional clause in NGO contracts that requires all staff to be trained on how to provide health services to those who have been subjected to GBV, including care, counseling, referral as needed, and reporting as per protocol. Institutional Strengthening 94. The project did much to strengthen the capacity of the Public Nutrition Department through financing key positions and upgrading it to Directorate level in the MOPH structure, which gave nutrition higher priority within Page 30 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) public health services. In addition to the institutional strengthening that resulted from Components 2 and 3, there was considerable learning-by-doing in the course of the project, from interacting with the project team over the years, the annual health policy seminars that each focused in depth on selected issues relevant to the project or ongoing policy discussions, and from participating in the Bank’s “Flagship Course on Health Systems Strengthening and Financial Sustainability”. The latter provided opportunities for MOPH teams to exchange ideas and experiences with teams from other countries, in addition to formal learning from WB and other global experts. During the Flagship course, country teams applied each day’s content to an important challenge in their country; the Afghan team worked diligently and enthusiastically each evening, and produced one of the best posters summarizing their work. The National Statistical Office benefitted from working with survey experts from the Royal Tropical Institute team. There were also learning events arranged by the Bank which brought the MOPH officers responsible for social accountability, gender and procurement together with their counterparts to learn “best practices” across Ministries, and from experts in other countries. Mobilizing Private Sector Financing 95. Some of the NGOs, especially those with International reach and partnerships, were able to mobilize additional grant funds directly from private sector (non-profit) funders such as the GFATM. Poverty Reduction and Shared Prosperity 96. Attention to poverty reduction and shared prosperity is an explicit part of the PDO which gives specific attention to enhancing services for the poor in particular. The project provided services to previously unserved or underserved parts of the country and population. The strong focus on basic health services enhanced the project’s contribution to reducing poverty, since these are the services most likely to be used by the poor (as well as those with higher incomes). Improvements in health that resulted from enhanced health services are likely to have increased productivity – especially important where agriculture dominates the economy as it does in Afghanistan. The project also generated employment by training and deploying thousands of community midwives, community nurses and nutrition counsellors throughout the country. Furthermore, most of the NGOs under Component 1 were local, or included local partners. Without recent data, it is not possible to judge whether SEHAT has helped reduce Afghanistan’s high out-of-pocket spending on health care. Other Unintended Outcomes and Impacts 97. No significant unintended outcomes or impacts in Afghanistan have been identified. However, the project offers a strong example for other conflict-affected countries and fragile states of effective service provision despite formidable obstacles. The survey data from households and health facilities and well-verified routine service data supported by the project could be a treasure-trove for researchers in Afghanistan and elsewhere. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 98. There was continuity in the project team, and a good balance of national staff with deep understanding of local conditions and international staff with broad experience. The project Task Team Leader and several other team members are seasoned national health experts, who have been part of the Afghanistan health sector since the beginning of WB lending in 2003. Their decade of project experience and deep understanding of the country (technical, political, cultural and contextual) informed the project design and preparation work. They are respected by government and DPs. There was good continuity of other team members as well, and deep local knowledge was Page 31 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) supplemented by staff and consultant experts with regional and global experience. This strong team contributed to a highly relevant design, with many “best practice” features, that has stood the test of time. 99. The design and preparation work built on and learned from the experiences and challenges during the previous projects. Several changes were made in the design and implementation arrangements for SEHAT, in response to the experience and lessons learned during implementation of the preceding SHARP project. An important example was that NGO contracts were changed from lump sum to part lump-sum and part performance- based as an incentive to maintain and continue performance improvements. This also drew on the results of a Results-Based Financing pilot implemented in 14 provinces during the last two years of SHARP, whose preliminary results showed indications of increasing coverage, quality and equity of service use. The SEHAT team devised new procedures for procurement of minor works and equipment, and for complaints handling, to replace procedures that had proved ineffective under SHARP, and also agreed with MOPH that Technical Departments would take on more responsibility for project oversight in their technical areas and would develop proposals for strengthening their functionality. Although the new procedures did not all prove successful, they were thoughtful efforts to improve and strengthen MOPH. Aspects of earlier projects that had worked well were retained, notably contracting- out to NGOs; contracting-in management support for the three provinces where services are delivered by the MOPH; and unusually strong arrangements for monitoring, data collection, analysis and reporting. 100. The project design was focused and simple. In a country where so much needs to be done to strengthen the health system and improve health outcomes, the team and MOPH maintained a clearly prioritized focus on the most important services, and kept the project design simple: the main component for contracting service delivery and M&E; a component of flexible design to enable MOPH to pursue activities to strengthen stewardship functionality and capability as plans were developed; and a small component to support project management. The PDO, project design, components, implementation and M&E arrangements were well aligned, consistent and complete, and provided flexibility to expand the geographic scope of the project if necessary. The project was clearly described in the PAD, with one exception. Component 2 was not expected to include all of the areas or activities listed; these were intended as an illustrative menu, only some of which would be funded when (if) the relevant Departments developed clear, acceptable proposals. The PAD (mis)used the word “including” in referring to the list of possible activities, instead of “such as” and did not clearly explain that activities in some but not all of the areas listed were likely to be funded. An alternative more critical view of component 2 is that rather than being deliberately flexible, it reflected an inability to decide on a selective set of capacity strengthening activities during project preparation. 101. The results framework was complete and thoughtful, and the monitoring plan and arrangements were very strong. (Details are discussed in Section IV Monitoring and Evaluation.) 102. Most risks were assessed realistically and thoughtful mitigation measures identified. The PAD described the overall project risk as substantial, noting the difficult security situation, uncertainty around the political transition, and the economic and other risks posed by the expected withdrawal of international military forces. The PAD gives a candid and accurate assessment of the level and nature of the various risks. Three areas were rated as Substantial: (i) stakeholder risk (the possibility that new national and sector leadership might not continue the high level of political support for the project and its approach, opposition from some within government to continuing to “contract out” provision of health services, staff shortages and turnover, and potential private sector resistance to regulatory reforms); (ii) Implementing Agency risk (low pay levels, weak capacity at provincial level, limited procurement capacity); and (iii) Governance (weak internal controls on financial management, widespread and pervasive corruption in the country). Risk in all other areas was rated as moderate, but moderate/substantial for Monitoring and Contract Management. One risk that was not explicitly noted or mitigated was that MOPH would fail to develop sound proposals for funding under component 2 within the first year (or even two) of the project. The team Page 32 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) adequately incorporated mitigation measures for each identified risk in the technical design and implementation arrangements during preparation. For instance, to mitigate the risks to procurement, the special services procurement unit (GCMU) was strengthened, procurement capacity in MOPH was assessed and plans developed for continued training and TA as needed, workshops for contractors on Bank procedures and guidelines were held, and careful supervision and support provided throughout. These measures enabled the all-important procurement for health services delivery to be successfully completed on time, even when the workload greatly increased. 103. A great deal of proactive work before effectiveness ensured readiness for implementation and a smooth transition between projects and rapid start to implementation once the project became effective in June 2013. Procurement of NGOs for services delivery is a notable example. Advanced procurement began a year before effectiveness, and was well under way by project approval on February 28, 2013. Proposals began to be evaluated in May 2013, and by the time of the first formal supervision mission in October 2013, 13 of the 27 contracts had already been signed, and all the others were on track to be signed by early November, giving contractors at least two months to prepare to begin operating in January 2014. (Services delivery during the early months of the project was carried out under contracts signed under the previous project and extended to the end of 2013 to ensure services continuity.) B. KEY FACTORS DURING IMPLEMENTATION Overall 104. Delivery, use and oversight of health services was hampered by worsening insecurity across large areas of the country. In spite of growing danger and difficulties, project implementation, government commitment and WB support and supervision did not falter, and many project targets were achieved or surpassed. Most importantly, the project succeeded in expanding the scope, coverage and quality of health services, especially to the poor. This is to the credit of the implementing agencies, MOPH, the Bank team, development partners, and dedicated health workers. In 2018, the Central Statistical Office reported that their latest survey of living conditions had found areas of the country with no school, mosque, or government building of any kind, but with a functioning health center. Factors outside the control of government and/or implementing entities. • Dangerous and deteriorating security situation throughout the project 105. In 2012, for the first time since the UN began keeping records in 2003, the number of security incidents fell, and there was optimism that this might be the start of a trend. But in 2013, incidents rose again and continued rising throughout project implementation, reaching new highs in 2016 and after. In 2017 there were nearly 24,000 incidents. Sometimes health workers and health facilities were targeted: In the final 6 months of the project, 63 incidents involving health workers and/or facilities were reported in 14 provinces, in which 5 health workers were killed or injured, 45 were detained or kidnapped (and 12 returned), 41 health facilities were closed or destroyed, 21 were reopened, and 6 were damaged or stolen from. Two provinces also reported incidences of intimidation of health workers and prevention of supplying health facilities with drug and other supplies (Reliefweb.int). Active conflict and bans on vaccinations by anti-government forces in areas they dominated denied access to immunization to over 500,000 children who needed them. • Economic stagnation, drought, deterioration in poverty and food-security, population movements 106. The ALCS 2016-17 found that economic growth and key socio-economic indicators had slowed or even completely stagnated since 2013. Poverty and food-security deteriorated sharply, made worse by periodic drought (early in the project, and an especially severe drought in 2018). By 2016/17, more than 54% of the population were living below the national poverty line, a sharp rise from around 35% at appraisal. Donor support and the number of international security forces decreased. Large flows of returning Afghans from neighboring Pakistan and Iran put added pressure on health services. In 2013, an estimated 631,000 people were internally displaced by violence; in Page 33 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 2016/17 the number had increased to more than 1.8 million people. Although the project provided extra contingency funds in NGO contracts to help them to respond to possible surges in returnees in provinces highly likely to need them, it was not always possible to scale up service delivery rapidly enough in response to unpredictable and large returnee flows. • Broader context is inimical to uptake of some health services, healthier behaviors and health outcomes 107. Afghanistan’s geography is a barrier to delivering and using health services: mountainous terrain, poorly developed infrastructure and low population density make travel to health facilities difficult in many parts of the country. This is made worse because the country is prone to landslides, earthquakes and floods, and parts of the country are snow-bound in winter. In 2016/17, 37% of the rural population lived further than 2km from an all- season road. The structural social factors hampering development – high population growth, high rates of poverty, low participation by women in society and the labor market, low quality of education, low literacy rates, and high unemployment – also have a negative impact on uptake of contraception and other health-promoting behaviors. There is low exposure to new information and many communities remain unaware or unpersuaded of the benefits of immunization and child spacing. Afghanistan’s conservative culture resists change; for example, a medical doctor noted that even he had found it difficult to persuade the women in his family to support exclusive breastfeeding during his infant son’s first 6 months. Despite recent improvements, access to sanitation and clean water remain poor: 64% of the population use improved drinking water sources, but fewer than one third use safely managed drinking water services, 77% of households (in a non-representative sample from 10 provinces) had E. coli in their drinking water. Around half of the population use an improved sanitation facility, only 41% use safely managed sanitation services. Most (72%) urban residents live in slums, informal settlements or inadequate housing, 44% of all households live in an overcrowded dwelling. A mere 25% of the population use primarily clean fuels and technology for cooking and only 4% for heating. (All statistics from CSO 2018.) Factors subject to government and/or implementing entities control • Changes in Government and in MOPH Leadership 108. The outcome of the April 2014 presidential election took six months to resolve. During more than 18 months of political uncertainty at the start of the project, MOPH leadership sensibly held off policy changes. However, inadequate attention was given during this time to developing acceptable proposals for activities for funding under Component 2. The current Minister of Public Health was appointed in February 2015 and took some time to assemble a leadership team. Political changes prevented several planned project activities from being implemented; notably the plans to give full autonomy to hospitals, and to strengthen the provincial level, because the new government put a hold on the policy of decentralization. • Strong engagement and leadership by MOPH 109. The new Minister for Public Health appointed in early 2015 ensured full continuity in national health policy. He had been part of the team that devised the BPHS and EPHS in his earlier post in the MOPH leadership team. As a medical doctor with managerial and global experience, his appointment had strong and broad support, and he is fully committed to the project, its objectives, and to sustaining the effort. • Competent, fully staffed MOPH Finance Department provided competent financial management for SEHAT 110. SEHAT’s financial management (FM) is rated one of the best of all WB-funded projects in Afghanistan by the WB FM specialist in Kabul. The MOPH Finance Department was fully staffed throughout the project, and turnover was very low among senior FM staff, who are experienced, competent and very knowledgeable about the project’s FM requirements, and MOPH, MOF, and donor agency FM procedures. They were diligent and responded quickly and well to any FM matters that emerged within the project and MOPH. Page 34 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) • Project budget/fund flows were generally smooth, with some delays 111. Overall the funds flow process was smooth. It was much faster to make local currency payments through the Designated Account (DA) than to process direct payments through the WB, so more local currency NGO payments were processed through the DA than had been expected. This caused a liquidity issue for a short time. In November 2015, the DA ceiling was increased which enabled smooth disbursements through the remainder of the project. 112. Throughout project implementation, budget allocations were delayed during the first month or two of each new fiscal year. This held up project disbursements, although they caught up in subsequent months. In anticipation of the annual delays, the contracts with NGOs avoided scheduling payments in the first two months of each fiscal year, which helped reduce the disruption to project implementation. In addition, MOPH used the process that is in place allowing them to request temporary budget funds from MOF to facilitate project disbursements, but the mechanism took time. Discussions among MOPH, WB and MOF about how to avoid these delays have, over time, resulted in a smoother and quicker process. There were complaints that MOPH was reluctant to release performance-related payments, MOPH noted the importance of paying only after Third Party verification of data. (The new project has included additional oversight procedure to try and avoid these situations.) 113. Issues and delays in 2018 resulted from changes in MOF forms and procedures in response to IMF requirements for reforms in payment procedures/processes. MOF staff found the changes complicated and the difficulties and delays difficult to resolve. Effective coordination between the MOPH and MOF was a critical factor for smooth disbursements because funds flow was centralized in MOF. MOPH was proactive in raising any issues with the Bank team, who actively facilitated, where needed, issues between MOPH and the MOF Treasury department. • National Procurement Policy changes 114. Changes in national procurement procedures in 2014 made accreditation far more difficult to achieve (only 2 procuring entities in the country have been accredited since 2014). Procurement reform has been a major priority of President Ghani, to address widespread and serious corruption. All large-scale procurement was centralized in the National Procurement Authority, and the President himself chaired weekly meetings of the National Procurement Commission to review all major procurements. A new law was enacted in 2016. There is consensus that much was achieved, but that the country still has a long way to go to meet good standards of procurement. While the strong central control has a clear rationale, it also had drawbacks, including occasional delays. • Proposals for activities to strengthen stewardship took 2-3 years to develop 115. SEHAT’s approach to strengthening MOPH stewardship capacity and functioning was to ask departments to consult with other development partners, and develop proposals for SEHAT funding. This was intended to ensure coherence, and MOPH ownership, avoid duplication and ensure complementarity with activities funded off-budget by other partners. However, it took three years before acceptable proposals were finalized. There were several reasons: (1) uncertainty during the long political transition noted above; (2) this was a new and difficult challenge for departments; (3) many departments were occupied with other projects and overlapping activities funded off- budget by USAID and other donors and did not have the capacity to propose or take on additional activities. The delay left too little time to implement most of the proposed activities. Factors subject to World Bank control • Flexibility of project design 116. The project design was deliberately flexible, anticipating the possible need to take over the funding flow and project responsibility for services provision in 13 provinces where USAID had been contracting directly with NGO providers, as US engagement in Afghanistan evolved. As noted above, from 2015, USAID changed from directly Page 35 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) funding services in 13 provinces to channeling the funding through the ARTF. This greatly increased the geographic scope and workload of the project and project team. The design of Component 2 can be seen as a pragmatic response to a situation where the MOPH and project team were unable to reach agreement on a well-defined program of activities to strengthen stewardship functions and capacity, and sensibly avoided the risk of duplication by waiting for bilateral projects that were under discussion to be ready to share, or can be criticized for not doing more than compiling a “menu” list of examples of possible kinds of activities that could be considered and proposed for funding during implementation. Whether regarded as a strength or failing of project design, it certainly contributed to the delays and reduced scale and scope of the component. • Continuous supportive supervision from WB Kabul office and Washington DC 117. The TTL and most other team members are based in Kabul, and were able to provide continuous, supportive supervision to MOPH. This was especially important given the very difficult security situation under which the project was implemented, the constraints on travel to Afghanistan by staff based elsewhere, and the dominant role of the project and ARTF in the health sector. There was unusually strong continuity in the team, both the Kabul-based team members and key senior staff with specific expertise (such as in contracting, nutrition, pharmaceuticals, health economics, etc.) based In Washington and elsewhere who worked on successive health projects and analytic work in Afghanistan. Given the limitations on travel to Kabul and within Afghanistan, an early priority was to provide video conferencing capability to all provinces. Regular face-to-face meetings were arranged in Kabul with provincial and NGO managers, since visits to the provinces were not possible. • Strategic, proactive approach to policy dialogue 118. There are three important examples of the WB taking a highly proactive and strategic approach to health policy dialogue. (1) The WB arranged annual health policy seminars, each focused on selected aspect/s of the project that needed additional focused attention, such as health system development, human resources, pharmaceuticals, behavior change communication, hospital management, and health sector performance management. The first seminar helped to bring the new Minister (appointed in early 2014) up to speed on the project. The 2015 and 2016 seminars helped prepare for the Mid-Term Review. Participants at these High-Level Policy Seminars included MOPH’s leadership and development partners (typically around 40 people in the years when the seminar was held outside the country, more in the alternating years when the seminar was in Kabul). As an example, the three-day 2016 Delhi seminar agenda covered health sector challenges and the way forward, human resources, ensuring the quality of pharmaceuticals, communications and public relations, improving management of national hospitals, and MOPH stewardship functions and strengthening service contract management. The Minister described the seminar as “very productive and useful”. The WB brought in top experts to describe global knowledge and experience and discuss how they might apply in Afghanistan. The seminars helped ensure that the Bank, other key sector partners, and MOPH were “on the same page”, and that MOPH leadership remained fully committed to SEHAT’s objectives and approach. They were also useful groundwork for conceptualizing the Sehatmandi project. (2) The second example is the analysis done in 2016/2017 to examine carefully and compare the extent to which good results were being achieved by the two models – contracting-out of services delivery to NGOs, and contracting-in managerial support in three provinces with service provision by the MOPH. There were differing opinions about which approach was better, and so the WB commissioned a strong analytic team to examine the data. The analysis found that the two approaches delivered comparable results, but that contracting-out had significant benefits in areas of high insecurity (World Bank 2018). This was an important example of informing policy discussion by analysis of data. (3) The third example is the Presidential Health Summit in 2017 at which these results were presented and discussed. The Summit was chaired by President Ghani. Careful preparations by the MOPH, Bank, and other partners over six months ensured evidence-based, focused, constructive discussion. Data on health services delivery and progress over 15 years were reviewed, key lessons Page 36 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) drawn, and areas in need of renewed efforts identified. The Summit concluded with discussion of a “Health Sector Development Framework 2017-2022” that was a useful basis for designing the Sehatmandi project. • Supporting BPHS/EPHS through a unified financing platform ensured a standardized, harmonized approach 119. SEHAT provided a harmonized, standardized approach for development partners’ support to the health sector in Afghanistan. Financing was able to flow through a single multi-donor Trust Fund, with one set of procurement and disbursement rules and procedures, and oversight arrangements. UNICEF especially appreciated how much this facilitated integrating and rolling out new nutrition services. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 120. On the whole, M&E design was strong, with some minor flaws. The theory of change is clear and the results chain short and direct. The logic of the results chain reflects strong global evidence on the most cost-effective, affordable and feasible interventions likely to have the greatest impact in improving health outcomes in Afghanistan, with a strong concern for equity in access and benefits. The expected results extrapolate from the previous ten years of experience, building on two successful previous health projects. 121. The PAD RF was thoughtful and selective, well-chosen SMART indicators were all drawn from the MOPH Strategic Plan for Health. There were enough indicators to enable all parts of the PDO to be tracked, but not so many as to dissipate focus and obscure project priorities (6 PDO indicators, 12 IR indicators). The RF included clear information on the data source for each indicator, responsible agency, and frequency of data collection. All six PDO indicators and all but one of the 12 Intermediate indicators had baseline and target values (for the end of the project and year 3, or annually where yearly data were collected) in the PAD. Baseline data for one IR indicator – the percent of children under 5 with severe acute malnutrition who received treatment, were being collected in a household nutrition survey, results of which were due in September 2013, soon after effectiveness. However, the survey data only became available a year into implementation, and baseline data and targets are in the October 2014 ISR. 122. One minor quibble is that three targets were set too high or too low. The contraceptive prevalence rate target was unrealistically ambitious: a review of all available DHS data on CPR trends (Arur et al 2011) suggests that even the global median increase of one percentage point per year would have been surprising, given Afghanistan’s social, economic and demographic situation (the target aimed for more than twice that increase).The project’s commitment to use national targets is admirable, but in this case, it compromised the project’s ability to achieve a goal. On the other hand, when two indicators were revised in 2015, targets for increases in the number of deliveries attended by skilled personnel and for the number of women receiving antenatal care were set obviously too low. 123. M&E arrangements were extremely strong. As in previous projects, an independent Third Party (TP) agency was contracted for M&E. Their contract included conducting four rounds of health facility surveys, seven rounds of verification of routine HMIS data and functionality of facilities, two national household surveys, a detailed evaluation of the RBF pilot, as well as analysis and dissemination of the data and survey results. This provided world- class expertise in surveys and analysis and state-of-the-art methods as well as sectoral expertise; it avoided an important conflict of interest since the TP had no financial interest in the data, and was committed to high quality data collection and analysis. It promoted data reliability and transparency. This arrangement had worked reasonably well in the previous projects, except where reliance was placed on surveys conducted by other agencies that were delayed and too late to serve their intended purpose. SEHAT’s M&E design avoided this: all the baseline data were Page 37 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) already in hand (with the single exception noted above), and the TP contract included collection of end-of-project data in the last project year. M&E Implementation 124. M&E implementation was very good, with only minor issues. Procurement of the M&E agent started the month after project effectiveness, but the contract was signed only in January 2015. KIT was able to make up for the delay and M&E activities began immediately upon signing. Initially, KIT worked with an Afghan subcontracted partner, but was not fully satisfied with their work even after providing training, mentoring and supervision. The sub-contract was terminated in October 2017 and KIT did all remaining tasks themselves. Over the course of the contract, KIT increased its local presence to 20 staff and 575 operational field staff, plus a data management unit of 30 people. The Kabul-based Team Leader post was unfilled for seven months, until KIT was able to recruit someone with the required technical qualifications and experience. KIT Amsterdam provided daily contact and regular monitoring and support (the security situation prevented international staff from visiting data collection sites). KIT recognized the need for systems of transparency, communication and close monitoring to compensate, and devised ways to improve tracking of field teams, quality screening of survey forms, and enhanced security management. 125. All planned data collection rounds were completed, with the exception of quality-testing of drugs from facilities stocks – samples had to be sent out of the country, and when the first round took an extremely long time, KIT and MOPH agreed to rethink the process. All survey data were collected and analyzed using best practice methods that are documented in survey reports, and discussed with stakeholders. Household survey reports are easily found on the internet. The household and facility survey data are the main sources for assessing project results in this ICR. Each survey report provides a full account of the sampling, survey team training, and data quality assurance mechanisms, which all follow best practice. The samples are large enough to allow robust provincial level estimates, and any impact of insecurity on administering the survey is noted. The descriptions of methodology and quality assurance and the presentation of the data analysis provide high levels of confidence in the reliability of the data. 126. The household surveys followed standard Demographic Health Survey (DHS) methodology. The AHS 2012/13, 2015 and 2018 were timed to provide indicator values for the start, middle and end of the project. This gave an unusually complete and reliable set of data. The AHS 2018 report provided useful analysis of trends using household surveys since 2003 (although the earlier ones are not all as fully comparable as the three that span the project). Survey samples were designed to yield representative information for most indicators for the whole country, for urban and rural areas, and for each of the 34 provinces, except for mortality estimates which are national only. The 2018 AHS sample consisted of 23,460 randomly selected households, in 1,020 clusters across all provinces. The teams were able to survey 19,684 households, in 912 clusters in 34 provinces. (2015 was of similar size.) KIT consulted with and reported to a steering committee of stakeholders including MOPH, major development partners, and the National Statistics and Information Authority. Efforts were made to learn from and improve upon previous surveys (see Annex 7 for details). Interviewers worked in pairs – one female and one male surveyor; three pairs, a female editor, a supervisor and a driver made up each team. Each team had a GPS tracker to help navigate, to enable their location to be tracked, and to collect cluster-level GPS locations to enable AHS coverage to be assessed. 127. The survey consisted of three questionnaires: 1) general questions on the demographic and health characteristics of the households; 2) specific questions on health and use of health services by women and children, including nutrition information; and 3) a mortality and verbal autopsy questionnaire. Comprehensive quality assurance included daily tracking of teams; the use of field-data editors; active- and post-monitoring of surveyor teams; and “bundle screening” in which cluster-wise data bundles from the field were screened, prior to data entry, to rapidly verify data quality. Page 38 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 128. Annual health facility surveys done by KIT’s sub-contractor in 2015, 2016, 2017 and by KIT in 2018 provided data for the Balanced Score Card (BSC). These national random samples assessed functionality and performance of health facilities in each province in delivering the BPHS, and of hospitals delivering the EPHS. As noted above, this is a “best practice” approach to quality assessment. The hospital surveys covered about 100 district, provincial, regional and national hospitals. Data for the health centers were collected from random samples of at least 25 facilities in each of the 34 provinces, with a total of 725 to 807 facilities (generally increasing over the years), observations of patient-provider interactions and exit interviews with 7,000-8,000 patients, and interviews with 2400-3,000 health workers and 489-1150 CHWs. (The ranges cover all 4 years of surveys.) Quality assurance measures included: 1) continuous supervision of each team by dedicated supervisors; 2) active and post-monitoring by monitors from KIT and MOPH; 3) regular contact with the field teams and phone calls to the health facilities; 4) collection of proofs of visit to health facilities such as signatures of health facility staff and a health facility stamp, photos of the teams in front of signboard of the health facilities, etc. (KIT BSC report 2018, p6). The only issue, noted earlier, was that the sub-contractor, Silk Route, was not fully satisfactory, and KIT terminated their sub-contract in 2017. There may be some systematic differences in data collection in 2018 compared to earlier years, such as applying criteria in slightly different ways. 129. The BSC reports provide average scores for each of 26 domains for each province, and also the median scores across all provinces. This provides a very good sense of trends, and strengths and weakness relative to other provinces. These annual independent observational surveys provide extensive measures of the functionality of facilities, including staffing, and the quality of services provided, and are a strong basis for targeting efforts to improve quality and services. The findings were discussed with facility and provincial managers each year. 130. HMIS Data. In addition to the household and facility surveys done by KIT, the project M&E also used routine HMIS data reported by facilities. Years of prior effort to improve HMIS reporting completeness and accuracy had borne fruit. In 2014 the MOPH undertook a rigorous assessment of the quality, completeness and use of HMIS data (MOPH 2014). The completeness of reporting, and consistency between the facility registers and HMIS entries (Data quality) of four indicators that reflect top MOHP and SEHAT priorities (Penta3, FP, ANC and out-patient visits) on very stringent criteria ranged between 70-75% (Figure 11). (Utilization is discussed in the sub-section below.) Figure 11: HMIS Data Quality, Knowledge and Use Indices (National Means), 2014 Source: MOPH 2014, p.8. Page 39 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 131. The scores for the quality of the data on these four indicators ranged from 42-100% across the provinces, and the average scores (mean and median) were 68-71% for Penta3 coverage and out-patient visits (OPD) and 74-78% for ANC and contraception use (FP) (Figure 11, and detailed data in MOPH 2014). The survey teams were able to verify 96% of the patient visits by tracing patients, and 96% of recorded institutional deliveries. This is relatively good performance, despite the fact that provincial officers who are supposed to verify the accuracy of reports collected in their provinces by visiting the health facilities themselves told US investigators in 2016 that “they rarely travelled outside the provincial capital and rarely verified the reports” (SIGAR 2017). The “knowledge” items measure whether the 1,664 providers sampled and interviewed had received HMIS training (basic or refresher) in the past year and could explain correctly which cases to include, and could list all the required activities in providing each service. Scores by province ranged from 38-100%, with national means and medians around 71-73%. 132. Since there is an obvious incentive to inflate HMIS data when they are used to calculate performance payments, KIT was required to do seven (bi-annual) rounds of HMIS verification. Facilities were randomly sampled and verification done on a rolling, year-round basis for greater unpredictability (and more efficient use of KIT staff). Verification involved comparing data entered into the HMIS with the registers and other records kept by the facility, inspecting the functionality of each facility visited and validating registers and records accuracy by interviewing random samples of patients. This improves the reliability of the HMIS data, and gives an important and clear message to health staff on the importance MOPH attaches to accurate and complete data that are used to monitor and improve performance. M&E Utilization 133. The data collected though the HMIS and facility surveys for the BSC were used to calculate the performance- based part of payments to the contractor NGOs. They were also used to compare performance across provinces, and against upper and lower benchmarks, defined by the 25th and 75th percentile performance on the BSC indicators in 2011/12. Regular on-line meetings between Kabul and provinces using the communications network provided by SEHAT included discussions of the latest data, and the progress and challenges in various provinces. ICRs for previous health projects commented on the “data and results” management culture at various levels of the MOPH. This has been encouraged and strongly supported by the WB. MOPH policy documents and guidelines refer to epidemiological and cost-effectiveness data to justify decisions. MOPH and other national documents frequently cite changes in health outcomes as vindicating the decision to keep a strong focus on the BPHS and EPHS, despite pressures to allocate more funding to tertiary and specialist hospitals. The regular six-monthly reports on the project submitted by the MOPH to the WB included updates of the results framework to reflect new HMIS or survey data as it became available, and cited detailed data on all aspects of the project. 134. Figure 11 above from the MOPH 2014 assessment of HMIS data includes a summary of metrics to assess how well data were being used. The data use index summarizes answers to nine questions that include being able to calculate coverage correctly (including estimating denominators), whether the facility staff had graphed trends in the data, compared year-to-year changes, and compared trends in different indicators to see if they provided a consistent picture, and could offer at least two reasons for any changes in trends. Average scores around 60% for this indicator are evidence of an impressive ability to interpret and use data at facility level, and importance given to data use by MOPH leadership. 135. Several other examples of use of data for making decisions during SEHAT are worth noting: • During project preparation, the preliminary results of the RBF pilot started during the SHARP project swayed the decision to adopt the approach across the country, and make part of the NGO contracts depend on results. Page 40 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) • Differing opinions and debate within MOPH on the relative merits of contracting out delivery of health services, or MOPH delivering services (with contracted-in managerial support) were resolved by careful analysis and comparison of the performance of the two types of arrangement, as discussed above. • In 2016, MOPH started a peer-reviewed national journal for publishing public health research, two issues of which are available on the MOPH website. The articles are of high quality. Some report on primary research intended to guide policies and the focus of interventions, others are detailed analyses of existing data sets (such as the 2010 mortality study, and national disease surveillance data). The MOPH website also contains years of weekly disease and outbreak surveillance reports. Both are unusual in a low-income and FCV country, and point to a culture of respect for and use of surveillance and monitoring data. Justification of Overall Rating of Quality of M&E 136. The quality of M&E is rated as High. Although some minor shortcomings were identified, the project stands out in the quality and seriousness of all aspects of M&E. All too often, lip service is paid to the importance of M&E; it is extremely unusual for a project to incorporate such extensive data collection, analysis and use, by an independent world-class agent, especially in a low-income country with severe security challenges. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental Issues – Healthcare Waste Management 137. Environmental safeguards. The project had an Environmental Assessment (EA) category of B and triggered Safeguard Policy OP/BP 4.01 because it funded healthcare services that generate medical waste. Two of the project’s conditions and legal covenants relate to Environmental Compliance: (1) To update the existing Healthcare Waste Management Plan (HCWMP) by the end of 2013, and (2) To carry out the project in accordance with the HCWMP, the Environmental and Social Management Framework, and the Environmental Management Plans. The HCWMP prepared under the preceding SHARP project had been partially implemented; SEHAT aimed for better implementation, and during appraisal, agreed on institutional arrangements and responsibilities and technical assistance needs which were intended “to ensure the plan is implemented as envisaged” (PAD p66). 138. Compliance with (1) was substantial (although 10 months late). A preliminary interim HCWMP was put in place in 2012, and an expert hired to work with the MOPH Environmental Health Directorate to develop “an environmentally sound, technically feasible, economically viable and socially acceptable … [HCWMP] … with cost implications and timeframe for implementation” (Terms of Reference, and HCWMP 2015 p.1). Completed in October 2014, the “Final Draft Comprehensive Health Care Waste Management Plan” (referred to here as HCWMP 2015) is available on the MOPH website (see Annex 6 for reference and url). It contains a very good summary of the importance and main elements of healthcare waste management; a careful and thorough analysis of the existing situation, including an assessment of existing policies and regulations, and findings from field visits to four provinces and discussions with stakeholders at all levels of the health system, as well as input from development partners, NGOs, landfill sites, municipalities, and regulatory bodies. There is a useful gap analysis listing the actions needed to move from the existing situation to the recommended situation; detailed specifications and construction guidelines for building simple pits for safe disposal of sharps, and for setting up common healthcare waste treatment facilities (initially as a pilot); a useful summary comparison of the advantages and disadvantages of different waste treatment methods; training plans; and a detailed implementation budget and timeline. The HCWMP 2015 is also strong on M&E: it includes a very good checklist for assessing whether a facility meets HCW disposal standards; and 8 indicators for monitoring HCWM with baseline values and clear targets (HCWMP 2015, p.76). The document is complete, well-considered and fully appropriate. However, it would have been useful to excerpt from this 78-page Page 41 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) “plan” – which is actually a consultant’s report containing a plan – a succinct, results-oriented, time-bound plan, as a more “user-friendly” guide to action for those responsible for its implementation. 139. There was modest compliance with (2), which required that the project be implemented in accordance with the HCWMP, the Environmental and Social Management Framework (ESMF) and Environmental Management Plan. The Environmental Health Department was responsible for HCWM, but consisted of only one official, and lacked adequate budget and motivation to monitor and take actions to improve HCWMP compliance. Reportedly, many health facilities do not properly segregate or safely dispose of their waste. 140. With repeated encouragement and urging from the Bank (and other development partners), including presentations and provision of materials on the reasons for and importance of proper HCWM, and escalating the issue to senior MOPH management, there was some progress. A private contractor was hired for the final year of the project to collect, and dispose of the HCW for 19 central tertiary hospitals and the Blood Bank. MOPH trained 90 staff at these facilities, and their Infection Prevention Committees formed HCWM committees and committed to following HCW protocols. A procedures manual for HCWM was distributed to all NGOs implementing the BPHS/EPHS, and NGOs were required to include waste segregation activities and issues in their quarterly reports. HCWM items were added to monitoring checklists. 141. The BSC includes a 9-item score that includes safe use of disposable injection syringes, presence of clean water and soap, regular use of sterilizers, disinfectants and incinerators, proper disposal of sharps, and whether medical waste is seen lying around in or near the facility. The national median score for these “Universal Precautions” (which weights all items equally and measures the percentage of facilities meeting the standards) rose steadily and significantly from 2012/13 through 2016. It faltered in the final years, but ended the project nearly 20% higher than at the start (Table 6). Seventeen provinces had scores that rose and then fell, the other 17 showed sustained improvement. Scores varied widely across provinces, from 29% to 91%. Initially, fewer than 80% of provinces were above the low benchmark (the worst quintile’s 2011 score), this rose well above 90% in 2015 and stayed above or near 90% for the rest of the project. This is clear evidence of improved HCWM in most provinces during most of the project. Table 6: BPHS Balanced Scorecard – National Median Scores for Universal Precautions 2012/13 2015 2016 2017 2018 62.1 72.9 80.2 78.0 73.5 Social Compliance 142. The Environmental and Social Management Framework (ESMF) was prepared well before project effectiveness and is available in WB files (see Annex 6). The project financed only very minor construction and refurbishments, and no environmental or social concerns related to these were raised in any of the supervision missions. The only social compliance issue relates to the grievance handling mechanism. The MOPH set up a system to enable citizens to submit complaints on a toll-free telephone line, by email, text message, Facebook and twitter, and put the name and contact information of a Focal Point in each province on the MOPH website (http://MOPH.gov.af/en/page/health-compliant-office-hco). When the system was launched, the Minister made numerous statements on TV, radio and newspapers to publicize it. However, only one person in MOPH was designated to handle complaints, and there was no system to triage or adequately follow-up. MOPH “Six monthly reports” in 2016 and early 2017 reported a total of 350 complaints and that most had been dealt with. The June 23, 2018 report by the Health Compliant Office states that between July 2017 and June 2018, 470 complaints from the capital and provinces were submitted through email, phone and form. The Office analyzed and shared the complaints with the relevant entities at national and sub-national levels for redressing purposes. The ICR mission Page 42 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) was informed that many complaints were left unresolved. This undermines confidence in the redress system. The MOPH policy and leadership’s commitment to transparency and probity may not be fully realized in practice. Fiduciary Compliance - Satisfactory 143. There was one fiduciary compliance issue: Government’s expected contribution of US$30 million towards project costs was not provided. Despite repeated requests to the MOF by MOPH, and comments by the Bank team and management during supervision missions and formal communications, the MOF did not allocate the funds. However, this did not constrain or hinder implementation in any way; there were ample other funds, especially considering the low disbursement under Component 2. The US$30 million was less than 5% of the final total project cost, and just over 7% of the initial project cost – more token than real contribution. 144. The Financial Management (FM) arrangements for SEHAT, including staffing, reporting and accounting, planning and budgeting, and audits, had been developed and improved during previous projects, and functioned well. Country systems were used for accounting, funds flow, and audit. Project accounts were maintained in the Afghanistan Financial Management Information System managed by the Ministry of Finance (MOF), and subsidiary records for the project were maintained by MOPH (initially in excel, and then Quick Books). Project Designated Accounts were maintained centrally by the Treasury Department of MOF, and provincial fund transfers were made through the mastufiats (MOF provincial branch). Annual (external) audits were carried out by the Supreme Audit Office of Afghanistan. FM systems, procedures and processes worked smoothly and effectively throughout the project, largely the result of continuity of highly capable, experienced FM staff. The head and several others in the Finance Department (development budget) had been in post during the SHARP project, and remain in post. They have a sound understanding of and long experience with government and donor-financed projects. Project FM was rated satisfactory in all but one ISR. (The one moderately satisfactory rating was due to a five months delay in updating the FM manual.) MOPH was proactive in dealing with internal FM issues and in coordinating with MOF. It was responsive to Bank observations and recommendations relating to FM from supervisions, Interim Financial Report (IFR) reviews and audit reviews. IFRs were submitted on time, in form and substance acceptable to the Bank. External audit reports were mostly received within the due dates, the few delays were beyond the control of MOPH. Audit observations were responded to quickly and appropriately, and the only issues raised were minor. Bi-annual internal audits by the MOPH Audit Department were often late, for two reasons: the MOPH Audit Department’s workload tended to exceed its capacity, and MOPH management delayed sending reports to the Bank if there were findings it wanted to follow up on. Although not a hindrance to achieving the PDO, there is no Inventory management system for fixed assets. Medical and other equipment bought by contracted NGOs is supposed to be reported to MOPH for later verification. However, reporting was poor, and no inventory control and management was done. A simple system is being set up under the new project. 145. Procurement. Most of the project procurement was of the NGOs to deliver the BPHS/EPHS. As noted above, MOPH has long had full accreditation for procurement of services through the Grants and Contracts Management Unit (GCMU). GCMU was well-staffed, competent and capable. It successfully completed several rounds of NGO contracting, including one round in which all 48 contracts, country-wide, had to be re-negotiated within just five months when MOF insisted on new contracts rather than contract extensions for NGOs with satisfactory performance, as the project had planned. This was an enormous task. Procurement was rated satisfactory in all but two ISRs, when continued delays in technical scoring and negotiation with the Third Party (TP) M&E agency resulted in one MU and one MS rating. Contract management was less satisfactory, and the PAD for the new project notes a “lack of standard procedures, (and) irrational performance parameters for termination of contracts…” (Sehatmandi PAD, p.23), which are being addressed under the new project. The Procurement Directorate in MOPH (responsible for procuring small works and goods including equipment and drugs) was weak. The Director’s post was vacant for a year, and then high turnover of a series of short-lived incumbents, all with inadequate capacity, Page 43 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) thwarted training efforts. (The current Director was chosen through a competitive process from qualified civil service applicants, and is expected to end this pattern.) Since the NGO contractors were able to undertake their own procurement, this unsatisfactory performance did not have much impact on project performance, but did cause repeated minor problems, and failure to meet the PDO indicator that required MOPH to achieve and maintain accreditation for procurement of goods and works. In summary, overall, project procurement performance was strong, albeit with some minor shortcomings. C. BANK PERFORMANCE Rating: Satisfactory Quality at Entry 146. As noted above, the PDO was relevant to the health situation and well-chosen national priorities, and consistent with the Bank’s assistance strategy for Afghanistan. The activities had a clear logical relationship to the expected results, and M&E design was exemplary. Implementation arrangements had been well tested in prior projects, and thoughtful improvements made on the basis of lessons of experience. The project preparation team included all the necessary expertise, and a very good balance of deep understanding and familiarity with the situation in Afghanistan, and relevant global experience. Preparation was inclusive, and the design made provision for the possibility of a major change in the support to the sector of a key development partner, to prevent a disruption in health services provision. Transition from the previous project was smooth, with no gap in funding or services provision. 147. As noted above, a positive view of the design of Component 2 is as a reasonable and strategic response to a difficult situation, that should have been described more accurately in the PAD and AF. A negative view is that 22% of the project funding was allocated to a component whose activities had not been specified. The judgement of this ICR tends to the former. There were many aspects of MOPH stewardship that needed strengthening, but limited capacity, and MOPH was not able or willing to agree on a small set of priority areas during project preparation. Elections were imminent which could (and did) result in new leadership in MOPH, and new national policies on decentralization, for example, with implications for MOPH stewardship functions. Moreover, the SEHAT team wanted to avoid duplicating activities being funded or planned by other donors, the details of which were not available to the team. The compromise was to list all ten areas that MOPH wanted (and needed) to improve, with examples and ideas of activities that might be undertaken in each, and ask all relevant MOPH departments to work closely with other development partners, and come up with proposals for consideration by the WB for funding under SEHAT. This kept MOPH in the “driving seat”, and provided a natural selection process of the most committed and capable departments for SEHAT support. However, the PAD and AF should have been more carefully and precisely worded, and the implementation risks more clearly identified. The project documents say that Component 2 would “include” the areas listed, rather than explicitly and more accurately explaining that these were a suggestive list of possible areas of activity, and that only some would go ahead, on the basis of realistic and sound proposals. The ability of departments to develop their own proposals was, in retrospect, overestimated. It took three years for a full set of proposals acceptable to the Bank to be ready, many of which were far too ambitious to be accomplished in the remaining time. As the team had anticipated might happen (but not reflected in the design of the component or discussion of risks to implmentation), many departments were fully occupied with other projects during SEHAT, such as USAID’s project to improve “Leadership, Management and Governance” in the MOPH (and Education Ministry), and the “Afghanistan Health Sector Resiliency” project. 148. No QER was performed which suggests confidence in the project design by the country and sector managers. Page 44 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Quality of Supervision 149. The Kabul-based project TTL and core team provided continual strong support and follow-up. The TTL and a key sectoral colleague remained the same throughout, and there was very little turnover in the rest of the team. The team is clearly respected and appreciated by the MOPH and other stakeholders, although the co-financiers would prefer more information about the project in addition to supervision documents. ISRs were complete, careful and informative, with the only shortcoming being that some data entries do not note their source or year. Aide Memoires (AMs) show a strong focus on development results, as well as careful attention to detail. They show evidence of the team being proactive and constructive. For example, Aide Memoire document efforts to expedite the delayed procurement of the TP including arranging with USAID to provide a procurement advisor to help in June 2014. 150. The thorough preparation for the mid-term review (MTR) deserves particular mention. This included the High- Level Policy Seminar the team arranged and the thoughtful and comprehensive Policy Options paper the team wrote in preparation for the MTR. The team was proactive in reallocating funds to Component 1 during the MTR for needed expansions in services. The workload of the project team increased greatly with the inclusion of the 13 additional provinces, but the quality of supervision remained extremely high. Changes in ISR ratings are explained and well- justified, and comments are frank. Transitions between projects have been smooth – especially important because the projects fund health services provision across most of the country. The one neglected item is regular filing of key documents in the project portal, which would be easy to remedy (documents were filed in WBdocs). 151. The team worked assiduously to support MOPH’s agenda on HCWM, Gender and Social Safeguards (grievance redress), often seeming more committed to the goals than some MOPH counterparts. These efforts included repeatedly providing support, arranging technical assistance, sharing good practices from other projects and other countries. For example, the Social Development Specialist held a workshop for about 50 focal points to share information across sectors and showcase good practices and systems. 152. The WB FM team took an active part in bi-annual implementation support missions, and worked closely with the team leads to find solutions to FM-related matters. The FM team also actively engaged with MOPH and MOF, and met with the project team whenever needed to follow through consistently on FM matters. Beyond project FM, the team was closely involved in discussions and provided advice on matters relating to MOPH’s fiduciary systems development. Justification of Overall Rating of Bank Performance - Rating: Satisfactory 153. A rating of satisfactory is well justified by the strong focus on results, well-defined PDO, focused project design and strong M&E, and by consistently helpful, proactive, professional, supportive supervision by a small, dedicated team, working in an unusually difficult situation. The shortcomings were in the imprecise description of Component 2 and inadequate recognition of the implementation risks; and in filing key project documents in WBdocs only and not in the project portal. D. RISK TO DEVELOPMENT OUTCOME 154. The risk to development outcome is High due to continued and escalating conflict in many provinces, and the possibility of declining donor funding. Attacks on health facilities and health workers may make it too dangerous to deliver services in some areas. In less extreme cases, the supply of drugs and other consumables can be affected, undermining the quality of care and further reducing confidence in public health services. Insecurity could further affect the ability of the MOPH and independent M&E agent to visit facilities to ensure quality of services provided. Page 45 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 155. The abstract to a recent analysis of prospects by a US professor summarizes the outlook as follows: “Seventeen years in, the war in Afghanistan presents a credible threat of disaster for all parties involved—or a settlement that could end the fighting on terms [the US] can accept” (Biddle, 2018). A presidential election is scheduled for early 2019; if it is held and the results disputed (as were the 2014 elections), an increasingly polarized and distrustful political system might result in factional violence rather than a smooth political transition or continuity. On the other hand, if recent steps toward exploring negotiated alternatives to conflict between the Taliban and government continue, a much more optimistic outcome might be possible. US political and economic engagement, including continued financial contributions to health care, security and other fiscal support are important and are by no means guaranteed. 156. WB, USAID and EU funding are secured for three more years through the new Sehatmandi project which began implementation as SEHAT was ending, and more follow-on projects are likely. Even if external funding cannot be guaranteed further ahead, the project has provided enduring benefits by saving lives and enhancing the health of the people of Afghanistan. V. LESSONS AND RECOMMENDATIONS Contracting with NGOs to deliver services is an important approach, especially in FCV settings. 157. The NGO contracting model used in health is an extraordinary achievement in Afghanistan, showing that it is possible to deliver a flexible, well-prioritized package of services efficiently and cost-effectively even in a FCV situation. NGOs had the flexibility and independence to be able to ensure services were delivered, expanded and improved, even in a seriously deteriorating security situation. They were able to forge alliances and agreements with local leaders, and adapt to specific local situations in ways that would not have been possible for government employees. Successive projects have achieved steady improvements in the alignment and harmonization of services, with SEHAT bringing all contracting, contract management and payment under a single process. There has been increasing reliance on performance-related funding, starting with a small pilot in SHARP, making 20% of payment performance-related under SEHAT, to linking full payments to performance under Sehatmandi. Capacity building takes time and is difficult. It is better not to attempt too much at once. It is critical to define clearly the purpose of capacity building. 158. Many projects aim to build capacity in a ministry or other organization. This is difficult, especially when there is considerable staff turnover. It often requires new systems and/or procedures, additional skills and new behaviors and norms. Capacity should not be thought of in the abstract, or as an aim in itself; the purpose needs to be clearly defined. This requires completing the phrase “…capacity to…” where specific tasks and outcomes are defined. The task is especially difficult when trying to change to the kind of ministry needed to manage a system where contracting-out is the dominant model, which needs very different kinds of capacity than a ministry whose main responsibility is delivering services. A ministry that manages a contracting-out approach needs smaller, more agile technical departments that provide guidance and help with contract management, and are able to help and support contractors to enhance emphasis as needed to achieve national goals, for example, to increase use of modern contraception. In light of this, experience over past years in trying to enhance MOPH stewardship capacity, and changes being implemented under the new Sehatmandi project (notably in performance management of NGO contracts) the first stage of a multi-stage functional review of the MOPH is underway, intended to help guide future capacity-building efforts. Page 46 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Token government contributions to project financing make little sense. 159. A coherent national health financing strategy incorporating realistic projections of allocations from tax revenues, external funds and out-of-pocket contributions is obviously extremely important, especially for a sector- wide project. In many countries, a key challenge is planning for steadily increasing domestic contributions to substitute for projected declines in aid flows to ensure sustainable financing for progress toward universal health coverage. And even in a more limited health project, it may be extremely important to include a government financing contribution, for example, to maintain a line-item in the national budget in case donor funds fail to materialize for essential items, or to ensure that the project is fully funded. However, in Afghanistan’s particular situation, although the Government’s commitment at appraisal to contribute US $30 million dollars to SEHAT’s financing was regarded as an important step, the MOF’s later decision not to allocate these funds was sensible. Fewer activities were undertaken under Component 2 than originally expected, and so the funds were not needed. There were many other very pressing needs for limited Government resources, and the government was already funding about one quarter of all non-private health care spending outside of SEHAT. In this context, with a project of nearly US$600 million, and adequate donor funds, the US$30 million could seem an unnecessary token Government contribution. It should not be seen as a sign of weak commitment to the project. Good quality monitoring and evaluation is worth investing in. 160. Adequate spending on rigorous, high quality, independent M&E can help ensure that investments in health systems provide the expected benefits. Contracting an Independent expert and impartial agency to undertake household and facility surveys, HMIS data verification, and data analysis and publication can ensure high quality, reliable data. It can also help avoid delays in making the data available. Complaints handling and grievance redress processes are not enough, complaints must be acted upon. 161. For a Grievance Redress System to function and fulfill its potential to contribute to better quality and more responsive health services, there must be sustained commitment and decisive corrective action by the Ministry. The public must have well-justified confidence that it is safe to make complaints and that warranted improvements will be made. An outstanding example is provided by Turkey’s complaint handling system under health Minister Recep Akdağ: the Minister gave out his personal mobile phone number to the public, and assiduously followed up on complaints himself. Health sector workers and the public knew that he was serious and determined. Despite the difficult and insecure environment, Afghanistan has several examples of good grievance handling Systems, in the Citizen Charter Project, National Horticulture and Livestock Project, and Kabul Municipal Development Project. A very strong team, and adequate resources are needed to manage an operation in an FCV environment. 162. SEHAT shows that a strong national team, well supported by the Bank’s top sectoral expertise and adequate resources can achieve success under extraordinarily difficult circumstances. The Afghanistan Country Management Unit has asked for the best health expertise the WB has to offer and been willing to cover the costs, and has retained an excellent and stable national team. The availability of Trust Fund resources to supplement Bank operational budgets has been essential. Ways have been found to enable supervision and support despite very limited ability to travel to the field. These include the unusually extensive data collection, which has resulted in unusually strong end-line data for a Bank operation (that these data show a downturn in many indicators should not be used to penalize the operation’s accomplishments). . Page 47 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To expand the scope of health services provided to the population Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of children under Percentage 24.00 55.00 77.00 five years with severe acute malnutrition who are treated 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Target surpassed at 171%. Data Source for Actual Achieved at Completion is the 2018 AHS. Objective/Outcome: To improve the quality of health services provided to the population Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Score on the balanced Percentage 55.00 70.00 59.30 scorecard examining quality of care in SCs, BHCs and CHCs 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Partially achieved (29% of target). 2017 value was higher (60% of target increase). This is a composite score on the BSC, average mean value across all provinces. Many individual items and the quality areas in particular show substantial improvement over the course of the project, with some fall off in 2018. Page 48 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Objective/Outcome: To expand the coverage of health services provided to the population Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Contraceptive prevalence rate Percentage 18.00 30.00 16.30 (any modern method) 25-Jan-2011 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Not met. Baseline from MICS 2010/11 was 18% for modern methods (19.5% reported in PAD was for all methods), but MOPH 2014 data was 13.8%, suggesting that the MICS 2010/11 may have overestimated, and CPR may not have fallen over the project. Questions about desired and actual family size and timing of pregnancies suggest substantial unmet demand. A fall in CPR would imply serious disruption in supply, but data on supplies do not indicate this. Data source for Actual achieved at Completion is 2018 AHS. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Births (deliveries) attended by Number 429305.00 566683.00 566683.00 890240.00 skilled health personnel (number) 15-Dec-2012 29-May-2015 30-Jun-2018 Comments (achievements against targets): Target surpassed by 336%. Target was too low, but the gain was strong - number of deliveries attended by skilled personnel more than doubled over the project. The percent of all deliveries attended by skilled personnel also increased from 47% in 2012 to 56% in 2018 (and higher in 2015). Data source for Actual Achieved at Completion is HMIS. Objective/Outcome: Ensure that the poor particularly benefit from the expansion in the scope, coverage and quality of health services Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 49 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) PENTA3 coverage among Percentage 28.90 60.00 45.00 children aged between 12 -23 months in lowest income 28-Jan-2011 28-Feb-2013 30-Jun-2018 quintile Comments (achievements against targets): Partially achieved at 52% of target. Baseline source was the MICS 2010/11. Value in 2015 AHS was 58%, well above the year 3 target of 35%. Final value in the 2018 AHS had fallen somewhat, but still 45% higher than baseline value. Data source for Actual Achieved at Completion is 2018 AHS. Objective/Outcome: Enhance the stewardship functions of the Ministry of Public Health Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion The accreditation for Yes/No N Y N procurement of goods and works achieved and maintained 28-Feb-2013 29-Feb-2012 30-Jun-2018 Comments (achievements against targets): Accreditation for procurement of goods and works not achieved. However, MOPH maintained its full accreditation for procurement of services, which accounted for most of the project expenditures. A.2 Intermediate Results Indicators Component: Component II: Building the stewardship capacity of the MoPH and system development Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of national hospitals Number 0.00 15.00 0.00 Page 50 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) with full budgetary autonomy 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Not met. The change in national decentralization policy under the National Unity Government after 2014 prevented the expected hospital autonomy program from going ahead. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of budget from the Percentage 0.00 70.00 0.00 Provincial Budgeting Initiative executed 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Not met. The change in national decentralization policy under the National Unity Government after 2014 prevented the provincial budgeting initiative from being implemented. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Capacity of drug quality control Yes/No N Y Y lab developed 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Met. Source: MoPH Report. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion HIS annual report prepared Yes/No Y Y Y Page 51 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) and disseminated 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Met. In addition to HIS reports, weekly surveillance reports, and periodic national survey reports also were produced and disseminated including on the MOPH website. Component: Component III: Strengthening program management Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion NGO contracts for BPHS/EPHS Number 27.00 27.00 48.00 76.00 service delivery signed and properly managed as per 28-Feb-2013 28-Feb-2013 30-Jun-2018 agreed timeline Comments (achievements against targets): Target surpassed by 233%. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Proportion of MOPH core Percentage 54.00 75.00 87.00 development budget executed 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Target surpassed by 157%. Especially strong performance compared with other ministries. Component: Component I: Sustaining and Improving BPHS and EPHS Indicator Name Unit of Baseline Original Target Formally Revised Actual Achieved at Page 52 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Measure Target Completion Health Facility Utilization Rate: Percentage 1.60 2.00 1.90 consultation per person per year 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): The HMIS "raw" count was 2.2, surpassing the target increase by 150%. The corrected HMIS value is 1.9, which is 75% of the target increase. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Score on the hospital balanced Percentage 69.00 77.00 78.00 scorecard that examines quality of care in public 28-Feb-2013 28-Feb-2013 30-Jun-2018 hospitals delivering EPHS Comments (achievements against targets): Target surpassed by 112%. Source is Balanced Scorecard data. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of pregnant and Percentage 0.00 50.00 58.00 lactating women who received counseling on infant and young 28-Feb-2013 28-Feb-2013 30-Jun-2018 child feeding (IYCF) Comments (achievements against targets): Target surpassed by 116%. Source: AHS 2018. Page 53 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion NGO contracts for BPHS/EPHS Number 27.00 27.00 48.00 76.00 service delivery signed and properly managed as per 28-Feb-2013 28-Feb-2013 30-Jun-2018 agreed timeline Comments (achievements against targets): Target surpassed by 233%. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion TB treatment success rate Percentage 89.00 90.00 90.00 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): HMIS system data report this target as met. However, this ICRR judges that it was not met, based on MOPH project progress reports to WB, and the reported rise of drug-resistant TB in the country, as well as the difficulty of providing treatment in conditions of escalating insecurity. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of IDUs reached by Percentage 27.00 50.00 0.00 Needle Syringe Program 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): It proved impossible to find a competent contractor to conduct the integrated bio-behavioral survey Page 54 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) needed to estimate this indicator. Data on the percentage of IDUs not available. However, there are data on the numbers of IDU reached with needle/syringe and other harm reduction services that show a large increase during the project. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Pregnant women receiving Number 723614.00 40.00 926226.00 1461781.00 antenatal care during a visit to a health provider (number) 28-Feb-2013 28-Feb-2013 30-Jun-2018 Comments (achievements against targets): Revised target surpassed by 364%. Source: HMIS. The original indicator and goal were to increase the percent of pregnant women in the lowest quintile who received at least one ANC care visit from 26% to 40%, which was exceeded by 135%. Source: AHS 2018 Page 55 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Expand the Scope of health services provided to the population in the project areas Outcome Indicator - Proportion of children under age five years with severe acute malnutrition who are treated Intermediate Results - Percentage of pregnant and lactating women who received counselling on infant and young child feeding Indicators 1. Three rounds of contracts signed with NGOs to provide the expanded packages of services including nutrition services. The first round of contracts covered BPHS in 21 provinces and EPHS in 10 provinces. From 2015 contracts covered BPHS nationwide in 34 provinces and EPHS in 15 provinces. Contracts for management consultants to support provision by MOPH of the expanded packages in 3 provinces, including urban Kabul. Under these contracts, by the end of the project, 1,028 (53 percent) of 1,922 health facilities across the country provided outpatient services for the treatment of severe acute malnutrition (SAM) and 145 inpatient facilities and 668 (34 percent) of outpatient facilities provided services to complicated SAM and moderate acute malnourished (MAM) children respectively. In 2017. 12 integrated mobile nutrition teams Key Outputs by Component provided services in hard to reach areas, with another 25 integrated mobile nutrition teams added in 2018. (linked to the achievement of 2. 292 new sub-centers and 48 new family Health Houses created the Objective/Outcome 1) 3. 259 facilities upgraded to be able to provide a wider range of services, including adding laboratories in many facilities, and procuring equipment 4. About 1800 female nutrition counsellors were trained, hired and deployed to health facilities in 18 provinces with most severe child nutrition. A physiotherapist was hired at each district hospital and a psychosocial counselor (nurse) at each community health center with funding for mental health interventions 6. In 2014, 236,121 acutely malnourished children were provided with life-saving treatment services, 315,890 in 2015, 400,488 in 2016 and 457,000 in 2017 6. NGO contracted to provide services to support the mental health hospital in Kabul Objective/Outcome 2: Expand the coverage of health services provided to the population in the project areas - Number of deliveries attended by skilled health personnel Outcome Indicators - Contraceptive prevalence rate (any modern method) Page 56 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) - Number of pregnant women who received antenatal care during a visit to a health provider Intermediate Results - Health facility utilization rate: consultation per person per year Indicators - Percentage of People Who Inject Drugs reached by needle/syringe program 1. About 2,000 additional community midwives and community nurses were trained. 2. The percent of facilities with at least one female staff rose to 99%. 3. Four contracts signed with NGOs to provide HIV harm reduction services (needle/syringes, condoms, counselling, testing, opioid substitution, etc.) in identified “hot spots” to people who inject drugs to reduce their elevated risk of HIV-infection. Providers were trained. Throughout the project, between 3,400 and 3,960 PWID were provided with regular harm reduction services, well over 7 million clean needle/syringes were handed out and over 6 million used needles were collected for safe disposal, and 207 people provided with opioid substitution therapy. 4. NGOs were contracted to provide health services specifically for marginalized populations (prisoners and nomads) 5. Innovations were devised to expand coverage tested with assistance from partners (e.g. Family Health House program with UNFPA), assessed and rolled out where successful 6. A mapping exercise identified “white areas” - populations with no access to health services or where the Key Outputs by Component provision of health services was limited, as a basis for decisions on where to set up new facilities and to (linked to the achievement of schedule mobile services the Objective/Outcome 2) 7. 292 new sub-centers and 48 new Family Health Houses were created to provide community health services in previously unserved areas 8. 259 facilities upgraded to provide a wider range of services, including adding laboratories in many facilities, and procuring equipment 9. In 2016, contraceptive implants were added to the Essential Drugs List, and a program of provider training and community mobilization to encourage uptake of modern contraception began to be implemented 10. The percentage of fully immunized children rose from under 30% in 2012 to nearly 60% in 2015, and was 50% in 2018. Coverage of individual vaccines varied from 47-80% in 2012, all rose in 2015 (ranging from 70%-84%), ending lower in 2018 but still well above the baseline (range of 60-77%) see Figure 4. 11. The number of health facility visits per person per year rose from 1.6 to around 2.0. 12. The number of births attended by skilled healthcare providers more than doubled, from 429,305 in 2012 to 890,240 in 2017/18. This was an increase in the percent of births with skilled attendance from 48% in 2012 to 58% in 2015 and 56% in 2018. Page 57 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 13. The number of women who received antenatal care (ANC) during a visit to a health provider rose from 723,614 in 2012 to 1,461,781 in 2017/18. In percentage terms, this was an increase from 54% in 2012 to 63.5% in 2018. Objective/Outcome 3: Expand the quality of health services provided to the population in the project areas Outcome Indicator - Score on the balanced scorecard examining quality of care in SCs, BHCs and CHCs Intermediate Results - Score on the hospital balanced scorecard that examines quality of care in public hospitals delivering EPHS Indicators - TB treatment success rate 1. Results-based financing pilot completed and evaluated. Results used as input for refining performance- based contracts with NGOs in 31 provinces to make 20% of the contract amount conditional on performance, including on quality of care. 2. New National Health Policy 2015-2020 developed by MOPH discussed quality of health services and care as one of the five most important health sector challenge (listing it third, after Governance and Institutional Issues, and Access and Equity) Key Outputs by Component 3. Community midwives and community nurses trained (linked to the achievement of 4. Continual in-service training provided to staff to enhance quality of care the Objective/Outcome 3) 5. BSC data were collected on the quality of care in four rounds of facility surveys. Results were discussed with NGO managers, provincial staff and MOPH technical departments to support improvements 6. Average, mean and range of scores on many areas of quality improved substantially, and despite some flattening or even falling scores, substantial improvements were maintained by the end of the project. 7. 259 facilities were upgraded, including adding laboratories in many facilities, which enabled better quality 8. Protocols and guidelines for providing services were developed, disseminated to facilities, and training provided in their use. Objective/Outcome 4: Expand the health services provided especially to the poor in the project areas - Births attended by skilled health personnel among lowest income quintile Outcome Indicators - PENTA3 coverage among children aged between 12 -23 months in lowest income quintile Intermediate Results Indicator - Antenatal care coverage - at least one visit among lowest income quintile Page 58 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 1. 292 new sub-centers and 48 new Family Health Houses, plus mobile services were established to reduce travel time to nearest health care point, with particular consideration for the poor who tend to live further from health facilities. 2. The percent of births with skilled attendance among women in the poorest quintile increased from 22% in 2012 to 35% in 2015 and ended the project at 30% in 2018. Key Outputs by Component 3.The percentage of pregnant women in the poorest quintile who received at least one ANC visit increased (linked to the achievement of from 41% in 2012 to 58% in 2015 and 46% in 2018. the Objective/Outcome 4) 4. PENTA3 coverage among children aged between 12 -23 months in the lowest income quintile rose from 29% in 2010/11 to 58% in 2015 and fell to 45% in 2018. Coverage of other vaccines showed a similar trend, but were higher. 5. Disparities in ANC coverage, SBA, facility deliveries and under 5 mortality all fell over the course of the project. 6. There were no differences across wealth quintiles in satisfaction with health services. Objective/Outcome 5: Enhance the stewardship functions of the Ministry of Public Health Outcome Indicator - MOPH accreditation for procurement of goods and works achieved and maintained - All NGO contracts for BPHS/EPHS service delivery awarded and signed as per timeline. - HIS annual report prepared and disseminated Intermediate Results - Number of national hospitals with full budgetary autonomy Indicators - Proportion of budget from the Provincial Budgeting Initiative executed - Capacity of drug quality control laboratory developed - Proportion of MOPH core development budget executed 1. Three rounds of contracts were signed with NGOs to provide the expanded packages of services including nutrition services. The first round of contracts covered BPHS in 21 provinces and EPHS in 10 provinces. From Key Outputs by Component 2015 contracts covered BPHS nationwide in 34 provinces and EPHS in 15 provinces. Contracts for (linked to the achievement of management consultants to support provision by MOPH of the expanded packages in 3 provinces, including the Objective/Outcome 5) urban Kabul. 2. “Contracting in” of management services completed smoothly for 3 provinces in which MOPH delivered health services. Page 59 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 3. Third party M&E firm recruited and, as per contract, undertook all activities: verified NGO performance data, verified drug stocks in BPHS and EPHS facilities, completed 2015 and 2018 AHS. 4. About 250 MOPH staff deemed essential for continuation of health services management and delivery, financed for last 3 years of SEHAT, providing bridge funding to fill gap created by long delays in the project for Capacity Building for Results Facility. 5. The proportion of the MOPH core development budget that was executed rose steadily from 54% in 2013 to 87% in 2017 (well above the average level across the government). 6. Annual Health Information reports, weekly surveillance reports, and periodic national survey reports were produced and disseminated including being posted on the MOPH website. 7. Good progress was made in implementing the plan for developing national capacity for drug quality control. 8. National policy documents articulate the importance of health and health services for all as a basis for national development and well-being of the population. 9. MOPH maintained a consistent and sustained focus on basic and essential services and achieved continued progress toward well-articulated national health goals that aim to improve health outcomes for all. The AHS 2018 estimated that infant, child and under-5 mortality continued to fall during SEHAT. 10. High level health sector policy seminars (with around 40 were held each year during the project, focused on strategically selected areas of the project, health policy or stewardship. Participants included H.E. the Minister of Public Health, Deputy Ministers and Senior Advisers of the MOPH, representatives from major SEHAT donors and implementing partners, UN agencies, World Bank team and experts. The discussions fed into policy and programming decisions. 11. A Presidential Health Summit was held in 2017, chaired by President Ghani, to take stock of health policies, programs, progress and challenges and to discuss a draft “Health Sector Development Framework 2017-2022”. 12. The average per capital cost of delivering the BPHS/EPHS was estimated at US$2.57 in 2012 and US$5.7 in 2018 for an expanded set of services, at the low end of the estimated cost for all low-income countries, indicating that service delivery is efficient. 13. Repeated training in procurement and technical assistance was provided to MOPH. 14. Teams of staff from MOPH participated in the WB Flagship course on Health System Strengthening and Sustainable Financing, and applied the contents to specific challenges in Afghanistan. Page 60 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS At Approval Stage SN Name Title Bank Staff 1. Ghulam Dastagir Sayed Senior Health Specialist, Task Team Leader 2. Abdul Mohammad Durani Social Development Specialist 3. Abdul Raouf Zia Senior Communications Officer 4. Aimal Sherzad Procurement Specialist 5. Asha Narayan Senior Financial Management Specialist 6. Asif Ali Senior Procurement Specialist 7. Asif Qurishi Team Assistant 8. Asta Olesen Senior Social Development Specialist 9. Chau-Ching Shen Senior Finance Officer 10. Cornelis P. Kostermans Lead Public Health 11. Inaam Ul Haq Senior Health Specialist 12. Julie-Anne M. Graitge Program Assistant 13. Kees Kostermans Lead Public Health Specialist 14. Lori A. Geurts Operations Analyst 15. Luc Laviolette Senior Nutrition Specialist 16. Mariam Haidary Program Assistant 17. Marjorie Mpundu Senior Counsel 18. Mohammad Arif Rasuli Senior Environmental Specialist 19. Mohammad Tawab Hashemi Health Specialist 20. Mohammad Yasin Noori Social Development Specialist 21. Muhammad Wali Ahmadzai Financial Management Analyst 22. Tekabe Ayalew Belay Sr Economist (Health) Non Bank Staff 1. Kristian Orsini Deputy Head of Operations, EU 2. Dr. Sefatullah Habib Project Officer Health, EU Page 61 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) At Supervision Stage: SN Name Title Bank Staff 1. Ghulam Dastagir Sayed Senior Health Specialist, Task Team Leader 2. Mohammad Tawab Hashemi Senior Health Specialist, Task Team Leader 3. Abdul Mohammad Durani Social Development Specialist 4. Abdul Raouf Zia Senior Communications Officer 5. Aimal Sherzad Procurement Specialist 6. Asha Narayan Senior Financial Management Specialist 7. Ahmad Rafi Otofat Program Assistant 8. Akbar Ali Mohammadi Financial Management Analyst 9. Andreas Seiter Lead Health Specialist 10. Asif Ali Senior Procurement Specialist 11. Asha Narayan Sr Financial Management Specialist 12. Chau-Ching Shen Senior Finance Officer 13. Juan Carlos Alvarez Senior Counsel 14. Luc Laviolette Senior Nutrition Specialist 15. Mohammad Arif Rasuli Senior Environmental Specialist 16. Nkosinathi Vusizihlobo Mbuya Sr Nutrition Specialist 17. Martha Vargas Program Assistant 18. Patrick M. Mullen Lead Health Specialist 19. Najla Sabri Social Development Specialist 20. Nazaneen Ismail Ali Senior Procurement Specialist 21. Adenike Sherifat Oyeyiola Financial Management Specialist 22. Nargis Mohammad Yousaf Financial Management Analyst 23. Syed Waseem Abbas Kazmi Sr Financial Management Specialist Page 62 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) At ICR Stage: SN Name Title Bank Staff 1. Ghulam Dastagir Sayed Senior Health Specialist, Task Team Leader 1. Mohammad Tawab Hashemi Senior Health Specialist, Task Team Leader 2. Abdul Mohammad Durani Social Development Specialist 3. Aimal Sherzad Procurement Specialist 4. Anand Kumar Srivastava Senior Procurement Specialist 5. Andre C. Medici Senior Economist (Health) 6. Bathula Amith Nagaraj Senior Operations Officer 7. Benjamin P. Loevinsohn Lead Public Health Specialist 8. Deepika Nayar Chaudhery Nutrition Specialist 9. Habibullah Ahmadzai Operations Officer 10. Mickey Chopra Lead Health Specialist 11. Minh Thi Hoang Trinh Program Assistant 12. Mohammad Arif Rasuli Senior Environmental Specialist 13. Mohammad Yasin Noori Social Development Specialist 14. Najla Sabri Social Development Specialist 15. Saeda Jeddi Team Assistant 16. Sayed Wasreen Abbas Kazmi Senior Financial Management Specialist 17. Yasuhiko Matsuda Program Leader Page 63 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 17.877 64,846.93 FY13 89.051 298,124.57 Total 106.93 362,971.50 Supervision/ICR FY13 3.390 11,027.12 FY14 86.348 343,302.51 FY15 131.478 526,999.34 FY16 136.859 678,030.47 FY17 165.309 533,866.59 FY18 106.238 397,414.63 FY19 2.225 15,714.58 Total 631.85 2,506,355.24 Page 64 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 3. PROJECT COST BY COMPONENT Amount at Revised Actual* at Actual as Actual as Components Approval Amount Project Closing Percentage of Percentage of (US$M) (US$M) (US$M) Approved (%) revised (%) Sustaining and improving BPHS and EPHS services 307.0 593.4 520.483 169.5 87.7 Building the stewardship capacity of MOPH and 90.0 38.5 14.920 16.6 38.8 system development Strengthening program management 10.0 22.0 13.604 136.0 61.8 Total 407.0 653.9 549.007 124.8 77.7 Actual disbursement data as of December 31, 2018, per Government records. Formal closing date was June 30, 2018, plus 6 months grace period for disbursements after project closing. Page 65 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 4. EFFICIENCY ANALYSIS The major focus of the project was funding delivery of the BPHS and EPHS across the country; prioritized sets of high-impact interventions with proven cost-effectiveness, well aligned with national health priorities and with the international health agenda of achieving Sustainable Development Goal 3: “Ensure healthy lives and promote well-being for all at ages”. The BPHS/EPHS aimed and have proved an effective way to respond to the basic health needs of rural and urban communities in Afghanistan. Afghanistan’s delivery of BPHS/EPHS through contracting out has become a model for other fragile states trying to rebuild their health system after emerging from conflict. In the health sector, the cost-effectiveness ratio (CER) is the gold-standard measure of efficiency. Unlike traditional economic evaluation that expresses costs and benefits in monetary terms, in CER, benefits are expressed in natural units (e.g. deaths averted). Disability-Adjusted Life Years (DALYs) averted are frequently used as a measure of benefit, especially when interventions are being compared. DALYs account for both premature deaths and years of disability avoided as a result of an intervention. Using evidence from low-and-middle income countries, the interventions in the BPHS are considered highly cost- effective in addressing infant, neonatal, and maternal mortality. CER estimates were obtained from systematic reviews conducted in developing countries (mostly in Afghanistan or other South Asian countries). We followed the threshold recommended by the World Health Organization Commission on Macroeconomics and Health (2001) for identifying cost-effective interventions: interventions with a cost per DALY below Afghanistan’s per capita gross domestic product (approx. US$550 in 2016) are considered very cost-effective, and a cost not more than three times the per capita GDP to be cost-effective. Cost-effectiveness of interventions in BPHS package Cost- CER/GDP BPHS interventions effectiveness Remarks on cost-effectiveness per capita (US$/DALY) Highly cost-effective. Model Modern family planning provision 130 0.3 parameters based on Afghanistan (Carvalho, Salehi and Goldie, 2013) Pregnancy package (ANC, family Highly cost-effective. Model planning, skilled birth attendance, 143 0.3 parameters based on Afghanistan information about access to transport, (Carvalho, Salehi and Goldie, 2013) referral facilities and quality of care) Highly cost-effective. In developing Treatment of Severe of Acute countries, ICER ranges from US$26- 26 <0.1 Malnutrition (SAM) 100, but evidence suggest lower ICER in poorer countries. HIV prevention programs for high-risk Highly cost-effective. Parameters groups (including PWID) 143 0.3 based on India (Vassall et al 2014). Expanded Program of Immunization Highly cost-effective. Based on the (EPI) <100 0.2 pooled estimates of Disease Control Priorities (DCP). Page 66 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Impact on human development and economic growth. In addition to promoting human development, improved access to maternal and child health services through BPHS/EPHS is likely to have had a positive impact on long- term economic growth by supporting human capital formation and increasing women’s labor supply and providing several thousand jobs in health care for women. Additional returns would have been realized by (a) reduced risk of impoverishment due to high health-related out-of-pocket expenditures since BPHS/EPHS services are provided without cost to patients, (b) reduced burden on households from lost income during illness-related incapacitation, and (c) improved learning capacity of children whose nutrition and health was improved by the project. According to the World Health Organization’s Global Investment Framework for Women’s and Children’s Health, investments in maternal and child health in lower-income countries such as Afghanistan produce social and economic returns that are up to 15 times greater than their costs. The project will have contributed to reduced morbidity and mortality of mothers and children by improving equitable access to both primary and hospital-based care. Technical efficiency can be assessed by estimating improvements in the delivery of health services compared with operating costs at the facility level. Estimates in the early years of the project found that health system performance, measured by the average composite aggregate score (ACS) for six domains of the Basic Scorecard (BSC) – client and community, human resources, physical capacity, quality of services provision, and management systems—improved faster than the increase in costs of BPHS implementation. Per capita BPHS costs for each point gained in the ACS was almost 8 percent lower in 2015 than 2014, indicating improvements in technical efficiency in the delivery of services by health facilities. Allocative efficiency was assessed by estimating disability-adjusted life years (DALYs) averted by the SEHAT project interventions. SEHAT focused on reducing communicable, maternal, neonatal, and nutritional causes of mortality and morbidity. In 2011, these causes accounted for 47 percent of the overall burden of diseases (BoD) in Afghanistan. SEHAT project support for delivering the BPHS/EPHS across the country contributed to a reduction of communicable, maternal, neonatal, and nutritional causes to 36 percent of the BoD, averting an estimated 8.5 million DALYs. Equity analysis compared access to services and one health outcome (under 5 mortality) across wealth quintiles. Health interventions provided by BPHS/EPHS improved access at least as much, if not more for people in the lowest economic quintile compared to other quintiles. For example, the Gini coefficient for institutional deliveries fell from 0.863 to 0.525 between 2010 and 2015; this indicates great progress in providing safe maternity-related care especially to poor mothers and their infants. (A Gini coefficient of 0 would indicate perfect equality, the more unequal services are, the closer the coefficient to a value of 1.) A significant proportion of interventions supported by BPHS under the SEHAT project were self-targeted to the poor, contributing to increased access. Equity analysis With antenatal care Family planning Facility-based delivery Skilled birth attendant Under-5 mortality rate from a skilled (modern) Quintile (%) (%) provider (%) (%) 2010 2015 2010 2015 2018 2010 2015 2018 2010 2015 2018 2010 2015 2018 Poorest 106 81 20.6 22.2 29.9 11.7 24.0 30.1 44.0 50.4 47.7 16.2 15.0 8.8 Q2 86 64 25.6 34.5 41.5 21.2 36.9 42.6 57.7 50.0 54.8 18.7 16.1 11.9 Q3 87 71 36.6 41.0 48.7 32.7 43.6 54.9 59.7 53.7 62 16.6 15.7 12.4 Q4 60 51 43.8 62.6 63.9 40.7 64.9 67.8 60.2 63.7 71.9 18.9 22.0 17.9 Richest 49 40 56.6 82.5 83.9 68.0 85.2 85.5 77.9 76.1 81.2 29.2 30.5 31.0 Q1/Q5 2.2 2.0 0.4 0.3 0.4 0.2 0.3 0.35 0.6 0.7 0.6 0.6 0.5 0.3 Source: AMS 2010, DHS 2015, AHS 2018 Page 67 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Borrower Comments Thank you for sharing SEHAT Project Implementation Completion Result (ICR) draft report. The MoPH team reviewed the ICR. Overall it’s well written and captured all aspects of the project. Additionally, we have found it reliable, evidence based, well organized and comprehensive for which we sincerely appreciate your objectivism. It has been a great pleasure that the MoPH in collaboration with development partners and BPHS/EPHS implementing NGOs has successfully completed SEHAT Project. In fact, the achievements were not only impressive in SEHAT project component-I, but also there were tangible progress in component II. A recent analysis of the component II which has probably not been shared before, indicates that, despite the progressive challenges and bottlenecks - lagging the progress and led to sub-optimal use of the opportunities, the ultimate results achieved in component II were encouraging. The MoPH analysis prevails that out of 65 agreed deliverables over 12 system development thematic areas under component II, 30 (46%) of those were successfully accomplished and 15 (23%) were partially achieved. There were 11 (17%) of the deliverables found to be overlapped interventions with off budget and/ or other government parallel activities. 7 (11%) of the deliverables were either dropped or failed to be accomplished, while 2 (3%) deliverables were carried forward to Sehatmandi project. In addition, the SEHAT project final fiscal status has just been updated as some of the pending payments were made during the project grace period. We have highlighted the points in the report in track change mode for your kind information. Being said that, we would kindly request you to please review our observations which may support the document finalization process. Availing the opportunity, we will once again thank you for your time and efforts in this endeavor. Best Regards, Sehatmandi Project Coordination Office ICR Author’s note: The MOPH final disbursement figures were confirmed by the WB project finance staff, and are reflected in the ICR. The information on the status of Component 2 deliverables was also added into the ICR text, and a list of all 65 deliverables and their status added in Annex 7. EU comments 1. SEHAT was established as the result of a joint collaboration between the Ministry of Public Health (MoPH), the World Bank (WB), the EU and USAID in the health sector of Afghanistan and represents a further step move toward a Sector Wide Approach (SWAp). It contributes to increased donor coordination, policy dialogues and government ownership of the sector. 2. Regular joint reviews of the project by the WB, MoPH and Development Partners have shown that SEHAT Page 68 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) made a substantial contribution to improving the health sector. Joint missions and third party monitoring identified challenges and barriers to service delivery and agreed on corrective actions and follow up to address the issues for the smooth implementation of the project. 3. PDO Part 1: Expand the scope of health services provided: 1) According to the revised BPHS, each Comprehensive Health Centre (CHC) should have at least one psychosocial counsellor (PSC). The EU financed training of more than 350 Psychosocial counsellors and all of them have been recruited in CHCs. 4. New services: The additional requirements included a physiotherapist in each district hospital and a psychosocial counselor at each Comprehensive Health Center. 5. The gains in nutrition services are not solely attributable to SEHAT: the EU earmarked EUR 17 million for invocations to improve nutrition interventions at the health facility and community through but it has not been mentioned. 6. The model is resilient in conflict situations because of implementing partners (NGOs) and their relationship with the local communities. NGOs have flexibility and decentralized management which provides the possibility of service delivery and access in hard to reach and conflict affected areas. However, there are concerns about the reporting of performance by NGOs under the Balanced Score Card and facility assessments. Due to performance-related pay, there is an incentive for NGOs to report good performance. The performance scores are not consistent with reports of high OOP and drug stock outs (which indicate poor quality of service). 7. The AHS 2018 did not provide estimates MMR and concerns have been raised by the MoPH and stakeholders. 8. The report mentioned establishing hundreds of additional HFs but does not provide information on the rational distribution of these health facilities (examining HMIS data, some HFs are extremely underutilized). 9. The AHS survey 2018 also showed a decline in some health services e.g. vaccination which further questions the role of these additional HFs. 10. The challenges in the procurement under SEHAT are not specifically highlighted. Some NGOs submitted low financial bids that may have been unrealistic and, therefore, -+ BPHS /EPHS may not have been fully implemented. 11. Concerns related to communication and non -timely sharing of information with stakeholders was raised and discussed as one of the main challenges on a number of occasions. Therefore, we were expecting that this would be mentioned in the report and recommendations made on how to improve communication during next round (SEHATMANDI). 12. The report does not highlight the provision of health services to IDPs and returnees. Although the report stated that a specific budget allocation was made for IDP and returnees (contingency budget), it does not provide information on coverage of health services to IDP and returnees. According to a number of humanitarian reports, a large number of IDP and returnees still receive health services from the humanitarian funding. This is an important area to be considered in the future. Considering the protracted situation of displaced population and reports from humanitarian donors, provision of services to IDP and returnees needs to be strengthened. 13. Significant changes during implementation: Based on projections during the MTR of the SEHAT it was reported that there will be some savings under Component 2 due to delays in the design. The mission advised the MOPH to set realistic disbursement projections and relocate the expected under- spend amount to Component 1. ICR Author’s note: Item 5 above has been reflected in the text of the ICR. Page 69 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) USAID Comments: ● Overall, the report would benefit from a methodology statement that clearly establishes who wrote the document, for what purpose and what methods were used to arrive at certain conclusions. ● Some or most of the data references are outdated. For example, number of health facilities are compared between 2002 and 2013. We may use the most up to date information where that is possible. More such data references can be found on page 13 under challenges. The situation may have changed a lot since MICS 2010. ● Page 13. A variety of statistics are used to demonstrate progress in health outcomes. However, none of those statistics has a citation. It is important for us to have source data to evaluate where the numbers are coming from. ● Page 20. Top of the page. USAID continued to manage health services in 13 provinces until June 2015 not March. ● Page 20. Total grant released from ARTF to SEHAT is $ 480 million. While the total estimated cost was around $650 million. The report doesn’t speak about any future planning/reprogramming the unspent balance. ● Beginning page 23, the report publishes ratings on PDO Part 1: “Expand the scope of health services provided”. There is discussion that the project expanded the scope of services in two ways: (1) by including additional services in the BPHS that NGOs were contracted to provide, with particular focus on nutrition services; (2) by upgrading facilities. However, there is no reference or citation providing quantitative evidence that additional services were added or that facilities are upgraded. And as the next suggestion notes, it’s uncertain what qualifies a rating as “high” or otherwise. ● Page 26, item 49 on IDU coverage contains a logical inconsistency. Initially, the argument is made that a qualified bidder could not be secured to complete the IBBS that would establish a denominator for IDU. But the very next sentence refers to coverage as a percentage of IDU, noting that the project could not double coverage from 27% to 50%. If there is no denominator, how are these percentages calculated? ● Rating is not very clear. For example what defines “High” or “substantial”. Additional description could help to understand the rating and its rational better. ● Page 37, item 86 (overall outcome rating and institutional strengthening) has not touched upon component two which was planned to strengthen MoPH stewardship capacity. Likewise, there would have been some factors affected the implementation under component two but no such information is found on page 41, item 101. It is also not very clear how/if the slow implementation of component two had affected the overall SEHAT implementation. ● Some indicators reported up to 360% over achievement. Will this information affect the target setting for SEHAT follow on -Sehatmandi. Whether it will be possible this ICR explain some potential reasons for this. ● Page 55 Results Framework and Key Outputs: The key outputs indicators list multiple examples of targets that were exceeded by very large percentages. For example, the “proportion of children under five years with severe acute malnutrition who are treated” was exceeded by 171%. The “Births (deliveries) attended by skilled health personnel (number)” was surpassed by 336%. Alternately, some indicators such as “Score on the balanced scorecard examining quality of care in SCs, BHCs and CHCs” achieved only 29% of the target. This suggests not that the SEHAT program was especially successful in certain areas, but rather the target setting on which objectives were based did not employ good data or that target setting practices were flawed. ● Page 83 Section 6. The report cites a 2017 SIGAR report to support the statement that “9 projects Page 70 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) funded (off-budget) by USAID during the implementation of SEHAT addressed issues that were also in the list of possible areas for support under Component 2.” Is the inference that USAID sought to duplicate World Bank projects? If that is the case, USAID would be very interested in examining project document dates to establish temporal order of implementation. ● Page 53. MoPH has often experienced challenges meeting the procurement deadlines. For example, delayed procurement of TMP under SEHAT and extended NGO contracts for service delivery in the past to allow additional time for MoPH/GCMU to complete the procurement (evaluation, negotiation, and contract). But this point is missing in recommendation and lessons learned. ICR Author’s note: Items have been addressed where feasible in the ICR. Page 71 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 6. SUPPORTING DOCUMENTS (some with links) Project Documents Afghanistan Reconstruction Trust Fund Grant No. TF015005 System Enhancement for Health Action in Transition Project Amendment to Amended and Restated Afghanistan Reconstruction Trust Fund Grant Agreement, Amendment April 03, 2017, and April 25, 2017. Amended and Restated Afghanistan Reconstruction Trust Fund Grant Agreement (Systems Enhancement for Health Action in Transition Project), ARFT Grant Number TF015005. August 27, 2013, Amended and Restated June 13, 2015. Amendment to the Financing Agreement, Grant Number H829-AF (System Enhancement for Health Action in Transition (SEHAT) Project)) June 13, 2015. Financing Agreement (2013), (System Enhancement for Health Action in Transition Project) between Islamic Republic of Afghanistan and International Development Association, May 6, 2013. Grant Number H829-AF. KIT Royal Tropical Institute (2018). The Balanced Scorecard Report – Basic Package of Health Services 2018, June 2018 – Final Draft 2018. KIT Royal Tropical Institute (2018). The Balanced Scorecard Report – Essential Package of Health Services 2018, Final 2018. KIT Royal Tropical Institute and Silk Route (2015). The Balanced Scorecard Report – Afghanistan Hospitals 2015, September 2015. World Bank (2018), Progress in the Face of Insecurity – Improving health Outcomes in Afghanistan, Washington DC. World Bank (2012), Afghanistan - System Enhancement for Health Action in Transition Project: environmental assessment (Vol. 2): Environment and social management framework (English). s.l. ; s.n.. http://documents.worldbank.org/curated/en/508791468185978310/Environment-and-social- management-framework World Bank (2014), Implementation, Completion and Results Report, Strengthening Health Activities for the Rural Poor (SHARP), February 2014. World Bank (various dates, 2013-2018), Afghanistan System Enhancement for Health Action in Transition Project (P129663) Implementation Status and Results Reports (ISRs) and associated mission Aides-Memoire, ISR Sequence 1 –11, May 16, 2013; November 26, 2013; May 31, 2014; November 9, 2014; May 5, 2015; November 5, 2015; January 23, 2016; July 05, 2016; January 10, 2017; September 19, 2017; June 26, 2018. The World Bank Team (2016). Afghanistan System Enhancement for Health Action in Transition (SEHAT) Project, Mid-Term Review Issue Paper, May 2, 2016. World Bank (2016), Afghanistan System Enhancement for Health Action in Transition Project (P129663) Mid-Term Review, March 2016, Mission Aide Mémoire. Page 72 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) World Bank (2013), Emergency Project Paper on a Proposed Grant in the Amount of SDR65.1 million (US$100 million Equivalent) to the Islamic republic of Afghanistan for the System Enhancement for Health Action in Transition Project, February 13, 2013 (Report No. 73645-AF) Other Official Documents - Government of Islamic Republic of Afghanistan, and World Bank Central Statistics Organization (2018), Afghanistan Living Conditions Survey 2016-17. Kabul, CSO Islamic Republic of Afghanistan, Ministry of Public Health (2010). A Basic Package of Health Services for Afghanistan – 2010/1389, Revised July 2010. Kabul, Afghanistan. Islamic Republic of Afghanistan, Ministry of Public Health. Comprehensive Health Care Waste Management Plan (HCWMP) For the System Enhancement For Health Action In Transition (SEHAT) Project, October 2014. Islamic Republic of Afghanistan, Ministry of Public Health. National Health Policy 2015 – 2020. Kabul, November 2015. Islamic Republic of Afghanistan (2017). Afghanistan National Peace and Development Framework (ANPDF) 2017-2020. Kabul, Afghanistan. http://extwprlegs1.fao.org/docs/pdf/afg148215.pdf MOPH (2014) Afghanistan HMIS Data Quality Assessment Report - Descriptive Report. General Directorate of Plan and Policy, Ministry of Public Health (MOPH), Kabul, Afghanistan, April 1, 2014. http://moph.gov.af/Content/files/Data%20Quality%20Assurance%20Assessment%202014.pdf Ministry of Public Health, Afghanistan. System Enhancement for Health Action in Transition (SEHAT) Project, MOPH 6 Monthly Reports, January-June 2015; July-December 2015; May–October 2016; Nov 2016-June 30 2017; July-Dec, 2017. Walters D, JD Eberwein, LB Schultz, J kakietek, H Ahmadzai, P Mustaphi, KMA Saeed, MY Zawoli and M Shekar (2018). An Investment Framework for Nutrition in Afghanistan: Estimating the Costs, Impacts, and Cost-Effectiveness of Expanding High-Impact Nutrition Interventions to Reduce Stunting and Invest in the Early Years. Health, Nutrition and Population (HNP) Discussion Paper, World Bank Group, Washington DC, April 2018 World Bank (2018), Afghanistan Fiscal Performance Improvement Support Project P159655, Implementation Status and Results Report (ISR) Sequence 2, November 29, 2018. World Bank Group (2018), Progress in the Face of Insecurity – Improving Health Outcomes in Afghanistan, World Bank, Washington DC, 2018. World Bank Group (2017), Bank Guidance: Implementation Completion and Results Report (ICR) for Investment Project Financing (IPF) Operations, OPS5.03-GUID.140, July 5, 2017. World Bank Group (2016), Partnership Framework for Islamic Republic of Afghanistan for the period FY 17 to FY 20, 2 October, 2016. Report No. 108727-AF World Bank (n.d.). World Bank’s Multi-Sector Nutrition Engagement in Afghanistan, Nutrition Presentation for the Afghanistan Reconstruction Trust Fund Strategy Group Meeting. World Bank (2014), Implementation Completion and Results Report, (ICA-H4690 IDA-H5810 TF-96362) on Grants in the Amount of SDR 51.9 million to the Islamic Republic of Afghanistan for a Strengthening Health Activities for the Rural Poor (SHARP). ICR2975, February 21, 2014. Page 73 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) World Bank (2013), Implementation Completion and Results Report for a grant in the amount of US$10 million to the Islamic Republic of Afghanistan for the Afghanistan HIV/AIDS Prevention Project (P101502), ICR2554, June 21, 2013. General Documents Allman, J, J Rohde, J Wray (1987) Integration and disintegration: the case of family planning in Haiti. Health Policy and Planning, 2 (3): 236–244, https://doi.org/10.1093/heapol/2.3.236 Ariely, D (2017). The Corruption Experiment. Short video summary, accessed on line on Nov 12, 2018 at https://boingboing.net/2017/01/30/dan-ariely-the-corruption-exp.html Arur A, Mohammed-Roberts R and Bos E (2011). Setting Targets in Health, Nutrition and Population Projects. HNP Discussion Paper, Health, Nutrition and Population Network, World Bank, Washington DC, September 2011. Gwatkin DR, Wagstaff A and Yazbeck A (2005), Reaching the Poor with Health, Nutrition and Population Services: What Works, What Doesn’t, and Why. World Bank, Washington DC. Gwatkin DR, Rutstein S, Johnson K, Suliman E, Wagstaff A, Amouzou A (2007), Socio-Economic Differences in Health, Nutrition and Population within Developing Countries: An Overview. 2007, Washington D.C: The World Bank Houweling TA, Kunst AE (2010), Socio-economic inequalities in childhood mortality in low- and middle- income countries: a review of the international evidence. Br Med Bull. 2010, 93: 7-26. 10.1093/bmb/ldp048. Independent Joint Anti-Corruption Monitoring and Evaluation Committee (2016). Vulnerability to Corruption Assessment in the Afghan Ministry of Public Health, June 4, 2016. Johns Hopkins University (2012) Integrated Behavioral & Biological Surveillance (IBBS) in Afghanistan Findings of 2012 IBBS survey and comparison to 2009 IBBS survey. Bloomberg School of Public Health HIV Surveillance Project, Submitted to Afghanistan MOPH National AIDS Control Program, November 2012. Lambsdorff JG (no date) Preventing Corruption by Promoting Trust – Insights from Behavioral Science. Discussion Paper V-69-15, Volkswirtschaftliche Reihe ISSN 1435-3520, Faculty of Business Administration and Economics, University of Passau, Innstr. 27, D-94032 Germany, Newbrander W, Ickx P, Feroz F & Stanekzai H (2014). “Afghanistan's Basic Package of Health Services: Its development and effects on rebuilding the health system”, Global Public Health, Vol 9(Suppl 1), S6- S28. Noori, Murtaza (2017), A Research on Afghanistan Public Procurement System’s Reform. Thesis Submitted to Faculty of Management Studies Indian School of Business Administration and Management. Qarizada AN, P Mustaphi, JA Oketch and S Safi (2018). Scale-up of Integrated Management of Acute Malnutrition (IMAM) services in Afghanistan. Field Exchange 57, March 2018. p.38. www.ennonline.net/fex/57/imamafghanistan Page 74 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Radovich E, el-Shitany A, Sholkamy H, Benova L (2018). Rising up: Fertility trends in Egypt before and after the revolution. PLoS ONE 13(1): e0190148. https://doi.org/10.1371/journal. pone.0190148 SIGAR (2017) Afghanistan’s Health Care Sector: USAID’s Use of Unreliable Data Presents Challenges in Assessing Program Performance and the Extent of Progress. Special Inspector General for Afghanistan Reconstruction, SIGAR 17-22 Audit Report/USAID Support for Afghanistan’s Health Care. Accessed on- line on November 12, 2018 at https://www.sigar.mil/pdf/audits/SIGAR-17-22-AR.pdf Tandon A, Murray C, Lauer J, Evans D (2001). Measuring Overall Health System Performance for 191 Countries. GPE Discussion Paper Series: No. 30 EIP/GPE/EQC World Health Organization UNFPA, USAID (2016) A Comprehensive Family Planning Needs Assessment in the Islamic Republic of Afghanistan Victora CG, Wagstaff A, Schellenberg JA, Gwatkin D, Claeson M, Habicht J-P (2003), Applying an equity lens to child health and mortality: more of the same is not enough. Lancet. 2003, 362 (9379): 233-241. 10.1016/S0140-6736(03)13917-7. WHO, USAID, Health Cluster (2018), Attacks on Health: December 2017 - June 2018. https://reliefweb.int/map/afghanistan/attacks-health-december-2017-june-2018 Yazbeck, Abdo (2009), Attacking Inequality in the Health Sector: a Synthesis of Evidence and Tools, World Bank, Washington DC. Page 75 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) ANNEX 7. ADDITIONAL INFORMATION 1. Per Capita Income, 2013. These data show Afghanistan’s per capita income as lowest in the region in 2013: Gross National Income Per Capita, 2013, Countries in South Asia, 2013 Country Current US$ Afghanistan 680 Bangladesh 1,010 Bhutan 2,300 India 1,520 Maldives 7,110 Nepal 720 Pakistan 1,360 Sri Lanka 3,490 Source: World Development Indicators. GNI, current US $, Atlas method. 2. WHO Definition of Stewardship Stewardship, sometimes more narrowly defined as governance, refers to the wide range of functions carried out by governments as they seek to achieve national health policy objectives. In addition to improving overall levels of population health, objectives are likely to be framed in terms of equity, coverage, access, quality, and patients' rights. National policy may also define the relative roles and responsibilities of the public, private and voluntary sectors - as well as civil society - in the provision and financing of health care. Stewardship is a political process that involves balancing competing influences and demands. It will include: maintaining the strategic direction of policy development and implementation; detecting and correcting undesirable trends and distortions; articulating the case for health in national development; regulating the behaviour of a wide range of actors - from health care financiers to health care providers; and establishing effective accountability mechanisms. Beyond the formal health system stewardship means ensuring that other areas of government policy and legislation promote - or at least do not undermine - peoples' health. In countries that receive significant amounts of development assistance, stewardship will be concerned with managing these resources in ways that promote national leadership, contribute to the achievement of agreed policy goals, and strengthen national management systems. While the scope for exercising stewardship functions is greatest at the national level, the concept can also cover the steering role of regional and local authorities. Source: http://www.who.int/healthsystems/stewardship/en/ Accessed on 09/23/2018 3. Description of Health System Structure in Afghanistan Health services in Afghanistan operate at three levels: 1) Primary Care Services at the community or village level, delivered through Community Health Workers (CHWs), Mobile Health Teams (MHTs), or at Health Posts, Sub-Health Centers (SHCs), and Basic Health Centers (BHCs); 2) Secondary Care Services at district level, through Comprehensive Health Centers (CHCs) and District Hospitals operating in the larger villages or communities of a province; and 3) tertiary care services at the provincial and national levels, in Page 76 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) provincial, regional, national, and specialty hospitals. The services, staffing, equipment and essential drugs at each of six standard types of health facility are defined in the Basic Package of Health Services (primary level), and the Essential Package of Health Services (EPHS) are similarly defined for hospitals. More than 20,000 Community Health Workers (CHWs) help deliver services in rural communities. These volunteer local community members are approved by a community council, and trained, supported and supervised by a health organization. CHWs link their community to the nearest health-care facility. More than half the CHWs are women. They offer basic curative care, diagnosing and treating minor illnesses and conditions common in children and adults, including malaria, diarrhea, community DOTS and acute respiratory infections such as pneumonia. CHWs manage 40-45% of Community Integrated Management of Childhood Illnesses, and refer severely sick children and pregnant women to health facilities. They give iron and folic acid tablets to pregnant women and distribute clean delivery kits. Deliveries are not part of CHWs’ job description, but female CHWs promote birth preparedness, safe home deliveries with a skilled birth attendant (when possible), awareness of the danger signs of pregnancy, and the need for urgent referral when delivery complications occur, and provide basic essential newborn care. CHWs provide 63% of all family planning services including condoms, oral contraceptives, and a first dose of injectable contraceptives. They are responsible for growth promotion and nutrition counseling and micronutrient supplementation; for raising awareness about disability and mental health, and for identifying persons with disabilities and mental conditions. Health Posts (HP). CHWs work from their own homes, which function as community health posts. A health post, ideally staffed by one female and one male CHW, covers a catchments area of 1,000–1,500 people, equivalent to 100–150 families. Health Sub-Centers (HSC) bridge the services gap between Health Posts and other BPHS levels of service delivery and increase access to health services for underserved populations living in remote areas. An HSC is intended to cover a population of about 3,000-7,000, and to ensure that no one lives more than 2 hours walking distance from a facility. HSCs are initially established in private houses. This is a precondition before construction of a permanent facility and requires commitment from the surrounding community. Priority HSC locations are determined by MOPH, specific locations are approved by the PHCC. HSCs should have two technical staff (a male nurse and a community midwife), and a cleaner/guard. They provide most of the BPHS services available in BHCs including health education, immunization, antenatal care, family planning, TB case detection and referral, and follow up of TB cases in coordination with community DOTS. They treat infectious diseases such as diarrhea and pneumonia and refer severe and complicated cases to higher level facilities. Where feasible, HSCs support health posts and CHWs, who provide a copy of their monthly reports to the HSC or the mobile team in their area. The DHO is required to supervise heath posts in their districts. Mobile Health Team (MHT): A limited number of mobile health teams in each province each serve a cluster of districts, to: 1) ensure provision of essential and basic health services in remote villages in geographically hard to access areas; 2) expand and strengthen community-based health care (CBHC) by identifying additional CHWs in hard to access areas and linking community level interventions with BPHS facility-based services; and 3) encourage greater community participation and community ownership of health services. Intervals between visits of the MHT depend on security, remoteness, and the needs of the population, but should occur at least once every two months. Based on the experience of the MHTs, the PHCC can revise the frequency of the visits. Planning for mobile health services needs to be done together with community leaders to gain their support and guidance, and assistance in providing secure Page 77 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) accommodations for overnight stays of the mobile team staff. EPI teams help the PHCC to determine appropriate sites for mobile health services. These services are an extension of BHC services and provide the services recommended for a BHC. The MHT ideally includes a male health provider (doctor or nurse), female health provider (community midwife or nurse), vaccinator, and driver. Basic Health Centers (BHCs) offer antenatal, delivery, and postpartum care; newborn care, non- permanent contraceptive methods; routine immunizations; integrated management of childhood illnesses; treatment of malaria and tuberculosis, including DOTS; and identification, referral, and follow- up care for mental health patients and persons with disabilities. BHCs supervise the activities of health posts in their catchment area. They cover a population of about 15,000–30,000, depending on the local geographic conditions and population density. In sparsely populated areas, BHC catchment population can be less than 15,000. Minimal staffing requirements are a nurse, a community midwife, and two vaccinators. Depending upon the scope of services provided and the BHC workload, up to two additional health care workers may need to be added to perform well-defined tasks such as CHW supervision and outreach activities. A male/female ratio of 1/1 is recommended, there must be at least one female health worker. The MOPH allows a physician to be at a BHC only to replace a midwife or a nurse when those positions are not filled and a physician is available and there is sufficient physician staffing at CHCs and district hospitals. The district hospital physiotherapist should visit BHCs on an outreach basis. Comprehensive Health Centers (CHCs) cover about 30,000–60,000 people. In addition to the services provided by BHCs, CHCs can handle certain delivery complications, grave cases of childhood illness, treatment of complicated cases of malaria, and outpatient care for mental health patients. Persons with disabilities and persons requiring physiotherapy services will be screened, given advice and referred to appropriate services in the area. The facility usually has limited space for inpatient care, but has a laboratory. CHC staff include male and female doctors, male and female nurses, midwives, one (male or female) psychosocial counsellor, and laboratory and pharmacy technicians. Physiotherapists visit CHCs on an outreach basis from the district hospital. District Hospitals (DHs) provide all BPHS services. Patients referred to DHs include those requiring major surgery under general anesthesia, X-rays, treatment of severe malnutrition, comprehensive emergency obstetric care, and male and female sterilizations. DHs offer comprehensive outpatient and inpatient care for mental health patients and rehabilitation for persons requiring physiotherapy with referral for specialized treatment when needed. They provide a wider range of essential drugs, renewable supplies and laboratory services than HCs. DH staff include a number of doctors, including a surgeon, anesthetist, pediatrician, female obstetricians/gynecologists, and a doctor who serves as a focal point for mental health; midwives; psychosocial counsellors/supervisors; laboratory and X-ray technicians; a pharmacist; a dentist and dental technician; and two physiotherapists (male and female). Each DH covers about 100,000–300,000 people. Flexibility is allowed in implementing the BPHS if an implementing agency faces local situations or problems that require innovation, modifications, or alternative approaches. Those include, inter alia, staff patterns, types of staff training, selection of brands and manufacturers of medical supplies, levels of health facilities, incentive schemes and on-call arrangements for relevant staff members. The implementing agency can also be flexible in response to changes in population growth or unusual population distribution. Flexible adjustments in BPHS implementation must observe the following principles: Page 78 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 1. Each adjustment should have a strong justification (such as gender equity, geography, security) and lead to tangible improvements in specific aspects of service delivery 2. Modifications should promote the availability and equitable access of BPHS 3. They should not undermine the quality of the BPHS services 4. They should be cost effective 5. They should be of limited nature, implemented only when and where necessary, to maintain the consistency of BPHS implementation 4. Additional Minor Changes not listed in the main text Changed Allocations for two expenditure categories The Amendment letter of June 13, 2015 made two other changes, related to the changes noted in the main text. (a) It changed the table of Eligible Expenditures in Schedule 2 Section IV A.2 of the FA for the IDA grant, as shown below, to remove small allocations [ (2) and (3) in the original FA] earmarked for activities in Component 2, for which there was a disbursement condition without specific dates for completion. The condition was that proposals acceptable to the WB had to be developed prior to disbursements. The activities covered under categories 2 and 3 were a small part of two of the ten thematic areas for health system development. Each area required a full proposal to be developed by MOPH and to be appraised and cleared by the Bank before any disbursement. There was no need for separate disbursement conditions or categories, and the amounts assigned would need to be revised in line with proposals. The project restructuring dropped the specific disbursement categories and condition (B. 1. (b) on p.14 of the Original FA). Since funds had already been disbursed under category 1, this category was retained, and the undisbursed amounts all moved to a single new expenditure category (4) (page 1, AF Agreement letter). Amount of the Financing Category Allocated (expressed in SDR) Original Amended (1) Goods, works, non-consulting services, consultants' services, Training and 64,100,000 31,475,000 Incremental Operating Costs for the Project (excluding Parts 1(d); (2)(c)(ii) and (iii); and 2(h)(ii) of the Project) (2) Goods, non-consulting services, consultants' services and works for Part 2(c)(ii) and 660,000 -0- (iii) of the Project (3) Goods, non-consulting services, consultants' services and works for Part 2(h)(ii) of 340,000 -0- the Project (4) Goods, works, non-consulting services, consultants' services, Training and (not in 33,625,000 Incremental Operating Costs for the Project (excluding Part 1(d) of the Project) Original) TOTAL AMOUNT 65,100,000 65,100,000 The project Parts 1 and 2 in the above table refer to the FA Project Description Schedule 1 (p. 4-5). Part 1(d) was the RBF pilot and evaluation, funded by an HRITF grant; Parts (2)(c)(ii) and (iii) were pharmaceuticals post-market surveillance and enhancing the inspection capacity of MOPH. Part 2(h)(ii) was integrated bio-behavioral (HIV) disease surveillance, for which it proved impossible to find an able and willing contractor. Page 79 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) (b) A related change was made to Section IV. B. Withdrawal Conditions; Withdrawal Period. Item 1. was changed to remove exceptions [for the activities referred to in categories (2) and (3) above] to the prohibition on withdrawals for payments made prior to the date on which the original AF was signed. The Amended Condition stated simply that: "1. Notwithstanding the provisions of Part A of this Section, no withdrawal shall be made for payments made prior to the date of this Agreement." (FA Amendment Letter p.2). 5. Additional details on equity of services Figure A7.1: Immunization and Vitamin A by Wealth Quintile, and Urban/Rural Residence Source: AHS 2018, p.44 Page 80 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Source: Afghanistan Living Condition Survey, 2016/17, p.184 Page 81 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 6. Projects funded off-budget by USAID that covered areas proposed by MOPH for inclusion in Component 2 The SIGAR team reported (in 2017) that USAID was funding or planned to fund 14 projects that provided “Development, capacity building, and direct assistance to the MOPH”, with a combined total budget of US$856.1 million (SIGAR 2017 p.10). The following 9 projects funded (off-budget) by USAID during the implementation of SEHAT addressed issues that were also in the list of possible areas for support under Component 2: Page 82 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) Project Timeframe Total Estimated Project Description Cost US million Central 2011-2022 US$25 million Simplifies mechanism for contraceptive procurement Contraceptive Procurement Health Sector 2015-2020 US$37.9 million Fosters a strengthened, reformed, and increasingly Resiliency self-reliant Afghan health system by helping the Afghan government prepare for a decreased donor support environment Strengthening 2011-2017 US$34.4 million Strengthen MOPH’s ability to regulate and assure Pharmaceutical quality of pharmaceutical products entering Systems Afghanistan and ensure essential medicines are available in public clinics Sustaining Health 2016-2018 US$6 million Seeks to harness the full potential of the private sector Outcomes through and catalyze public-private engagement to improve the Private Sector health outcomes in family planning, HIV/AIDS, Plus maternal and child health, and other areas Health Policy Project 2012-2015 US$29.8 million Build the MOPH’s ability to regulate the health sector and improve financial management inside the ministry Partnership for 2009-2015 US$1.5 million A global USAID mechanism for procurement of Supply Chain essential medicine commodities Management Health Care 2009-2013 US$14.0 million Work with the MOPH and the private sector to increase Improvement Project the quality of health services by developing health capacity and infrastructure at the national and provincial levels, with a focus on maternal and newborn care Leadership, 2012-2015 US$38.5 million Intended to build in-country knowledge of health care Management, and system capacity by increasing leadership, management, Governance and governance of health care providers and managers Promoting Quality of 2016-2017 US$4.5 million Strengthen medicine quality assurance and quality Medicines control programs Source: SIGAR (2017) Afghanistan’s Health Care Sector: USAID’s Use of Unreliable Data Presents Challenges in Assessing Program Performance and the Extent of Progress. Special Inspector General for Afghanistan Reconstruction, SIGAR 17-22 Audit Report/USAID Support for Afghanistan’s Health Care. Accessed on-line on November 12, 2018 at https://www.sigar.mil/pdf/audits/SIGAR-17-22-AR.pdf 7. Improvements in the AHS 2018 as Described in the Survey Report, p.13 To ensure the engagement of stakeholders involved in SEHAT, a steering committee (SC) for AHS 2018 was formed, including representatives from the Ministry of Public Health (MoPH), the World Bank, USAID, UNICEF, the World Health Organization, UNFPA, and the National Statistics and Information Authority (NSIA). Furthermore, taking stock of technical lessons learned from the AHS 2015 and further recommendations from key stakeholders, the AHS 2018 has undergone a number of adaptations. These adaptations aimed to ensure sustained relevance for policy-making, as well as quality of operations. Thematic adaptations include: • Collaboration with UNICEF for the inclusion of a nutrition component, so to provide estimates of Page 83 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) children under-five stunting and wasting; • Liaising with the MoPH Counter Narcotics Department to include questions on smoking and drug usage; • Collaboration with the Expanded Program on Immunization department for the inclusion of EPI coverage survey indicators Technical changes include: • Engagement with the Afghanistan NSIA for sampling and definition of household listings, to ensure using the most updated version; • Thorough review and updating assumptions for sample size calculations (using updated input data from both the AHS 2015 and the DHS 2015); • Use of GPS trackers to collect cluster-level GPS locations, so to be able to monitor and evaluate the coverage of the AHS Logistical changes include: • Implementing new training approach for field staff, including Training of Trainers, division of classes for Pashto and Dari speaking, working in smaller classes, and pilot testing; • Strengthening structure of the field teams, by hiring field editors who review filled forms on-site; • Improving transparency by instituting external monitoring teams including representatives from the NSIA, UNICEF, and MoPH’s EHIS and Public Nutrition Department; • Piloting the use of hand-held devices for a portion of data collection to be done electronically; • Implementation of a comprehensive quality assurance plan following the KIT standards for Good Epidemiological Practice. 8. Component 2 Deliverables and Status at Project End. Thematic No. Deliverables Status Area 1 1. Administrative and financial procedure manual developed Completed social accountability Governance and 2. Transparency enhanced in the service delivery and Completed 2 administrative procedures of the MoPH 3. A functional grievance redress mechanism (complaints Completed 3 handling mechanism) established 4 4. Anti-corruption strategy and anti-corruption measures Completed 1. The planning and reporting activities at provincial level Completed Sub-national government 5 improved and provincial budgeting initiative implemented 2. Establish a functional coordination mechanism at the Completed 6 provincial and central levels, including effective feedback mechanism 3. A functional security alert mechanism linking HFs, districts 7 and provinces with the central MoPH established Not Completed 8 4. Strengthen Provincial Governance Partially Completed Page 84 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 9 1. SOP for procurement of goods and works developed Completed 10 2. E-Procurement system established Partially Completed Procurement 3. System for monitoring and tracking of procurement process 11 established Partially Completed 12 4. Asset management system established Partially Completed 13 5. Strengthen MoPH capacity for pharmaceuticals procurement Partially Completed 6. Procurement department accredited for procurement of 14 goods by the NPC Not Completed 1. Public expenditure tracking survey (PETS) conducted Carried forward to 15 Sehatmandi project Health financing 16 2. National health accounts (NHA) institutionalized Partially Completed 17 3. Results-Based Financing project is evaluated Partially Completed 4. Provider payment mechanism piloted at least in one hospital 18 Partially Completed 5. Government core budget contribution to health sector 19 enhanced Partially Completed 1. Community based monitoring system established Dropped (lengthy process, 20 too little time to implement) 2. M&E system for the private health facilities established Dropped (lengthy process, 21 too little time to implement 3. IDSR established Dropped (lengthy process, 22 too little time to implement 4. Data warehouse established (open data initiatives) Completed Health Information System 23 5. In collaboration with the World Bank, an in-depth study Completed 24 comparing service delivery modalities conducted by independent third party 25 6. RNMCH score card further developed and maintained Completed 7. PHO monitoring and oversight of service delivery enhanced 26 and maintained Partially Completed 8. Functioning internet services in both central and provincial Completed 28 deps of the MoPH ensured 9. Policy and standards on maintenance of IT 29 equipment/software and anti-virus software developed Completed 10. MoPH IT equipment well-maintained and software update – 30 MoPH could consider contracting out this function Not Completed 31 11. The capacity of MoPH staff built in IT related issues Completed 1. MQCL renovated and upgraded Completed Phar mac euti cals 32 Page 85 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 2. Pharmaceutical inspection capacity enhanced and the system 33 improved with clear role and responsibilities of other related departments Partially Completed 34 3. Post market surveillance system established Completed 35 4.National Medicine Regulatory Authority established Completed 36 1. Private health sector assessment report completed Completed 37 2. Transaction adviser recruited for two hospitals Partially Completed 3. At least one tertiary hospital gone through PPP arrangement Private Sector 38 Partially Completed 39 4. Small scale PPP interventions Partially Completed 5. Registration and licensing process for private health facilities 40 simplified Completed 6. Independent-semi autonomous accreditation body for 41 private health facilities established Partially Completed 1. Revised structure of financial management department to be 42 more responsive to current needs Completed 43 2. MoPH FM SOP/manual developed Partially Completed 3. AFMIS access in central, provincial and Kabul hospitals Fiduciary systems 44 assured Completed 45 4. MoPH internal audit strengthened Partially Completed 46 5. FM system upgraded to web-based system Partially Completed 47 6. Payment process simplified Partially Completed 7. FM tracking system to figure out bottlenecks and delays in 48 payments established Partially Completed 8. International firm hired to develop PFM manual, guidelines, 49 monitoring tools, functionalize web-based system Partially Completed 50 1. A functioning HRMI established Completed Human resources for 2.The IHS curricula revised; the proposed amount of $240,600 Dropped (lengthy process, 51 may be too high for this task and needs to be revisited too little time to implement) 52 3.National Health Workforce Plan updated/improved health Partially Completed 4.Relations and coordination with MoHE improved through Completed 53 definition of clear indicators 5.Deployment and retention of female health workers Completed 54 improved 1. Hospital autonomy is effectively implemented and further 55 improved Partially Completed Hospitals 56 2. System for user fees developed and implemented Partially Completed 57 3. Health care waste management improved Completed 58 4. Clinical guidelines and patient safety protocols developed Completed Page 86 of 88 The World Bank Afghanistan: System Enhancement for Health Action in Transition Project (P129663) 59 5. Provider payment mechanism established Completed 6.Patient and provider safety standards developed and Completed 60 implemented 1. All projects and programs of the health sector rendered Completed 61 gender friendly 2. Gender related knowledge and behavior improved in the Completed 62 Gender MOPH 3. GBV case management and referral improved in all health Completed 63 facilities. 4. GBV related Grievance Redress System established within Completed 64 MOPH Promotion Health 65 1. Formative Research and Umbrella Campaign Conducted Carried forward to Sehatmandi project Page 87 of 88