Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004341 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON CREDITS IN THE AMOUNT OF SDR 48.7 MILLION (US$ 75 MILLION EQUIVALENT) TO THE REPUBLIC OF MADAGASCAR FOR AN MADAGASCAR EMERGENCY SUPPORT TO CRITICAL EDUCATION, HEALTH AND NUTRITION SERVICES PROJECT ( P131945 ) January 26, 2017 Health, Nutrition & Population Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective as of January 22, 2018) Currency Unit = Malagasy Ariary (MGA) MGA 3,251.50 = US$ 1 US$ 1 = SDR 1.442100 FISCAL YEAR July 1 - June 30 Regional Vice President: Makhtar Diop Country Director: Mark R. Lundell Senior Global Practice Director: Timothy Grant Evans Practice Manager: Magnus Lindelow Task Team Leader(s): Jumana N. Qamruddin, Rary Adria Rakotoarivony ICR Main Contributor: Dorothee Chen ABBREVIATIONS AND ACRONYMS AIS Activity Initiation Summary BCR Benefit-Cost Ratio CBA Cost-Benefit Analysis CCP Cellule de Coordination du Projet CRESED Crédit de Renforcement du Secteur de l’Education (Education Sector Reinforcement Credit) DHS Demographic and Health Survey EFA-FTI Education for All-Fast Track Initiative ENSOMD Enquête Nationale sur le Suivi des Objectifs du Millénaire pour le Développement (Millenium Development Goals National Monitoring Survey) FY Fiscal Year GDP Gross Domestic Product GPE Global Partnership for Education HE Health Expenditure ICR Implementation Completion and Results Report (ICR) IDA International Development Association IMF International Monetary Fund IRR Internal Rate of Return ISN Interim Strategy Note IT Information Technology MDGs Millennium Development Goals MICS Multiple Indicator Cluster Survey MMR Maternal Mortality Ratio MTR Mid-Term Review MUAC Mid-Upper Arm Circumference MWMP Medical Waste Management Plan NGO Non-Governmental Organization NPV Net Present Value NTDs Neglected Tropical Diseases OP/BP Operational Policy and Bank Procedures PDO Project Development Objective PIU Project Implementation Unit SDGs Sustainable Development Goals SDI Service Delivery Indicators SSA Sub-Saharan Africa TF Trust Fund TTL Task Team Leader UAT-EPT Unité d’Appui Technique-Education pour Tous (Education PIU) UGP Santé Unité de Gestion des Projets de Santé (Health PIU) UNOPS United Nations Office for Project Services UPNNC Unité Programme National de Nutrition Communautaire (Nutrition PIU) U5MR Under-Five Mortality Rate WDI World Development Indicators WDR World Development Report WFP World Food Program TABLE OF CONTENTS DATA SHEET .................................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ...................................................................5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME ................................................................................................................................ 14 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 21 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 24 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 24 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME............................................. 26 A. KEY FACTORS DURING PREPARATION ................................................................................... 26 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 27 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME ............... 30 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 30 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 31 C. BANK PERFORMANCE ........................................................................................................... 32 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 35 V. LESSONS AND RECOMMENDATIONS ......................................................................................... 35 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................................ 37 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ...................................... 51 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................................ 53 ANNEX 4. EFFICIENCY ANALYSIS ....................................................................................................... 54 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ................. 72 ANNEX 6. SUPPORTING DOCUMENTS ............................................................................................... 73 ANNEX 7. SUMMARY OF CLIENT’S IMPLEMENTATION COMPLETION & RESULTS REPORTS (ICRS) ........ 77 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name MADAGASCAR EMERGENCY SUPPORT TO CRITICAL P131945 EDUCATION, HEALTH AND NUTRITION SERVICES PROJECT ( P131945 ) Country Financing Instrument Madagascar Emergency Recovery Loan Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Additional Financing P148749-AF-Emergency 27-Feb-2014 IBRD/IDA Support Critical Education Health and Nutrition Services Organizations Borrower Implementing Agency Ministry of Finance and Budget UPNNC, UAT-EPT, UGP-Sante Project Development Objective (PDO) Original PDO The Project Development Objective (PDO) is to preserve critical education, health and nutrition service delivery in targeted vulnerable areas in the recipient’s territory. Page 1 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 65,000,000 62,917,077 59,074,647 IDA-51860 10,000,000 10,000,000 9,038,005 IDA-53820 Total 75,000,000 72,917,077 68,112,652 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 75,000,000 72,917,077 68,112,652 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 29-Nov-2012 25-Apr-2013 27-Oct-2014 31-Jul-2016 30-Jul-2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 27-Mar-2014 16.77 Additional Financing Change in Results Framework Change in Components and Cost Other Change(s) 21-Jul-2015 36.81 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule 14-Jun-2016 60.14 Reallocation between Disbursement Categories 21-Jul-2016 61.04 Change in Loan Closing Date(s) Reallocation between Disbursement Categories Page 2 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 28-Sep-2013 Satisfactory Satisfactory 9.05 02 30-Mar-2014 Satisfactory Satisfactory 16.77 03 29-Sep-2014 Satisfactory Satisfactory 24.88 04 18-Mar-2015 Satisfactory Satisfactory 32.86 05 21-Sep-2015 Satisfactory Satisfactory 40.78 06 23-Mar-2016 Satisfactory Satisfactory 55.50 07 27-Sep-2016 Satisfactory Satisfactory 63.97 08 30-Mar-2017 Satisfactory Satisfactory 69.39 09 27-Jul-2017 Satisfactory Satisfactory 68.11 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 100 Primary Education 40 Health 100 Health 60 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Human Development and Gender 101 Health Systems and Policies 43 Health System Strengthening 16 Reproductive and Maternal Health 10 Child Health 17 Education 40 Access to Education 20 Education Financing 20 Nutrition and Food Security 18 Nutrition 9 Food Security 9 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Makhtar Diop Country Director: Haleh Z. Bridi Mark R. Lundell Senior Global Practice Director: Ritva S. Reinikka Timothy Grant Evans Practice Manager: Sajitha Bashir Magnus Lindelow Andreas Blom, Jumana N. Jumana N. Qamruddin, Rary Task Team Leader(s): Qamruddin Adria Rakotoarivony ICR Contributing Author: Dorothee Chen Page 4 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context Political and Economic Developments 1. In early 2009, a “de facto” government came into power in Madagascar, which triggered the application of Operational Policy and Bank Procedures (OP/BP) 7.30 (Dealing with De Facto Governments). As a result, preparation and approval of new lending was put on hold under the Madagascar portfolio in March 2009, as well as disbursements under existing operations.1 The political and economic crisis was impervious to international mediation efforts for years, exacting a heavy toll on one of the poorest populations in the world. 2. The economic cost of the crisis was substantial, and the fiscal policy response resulted in drastic cuts in the national budget for the social sectors. Overall economic growth was flat from 2009 to 2012, but with the high population growth (2.9 percent), income per capita in 2012 returned to its 2003 level. The economic and social effects of the crisis were intensified by the suspension of many donor activities. In a country where international aid represents at least 50 percent of the government budget, this led to significant cuts in investments and a sharp decline in the delivery of services. 3. In 2012, several years into the crisis and with most aid suspended, Madagascar was progressively sliding into greater fragility and was experiencing dramatic increases in poverty levels and worrisome deterioration in the overall governance environment. Poverty levels increased by 9 percent between 2005 and 2010, reaching 77 percent of households, the highest rate in Africa according to the World Development Indicators (WDI). In health, education and food security, there was an emergency situation as the public service delivery system was at risk of paralysis and humanitarian aid bypassing public institutions was showing its limitations. In 2012, the attainment of the Millennium Development Goals (MDGs) had become an increasingly remote target. Health, Nutrition and Education 4. The impact of the crisis in the health and nutrition sector was pervasive and was reversing progress in improving the well-being of the population that had been occurring over a decade. The public health service delivery system was increasingly under threat with a dwindling supply of basic services, and increased fragmentation of scarce donor resources through parallel service delivery systems. Between 2008 and 2010-11 the following trends had been observed: (i) the government reduced the budget covering operational costs of health centers by 30 percent; (ii) the prescription satisfaction rate, a key indicator for drug availability, sharply declined from 69 percent to 58.9 percent at the facility level; (iii) the utilization rate of basic health centers decreased by 20 percent; and (iv) the prenatal consultation rate in primary health care facilities went from 74 percent to 62 percent. A lack of human and financial resources caused the closure of 339 primary health care facilities out of 2,500 nationwide. In addition, the nutrition status of the population was alarming due to an increased rate of acute malnutrition from 4.7 percent in 2008 to 7.4 percent in 2011 in some of the most food insecure and poorest districts in the country (southeast, southwest, and central regions). 1In November 2009, disbursements for certain projects in Madagascar were exceptionally allowed to resume on humanitarian grounds. The full resumption of disbursements in April 2011 finally allowed for a major portfolio restructuring completed in 2012, as well as the preparation of two emergency operations, including the Emergency Support to Critical Education, Health, and Nutrition Services Project. Page 5 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 5. The impact of the crisis in the education sector was also high. For example, between 2009 and 2011, enrollment in primary education stagnated at 4.3 million pupils, which was at odds with the overall context of rapid population growth, and in sharp contrast with the period immediately preceding the crisis when the annual growth rate averaged 7.8 percent over 10 years and 6.2 percent over 3 years. In addition, public expenditures on education as a share of GDP decreased from 3.6 percent in 2008 to 2.6 in 2010, which represented a 30 percent reduction in constant 2010 prices. This translated into an increasing share of education costs being borne by poor families (for example, as much as two- thirds of primary school teachers were then hired by communities). 6. Few donor-funded programs in health, nutrition, and education that were implemented through established Project Implementation Units (PIUs) embedded in the Ministry of Education, the Ministry of Health, and the National Nutrition Office continued to operate. The main donor-funded program in education, the Global Partnership for Education supporting the Education for All program (GPE-EFA) came to an end in June 2012.2 The main donor-funded programs in health and nutrition were: (i) the Bank-funded Second Multisectoral STI/HIV/AIDS Prevention Project Additional Financing approved in June 2012 (US$6 million), which was designed to serve “as a bridge financing arrangement” with a closing date in September 2014;3 and (ii) the Agence Française de Developpement-supported project, which maintained a minimum of bridge financing (US$10 million for 2012-13) to address basic health needs.4 7. The national priorities in education, health and nutrition were to preserve critical service delivery. The crisis placed implementation of the five-year development plan—the Madagascar Action Plan—on hold. However, interim sectoral strategies for education, health and nutrition had been, or were being developed. These strategies focused on the short-term, aiming to protect access to and utilization of services. In education, the government was preparing an interim education sector plan for 2013-16, which focused on improving access to education and enhancing the learning environment for better results. In health, the government was finalizing a strategic development plan for 2012–2015, which primarily focused on maternal and child health. A national nutrition strategy centered on ensuring adequate nutrition services for children between zero and five years of age, with a focus on the period between conception and two years old, was about to be adopted. Rationale for Bank Support at the Time of Approval and Contribution to Higher-level Interim Strategy Note (ISN) Objectives 8. To help mitigate the risk of Madagascar falling further into fragility, guidance provided by the 2011 World Development Report (WDR) on Conflict, Security and Development informed the development of the Interim Strategy Note (ISN) for Fiscal Years 2012 and 2013 (FY12-13). The ISN highlighted that the cost of continuing donor inaction or suspension of ongoing projects due to the crisis could result in: (i) an increased risk of reversal of a number of critical reforms due to the lack of formal policy dialogue; (ii) the collapse of key public programs, jeopardizing years of achievements and capacity building ; and (iii) ultimately, a high cost of reengagement due to the continued deterioration of the socio-economic situation and the disruption in project management on the government’s side. As a consequence, the ISN proposed, among other things, to provide a cautious, strategic and selective approach to new lending in order to address emergency situations in social sectors, and in particular to reverse the aforementioned negative trends in education, health, and nutrition. The ISN also recommended learning from and capitalizing on the experience of existing PIUs in designing and implementing support programs. 2 UNICEF was the supervising entity for this operation. 3 For more information about the Second Multisectoral STI/HIV/AIDS Prevention Project and its Additional Financing (P090615/Credit No IDA-41040 and IDA-51240), see World Bank Group 2014g. 4 This operation was initially supposed to be co-financed by the Bank for an amount of US$63 million, which was not approved because of the start of the crisis. Page 6 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Theory of Change (Results Chain) 9. The Emergency Support to Critical Education, Health and Nutrition Services Project (P131945, Credit No. 5186- MG) approved on November 29, 2012, in the amount of SDR 42.2 million (US$65 million equivalent) focused on the provision of financial assistance for human development—basic education, health, and nutrition—service delivery in vulnerable communities to help preserve access to and maintain utilization of basic social services. The Additional Financing (Credit No. 5382-MG), approved on February 27, 2014, in the amount of SDR 6.5 million (US$10 million equivalent), which was processed due to a locust infestation that had resulted in food insecurity, supplemented the initial credit with a focus on nutrition activities. 10. As per OP/BP 8.00 (Rapid Response to Crises and Emergencies), the operation aimed at preserving essential services and not addressing long-term economic issues. To this end, interventions included supply-side activities in each sector, as well as demand-side activities contributing to reduction and removal of geographic and financial barriers (Figure 1). 11. In education, activities included: subsidization of community teacher salaries; school grants to meet basic operational needs such as notebooks, paper, chalk, and urgent minor infrastructure repairs; training of communities on governance of school grants and community teacher salaries to ensure that primary school services were available and functional; and school health and nutrition interventions, i.e. the distribution of treatment against Neglected Tropical Diseases (NTDs) to stimulate demand for primary school services. The expected outcome of these interventions was that critical education services be preserved—indicators associated with this outcome were the number of functional schools and the number of students enrolled, to be maintained at the 2012 levels.5 12. In health, supply-side activities included: the rehabilitation and equipment of primary health care facilities; training of medical staff; the recapitalization/restocking of public pharmacies; and capacity building of the Ministry of Health’s district services to ensure that primary health care services were available and functional. Demand-side activities designed to reduce financial and geographic barriers and to stimulate utilization for primary health care services included: the establishment of a fee exemption system for maternal and child health interventions to remove out-of- pocket payments; the distribution of free safe delivery kits; and the provision of transportation (motorcycles) and equipment to expand the radius of community outreach services (i.e. consultation and immunization). The expected outcome of these interventions was that critical health services would be preserved. Indicators associated with this outcome were the number of births attended by skilled health personnel and the number of children immunized, to be increased and reach pre-crisis levels. 13. In nutrition, supply-side activities included the functionalization of new and existing local community nutrition sites under the National Community Nutrition Program to deliver nutrition services (i.e. weighing, nutritional counseling, and cooking demonstrations), and the recruitment of non-governmental organizations (NGOs) to support and supervise community nutrition agents at community nutrition sites. Demand-side activities included targeted household visits for children with faltering weight. Activities also included support for the design and evaluation of various interventions to improve the quality of nutritional counseling services. The expected outcome of these interventions was that critical nutrition services would be preserved. Indicators associated with this outcome were the number of people with access to a basic package of health, nutrition or reproductive health services, and the number of children benefiting from 5 The number of functional schools could have been an output/intermediate-result indicator instead of an outcome indicator. Page 7 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) improved child feeding practices, to be increased and reach pre-crisis levels.6 For further details about activities, see paragraph 16 and Annex 1. Figure 1: Results Chain 6The number of people with access to a basic package of health, nutrition or reproductive health services could have been an output/intermediate-result indicator instead of an outcome indicator. Page 8 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Project Development Objectives (PDOs) 14. The formulation of the Project Development Objective (PDO) in the Emergency Project Paper (Original Credit) and the Additional Financing Project Paper, as well as in both financing agreements, was the same. The PDO was to “preserve critical education, health and nutrition service delivery in targeted vulnerable areas in the recipient’s territory”.7 Key Expected Outcomes and Outcome Indicators 15. The key expected outcomes were the following: • Critical education service delivery is preserved, with the following outcome indicators: o Number of students enrolled in primary schools in targeted regions (of which percentage of female) o Number of schools receiving school grants funded by the project8 • Critical health service delivery is preserved, with the following outcome indicators: o Number of births / deliveries attended by skilled health personnel in project areas o Number of children immunized • Critical nutrition service delivery is preserved, with the following outcome indicators: o Number of people with access to a basic package of health, nutrition or reproductive health service9 o Number of children under the age of 24 months benefitting from improved infant and young child feeding practices • In addition, the Results Framework included one cross-sector outcome indicator: the number of direct project beneficiaries.10 Components 16. Each outcome was associated with one component, described in the Project Papers as follows: Component 1 consisted in “preserving critical education services” and included four subcomponents/activities: • Subsidization of community teacher salaries; • School grants; • School health and nutrition interventions, i.e. the distribution of treatment against NTDs; and • Capacity strengthening, project management, and monitoring and evaluation. 7 Five regions were targeted under the Original Credit, among the 22 regions of the country. These five regions had the highest poverty rates according to the 2011 Poverty Map, and low social sector outcomes confirmed by population surveys and routine data in the education, health, and nutrition sectors. Two of these five regions and three additional regions were targeted under the Additional Financing, which focused on the technical and geographical expansion of Component 3 (nutrition). So, the Additional Financing brought the geographical scope of the Project to eight regions (three of them benefitting only from Component 3). Finally, during the Mid-Term Review (MTR), Component 3 was extended to one more region, for a total of nine regions (four of them benefitting only from Component 3). 8 This indicator was a proxy for the number of functional schools in the regions targeted under the project. As mentioned, this could have been an output/intermediate-result indicator instead of an outcome indicator. 9 As mentioned, this indicator could have been an output/intermediate-result indicator instead of an outcome indicator. 10 As mentioned, this indicator could have been an output/intermediate-result indicator instead of an outcome indicator. Page 9 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Component 2 consisted in “preserving critical health services” and included two subcomponents: • Support to the delivery of a critical package for pregnant women and children under five at the health facility level. As mentioned in Figure 1, this sub-component included the following activities: (i) rehabilitation of and procurement of basic equipment for primary health care facilities; (ii) procurement of essential obstetric and neonatal equipment; (iii) training on obstetric and neonatal care for medical staff at the facility, district and regional levels; (iv) procurement of safe delivery kits; (v) procurement of transport and medical equipment for community outreach; (vi) recapitalization/restocking of public pharmacies at the facility level; (vii) establishment of a fee exemption system for pregnant women and children under 5 years old; and (viii) distribution of treatment against NTDs through participation in school health and nutrition interventions, including the procurement of drugs, and through mass drug administration at the community level; and • Project management and monitoring and evaluation. Component 3 consisted in “preserving critical nutrition services” and included two sub-components: • Support to basic community nutrition services. As mentioned in Figure 1, this sub-component included the following key activities: (i) establishment of new community nutrition sites (identification of vulnerable communities, awareness campaign, and selection of community nutrition workers); (ii) support to existing and new community nutrition sites, including the development and distribution of manuals, posters, and other communication materials to support awareness presentations delivered by community nutrition workers, as well as the procurement of cooking kits for culinary demonstrations, scales, and so forth; (iii) the recruitment of NGOs to supervise and support community nutrition sites; (iv) the testing of different approaches, namely (a) intensive counselling through household visits, (b) lipid-based nutrient supplementation for kids, (c) lipid-based nutrient supplementation for pregnant and breast-feeding women, and (d) early childhood stimulation, which were evaluated through a randomized controlled trial;11 and (v) participation in the distribution of treatment against NTDs, including provision of small meals prior to administering the NTD medication and sensitization of the communities on the NTD campaigns. In addition, a new set of food security activities at the community and household levels, including home gardens, short-cycle farming and agriculture, and food conservation, were introduced under the Additional Financing; and • Project management and monitoring and evaluation. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 17. The PDO was not revised during implementation. 18. Target values for three outcome indicators were revised (increased) under the Additional Financing dated March 27, 2014. Only nutrition-related outcome indicators were affected since the Additional Financing focused on the expansion of Component 3. One of the three increased target values was subsequently decreased through a third restructuring in July 2016 (for more details, see paragraph 19). 11The randomized control trial was financed by the Strategic Impact Evaluation Fund (SIEF), the Early Learning Partnership Program (ELP), the World Bank Innovation Grant, the Grand Challenges Canada, the World Bank Research Committee, the Japan Nutrition Trust Fund, and the Power of Nutrition Trust Fund. Page 10 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Revised PDO Indicators 19. End target values for the following outcome indicators were revised under the Additional Financing: • Number of direct project beneficiaries was increased, but subsequently decreased through the third 2016 restructuring; • Number of people with access to a basic package of health, nutrition, or reproductive services and two breakdown indicators (number of pregnant/lactating women, and number of children under 5) were increased; and • Number of children under the age of 24 months benefiting from improved infant and young child feeding practice was increased. Revised Components 20. Components of the project were not revised. However, a few activities were introduced under existing components (for further details, see paragraphs 21 to 25). Other Changes 21. The Additional Financing focused on the expansion of Component 3 (nutrition). The main changes introduced were as follows: • Nutrition activities already included in the initial credit were geographically expanded to three regions;12 • Food security activities were introduced in five regions (the three new regions and two regions—out of five—included in the initial credit); • In addition to increases in end target values for three nutrition-related outcome indicators (see paragraphs 18 and 19), target values were increased for three intermediate results indicators: o Number of children under 2 years old enrolled in the growth monitoring program; o Number of children between 2 and 5 years old enrolled in the mid-upper arm circumference (MUAC) program; and o Number of community nutrition agents trained to provide health and nutrition education. • An intermediate results indicator related to new food security activities was introduced: number of households receiving support kits for short cycle agriculture and livestock activities (World Bank Group 2014a). 12Five regions were targeted under the Original Credit. Two of these five regions and three additional regions were targeted under the Additional Financing, which focused on the technical and geographical expansion of Component 3 (nutrition). Thus, a total of eight regions were covered by the project at this stage (three of them benefitting only from Component 3). Page 11 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 22. During the Mid-Term Review (MTR) (October 27 to November 7, 2014), the government and the Bank team concurred that the project was on track to achieve its objectives and beyond due to the progress on the results framework indicators and the fast disbursement rate.13 The amount that had been initially unallocated, so that the project could be responsive to the emergency context, was still available, as were savings from existing activities.14 Consequently, the government and the Bank agreed to introduce new, more innovative activities focused on improving quality. Moreover, there was an agreement to immediately further expand the geographical scope of nutrition activities under Component 3 to one more region (World Bank Group 2014d).15 23. Based on the agreement reached during the MTR, a first restructuring was approved by the Bank on July 21, 2015, with countersignature by the client on August 19, 2015. The following activities were introduced: • Under Component 1: purchasing of school manuals, and delivery of intensive trainings for community teachers; • Under Component 2: results-based financing to continue a pilot that had been initiated under the Second Multisectoral STI/HIV/AIDS Prevention Project (P090615), which preceded and overlapped with this project; and • Under Component 3: behavior change communication to improve utilization of nutrition messages at the household level (human centered design). In addition, the following changes were introduced: • Extension of the closing date of the Additional Financing by one year, from July 31, 2016 to July 30, 2017; and • Change in one indicator: “Number of syphilis treatments distributed to pregnant women in public health centers” was replaced with “Percentage of women attending antenatal care who are tested for syphilis in project areas” (World Bank Group 2015a). 24. In June 2016, a second restructuring resulted in the reallocation of the remaining unallocated category (US$702,000) to Component 3 to intensify emergency response efforts in the South of Madagascar through the identification of children under 5 years old who had acute malnutrition, and the treatment of malnutrition.16 This new activity was planned in partnership with the World Food Program (WFP) (World Bank Group 2016a). 25. A third and final restructuring in July 2016 introduced the following changes:17 • Extension of the closing date of the Original Credit by 5 months, from July 31, 2016 to December 31, 2016; and • In addition to the decreased target value for the outcome indicator on direct beneficiaries of the project (see paragraphs 18 and 19), decreases in target values for two intermediate results indicators: o Number of children under 2 years old enrolled in the growth monitoring program; and o Number of children between 2 and 5 years old enrolled in the MUAC program (World Bank Group 2016b). 13 As of September 2014, US$ 24.2 million (approximately 40 percent of the Original Credit) had already been disbursed (World Bank Group 2014g). 14 Saving were mainly due to: a favorable exchange rate, actual costs that were lower than initial estimated costs (for example, for vehicles), unexpected financing from the government for social security contributions for community teachers, and the delayed finalization of contracts with NGOs for nutrition activities. 15 As a consequence, a total of nine regions were covered by the project from this date (four of them benefitting only from Component 3). 16 The treatment was administered over two months and included sachets of Plumpy Sup for the affected child and food rations for the family to ensure adherence/compliance. 17 The second and third restructurings were processed in rapid succession because there was a need to address urgently the emergency in the South of the country due to the prolonged drought while the discussions about the extension of the closing date and the target values for nutrition indicators required extended discussions with the Government. Had the decision to have two separate restructurings in 2016 not been made, the resources for the South would have been delayed by five weeks, a critical time-period for emergency activities. Page 12 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Rationale for Changes and Their Implication on the Original Theory of Change 26. The rationale for the Additional Financing was to address acute malnutrition in regions that were severely affected by a locust infestation by undertaking a geographical extension of nutrition activities implemented under the initial credit, as well as to introduce a new set of food security activities at community and household levels, including home gardens, short-cycle farming and agriculture, and food conservation. 27. The rationale for further extension of the geographical scope of Component 3 during the 2014 MTR (adding one more region) was to address the expansion of the locust infestation and food insecurity to this region. 28. The rationale for the introduction of new activities through the first restructuring was to implement some activities that focused on quality in view of the significant progress of the project in maintaining/increasing services as evidenced by actual values for results framework indicators at mid-term. In addition, some innovative activities were introduced to inform future non-emergency operations that would go beyond the preservation of service delivery since the country had returned to constitutional order in early 2014. The extension by one year of the closing date of the Additional Financing was designed to address the delay of effectiveness due to the late signature of the financing agreement given the transition to the new government after the December 2013 elections. Finally, the reason for the change in the results framework was to incentivize and better monitor preventive syphilis services instead of just curative services related to syphilis. 29. The rationale for the introduction of new emergency activities and the extension of the geographical scope of Component 3 through the second restructuring in 2016 was to prevent and treat child malnutrition that has occurred because of the drought and the associated food insecurity in the South of Madagascar. 30. Finally, the main rationale for the third and final restructuring was to extend the closing date of the initial credit to allow for the completion of activities under Component 1 (procurement and delivery of school manuals), and Component 2 (including rehabilitation of health care facilities, and an epidemiological study). In addition, the end target values were lowered for one outcome indicator and two intermediate indicators related to Component 3 to reflect changes in population projections.18 31. These changes had a limited impact on the original theory of change of the project. They only affected the list of activities implemented under the project (see paragraphs 21 to 24), target values for three outcome indicators (see paragraphs 18 and 19) and for three intermediate results indicators (see paragraphs 21 and 25), and the formulation of one intermediate results indicator (see paragraph 23). 18Number of direct project beneficiaries; number of children under 2 years old enrolled in the growth monitoring program; and number of children between 2 and 5 years old enrolled in the MUAC program (for more details, see the section on “Bank Performance, Compliance Issues, and Risk to Development Outcome”, “Quality of Monitoring and Evaluation (M&E)”, “M&E Implementation”). Page 13 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 32. As mentioned in paragraphs 1 to 8, the PDO was highly relevant to the FY12-13 ISN and the country context. Country development challenges did not significantly change during implementation, and therefore the operation’s development objectives remained highly relevant to the Country Partnership Framework (CPF) for FY17-FY21.19 Under the first area of focus of the FY17-FY21 CPF—“Increase resilience and reduce fragility”—the first objective was to strengthen children’s human development, and the rationale for this prioritization was as follows: “Over the long term, human resilience and productive potential will be strengthened by a combination of health, nutrition, and education investments in children’s ‘Early Years’.” The operation was associated with the first objective of the CPF in the Results Matrix, along with follow-up operations in education, and health and nutrition (World Bank Group 2017a). 33. The priorities of the FY17-FY21 CPF reflected the 2015 Systematic Country Diagnostic recommendations (World Bank Group 2015b), which highlighted the need to: • Improve the quality and equity of education by investing in pre- and in-service training for all teachers, improving the distribution of teachers across the country, and gradually integrating trained community teachers in the system while maintaining local accountability mechanisms; • Prioritize and invest in first-level rural health facilities, train and distribute health personnel across the country, and remove out-of-pocket costs at facility levels, particularly for poorest communities; and • Renew the focus on nutrition, and scale-up interventions that tackle stunting. 34. Finally, the development objectives were aligned with the Government of Madagascar’s 2015-19 National Development Plan, supported by the CPF, which included an “adequate human capital for the development process” as one among five strategic areas (Bank Group 2017a; Government of Madagascar n.d.). 35. Considering these factors, the relevance of the operation’s development objectives is High. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 36. There was no implementation of non-project activities in the targeted areas that could have affected the observed outcomes, such as interventions supported by other donors and policy changes unrelated to the operation. The operation was closely coordinated with the other donor operations in education, health, and nutrition, to avoid geographical overlaps. In addition, the Government of Madagascar did not implement any other reforms/programs in the targeted areas. 37. There is no need for a split rating despite the restructurings since all target values for outcome indicators would have been met in any event, and actual values for three intermediate results indicators would have been below the target values (with two of them very close to the target values) had the third restructuring not happened instead of two (with one of them very close to the target value)—for further discussions about this, see paragraph 52 and Table 5. 19 The FY17-21 CPF came immediately after the ISN for FY12-FY13 and was effective when the operation was closed. Page 14 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 38. One joint outcome indicator was directly linked to the three distinct outcomes: the number of direct project beneficiaries. The target value, which was revised twice—increased through the Additional Financing and subsequently decreased through the third restructuring—was exceeded. The actual end value was above the final target value as well as the highest target value set through the Additional Financing (see Table 1). Table 1: Cross-Sector Indicators—Target and Actual Values Revised Target Value Initial Target (2014 Revised Value Additional Target Value Baseline (Emergency Financing (2016 Third Actual End Indicator Value (2012) Project Paper) Project Paper) Restructuring) Value Outcome Indicators Number of direct project beneficiaries 0 1,916,726 2,603,603 2,139,153 2,642,180 of which percentage of female NA 55 NA NA 59 Outcome 1: Critical education service delivery is preserved in targeted vulnerable areas 39. The following activities were effectively implemented under the operation, as initially planned: (i) transfer of school grants; (ii) subsidization of community teacher salaries; (iii) delivery of school health and nutrition interventions, i.e. the distribution of treatment against NTDs; and (iv) development of school supervision by communities and decentralized services of the Ministry of Education. In addition, the following activities were implemented after the MTR: (i) the purchasing of school manuals; and (ii) the delivery of intensive trainings for community teachers. 40. To monitor and evaluate achievements under this objective, relevant indicators available in administrative databases were included in the results framework: five intermediate results indicators, and two outcome indicators. All the end targets set in the Project Paper, which were not revised, were exceeded (see Table 2). Page 15 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Table 2: Education Component Indicators—Target and Actual Values Initial End Target (Emergency Baseline Project Paper, Actual End Indicator Value (2012) unchanged) Value Outcome Indicators Number of students enrolled in primary schools 974,300 974,300 1,131,353 of which percentage of female 49 49 50 Number of schools receiving school grants from the project* 0 6,050 6,682 Intermediate Results Indicators Number of community teachers certified to be in service and paid 0 10,000 16,999 of which percentage of female NA 50 51 Percentage of school grants paid on time 0 95 100 Number of children receiving anti-helminth treatment 0 667,944 1,804,964 Number of parents' associations / school management committees trained on teacher accountability process and use of school grants 0 6,050 6,688 Number of teachers trained in school health and nutrition 0 3,750 19,852 * This indicator was a proxy for the number of functional schools in the regions targeted under the project. As mentioned, it could have been an output/intermediate-result indicator instead of an outcome indicator. 41. Independent evaluations conducted under the project confirmed that school grants and subsidies to community teacher salaries reported in administrative databases were effectively transferred (Adris 2014; Cabinet ECR 2015; Mahomby 2016). However, the evaluations did not look at other indicators in and beyond the results framework. A Service Delivery Indicators (SDI) Survey for Education was also financed by the project. However, it cannot be used for triangulation against the results framework to confirm achievements under this objective because the SDI survey focused on the quality—rather than the quantity—of services delivered. Outcome 2: Critical health service delivery is preserved in targeted vulnerable areas 42. The following activities were effectively implemented under the operation, as initially planned: (i) rehabilitation of and procurement of basic equipment for primary health care facilities, (ii) procurement of essential obstetric and neonatal equipment; (iii) training on obstetric and neonatal care for medical staff at the facility, district, and regional levels; (iv) distribution of safe delivery kits; (v) procurement of transport and medical equipment for community outreach; (vi) recapitalization/restocking of public pharmacies at the facility level; (vii) establishment of a fee exemption system for pregnant women and children under 5 years old in collaboration with NGOs; (viii) distribution of treatment against NTDs through participation in the school health and nutrition interventions, including the procurement of drugs, and through mass drug administration at the community level; and (ix) provision of technical support to the Ministry of Health’s district services. In addition, the continuation of a results-based financing pilot was supported after the MTR. 43. Relevant indicators available in administrative databases were included in the operation’s results framework to monitor and evaluate achievements under Outcome 2: four intermediate results indicators, and two outcome indicators. The target values set in the Emergency Project Paper for the two outcome indicators and the three intermediate indicators that were not revised were exceeded. The target value for one intermediate result indicator that was revised through the first restructuring was not achieved (see Table 3). The target value for the intermediate indicator on syphilis was not achieved due to a longer than expected transition between the project and the Global Fund’s project which was to take over the delivery of syphilis tests and treatments. This resulted in a drop in the percentage of women attending an Page 16 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) antenatal clinic who were tested for syphilis from 72 percent in March 2016 to 48 percent in December 2016 (the revised closing date for the Original Credit). Syphilis tests and treatments were procured as an emergency measure under the project; however, these inputs were not delivered in time to avoid the shortages created by the delay of the Global Fund resources. Table 3: Health Component Indicators—Target and Actual Values Initial End Revised End Target Target (First Baseline (Emergency Restructuring Actual End Indicator Value (2012) Project Paper) —2015) Value Outcome Indicators Number of deliveries attended by skilled health personnel*, ** 0 12,600 NA 131,431 Number of children immunized (under 12 months immunized against DTP3)*, ** 0 18,300 NA 286,194 Intermediate Results Indicators Number of pregnant women receiving antenatal care during a visit to a health provider*, **, *** 0 18,283 NA 113,131 Number of health facilities constructed, renovated, and/or equipped** 0 347 NA 347 Percentage of women attending antenatal clinic tested for syphilis (which replaced number of syphilis treatments distributed to pregnant women in public health 36 centers)*** (2013) NA 90 48 Percentage of facilities visited by the district technical assistants 0 95 NA 100 * The initial target values for these indicators were ambitious even though, retrospectively, they look low compared to the actual end values. As per OP/BP 8.00, the project aimed to preserve service delivery, and the initial target values for these indicators could have been aligned with actual values at appraisal. The initial target values for these indicators were actually aligned with actual values achieved before the crisis, which were higher than the actual values at appraisal, because the package of interventions included activities to be implemented for the first time that were rightly expected to increase rather than maintain the demand for health services, including the establishment of a fee exemption system. However, the impact of the package of new interventions on service utilization in a very challenging context could not be accurately predicted at appraisal, and the actual impact was even more massive than expected. ** These indicators had ‘0’ under the baseline value since they were core sector indicators—as per Guidance Notes on Core Sector Indicators (World Bank Group 2012b). *** These indicators could have been outcome indicators related to effective service delivery like the outcome indicator on deliveries. 44. There were no evaluations in the health sector under the project that could be used for triangulation against the results framework to confirm achievements related to health service delivery. Surveys were conducted under the operation, including two prevalence surveys on NTDs (Institut Pasteur de Madagascar 2015; Faculté de Médecine d’Antananarivo 2017). However, these focused on the prevalence of NTDs and did not look at service delivery indicators, e.g. treatments effectively administered. A SDI Survey for Health was also financed by the project. However, as mentioned in paragraph 41, it cannot be used to assess achievements under this objective due to the SDI’s focus on the quality of services delivered. An independent evaluation of fee exemption systems on the use of health care services, published in Health Affairs, confirmed that the project had a significant positive impact on increasing antenatal consultations, child care consultations, and all outpatient consultations at a low cost.20 20In particular, the study estimated that prenatal consultations increased by 28 percent between 2013 and 2015 due to the project implementation (Garchitorena et al. 2017). Page 17 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Outcome 3: Critical nutrition service delivery is preserved in targeted vulnerable areas 45. The following activities were effectively implemented under the operation, as initially planned: (i) the establishment of new community nutrition sites; (ii) support to new and existing community nutrition sites; (iii) the recruitment of NGOs to supervise and support community nutrition sites; (iv) the testing of different approaches (intensive counselling, supplementation for kids and pregnant and breast-feeding women, and early childhood stimulation); (v) participation in the distribution of treatment against NTDs, including provision of small meals prior to administering the NTD medication and sensitization of the communities on the NTD campaigns; and (vi) capacity building in project management, including the procurement of transportation equipment for central and decentralized services of the National Nutrition Office and IT equipment for these services as well as NGOs. Food security activities at community and household levels, including home gardens, short-cycle farming and agriculture, and food conservation, supported by the Additional Financing, were also implemented. In addition, an innovative behavior change approach called ‘human centered design’ was developed and piloted after the MTR. 46. To monitor and evaluate achievements under this objective, six intermediate results indicators and two outcome indicators available in administrative databases were included in the results framework. All target values for the nutrition- related indicators were achieved or exceeded, except for one intermediate results indicator (percentage of nutrition sites submitting monthly reports through mobile phone within 5 days after the end of the month). however, the actual end value for the latter was close to the target value: 92 percent of community nutrition agents submitted monthly reports within 5 days after the end of the month against a target value of 95 percent (see Table 4). 47. As previously mentioned, target values for two intermediate result indicators were revised twice. For one of these, the actual end value was above both the final target value as well as the highest target value set through the Additional Financing (number of children under 2 years old enrolled in the growth monitoring program). For the other one, the actual end value was above the final target value and close to the highest target value set through the Additional Financing: 515,764 children under 2 years old were actually enrolled in the growth monitoring program by the closing date of July 30, 2017, i.e. 86 percent of the children targeted under the Additional Financing.21 21The rationale for this change was to reflect revised population projections (for more details, see the section on “Bank Performance, Compliance Issues, and Risk to Development Outcome”, “Quality of Monitoring and Evaluation (M&E)”, “M&E Implementation”). Page 18 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Table 4: Nutrition Component Indicators—Target and Actual Values Revised End Initial End Target (2014 Revised End Target Additional Target (Third Baseline (Emergency Financing Restructuring Actual End Indicator Value (2012) Project Paper) Project Paper) —2016) Value Outcome Indicators Number of people with access to a basic package of health, nutrition or reproductive health services*, ** 0 932,426 1,619,303 NA 2,268,854 of which number of pregnant/lactating women*, ** 0 182,161 311,350 NA 508,312 of which number of children under 5*, ** 0 750,265 1,307,953 NA 1,760,542 Number of children under the age of 24 months benefiting from improved infant and young child feeding practices* 0 164,220 289,340 NA 425,360 Intermediate Results Indicators Number of children under 2 years old enrolled in the growth monitoring program*** 140,319 234,600 413,343 321,999 462,315 Number of children between 2 and 5 years old enrolled in the MUAC program***, **** 207,368 341,000 600,810 433,204 515,764** Number of community nutrition agents trained to provide health and nutrition education 1,484 2,000 2,837 NA 3,582 Percentage of nutrition sites submitting monthly reports through mobile phone within 5 days after the end of the month 65 95 NA NA 92 Number of schools supported by community nutrition agents during deworming sessions 0 2,500 NA NA 6,587 Number of households receiving support kits (inputs and equipment) for short cycle agriculture and livestock activities 0 NA 23,114 NA 23,114 * These indicators had ‘0’ under the baseline value since they were core sector indicators—as per Guidance Notes on Core Sector Indicators (World Bank Group 2012b). ** As mentioned, this indicator could have been an output/intermediate-result indicator instead of an outcome indicator. *** These indicators could have been outcome indicators related to effective service delivery like the outcome indicator on children benefitting from improved infant and young child feeding practices. **** The number of children between 2 and 5 years old actually enrolled in the MUAC program represent 86 percent of the highest end target value set through the Additional Financing and subsequently decreased through the third restructuring. 48. There were no independent evaluations in the nutrition sector under or beyond the project that could be used for triangulation against the results framework indicators related to nutrition service delivery in targeted areas. Page 19 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Justification of Overall Efficacy Rating 49. The operation exceeded or fully achieved the target values for all the outcome indicators. One outcome indicator (number of direct beneficiaries) had a target value that was revised twice—first increased and then subsequently decreased. However, even the highest target value (ultimately decreased) was exceeded (see paragraph 38). 50. In addition, most of the final target values for intermediate results indicators included in the results framework were fully achieved or exceeded. Among the 15 intermediate results indicators included in the results framework, 13 final target values were fully achieved or exceeded. For one of the indicators for which the final target value was not achieved, the actual value was extremely close to the final target value (92 percent of monthly reports shared on time against a target of 95 percent). For the other one, the gap between the actual end value and the final target value (48 percent of pregnant women tested for syphilis against a target of 90 percent) deteriorated during the last months of implementation due to an inadequate planning of activities under the operation that were linked to the support of the Global Fund. 51. Two intermediate results indicators had target values that were revised twice—first increased and then subsequently decreased. For one of those indicators (number of children under 2 years old enrolled in the growth monitoring program), the highest target (ultimately decreased) was in any event exceeded. For the other one (number of children between 2 and 5 years old enrolled in the MUAC program), the actual end value was close to the highest target (ultimately decreased). 52. As summarized in Table 5, the respective conclusions for outcome indicators and intermediate results indicators are similar for all phases of the project. Therefore, applying a split rating is not relevant in this case. Table 5: Number of Indicators with Target Value Achieved or Exceeded Target Value Achieved or Exceeded Revised Target Values Initial Target Values (Additional Financing Project Final Target Values (Third (Emergency Project Paper) Paper) Restructuring) Outcome Indicators 7/7 7/7 7/7 Intermediate Results 13/15 12/15 13/15 Indicators (with one actual value close (with two actual values close (with one actual value close to the unachieved target to the unachieved target to the unachieved target value) values) value) 53. Considering the above, the overall efficacy is rated Substantial. Given that the outcome indicator targets were fully met, there could be a justification for the overall efficacy to be rated High. However, the Substantial rating is more appropriate because the intermediate result indicator for which there was an important gap between the actual value and the target value could have been an outcome indicator (percentage of women attending antenatal clinic tested for syphilis). Page 20 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) C. EFFICIENCY Assessment of Efficiency and Rating Economic analysis at appraisal 54. Due to the emergency situation in Madagascar and the condensed preparation timeline, a prospective economic analysis of this project with assumed impact and projected beneficiaries was not conducted at appraisal. Rather, the existing evidence on the benefit-cost ratios (BCR) of health, nutrition, and educational interventions was presented. For this reason, a direct comparison of the benefit-cost findings from the Implementation Completion and Results Report (ICR) analysis with the findings at appraisal is not possible. Economic analysis for the ICR 55. A cost-benefit analysis (CBA) was conducted for the ICR to assess the returns on investment, in terms of internal rate of return (IRR), estimates of net present value (NPV) and the BCR, of the Emergency Support to Critical Education, Health and Nutrition Services Project (see Annex 4 for details). Standard methods were used to estimate the economic benefits generated through the preservation of critical health, nutrition, and education services to assess whether the investment is justified on economic grounds. The analysis was conducted using regional-level data where possible, and then aggregated into standard measures of economic return. For each sector, the CBA followed a three-step process: first, it estimated the change in service utilization that can be attributed to the project; second, it translated the increase in service utilization into health and educational benefits; and third, it assigned a monetary value to these benefits. Benefits were estimated in terms of the number of maternal lives saved, the number of lives saved among children below age 5, and the number of children school-years preserved. 56. For the health and nutrition interventions, we used findings from a rigorous impact evaluation that estimated the increase in maternity health service utilization (14 percent per year) and outpatient services for children under the age of 5 (29 percent per year) resulting from this Emergency Project (Garchitorena et al. 2017). Higher health service utilization was then associated with the number of maternal and child lives saved based on existing evidence (Graham 2001 ; McGuire 2006 ; Kakietek 2016). 57. For the education interventions, we assumed benefits were achieved through two main pathways: (i) educational interventions aiming at keeping schools open and functional to increase the enrollment of children in primary school, and (ii) deworming treatment aiming at decreasing absenteeism (i.e. increasing participation). The project’s impact on school enrollment was defined as the difference in the number of children enrolled in primary school in 2017 and the counterfactual number of children that would have been enrolled had the enrollment rate increased at the same pace as in the preceding period (2009-12), by region. Benefits from the deworming program were quantified in terms of increased school attendance for all children receiving three years of deworming treatment (+0.294 school years) (Baird et al. 2015). Existing evidence was used to translate the benefits from the increased number of children school-years into preserved work productivity and future wages (Montenegro and Patrinos 2014). 58. All costs related to the project were included in the analysis, i.e. the initial credit and the additional financing. The impact of health and nutrition status and educational attainment of project beneficiaries was then translated into economic benefits in terms of average preserved future earnings of an individual (gross domestic product (GDP) per capita), adjusted for the percentage of mothers working22 and the percentage of GDP from wages in Sub-Saharan Africa (SSA) (Lübker 2007). We assumed that benefits accrue over the working lifetime of pregnant women (22-65) and children (18-65), and that the economy will grow at an even rate of 2.24 percent (the long-term economic growth in SSA) over the 22 WDI, 2016. Page 21 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) time horizon of the analysis. All benefits and costs were discounted with a 9.5 percent rate, the current interest rate from the Madagascar Central Bank (current opportunity costs of capital). Finally, the costs were compared to the benefits, and the IRR and the BCR were calculated. 59. The total investment through the project was of US$68.2 million over 3.5 years (2013-17). The estimated number of maternal lives and of lives of children under age 5 saved with the interventions in the health sector were 10,270 and 52,203, respectively, while the estimated number of lives saved among children under age 5 with nutrition interventions was 1,735. Moreover, the project was estimated to have increased the number of children in primary school by 33,070. The CBA also captured the benefits from the deworming program in terms of increased school attendance for all children receiving three years of deworming treatment (+0.294 school years) (Baird et al. 2015). 60. The project yielded an IRR of 17 percent and generated economic benefits with a NPV of US$108 million. With an IRR higher than the discount rate, the project yielded an economic return higher than the return on investment from alternative investments in the financial market. The investment had an attractive 3.1 BCR, suggesting that each US$ invested yielded an economic return of US$3.1 for the population of Madagascar. Sensitivity analyses showed that the results of the CBA were sensitive to changes in key modelling assumptions. However, the main conclusion was not affected, namely that the investment was justified on economic grounds. Even under more conservative assumptions of no economic growth and a higher discounting factor, the benefits still outweighed the costs. 61. The CBA is likely to have underestimated the benefits of the project for a number of reasons. First, the CBA did not capture changes in disability that are likely to have resulted from the health and nutrition interventions. Instead, the analysis focused on the number of lives saved. Second, the analysis only captured the impact of nutrition interventions on under-5 mortality, but not on maternal mortality. Third, nutrition-specific interventions are likely to increase overall labor productivity, while the analysis only captured the impact on wages. Fourth, the CBA captured the direct benefits from the deworming treatment on the treated children, while studies have shown that deworming has significant positive externalities on nontreated pupils in the same school as well as on pupils in neighboring schools. Finally, conservative assumptions were used throughout the analysis. Operational efficiency 62. The time from approval to effectiveness (4.9 months) and from effectiveness to first disbursement (1.3 months) was below the average of other projects implemented in the country and generally in Africa (Table 6). The total time from concept to first disbursement was 10.4 months, compared to 24-26 months on average. The shorter preparation time reflected the fact that because this was an emergency project, an expedited process was used. In fact, OP/BP 8.00 specifies that for simple new emergency recovery loans, the task team should aim to prepare the operation (from concept to approval) within 2.5 months. However, given the multi-sectoral scope of the operation, the time for the preparation of this operation was reasonable. The time for the consecutive phases of the project (to approval and to first disbursement) was significantly shorter than the averages for the country, the region, and the Bank. Table 6: Time for Project approval, effectiveness, and first disbursement compared to Bank’s average time Page 22 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 63. The level of readiness for implementation was high in Madagascar, therefore the project could leverage existing interventions and institutional arrangements. The creation of a cross-sectoral steering committee created synergies between sectors and levels of service delivery that contributed to the smooth implementation and the fast disbursement rate (US$68 in 3.5 years, including the five-month extension of the Original Credit closing date and the twelve-month extension of the Additional Financing credit closing date). 64. The in-country staff turnover rate was relatively low, although the project experienced some delays in in-country staff replacement (e.g. the Director position at the Nutrition PIU was vacant for one year). The Bank staff turnover was very low, which minimized learning costs for new staff. 65. The in-country operating costs made up 12 percent of the total project costs. Although this percentage was slightly higher than expected (5-10 percent), operating costs were considered reasonable due to the multi-sectoral nature of the project (three PIUs), and the emergency situation in Madagascar resulting in the need for more technical assistance. Financial Analysis 66. With a per capita GDP of US$401 and with 77.8 percent of the population living in poverty, Madagascar is among the poorest countries in SSA. In recent decades, the nation suffered the consequences of multiple political and macroeconomic crises, which resulted in volatile economic growth, fiscal revenue, and government spending hindering the achievement of the MDGs. 67. The recent political crisis that started in 2008-09 hampered the macroeconomic stability, affecting fiscal space for government spending and consequently the delivery of critical health, nutrition, and education services. For example, the GDP growth dropped from 7 percent in 2008 to negative 4 percent in 2009, and many donors’ activities were suspended. 68. The 2009-13 political crisis impacted the budget for health, nutrition, and education services. The drop in resources was mainly driven by the withdrawal of external aid, while internally funded expenditures remained relatively stable in real terms until 2014. The improving macroeconomic conditions are expected to increase the fiscal space for funding these critical services. Government spending (internally and externally funded) remains well below the resources needed to provide these critical services. 69. The political crisis ended in 2014, and the new economic and social development strategy (the National Development Plan) began to bear fruit. The GDP growth reached 4 percent in 2016, the tax burden also gradually increased, from 9.9 percent in 2014 to 11.6 percent in 2018; and international reserves have also been accumulating (representing 4 months of imports in 2016). 70. However, the success of the National Development Plan and developmental progress hinges on domestic political and macroeconomic stability, which remains a concern based on a recent International Monetary Fund (IMF) report (IMF 2017). Therefore, the new strategies put in place by the government, the catalytic role of International Development Association (IDA) financing (including the Bank’s new project on stunting (P160848)), and the growing commitment of development partners will be critical to support the government of Madagascar in achieving the Sustainable Development Goals (SDGs) in the health, nutrition, and education sectors. 71. Considering the above, the efficiency is rated Substantial. Page 23 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) D. JUSTIFICATION OF OVERALL OUTCOME RATING 72. As per Bank Guidance on ICRs, the overall outcome rating is Satisfactory (Table 7).23 Table 7: Justification of Overall Outcome Rating Sub-Ratings Overall Outcome Rating Relevance Efficacy Efficiency High Substantial Substantial Satisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 73. The operation had gender dimensions. The gender gap in Madagascar is evidenced by the high maternal mortality ratio. Component 2, under which almost all outcomes were fully achieved or exceeded, focused on, inter alia, activities increasing access and utilization of maternal health care services and contributing to close the gender gap, including: (i) procurement of essential obstetric equipment; (ii) training on obstetric care for medical staff at the facility, district and regional levels; (iii) recapitalization/restocking of public pharmacies, and (iv) development of a fee exemption system for pregnant women. Nutrition interventions were also centered on mothers, aiming to improve their knowledge about nutrition and increase their assets under food security activities. 74. In addition, the results framework comprised several indicators that included a gender dimension: (i) the percentage of females in the total number of project beneficiaries; (ii) the percentage of females in the number of students enrolled in primary schools; (iii) the percentage of females in the number of community teachers certified to be in service and paid. The end targets for these gender-related supplement indicators were all exceeded (for more details, see Table 1, and Table 2). Institutional Strengthening 75. Despite the preparation of the project under OP/BP 7.30, close collaboration with the Government was ensured at the technical level through both the preparation and the implementation phases, strengthening technical stakeholders’ capacities to design and implement priority interventions. In particular, the three existing PIUs that had been in charge of the main donor programs in the education, health, and nutrition sectors for years, which were embedded in government services, were deeply involved in the project preparation and were designated as the implementing agencies respectively for the education, health, and nutrition components of the project: • The Education PIU (Unité d’Appui Technique-Education Pour Tous—UAT-EPT), mapped to the Ministry of Education, had been in place since 2002 and had been responsible for the implementation of, inter alia, the IDA project “Crédit de Renforcement du Secteur de l’Education” (CRESED), and the multi-donor trust fund for the Education for All-Fast Track Initiative (EFA-FTI)—now GPE—for which the Bank had been the supervising entity prior to the crisis in 2009. 23 For details about how to derive the Overall Outcome Rating, see World Bank Group 2017f, and in particular Appendix H. Page 24 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) • The Health PIU (Unité de Gestion des Projets de Santé—UGP Santé), mapped to the Ministry of Health, had been operating since 1999 and had been responsible for the implementation of multiple donors’ interventions, including the IDA Multisectoral STI/HIV/Aids Prevention Projects. • The Nutrition PIU (Unité Programme National de Nutrition Communautaire—UPNNC), mapped to the National Nutrition Office, had been operating since 1992 and had been responsible for the implementation of Community Nutrition Projects until 2011, and had been involved in the IDA Multisectoral STI/HIV/Aids Prevention Projects. 76. Supervision capacities of the PIUs and other services of the Ministry of Education, the Ministry of Health and the National Nutrition Office were strengthened under the project through tailored technical and financial support. For example, the Ministry of Education received motorcycles, as well as computers, for regional, district, and local services. Similarly, the General Secretary of the Ministry of Health received computers, while regional services received vehicles, and district services received full-time technical support from consultants and computers. Central and decentralized services of the National Nutrition Office received vehicles and computers, contracted NGOs received motorcycles, community nutrition workers received mobile phones, and an organizational audit was conducted to inform future capacity building activities. 77. Furthermore, ad hoc cross-sector coordination mechanisms were established, which contributed to ensuring the ownership of key decision-makers and strengthening the government’s capacity to implement interventions involving Education, Health and Nutrition Services. A Technical Steering Committee made up of the General Secretaries of the Ministry of Education, and of the Ministry of Health, and the National Coordinator of the National Nutrition Office, and chaired by the General Secretary of the Ministry of Finance, was put in place and met bi-annually—except during the first year of project implementation. In addition, a coordination unit (Cellule de Coordination du Projet—CCP) comprising a part-time, and later full-time coordinator and an administrative assistant was put in place to support the Technical Steering Committee and to facilitate the coordination of cross-sector activities, including mass drug administration at the community level and school health and nutrition interventions, which were implemented for the first time under the project. Mobilizing Private Sector Financing 78. Not applicable. Poverty Reduction and Shared Prosperity 79. As stated in the PDO, the operation targeted vulnerable areas. In particular, the 5 regions covered under the initial credit—among the 22 regions of the country—were selected based on the following criteria: (i) they had the highest poverty rates according to the 2011 Poverty Map; and (ii) they also had low social sector outcomes. By reducing household out-of-pocket payments for health and education services and increasing assets of the most vulnerable households under food security activities, the project contributed to poverty reduction and shared prosperity. Other Unintended Outcomes and Impacts 80. Not applicable. Page 25 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 81. Objective. As per OP/BP 8.00, the operation was aimed at preserving essential services but not at addressing long-term economic issues. The objective of the operation was to preserve essential services and reverse the downward trends in education, health, and nutrition service delivery in targeted regions. This objective was realistic and set at the right level of ambitiousness for an emergency project prepared in the context of a long-lasting political and economic crisis. 82. Design. Recognizing the critical importance of speed, flexibility, and simplicity for an effective rapid response highlighted in OP/BP 8.00, the operation was: • Prepared in a timely manner as follows: Activity Initiation Summary (AIS) signed off: May 31, 2012; Concept Review: July 25, 2012; Decision Review: October 5, 2012; and Approval: November 29, 2012.24 • Flexible, with a significant unallocated amount representing 9.23 percent of the total credit amount. • Simple: although the operation covered three sectors, it focused on interventions that had already been implemented and tested under previous operations, using the same procedures, and a few innovative interventions for which there was strong ownership from counterparts (including school health and nutrition interventions, a fee exemption system in primary health care facilities, and various innovative interventions on nutrition to be tested and evaluated). Also, for those activities that included operational requirements, the sequencing was well defined, e.g. for school grants and community teacher salary subsidies, an appropriate timeline and sequencing was defined in the Project Papers and the Financing Agreements for the adoption of the needed regulation and for contracting appropriate operators. 83. Institutional Arrangements. As mentioned in paragraph 75, three PIUs had a demonstrated track record of satisfactory project implementation in education, health, and nutrition in collaboration with Government services and NGOs at the subnational level. These institutions were fully involved in the project preparation, ensuring strong ownership from the government’s side. In addition, the level of readiness was high. In particular, the PIUs were almost fully staffed, their existing operations manuals only needed to be updated, and twelve-month procurement plans were prepared and incorporated in the Project Papers (Original Credit and Additional Financing) for the three components. Because cross- sector coordination mechanisms were missing, it was agreed to establish a cross-sector Technical Steering Committee and a coordination unit to support the Technical Steering Committee and to facilitate the coordination of cross-sector activities. 24 OP/BP 8.00 specify that for simple new emergency recovery loans, the task team should aim to prepare the operation (from initiation to approval) within ten weeks. The timeframe for the preparation of this operation was reasonable though, given the scope of the operation, which covered three sectors in Madagascar. Page 26 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) B. KEY FACTORS DURING IMPLEMENTATION 84. During implementation, when challenges emerged, mainly in the areas of specific procurements, environmental safeguards, and personnel turnover within the PIUs, the government and the Bank largely addressed them in a timely manner, which limited their negative impact on project implementation and outcome. At the end, there was a need to extend (i) the closing date of the Original Credit by only five months from July 31, 2016 to December 31, 2016 to allow for the completion of a few activities, and (ii) the closing date of the Additional Financing by twelve months from July 31, 2016 to July 31, 2017—mainly to address the deferral of effectiveness and implementation due to the delayed signature of the financing agreement by the new government after the December 2013 elections. In addition, as mentioned, most expected outcomes were fully achieved and/or exceeded. Factors Subject to Government and/or Implementing Entities Control 85. Institutional Arrangements. Key stakeholders complied with the adequate institutional arrangement agreed upon during preparation. In particular, the Technical Steering Committee met biannually, in accordance with the Financing Agreement, including to approve annual work plans and review project implementation progress. The Technical Steering Committee was supported by the CCP, which also facilitated coordination between the three PIUs for cross-sector activities, including school health and nutrition interventions, and in particular the distribution to students of drugs for NTDs accompanied by food distributions, which were implemented in a timely manner. On the other hand, challenges related to staffing in PIUs emerged, which contributed to delays in project implementation that led to the extension of the closing dates. In particular, the Director position at the Nutrition PIU was vacant from April 2013 to August 2014, resulting in the National Coordinator for the National Nutrition Office also serving as the acting PIU Director. In Spring 2016, the Head of the Health PIU was replaced. The transition took two months, and as a consequence there was no authorized signatory during that period. 86. Procurement. Complex procurement of vehicles was performed through the United Nations Office for Project Services (UNOPS) in accordance with the Financing Agreement, as well as the procurement of information technology (IT) equipment. Similarly, the procurement of drugs was delegated to the national drug purchasing agency. Feedback on these procurement methods during interviews conducted for the preparation of the ICR was positive—both from the Bank and from the government—because these methods allowed for the timely completion of procurement processes and lower costs than expected. On the other hand, there were several challenges related to procurement performed by the government during implementation, which contributed to the aforementioned delays in implementation, including: • Under Component 1, iron/folic acid to be distributed to students was not purchased during the first year of implementation due to ambiguous government medical guidelines. The government then launched a competitive bidding process as recommended by the Bank, but the procurement was unsuccessful. The government eventually managed to complete a single source selection, contracting with the national drug purchasing agency, and two distributions were finally completed.25 The procurement of school manuals was also delayed, primarily due to a bidder’s complaint—which was handled in accordance with the Bank’s procurement guidelines, including consultations with the Integrity Vice-Presidency, responses to the complainant, and briefing of the complainant. 25The first distribution was conducted by the Education PIU in collaboration with the decentralized services of the Ministry of Education while the second one was taken over by the Nutrition PIU in coordination with the decentralized services of the Ministry of Education, as the deadlines were too tight for the Education PIU. Page 27 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) • Under Component 2, the procurement of solar-powered refrigerators for health facilities located in remote areas was also delayed. In particular, the competitive bidding process recommended by the Bank led to complaints because the technical requirements were specific. The government and the Bank eventually confirmed that specific specifications were necessary because the challenging environment required sophisticated devices and, as a consequence, they agreed that a single source selection was required. • Under Component 3, NGOs were contracted with a one year delay, which was mainly due to the late recruitment of a procurement specialist within the PIU and limited guidance from the Bank on the contract model. In addition, the procurement of poor quality livestock by regional government services with limited capacities led to complaints from beneficiaries. The government decided to indemnify the beneficiaries with its own resources and, with the Bank’s agreement, subsequently handed over the responsibility for such procurements to NGOs. 87. Environmental Safeguards. The government complied with safeguards requirements, although at a slower pace than initially expected. During the preparation, the project was classified as “Category B, Partial Assessment”. Activities to be supported by the project were expected to have some relatively minimal and site-specific adverse environmental and social impacts that would be easily manageable and would not involve land acquisition, leading to involuntary resettlement and/or loss of access to resources or livelihoods. Therefore, only the environmental assessment policy— OP/BP 4.01 (Environmental Assessment)—was triggered, and a revision of the existing medical waste management plan (MWMP) was required. The updated MWMP was prepared by the government based on a consultative process involving all stakeholders at the regional and national level in the health sector and with limited support—the government only received advisory support from the Bank and did not rely on consultants to perform this task. On the one hand, this process allowed for the government’s full ownership of the MWMP, which was implemented and monitored in a manner that was systematically deemed satisfactory by the Bank. On the other hand, this process contributed to delays in the area of Environmental Safeguards: the updated MWMP was published in the country and disclosed through the Bank’s Infoshop in December 2014 even though the disclosure had been due by February 2013 under the terms of the Financing Agreement, which affected the implementation of mitigation measures.26 In particular, delays in the systematic installation of secured pits in primary health care facilities was one of the contributing factors for extending the closing date of the initial project by five months. Factors Subject to World Bank Control 88. The Bank provided the government with close support over the entire implementation period. Formal implementation support visits involved experts from each sector, fiduciary team members and environmental safeguards specialists. These visits were held twice each FY, with adjustments in FY15-16 to accommodate budget constraints—there was one visit in FY15, which was offset by three visits in FY16, including one at the very beginning of the FY. The MTR review was conducted on time (October 27-November 7, 2014). All implementation support visits systematically included a thorough review of the implementation status, including joint sessions to update the results framework. In addition, after the MTR, monthly videoconferences with the Government were held to monitor progress and to address issues in a timely manner. Finally, education and health experts and consultants were based in Antananarivo during the entire implementation period, ensuring just-in-time dialogue and daily follow-up with the government, as well as monthly field visits. 26As mentioned in the next section on “Factors Subject to World Bank Control”, the Bank’s limited support and guidance also contributed to some of these delays. Page 28 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 89. Continuity in the Bank team composition was ensured despite a few changes in the roles of the team members. There were formally three Task Team Leaders (TTLs) in charge of the operation: • The initial TTL was an education staff in charge of the project preparation, with a health staff serving as co-TTL; • Subsequently, the TTLship was assumed by another education staff in charge of project implementation for one year, with the same health staff serving as co-TTL; • Finally, the TTLship was assumed until the project closing date by the health staff who had been the co-TTL since the beginning of the project, with an education staff serving as co-TTL. 90. The health staff, as well as the procurement and safeguards specialists based in Antananarivo, were involved as team members during both the preparation and implementation stages as team members, along with two financial management specialists. There was a core team of consultants involved, as was noted in the aide-memoires. 91. The quality of reporting, which was based on the government’s progress reports and intermediate financial reports, was high. The aide-memoires systematically included at least a section on implementation progress and issues, a section on fiduciary aspects, a section on environmental safeguards, a section on the results framework, an update about the issues discussed in the previous aide-memoire, and a section on pending issues with responsibilities and deadlines. These comprehensive documents were then candidly summarized in Implementation Status and Results reports, which were submitted on time. 92. As mentioned in paragraph 86, the use of multiple and flexible procurement arrangements, with the Bank’s support, contributed to the timely implementation of most of the important procurements packages under the Project. However, the Bank could have been more responsive to and flexible with a few procurement challenges: • Iron/folic acid to be distributed to students. The Bank resisted the government’s suggestion to make a single source selection with the national drug procurement agency and instead recommended international competitive bidding. However, the procurement was unsuccessful due to specificities in the technical specifications that did not match suppliers’ products on the international market. Finally, the government managed to complete a single source selection by contracting the national drug purchasing agency with the Bank’s approval; • The procurement of solar-powered refrigerators for health facilities located in remote areas. As mentioned, the Bank initially insisted that a competitive bidding process was adequate, before agreeing with a single source selection; and • The preparation of the model for contracts with NGOs under Component 3. Limited support to the new government team, which needed capacity building and guidance, contributed to the deferral of this preparation. 93. As mentioned in paragraph 87, the close collaboration between the government and the Bank contributed to strong government ownership and a satisfactory implementation of Environmental Safeguards. However, the Bank could have been more responsive, providing more timely support and clear guidance to avoid delays, including with the disclosure of the updated MWMP, and implementation of mitigation measures. Factors Outside the Control of Government and/or Implementing Agencies 94. Changes in the exchange rate positively affected project implementation and outcomes. The exchange rate changed in a favorable manner with regard to project resources, with the Madagascar Ariary devaluation leading to an increase in local currency resources while most transactions were paid in local currency. Ultimately, most of the expected outcomes were fully achieved or exceeded with part of the credits that remained undisbursed (9.19 percent of the original amount / 6.59 percent of the revised amount). Page 29 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 95. As discussed in paragraphs 9 to 16, the operation’s theory of change included in the Project Papers was clear and sound. The objectives were clearly specified and all expected outcomes were reflected in results framework indicators, which also adequately captured the operation’s activities and outputs contributing to the achievement of outcomes. Despite a challenging environment in terms of data collection in the context of a long-lasting crisis that contributed to the deterioration of public services, administrative databases were still populated with data on service utilization. Consequently, it was possible to agree upon a results framework including initially 22 indicators—plus supplement/breakdown indicators—and 23 after the Additional Financing that were aligned with the operational objectives. The results framework indicators were rightly selected among indicators available in existing administrative databases and, as a consequence, they systematically included baselines and targets.27 The results framework focused on changes over time, which was in line with the objective of the project to preserve service delivery in targeted regions. 96. Some of the framework indicators and their formulae were not accurately defined at the preparation stage.28 In addition, two indicators were included as outcome indicators when they should have typically been listed as output/intermediate results indicators (number of schools receiving school grants from the project; and number of people with access to a basic package of health, nutrition or reproductive health services), and four indicators were included as intermediate results indicators while they could have been outcome indicators (number of pregnant women receiving antenatal care during a visit to a health provider; percentage of women attending antenatal clinic tested for syphilis; number of children under 2 years old enrolled in the growth monitoring program; and number of children between 2 and 5 years old enrolled in the MUAC program). M&E Implementation 97. The high turnover of the monitoring and evaluation specialist position in the Nutrition PIU during implementation intensified the challenge noted in paragraph 96 regarding indicator definitions and their formulae.29 As a consequence, there were uncertainties and changes in the formulae during the first phase of implementation for some indicators, and beyond the MTR for one indicator—i.e. number of direct project beneficiaries—and the government and the Bank had to make tremendous efforts to address this challenge, including during implementation support visits. The definitions and formulae for most indicators were agreed upon before the MTR, and for the number of direct project beneficiaries an agreement was reached after the MTR but before the end of the project.30 27 Only indicators that were core sector indicators had ‘0’ as a baseline as per Guidance Notes on Core Sector Indicators, and indicators that were directly linked to the project. 28 Accurate definitions and formulae were not included either in Annex 2 of the Project Document, or in the Supplemental Letter of the Original Credit. 29 The monitoring and evaluation specialist who contributed to the preparation of the nutrition component of the results framework left the nutrition PIU in February 2013; the second monitoring and evaluation specialist worked from July 2013 to December 2014; the third one worked from June to November 2015; the fourth one worked from December 2015 to April 2016; the fifth one worked from May 2016 to January 2017; the monitoring and evaluation responsibilities were then taken over by the existing staff of the PIU until the closing date. 30 Consequently, the definitions and the formulae of indicators were agreed on and spelled out in the mission aide-memoires. Page 30 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 98. As discussed in paragraph 88, all implementation support visits systematically included a thorough review of implementation status, including joint sessions to update the results framework. The results of these joint work sessions were systematically reflected in the mission aide-memoires and the subsequent Implementation Status and Results reports. Hence, evidence of achievement of both outcomes and activities/inputs was regularly monitored by all key stakeholders. In addition, independent verifications that were conducted for two indicators that were directly related to transfers under the project (number of schools receiving school grants funded by the project, and number of community teachers certified to be in service and paid) confirmed that administrative data was reliable. 99. There was a delay in the update of the end targets for nutrition indicators set through the Additional Financing in the Bank’s reporting system (aides-memoires and Implementation Status and Results reports). The results framework was not updated automatically because the Additional Financing was processed under the Bank’s previous operations information system (“Portal”); the team updated the results framework manually in March 2016, instead of September 2014 at the time of approval. However, this delay did not affect the assessment of the project achievements since this shortcoming was addressed before completion. 100. The third and final restructuring conducted in July 2016 included, inter alia, decreases in the target values for three indicators (number of direct project beneficiaries; number of children under 2 years old enrolled in the growth monitoring program; and number of children between 2 and 5 years old enrolled in the MUAC program). The rationale was to reflect revised population projections. However, this change did not affect the assessment of the project achievements, and in particular did not involve a split rating, since the conclusions are similar for all phases of the project (see paragraph 52). M&E Utilization 101. Data on performance and results progress was used to inform project management and decision-making, particularly at mid-term. During the MTR, the government and the Bank made the decision to introduce new activities to inform future operations while there was evidence that the initial project was implemented smoothly and was on track to achieve its objectives and beyond due to the progress on the results framework indicators and the high disbursement rate. Consequently, the government and the Bank agreed to introduce new, more innovative activities focused on improving quality. Progress on the results framework indicators related to Component 3 (nutrition) also contributed to the decision to extend the closing date of the Additional Financing by one year, from July 31, 2016 to July 30, 2017. Justification of Overall Rating of Quality of M&E 102. The rating of quality of monitoring and evaluation is Substantial. The M&E system as designed and implemented was sufficient to assess the achievement of the objectives and test the links in the results chain despite moderate shortcomings in the design and implementation of the system. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 103. Environmental Safeguards. As mentioned in paragraph 87, the Government complied with safeguards requirements, although at a slower pace than had been initially expected. During the preparation phase, the project was classified as “Category B, Partial Assessment”. Only the environmental assessment policy—OP/BP 4.01 (Environmental Assessment)—was triggered, and the revision of the MWMP was required. The updated MWMP was prepared by the government with limited support. This process allowed for the government’s full ownership of the MWMP, which was implemented and monitored in a manner that was systematically deemed satisfactory by the Bank, despite a delayed disclosure through the Bank’s Infoshop and short delays in the implementation of mitigation measures. Page 31 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 104. Procurement. As mentioned in paragraph 86, procurement challenges emerged during implementation, including delays in the procurement of school manuals under Component 1 due to a complaint from a bidder. The complaint was handled in accordance with the Bank’s procurement guidelines. 105. Financial Management. Financial management reports were prepared and transmitted in a timely manner, including (i) interim unaudited financial reports that were deemed acceptable by the Bank, and (ii) audit reports on the project financial statements with unqualified opinion. One key factor that could have affected project implementation was the failure in 2014 of one of the microfinance institutions that was contracted by the government to deliver subsidies to community teacher salaries. Loan proceeds that had been transferred to the microfinance institution were collected by the microfinance institution’s creditors, which could have led to the exercise of remedies by the Bank such as a suspension of the credit because these were not eligible expenditures. However, in 2015, the government substituted its own resources to the loan proceeds, and managed to contract with alternative finance institutions to continue project implementation. The financial management rating was temporarily downgraded to “moderately satisfactory” in the Bank’s financial management information system, and in the Implementation Status and Results Report, without implications for the pace of project implementation. 106. At the end of project implementation, the following shortcomings were identified during Bank’s financial management supervision field visits: (i) shortcomings in budget monitoring by the Education PIU resulting in unexpected undisbursed resources at the end of the project, which contributed to a final disbursement level of 90.81 percent of the approved amount / 93.41 percent of the revised amount;31 (ii) weaknesses in contract management under Component 3 related to food security activities; and (iii) delays in the collection of required documentation on expenditures. As a consequence, the financial management risk was increased in April 2016 from ‘Moderate’ to ‘Substantial’ and the financial management rating was downgraded in April 2017 from ‘Satisfactory’ to ‘Moderately Satisfactory’ in the Bank’s financial management information system. However, these shortcomings did not lead to the establishment of mitigation measures that could affect project implementation and outcome since the government still complied with regular financial management reporting and also since these challenges emerged at the end of the implementation period. C. BANK PERFORMANCE Quality at Entry 107. As discussed in paragraphs 81 to 83 and 95, despite the short preparation period, the Bank managed to successfully identify, facilitate the preparation of and appraise the operation, including the following aspects: • The objective of the operation was clearly set out in the Project Papers and at the right level of ambitiousness for an emergency project, particularly in the context of a long-lasting political and economic crisis; • The design of the operation was kept simple. Although the operation covered critical service delivery in three sectors, it focused on activities that had already been implemented in an efficient manner and a few innovative interventions for which there was strong ownership from counterparts. In addition, the sequencing was well defined in the Project Papers of the Original Credit and the Financing Agreement; 31The exchange rate changed in a favorable manner with regard to project resources, with the Madagascar Ariary devaluation leading to an increase in local currency resources while most transactions were paid in local currency. The Education PIU identified the remaining resources shortly before the closing date, so these could not be reallocated to additional activities. However, most of the expected outcomes were fully achieved or exceeded even though part of the credits remained undisbursed (9.19 percent of the original amount / 6.59 percent of the revised amount). The revised amount reflects the amount taking into account undisbursed funds and is generated automatically by the World Bank’s information system. For further information, see data sheet for the project (page 2). Page 32 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) • The institutional arrangement was pragmatic, ensuring government ownership and cross-sector coordination. In addition, plans for appropriate capacity strengthening in PIUs were included;32 • The quality of the monitoring and evaluation system was substantial. 108. In addition, the operational risk assessment framework of the Emergency Project Paper was comprehensive. The overall implementation risk was appropriately rated “substantial” because the rating was “substantial” for five risks (Stakeholder, Governance, Program and Donor, Political Interference, and Delivery Monitoring and Sustainability), “moderate” for two risks (Capacity and Design), and “low” for one risk (“Social and Environmental”). Appropriate mitigation measures were identified and reflected in the Financing Agreement, e.g. mitigation measures for substantial country fiduciary risks such as the close supervision of fiduciary issues, and strengthened controls and audits through the development of internal audits within the PIUs; mitigation measures for the risk of political interference such as the selection of regions of intervention based on poverty maps and low levels of social sector indicators, capacity building of communities to enhance governance through trainings of parents’ associations on governance of school grants and teacher salary subsidies, and so forth.33 109. The Emergency Project Document included a full annex describing lessons learnt from previous Bank operations in the region and in the country. In particular, the project design reflected the need to: include in the results framework available data; build capacities of communities; collaborate with NGOs; align with the Government’s vision and priorities; keep maximum flexibility; maintain support to decentralized health and nutrition services; cultivate complementarity with other donors, including in terms of geographical areas of intervention; and ensure sound institutional arrangements, including cross-sector coordination mechanisms and intense dialogue with technical counterparts.34 110. The project design was also based on the analysis and recommendations of several pieces of Economic and Sector Work jointly undertaken by the World Bank and the government, as well as other sector studies prepared by development partners and local and international researchers, such as the World Bank study on the impact of the crisis on the education sector, a health country status report, the government’s sector strategies, and so forth.35 Quality of Supervision 111. As mentioned in paragraph 88, the Bank provided the government with close support over the entire implementation period. Formal implementation support visits included experts from each sector, fiduciary team members, and environmental safeguards specialists. Overall, these visits were held every six months. In addition, monthly videoconferences were held after the MTR. Finally, education and health staff as well as consultants were based in Antananarivo during the entire implementation period, ensuring a just-in-time dialog and daily follow-up with the government, as well as monthly field visits. Despite a few changes in the roles of the team members, continuity in the Bank team composition was ensured. 32 The positions required in the PIUs as described in the Financing Agreement included: (i) a national director in each PIU; (ii) an internal auditor in each PIU; (iii) a procurement specialist in each PIU; (iv) a financial management specialist in each PIU; (v) an accountant in each PIU; (vi) a monitoring and evaluation specialist in each of the Health and the Nutrition PIUs; and (vii) a safeguard specialist in the Health PIU. 33 For more details, see the Operational Risk Assessment Framework, Annex 4 of the Emergency Project Document. 34 For more details, see Annex 11 of the Emergency Project Document. 35 For more details, see main text and Annex 13 of the Emergency Project Document. Page 33 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 112. However, as mentioned in paragraphs 86 and 92, the Bank could have been more responsive to and flexible with a few procurement challenges, including: (i) the procurement of iron/folic acid; (ii) the procurement of solar-powered refrigerators; and(iii) the preparation of the model for contracts with NGOs under Component 3. In addition and as mentioned in paragraph 87 and 93, the Bank could have provided more intensive support in the area of Environmental Safeguards, particularly in terms of clear guidance and timely support during the preparation of the MWMP. 113. As mentioned in paragraph 91, the quality of reporting, based on the government’s progress and intermediate financial reports, was high. The aide-memoires systematically included sections on implementation progress and issues, fiduciary aspects, environmental safeguards, the results framework, an update about the issues discussed in the previous aide-memoire, and pending issues with responsibilities and deadlines. These comprehensive documents were then candidly summarized in detailed Implementation Status and Results reports that were also informed by discussions with the Country Management Unit and Health, Nutrition and Population/Education management. Implementation Status and Results reports were submitted on time. Management comments confirmed the key issues that the team highlighted in the Implementation Status and Results reports and summarized guidance that had been discussed with the team as part of an active dialogue on the operation. The only shortcomings in the area of reporting were: (i) a discrepancy between financial management ratings contained in the Bank’s financial management information system and those outlined in the Implementation Status and Results Report at the end of the implementation period (for further details about Financial Management ratings, see paragraph 106); and (ii) the delayed update of the end targets for nutrition indicators set through the Additional Financing in the Bank’s reporting system (for further details about this issue, see paragraph 99). However, these shortcomings did not affect project implementation and outcome. 114. Finally, the Bank aimed to ensure adequate transition arrangements by taking the following actions: • Supporting the development of the government’s education, health and nutrition strategies: the National Health Policy and the Universal Health Coverage Strategy approved in 2016, the Education Sector Plan adopted in 2017, and the National Nutrition Strategy adopted in 2017. As a result, these strategies include regular operation and continuation of activities that were supported under the Project; and • Launching the preparation of follow-up operations in the education sector and in the health and nutrition sector, and building these operations on activities that had been successfully implemented under the project but that still needed technical and financial support after credit closing. The follow-up education project under preparation includes teacher training as well as school health and nutrition interventions. The new health and nutrition operation includes: continued support for the distribution of safe delivery kits; community outreach; fee exemption systems; results-based financing; support for school health and nutrition interventions; and the roll-out of lipid-based nutrient supplementation for kids, and early childhood stimulation.36 36 The Multiphase Programmatic Approach to Improving Nutrition Outcomes in Madagascar was approved on December 12, 2017. This program is built upon and expands several successful implementation modalities under the Emergency Support to Critical Education, Health, and Nutrition Services Project which will help to ensure uptake of quality services in the first years of the Program: (i) the fee-exemption system, which ensures that subsidized services are free to beneficiaries; (ii) community outreach, which now includes antenatal care and family planning services in addition to immunization and consultations; (iii) assisted delivery kits; (iv) key results-based approaches including performance-based financing that will be scaled at primary care and community levels; (v) the multisector interventions related to NTDs, including deworming at school and community levels through a tripartite arrangement between the health, nutrition and education sectors; and (vi) the program arrangements at central level, with a multisectoral steering committee chaired by the Ministry of Finance to ensure government ownership. As mentioned, under the Emergency Support to Critical Education, Health and Nutrition Services Project, different approaches were tested, namely: (a) intensive counselling through household visits, (b) lipid-based nutrient supplementation for kids, (c) lipid- based nutrient supplementation for pregnant and breast-feeding women, and (d) early childhood stimulation, which were evaluated through a randomized controlled trial. Cutting edge research that informed adaptive learning in nutrition under the Page 34 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Justification of Overall Rating of Bank Performance 115. Based on the above, the rating of Bank performance is satisfactory since there were only minor shortcomings in the quality of supervision. D. RISK TO DEVELOPMENT OUTCOME 116. At the time of completion, the government had committed to achieving sustainable improvements in education, health, and nutrition, including through its sector strategies. The technical risk that outcomes that were achieved may not be maintained is low since the Government’s capacities were strengthened during project implementation (see paragraphs 75 to 77). The social risks, as well as risks related to stakeholders’ ownership and commitment, are also low due to the fact that there is an important demand from the population for such interventions. Finally, the financial risk is low because after completion, the government continued to implement activities that were supported under this project (e.g. subsidization of community teacher salaries, transfer of school grants, and stationary and mobile maternal and child care consultations). While the implementation of other activities was interrupted after the completion of the project (e.g. school health and nutrition interventions, the fee exemption system that was supported by NGOs, and results-based financing for primary health care facilities), the government confirmed its interest in these activities and looked for resources to continue their implementation. In particular, as mentioned in paragraph 114, two Bank projects in education and in health and nutrition will actually support such activities. 117. On the other hand, the development outcome will continue to face risks linked to the country’s cyclical fragility, i.e. political turmoil, weak governance, and weather related crises. However, the inclusion of a Contingent Emergency Response Component in the new Health and Nutrition operation and in the education operation under preparation will mitigate these risks by facilitating rapid responses to unforeseen challenges. V. LESSONS AND RECOMMENDATIONS 118. It is important for an emergency operation to have a realistic objective and a simple and flexible design. Setting the objective at a realistic level of ambition was key to the successful implementation and outcomes of this operation, namely preserving essential services instead of addressing long-term economic issues or more complex development outcomes. Maintaining flexibility was also essential: there was a significant unallocated amount of resources, which was ultimately used to address emerging challenges (such as natural disasters), and to implement innovative activities. Finally, the simplicity of the design of the operation was critical, in particular its focus on interventions that had already been implemented or tested under previous operations and a few innovative interventions for which there was strong ownership from counterparts. Such innovations were successfully implemented before the MTR, including school health and nutrition interventions, the establishment a fee exemption system in primary health care facilities, and various innovative interventions on nutrition. After the MTR, additional innovative interventions were introduced, including the distribution of school manuals, intensive trainings for teachers, a results-based financing pilot, and human centered design to improve the effectiveness of nutritional messaging and counseling. These innovative activities informed follow up operations, i.e. the education project under preparation, and the Multiphase Programmatic Approach to Improving Nutrition Outcomes in Madagascar that was approved on December 12, 2017. Emergency Support to Critical Education, Health, and Nutrition Services Project is a central part of the aforementioned Multiphase Programmatic Approach. Page 35 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 119. A pragmatic and stable institutional arrangement is a key factor to successful project implementation and the achievement of the targeted outcomes.37 The operation capitalized on the experience of existing PIUs, which was crucial for the high level of implementation readiness and the timely implementation of sector-specific interventions. The establishment of ad hoc coordination mechanisms, the Technical Steering Committee and the CCP, was also a key success factor, including with respect to the successful implementation of cross-sector activities. In fact, the cross-sector steering committee chaired by the Ministry of Finance and comprising line ministries, a new arrangement under this operation, is part of the institutional arrangements for the follow-up operation in the health and nutrition sector. Moreover, supervision capacities of decentralized services of the Ministry of Education, the Ministry of Health and the National Nutrition Office, as well as NGOs and communities were strengthened under the project through tailored technical and financial support, which was key to the successful and timely implementation of all the activities. It is also important to minimize changes both on the Government’s side and within the Bank team. On the Government’s side, it is particularly important to staff key positions in the PIUs in a timely manner and limit changes, including for the heads of the units. At the very least, it is critical that there is a transition period and a proper handover of responsibilities. An inadequate nomination and transition process of the head of the Health PIU during the last quarter of the project’s original credit contributed to delays that resulted in the need to extend the closing dates. Conversely, on the Bank’s side, limited changes in the TTLs and the core team members from preparation to completion, including staff and consultants based in the field, allowed for a constantly close project supervision. 120. During implementation, both flexibility and Bank support are also critical. The possibility to delegate procurement was crucial to the successful and timely implementation of complex procurement of transport and IT equipment. The successful purchasing of health products and equipment (e.g. iron/acid folic and solar-powered refrigerators for health facilities located in remote areas) also required flexibility, in particular the possibility to conduct single-source selections, including with the national drug purchasing agency that had been established with the Bank’s support under previous operations. In addition, Bank support was critical in areas of procurement where the Government had limited experience, e.g. new contracts with NGOs. The careful and timely extension of the closing dates was also key to the completion of all important procurement packages and environmental safeguards within reasonable deadlines. 121. Depending on the context and the thoughtfulness of the design, a multi-sectoral operation has a value-added. This emergency project was prepared in a challenging context, where there was scope for a very limited number of new operations to be prepared to help the government address the crisis. As a result, the social sectors that were at risk were packaged in one project with the recognition that some activities would be jointly implemented even though most of the interventions would be sector-specific, and the government and the Bank coordinated extensively to ensure a tailored, flexible design that maximized the multi-sectoral aspects of the project where possible. This decision resulted in several benefits that for the project and have informed follow up operations in the social sectors, including: (i) as mentioned in paragraph 119, the establishment of effective cross-sector coordination mechanisms with the involvement of the Ministry of Finance (a new arrangement in the country); (ii) the implementation of school health and nutrition interventions and mass drug administration at the community level for the first time in the country; and (iii) strengthened cross-sector social sector collaboration under the project to ensure joint planning and supervision, and beyond the project, e.g. analytic work such as the public expenditure review, the SDI survey, and analytical work on early childhood development. Importantly, the joint missions also fostered synergies on the Government’s end in its dialogue with the Bank and made the relationship more streamlined and efficient for the client. On the other hand, the limited adjustment of the Bank’s supervision resources to the multi-sectoral nature of the operation and the fragile context required intensive efforts to raise resources beyond regular supervision allocations, i.e. trust funds. . 37In fact, the use and strengthening of existing institutions not only contributed to prevent their collapse in a crisis situation, but also allowed for the development of the building blocks of the sector systems, including the governance and stewardship functions. Page 36 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Preserve critical education service delivery Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 1916726.00 2139153.00 2642180.00 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Female beneficiaries Percentage 0.00 55.00 59.30 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Comments (achievements against targets): The target value was exceeded (124 percent). The target value for the supplement indicator (female beneficiaries) was also exceeded (108 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Students enrolled in primary Number 974300.00 974300.00 1131353.00 schools in targeted regions 31-Jul-2012 29-Nov-2012 31-Dec-2016 Female beneficiaries Percentage 49.00 49.00 49.90 Page 37 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (116 percent). The target value for the supplement indicator (female beneficiaries) was fully achieved (102 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Total number of schools Number 0.00 6050.00 6682.00 receiving school grants funded by the project 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (110 percent). Objective/Outcome: Preserve critical health service delivery Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 1916726.00 2139153.00 2642180.00 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Female beneficiaries Percentage 0.00 55.00 59.30 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Comments (achievements against targets): The target value was exceeded (124 percent). The target value for the supplement indicator (female beneficiaries) was also exceeded (108 percent). Page 38 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Births (deliveries) attended Number 0.00 12600.00 131431.00 by skilled health personnel (number) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (1,043 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Children immunized Number 0.00 18300.00 286194.00 (number) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Children immunized - under Number 0.00 18300.00 286194.00 12 months against DTP3 (number) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (1,564 percent). Objective/Outcome: Preserve critical nutrition service delivery Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 1916726.00 2139153.00 2642180.00 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Page 39 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Female beneficiaries Percentage 0.00 55.00 59.30 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Comments (achievements against targets): The target value was exceeded (124 percent). The target value for the supplement indicator (female beneficiaries) was also exceeded (108 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People with access to a basic Number 0.00 932426.00 1619303.00 2268854.00 package of health, nutrition, or reproductive health 31-Jul-2012 29-Nov-2012 27-Mar-2014 30-Jul-2017 services (number) Pregnant/lactating women Number 0.00 182161.00 311350.00 508312.00 31-Jul-2012 29-Nov-2012 27-Mar-2014 30-Jul-2017 Children under 5 Number 0.00 750265.00 1307953.00 1760542.00 31-Jul-2012 29-Nov-2012 27-Mar-2014 30-Jul-2017 Comments (achievements against targets): Based on the reported data, the target value was exceeded (140 percent). The target values for the breakdown indicators were also exceeded (respectively 163 percent and 135 percent). However, unexpected constant changes in reported values over the project implementation period suggest that the actual values should be interpreted with caution. Page 40 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Children under the age of 24 Number 0.00 164220.00 289340.00 425360.00 months benefiting from improved infant and young 31-Jul-2012 29-Nov-2012 27-Mar-2014 30-Jul-2017 child feeding (IYCF) practices Comments (achievements against targets): The target value was exceeded (147 percent). A.2 Intermediate Results Indicators Component: Preserving critical education services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of community Number 0.00 10000.00 16999.00 teachers certified to be in service paid 31-Jul-2012 29-Nov-2012 31-Dec-2016 of which % of female Percentage 49.00 50.00 50.50 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (170 percent). The target value for the supplement indicator was fully achieved (101 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 41 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) School grants paid by the Percentage 0.00 95.00 99.90 project on time 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (105 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of children receiving Number 0.00 667944.00 1804964.00 anti-helminth treatment (school age children) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (270 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of parents' Number 0.00 6050.00 6688.00 associations/school management committees 31-Jul-2012 29-Nov-2012 31-Dec-2016 trained on teacher accountability process and use of school grants year Comments (achievements against targets): The target value was exceeded (111 percent). Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 42 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Target Completion Number of teachers trained Number 0.00 3750.00 19852.00 in school health and nutrition activities 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (529 percent). Component: Preserving critical health services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pregnant women receiving Number 0.00 18283.00 113131.00 antenatal care during a visit to a health provider 31-Jul-2012 29-Nov-2012 31-Dec-2016 (number) Comments (achievements against targets): The target value was exceeded (619 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Health facilities constructed, Number 0.00 347.00 347.00 renovated, and/or equipped (number) 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was fully achieved (100 percent). Page 43 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of women Percentage 36.00 90.00 48.00 attending antenatal clinic who are tested for syphilis 31-Jul-2013 21-Jul-2015 31-Dec-2016 Comments (achievements against targets): The target value was not achieved (53 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of facilities Percentage 0.00 95.00 100.00 visited by the district technical assistants 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (105 percent). Component: Preserving critical nutrition services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of children under 2 Number 140319.00 234600.00 321999.00 462315.00 years enrolled in the growth monitoring program 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Comments (achievements against targets): The target value was exceeded (144 percent). Page 44 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of children enrolled Number 207368.00 341000.00 433204.00 515764.00 in the MUAC program between 2-5 years of age 31-Jul-2012 29-Nov-2012 21-Jul-2016 30-Jul-2017 Comments (achievements against targets): The target value was exceeded (119 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Community Number 1484.00 2000.00 2837.00 3582.00 Nutrition Agents trained to provide health and nutrition 31-Jul-2012 29-Nov-2012 27-Mar-2014 30-Jul-2017 education Comments (achievements against targets): The target value was exceeded (126 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion percentage of Nutrition sites Percentage 65.00 95.00 92.00 Monthly Report submitted within +5 days of the end of 31-Jul-2012 29-Nov-2012 30-Jul-2017 the month through mobile phones Comments (achievements against targets): The actual value is very close to the target value (97 percent). Page 45 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of schools Number 0.00 2500.00 6587.00 supported by ACN during de- worming 31-Jul-2012 29-Nov-2012 31-Dec-2016 Comments (achievements against targets): The target value was exceeded (263 percent). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of households Number 0.00 23114.00 23114.00 receiving support kits (inputs and equipment) for short 29-Nov-2013 27-Mar-2014 30-Jul-2017 cycle agriculture and livestock activities Comments (achievements against targets): The target value was fully achieved (100 percent). Page 46 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1: Preserve critical education service delivery 1. Students enrolled in primary schools in targeted regions (of which % of female) Outcome Indicators 2. Total number of schools receiving school grants funded by the project 1. Number of community teachers certified to be in service paid (of which % of female) 2. School grants paid by the project on time 3. Number of children receiving anti-helminth treatment (school age Intermediate Results Indicators children) 4. Number of parents' associations/school management committees trained on teacher accountability process and use of school grants year 5. Number of teachers trained in school health and nutrition activities 1. Subsidies of community teacher salaries (2 payments per teacher and per year, covering 4 months—8 months per year being covered by the government): 89,129 2. Annual school grants: 20,158 3. Training sessions on school grants and subsidies to community Key Outputs by Component teacher salaries for each school, including parents’ associations: 5 (linked to the achievement of the Objective/Outcome 1) 4. Annual training sessions on school health and nutrition for school staff of each school: 3 5. Annual distributions of treatments against NTDs, accompanied by food distribution: 3 6. Distributions of iron/folic acid: 2 7. School manuals procured and distributed: 626,583 Page 47 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 8. Motorcycles procured for decentralized (regional, district and local) services of the Ministry of Education: 573 9. Computers procured for decentralized (regional, district and local) services of the Ministry of Education: 145 11. Annual surveys (verification of outcome indicator 2 and intermediate results indicators 1 and 2): 3 Objective/Outcome 2: Preserve critical health service delivery 1. Births (deliveries) attended by skilled health personnel (number) Outcome Indicators 2. Children immunized (number) 1. Pregnant women receiving antenatal care during a visit to a health provider (number) 2. Health facilities constructed, renovated, and/or equipped (number) Intermediate Results Indicators 3. Percentage of women attending antenatal clinic who are tested for syphilis 4. Percentage of facilities visited by the district technical assistants 1. Kits of essential medical equipment (64 items) procured: 347 2. Solar-powered refrigerators procured for facilities located in remote areas: 296 3. Safe delivery kits procured: 157,251 4. Laboratory tests and disposable items purchased: 12,139 boxes of 30 Bioline tests, 5,764 kits of 100 Determine tests, 440 kits of 20 Key Outputs by Component Unigold HIV tests, 1,054 boxes of 50 Benzathine Penicilline, 527 boxes (linked to the achievement of the Objective/Outcome 2) of syringes, etc. 5. Training sessions for medical staff on: obstetrical and neonatal care (347 head of primary health care facilities); fee exemption system (347 head of primary health care facilities, 487 NGO employees); medical waste management (347 head of primary health care facilities); administrative data collection (347 head of primary health care facilities) Page 48 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 6. Community outreach motorcycles and equipment purchased: 347 7. Lots of 21 essential drugs purchased: 347 8. NGOs recruited to manage fee exemptions (delivery of certificates and reimbursement of pharmacies): 60 9. Primary health care facilities contracted under the results-based financing pilot: 29 10. Technical assistants recruited to support district services of the Ministry of Health: 19 11. Vehicles purchased for regional services of the Ministry of Health to facilitate supervision visits: 5 12. Computers purchased for central and district services of the Ministry of Health: 21 13. Plumpy Sup procured: 125 tons (under third restructuring) Objective/Outcome 3: Preserve critical nutrition service delivery 1. People with access to a basic package of health, nutrition, or reproductive health services (number) Outcome Indicators 2. Children under the age of 24 months benefiting from improved infant and young child feeding (IYCF) practices 1. Number of children under 2 years enrolled in the growth monitoring program 2. Number of children enrolled in the MUAC program between 2-5 years of age 3. Number of Community Nutrition Agents trained to provide health Intermediate Results Indicators and nutrition education 4. Percentage of nutrition sites with a monthly report submitted within 5 days after the end of the month through mobile phones 5. Number of schools supported by ACN during de-worming 6. Number of households receiving support kits (inputs and equipment) for short cycle agriculture and livestock activities Page 49 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 1. Establishment of new community nutrition sites (identification of vulnerable areas, awareness campaign, election of community nutrition workers by the population): 1,005 2. Support to community nutrition sites: 1,008 scales/MUAC bands/measuring rods; 1,008 manuals; 1,100 posters; 1,980 cooking kits; 1,980 gardening kits; 3,582 working clothes; 3,582 calendars Key Outputs by Component annually. (linked to the achievement of the Objective/Outcome 2) 3. NGOs recruited to supervise and support community nutrition sites: 41 4. Training sessions: 3,582 community nutrition workers 5. Office equipment, including IT equipment, procured for central and decentralized services of the National Nutrition Office: 4 cars, 127 motorcycles, 18 computers Page 50 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Jumana N. Qamruddin, Rary Adria Rakotoarivony Task Team Leader(s) Sylvain Auguste Rambeloson Procurement Specialist(s) Maharavo Harimandimby Ramarotahiantsoa Financial Management Specialist Pauline Ravalisoamampianina Team Member Christian A. Rey Team Member Yvette M. Atkins Team Member Siobhan McInerney-Lankford Counsel Louis Jean De Marigny Team Member Paul-Jean Feno Environmental Safeguards Specialist Peter Lafere Social Safeguards Specialist Voahirana Hanitriniala Rajoela Team Member Rija Lalaina Andriantavison Team Member Laura Di Giorgio Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 4.000 35,856.81 FY13 34.232 285,841.21 Page 51 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Total 38.23 321,698.02 Supervision/ICR FY13 12.050 42,783.01 FY14 27.028 107,838.72 FY15 68.997 236,763.02 FY16 66.002 276,731.97 FY17 33.648 153,709.98 FY18 16.563 100,717.37 Total 224.29 918,544.07 Page 52 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Preserving 23.50 22.02 31.33 Critical Education Services Component 2: Preserving 25.00 24.81 33.33 Critical Health Services Component 3: Preserving 20.50 21.28 27.33 Critical Nutrition Services Unallocated 6.00 8.00 Total 75.00 68.11 99.99 Page 53 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 4. EFFICIENCY ANALYSIS Introduction and Purpose 1. The economic analysis conducted for the ICR of the Emergency Support to Critical Education, Health and Nutrition Services Project in Madagascar followed the most recent Bank’s guidelines (2014) and it included, in addition to a standard economic analysis, a financial analysis and a discussion of the operational efficiency of the Project. 2. An economic analysis aims to assess whether the dollar benefits of a program/intervention outweigh its dollar costs. By comparing the pros and cons of policies and programs, it helps policymakers to identify the most valuable options to pursue. The main advantage of the cost-benefit analysis (CBA) is that it monetizes all major benefits and all costs associated with a project so that alternative projects can be directly compared with each other. For this reason, CBA is often considered the “gold standard” method for evaluating programs. 3. Standard CBA provide three measures of the economic value of a project: the benefit to cost ratio (BCR), the internal rate of return (IRR), and the net present value (NPV) of the economic benefits resulting from the project. The BCR is defined as the ratio between the monetized benefits and the costs, both expressed in discounted present values. A BCR greater than 1 indicates that the NPV of the project benefits outweighs the NPV of the costs and it assesses the economic return generated for each dollar spent. Another very important output of the CBA is the IRR, i.e. the discount rate at which the NPV of all project cashflows (benefits minus costs) is equal to 0. The IRR is compared to the costs of capital to assess the project’s profitability. Finally, the NPV of the project is expected to be greater than 0 in order for a project to be economically attractive. Detailed Write-up on Methods Project Development Objective and Link with the Economic Evaluation 4. The project PDO was to “preserve critical education, health, and nutrition service delivery in targeted vulnerable areas in the recipient’s territory” that was deteriorating as a consequence of the crisis. To this purpose, the project took a multi-sectoral approach by supporting the health, nutrition, and education sectors and focused on the most vulnerable population (pregnant women, children under the age of 5, and children of primary school age). It supported the delivery of a comprehensive package of interventions in the three sectors to ensure that the delivery of critical services continued. The high-level outcome of the project was to preserve critical human capital in the country that would otherwise be lost as a consequence of the crisis, leading to long-term economic consequences for the current and future generations. 5. The economic analysis built directly on the PDO, while recognizing the greater complexity of the underlying causal chain of the interventions on the outcomes and economic benefits. The high-level outcome, to preserve critical human capital, included two components: (i) the number of maternal and child lives saved, which in monetary benefits translate into the number of active and productive individuals saved; and (ii) the number of children whose potential work productivity was preserved through continued attendance and participation in schools (in terms of cognitive development). 6. There are three key steps to undertake when conducting an economic analysis. First, the economic analysis needs to be linked directly to the PDOs, to the extent possible, because the PDOs define the key objectives of the project. Second, it requires assessing what portion of the changes in the PDOs can be attributed to the project, and what portion instead resulted from factors beyond the project’s control. The PDOs tracked during the project do not always imply attribution, and therefore necessary steps were taken to adjust for this difference (see the methods section). Third, these benefits need to be translated into monetary value. Page 54 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 7. Figure A.1 below provides an overview of how the project’s impact was modelled in the CBA for the ICR, and how the benefits were translated into monetary value. For the health sector, we estimated the number of maternal deaths averted (i.e. reduction in maternal mortality, MMR) and the number of deaths averted among children under the age of 5 (U5MR) that resulted from increased service utilization. A similar approach was used for the nutrition sector, although only the impact on U5MR was estimated because it related to the tracked PDO (“Number of children under the age of 24 months benefitting from improved feeding practices”). Finally, for the education sector we assumed that the impact occurred through two main channels: deworming treatment reduced school absenteeism (i.e. increased attendance to school) while other project’s interventions for the education sector increased school participation, i.e. the number of children enrolled in primary school. 8. It is important to highlight that, given the multitude of interventions supported by the project and the lack of detailed data at the regional level (including historical data to control for trends), it was not possible to measure the impact of each single intervention (e.g. increase in antenatal care, increase in immunizations, increase in vitamin A, and so forth). Moreover, modelling the impact of each component separately and assuming that they are additive would likely overestimate the benefits. 9. The economic analysis for the ICR reflected the sectoral approach taken by the project. The benefits of the project were modelled separately for each sector and region (where data was available), and then translated into standard aggregated measures of economic return on investment (Figure A.1). 10. An economic evaluation generally compares an intervention approach (such as this project) with the status quo, i.e. what would have happened without the project. In this CBA, the situation at the end of the project (2016/17) was compared to the situation before the project’s implementation (2013). 11. Since regions covered by the project were selected to avoid overlap between the Bank’s support and the support from other organizations (especially the European Union and the French Development Agency), all the costs and benefits estimated were assumed to result from the Bank-funded Emergency Project. Page 55 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Figure A.1: Project’s Impact and Economic Benefits Modelled in the CBA ↓MMR Health ↑ Service utilization Number of active and productive individuals preserved Nutrition ↑ Service utilization ↓U5MR Economic benefits ↑ School attendance Number of Education (deworming) children whose potential productivity was ↑ Participation preserved (other interventions) Notes: MMR = Maternal Mortality Ratio; U5MR = Under-5 Mortality Rate Project Costs 12. The costs included in a CBA should reflect the marginal costs of the program compared to the status quo. Costs should include all the resources added to the system, in this case the entire amount of the initial IDA grant (P131945) and the additional financing. The project costs thus reflected the actual total disbursement in US$ million (US$ M) per year, as shown in Table A.8. The total costs were US$68.2 million and the present value of the total disbursement was US$51.7 million. Table A.8: Total Project Disbursement in US$ Million per Year 2013 2014 2015 2016 2017 Total Disbursement 9.0 13.6 14.2 23.7 7.7 68.2 (US$ M) Project Beneficiaries and the Effectiveness 13. The effectiveness of a program is often measured through the PDOs as the change in the rate of service utilization. For example, the Emergency Project tracked the number of deliveries assisted by qualified providers or the number of children enrolled in primary schools in the targeted regions. However, measuring the effectiveness in terms of lives saved or disability-adjusted life years (DALYs) allows policy makers to compare a program to other initiatives, since this provides a standardized outcome measure and can be translated in monetary terms, which are then used as input into the CBA. To this purpose, it was assumed that the increase in utilization of services will bring about health benefits to the target populations through preventing occurrence of diseases, effectively managing illnesses, or reducing risks of death. Page 56 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Duration 14. When conducting a CBA, the year in which costs occur and interventions are deployed has an impact on the CBA results, due to the discounting of costs and benefits. For this analysis, costs and interventions’ deployment were assigned to the year in which disbursement occurred, based on the Bank’s operations system. For the interventions covered by the initial credit, the interventions were assumed to be deployed from mid-year 2013 up to the end of the year 2016 (3.5 years total). Instead, for the interventions covered under the additional financing, interventions were assumed to be deployed for 3 years from mid-year 2014 until mid-year 2017 (see Table A.9). Table A.9: Illustration of Project’s Timing. 2013 2014 2015 2016 2017 Initial credit Additional Financing Population Size per Year and Region 15. To quantify the number of lives saved in each year as a consequence of the health and nutrition interventions, the number of beneficiaries by region and year was estimated. To do so, data was used from the PIUs on the population size in 2017 by region (total population size, number of pregnant women, and number of children under the age of 5) and these statistics were estimated backward until year 2013, when the project began. In each region, the number of children under the age of 5 and the number of pregnant women was estimated applying the relative percentage from year 2017 (i.e. from actual data) to the total population size over the years 2013-16, in each region. The assumption was made that the proportion of children under the age of 5 and the proportion of pregnant women remained the same over the last four years. Conducting the analysis at the regional level was very important because these population statistics vary significantly by region. Using national population statistics would have likely led to an underestimation of the project’s benefits because the project was implemented in the regions with disadvantaged socioeconomic status (e.g. crude birth rate is higher than the national average). Impact of Health and Nutrition Interventions 16. For the health and nutrition interventions, an estimate was made of the number of maternal and child lives saved using findings from a rigorous impact evaluation of this Emergency Project (Garchitorena et al, 2017). The study estimated that the Bank’s project increased maternity care service utilization by 14 percent per year, and it increased outpatient service utilization for children under age 5 by 29 percent per year. The authors used interrupted time series analysis with a control group, and controlled for linear and seasonal trends in service utilization, as well as several other factors that may affect service utilization. The analysis was conducted for the region of Vatovavy-Fitovinany, and it was assumed that the interventions yielded the same impact in the other regions targeted by the project, given that the regions had a similar socio-economic profile. The impact evaluation was conducted only over a 2-year period. Therefore, a conservative assumption was made that service utilization remained constant afterwards because it was not known if service utilization would have increased further. 17. With respect to maternal lives saved from the health interventions, the PDO indicator “number of deliveries assisted by qualified health personnel” was used as a proxy for the increase in maternal health service utilization. The percentage of deliveries assisted before the project’s implementation (2013) was provided by the health PIU (Androy: 20.8; Atsimo Atsinanana: 20.3; Vatovavy Fitovinany: 20.7; Haute Matsiatra: 38.4; Amoron’i Mania: 40.238). The total 38For Amoron’i Mania, the percentage of assisted deliveries was taken from the MDG survey data because the number provided by the PIU seemed unrealistic (89.7 percent). Page 57 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) number of deliveries assisted by skilled health personnel in the regions over the project’s lifecycle was 131,431. However, health facilities were functional before the project’s implementation, and therefore the total number of assisted deliveries cannot be fully attributed to the project. In addition to that, there is need to control for the increasing number of pregnant women over time resulting from a growing population. Therefore, the findings from the impact evaluation study were used to calculate the number of additional assisted deliveries, i.e. the difference between the number of assisted deliveries that would have been observed if the percentage of assisted deliveries remained the same as in 2013, and the number of assisted deliveries under the estimated impact of the project. The reduction in maternal mortality resulting from the increase in the number of assisted deliveries was then estimated using findings by Graham (Graham 2001). The authors estimated a reduction in maternal deaths of between 16 to 33 percent through the prevention of complications such as obstructed labor, eclampsia, puerperal sepsis, and obstetric hemorrhage. To be conservative in estimates, the minimum estimate (16 percent) was applied. The number of additional assisted deliveries attributed to the project was estimated to be 68,874, which made up 52 percent of the total number of deliveries assisted in the target regions over the Project period. This resulted in 10,270 maternal lives saved (Table A.10). 18. To estimate the number of lives saved from the health interventions among children under the age of 5, findings were used from a study that estimated the impact of increasing basic maternal and infant health services on under-5 mortality (U5MR) using a large dataset from developing countries (McGuire 2006). Coverage of health service delivery was measured using expert ratings on health service coverage. The study found that an increase in maternal and infant program efforts by 35 percent would decrease U5MR by 18 per 1,000. Applied to Madagascar, where the impact evaluation found an increase in service utilization by 58 percent, this finding translated into a reduction in U5MR by 30 per 1,000. To calculate the number of deaths averted among children under 5, we assumed a decrease in U5MR by 30 per 1,000 over the years 2013-16 proportional to the project’s disbursement over time. For Androy and Atsimo Atsinanana, data on U5MR for the year 2013 was taken from the 2013 Multiple Indicator Cluster Survey (MICS), while for the other regions we assumed that U5MR remained the same as in year 2009 (2008-09 Demographic and Health Survey—DHS). The reasons for this choice were threefold. First, the MICS survey was only conducted for Southern Madagascar, and did not measure U5MR in the other three regions covered by the project. Second, U5MR in Androy and Atsimo Atsinanana changed slightly compared to the year 2009 (from 122 to 120 in Atsimo Atsinanana and from 77 to 85 in Androy). Assuming the regions covered by the project had a similar socioeconomic profile, it seemed plausible to assume a similar pattern in U5MR over a short time. Third, data from the only source of U5MR that covered all regions (2012-13 Millenium Development Goals National Monitoring Survey—Enquête Nationale sur le Suivi des Objectifs du Millénaire pour le Développement (ENSOMD)) was not comparable with DHS and MICS data. For example, the U5MR in 2013 in Atsimo Atsinanana was estimated at 120 in the MICS survey and 64 in the ENSOMD survey, while U5MR in 2009 was 122 from DHS data. The project was estimated to have averted 52,203 deaths among children under 5. The number of lives saved at the regional level is summarized in Table A.10. Page 58 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Table A.10: Estimated Number of Maternal and Child Lives Saved by the Project, by Year 2013 2014 2015 2016 Total Androy 66 270 277 285 898 Maternal deaths averted Deaths averted among 352 1,817 3,041 5,130 10,341 children under 5 Atsimo 92 376 387 397 1,252 Atsinanana Maternal deaths averted Deaths averted among 375 1931 3233 5454 10,993 children under 5 Vatovavy 176 724 743 764 2,407 Fitovinany Maternal deaths averted Deaths averted among 508 1310 2194 3700 7,712 children under 5 Haute 253 1,040 1,068 1,097 3,458 Matsiatra Maternal deaths averted Deaths averted among 977 2519 4216 7112 14,824 children under 5 Amoron’i Maternal deaths averted 165 678 696 716 2,255 Mania Deaths averted among 549 1416 2370 3998 8,333 children under 5 Total Maternal deaths averted 752 3,088 3,171 3,259 10,270 Deaths averted among 2,761 8,993 15,055 25,395 52,203 children under 5 19. Finally, we estimated the number of lives saved among children under the age of 5 resulting from the additional nutrition interventions (Table A.11). To this purpose, we assumed that a 1 percent increase in service coverage (such as increased breastfeeding promotion and complementary feeding education) saved 62 lives based on the findings from a recent study on the benefits of scaling-up nutrition services in Madagascar (Kakietek 2016). The CBA focused on the benefits from increased breastfeeding promotion and complementary feeding education, since the benefits from other nutrition-specific interventions (e.g. Vitamin A supplementation, and zinc for treatment of diarrhea) were likely to already have been captured as part of the benefits from increased health service utilization. Given a 28 percent increase in service utilization estimated by the impact evaluation, this resulted in 1,735 lives saved among children under 5. These were allocated over time proportionally to disbursement. Table A.11: Number of Deaths Averted through Nutrition-Specific Interventions among Children under Five 2013 2014 2015 2016 2017 Total Total 229 346 362 602 196 1,735 Impact of Education Interventions 20. The benefits of the education interventions were estimated using the PDO indicator on “The number of students enrolled in primary schools in the targeted regions” and “The number of children receiving anti-helminth treatment”. The objective was to capture two channels through which educational interventions preserved educational attainment: Page 59 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) first, by providing school grants, subsidies, and training and supervision, the project ensured that schools remained open and functional, thus keeping children in school (“participation”-effect). Second, by providing anti-helminth treatment to children enrolled in school, the Project decreased absenteeism (“attendance”-effect). 21. To estimate the impact on children’s participation in school attributable to the project, the difference was calculated between the observed number of children enrolled in primary school (number of children enrolled in 2016) and the number of children that would have been enrolled had the project not been implemented (number of children enrolled under the counterfactual scenario). The counterfactual scenario was estimated by assuming that, in the absence of the Bank’s engagement, the number of children enrolled in each region would have increased at the same rate over the project years (2013-16) as over the preceding years (i.e. from 2009 to 2013). The estimated project’s benefits reflected this difference (Table A.12). The additional number of children enrolled in primary school was allocated over time proportionally to disbursement. In Androy, the number of children enrolled in 2016 was below the counterfactual scenario, therefore the impact was zero. Table A.12: Estimated Project’s Impact on the Number of Children Enrolled in Primary School Atsimo Vatovavy Haute Amoron’i Androy Total Atsinanana Fitovinany Matsiatra Mania # of children 190,342 226,716 351,349 220,877 142,069 1,131,353 enrolled in 2016 # of children enrolled – 197,775 204,985 343,582 218,059 141,315 1,105,716 counterfactual Additional # of 21,731 7,767 2,818 754 33,070 - children enrolled Note: # = Number. 22. Finally, we assumed that the benefits from the deworming treatment were entirely attributable to the project. The total number of children receiving anti-helminth treatment (intermediate PDO indicator tracked by the project) was allocated over time proportionally to disbursement. Conversion of the Project’s Benefits in Monetary Terms 23. The last step of the CBA consists in converting benefits (estimated number of child and maternal deaths averted and preserved productivity) into (present) monetary value. The assumptions for the main results (base-case scenario) were summarized hereafter. 24. Each life year saved (maternal or among children under 5) was valued one time the gross domestic product (GDP) per capita in Madagascar, and benefits were calculated over the productive life of children and mothers benefitting from the interventions. In the case of maternal lives saved, per capita GDP was adjusted for the percentage of women currently employed (84 percent).39 This method reflects the human capital approach, i.e. the average lost future earnings of an individual in Madagascar. 25. Alternatively, benefits yielded from the education interventions were measured in terms of preserved increases in wages. The monetary benefits from the increased school participation were calculated as follows: for each additional child ( ) enrolled in primary school over the years (), the number of additional children in school was multiplied by the per capita GDP ( ) in Madagascar over the years of productive life (), adjusted by the percentage of GDP from 39 WDI, 2016. Page 60 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) wages in Sub-Saharan Africa (% = 30) (Lübker 2007), multiplied by the economic return from each additional year of primary school (), which was shown to be 12 percent in SSA (net of the foregone earning while studying) (Montenegro and Patrinos 2014). Finally, the additional number of school-years was adjusted for the average drop-out rate () in Madagascar (30 percent).40 _ℎ = ∑ ∗ % ∗ ∆ℎ ∗ ∗ ,, 26. A similar approach was used to estimate the benefits from deworming (school attendance). An assumption was made that after three years of treatment, the number of years in school increased by 0.294 per pupil (∆ℎ), based on the study by Baird and colleagues (2015). This assumed an economic return from primary school () by 12 percent, and it was adjusted with the percentage of GDP from wages in SSA (30 percent). Since the impact estimated by Baird and colleagues assumed a three-year treatment duration, benefits for children in the last two years of primary school were excluded since these would receive only one or two years of treatment. _ℎ = ∑ ∗ % ∗ ∆ℎ ∗ ,, 27. Once benefits are translated into monetary value, three key assumptions need to be made to estimate the NPV, i.e. the value of the costs and benefits at project start given that benefits are yielded much further out in the future. These assumptions defined the base-case scenario: • Time horizon of the CBA: benefits were accrued over the productive life of the beneficiaries. While people working in the formal sector may have a predefined retirement age, the poorest beneficiaries of the programs are likely to continue working all their life. Given a life expectancy at birth in Madagascar of 65 years,41 an assumption was made that children would start earning at age 18 and continue earning until the age of 65. In addition, it was assumed that pregnant women would continue working from the delivery age of 22, i.e. the median age at first birth (20 years, 2008-09 DHS) plus half the fertility rate (four children based on the most recent DHS data).42 Therefore, the benefits of the project for mothers reflected lost earning from age 22 to age 65. • Economic growth: using a conservative approach, it was assumed that there would be an economic growth equal to 2.24 percent, the average annual growth rate since 1997 in Sub-Saharan Africa. This rate was used because of the extended time horizon of the analysis and the uncertainty related to the political and macroeconomic situation in the country. • Discount rate: this is the rate at which benefits and costs are discounted over time. Most economic evaluations conducted so far assumed a 3 percent discount rate for costs and benefits from health interventions (NICE, 2014). In the base-case scenario of this CBA an assumption was made of a discount rate of 9.5 percent, the current interest rate from the Madagascar Central Bank, as this reflected the current opportunity costs of capital investments in the country. 40 UNESCO Institute for Statistics. 41 WDI, 2015. 42 2008-09 DHS. Page 61 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) CBA results 28. Under the base-case scenario, the Emergency Project yielded a ratio of benefits to costs of 3.1, suggesting that for every dollar invested, the project yielded an economic return of 3.1 dollars, i.e. more than three times higher (Table A.13). The initial investment of about US$68 million generated economic benefits with a net present value of US$108 million. The internal rate of return was 17 percent, which was significantly higher than the current costs of capital in Madagascar (9.5 percent). Sensitivity Analysis 29. Due to the uncertainty in the assumptions made in the base-case scenario, principles of economic analysis indicate that sensitivity analyses need to be conducted to assess the sensitivity of the CBA results to changes in key parameters. The key parameters were varied in sensitivity analyses and are summarized in Table A.13, where the BCR, the IRR, and the NPV of economic benefits are reported. 30. Sensitivity analyses showed that the results of the CBA were sensitive to changes in key modelling assumptions, but the main conclusions remain unchanged, i.e. that the investment was justified on economic grounds (Table A.13). Even under more conservative assumptions of no economic growth and higher discounting factor, the benefits still outweighed the costs. A higher discount rate of 12 percent reduced the BCR and NPV (BCR:1.9, NPV: US$45 million). To reflect uncertainty in economic growth, especially in a country with considerable political instability, the analysis was ran assuming no long-term economic growth. Even under this unlikely scenario, the CBA showed that the investment yielded positive economic returns: a BCR of two per dollar spent and a NPV of US$52 million. Under weaker restrictions in the key assumptions, the economic return was higher. For example, a lower discount rate at 5 percent (commonly used for CBA purposes) resulted in a BCR of 9.3 and NPV of US$488 million. Instead, assuming the economy would grow at the IMF-forecasted rate of 4.8 percent over the time 2016-22, and drop to 2.2 percent afterwards, yielded a BCR of 3.6 per dollar invested, and a NPV of the investment of US$137 million. Table A.13: Results from Cost Benefit Analysis – Base-Case Scenario and Sensitivity Analyses BCR IRR NPV Base-case scenario 3.1 17% US$108 M Sensitivity analysis Discount rate Alternative discount rate used in economic evaluations (5%) 9.3 17% US$488 M Alternative discount rate used in economic evaluations (12%) 1.9 17% US$45 M GDP growth No growth rate in per capita GDP 2.0 14% US$52 M Growth rate 2016-2022 of 4.8% (IMF Projections) 3.6 18% US$137 M Limitations 31. Given that a prospective CBA was not conducted at appraisal due to the emergency situation in the country, the findings from the CBA for the ICR cannot be directly compared with expected results at appraisal for this specific project. Furthermore, there are no assumptions to be validated in terms of expected impact/benefits. 32. Ideally, the situation in 2016-17 would have been compared with what would have happened had the country not received support from the Bank through this Emergency Project. It seems plausible to assume that the provision of health, nutrition, and education services would have deteriorated further, thus potentially increasing the negative Page 62 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) consequences on maternal and uder-5 mortality, and on cognitive development. However, it was not possible to model this counterfactual scenario due to the lack of data. Moreover, no impact evaluation of the entire project was conducted (across all regions and over the entire duration of the project). 33. The CBA underestimated the benefits of the project. First, the CBA did not capture changes in morbidity/disability that are likely to have resulted from the health and nutrition interventions. Instead, the analysis focused on the number of lives saved. Second, the analysis only captured the impact of nutrition interventions on under- 5 mortality, but not on maternal mortality. Third, nutrition-specific interventions are likely to increase overall labor productivity, while the analysis focused on the impact on wages only. Fourth, the CBA only captured the direct benefits from deworming treatment on the treated children, while studies have shown that deworming has significant positive externalities for non-treated pupils in the same school as well as for pupils in neighboring schools. Finally, conservative assumptions were used throughout the analysis. Operational Efficiency Aspects of Design and Implementation 34. The project built mainly on existing interventions and institutional arrangements for its design and implementation, especially leveraging the three PIUs that for years had been implementing donors’ interventions in the education, health, and nutrition sectors and that were embedded in government services. In addition, the level of readiness was high: the PIUs were almost fully staffed, existing operations manuals only needed to be updated, and twelve-month procurement plans were prepared and incorporated in the Project Papers. The main addition of the project was the establishment of a cross-sectoral Technical Steering Committee and a coordination unit to support the Technical Steering Committee and facilitate the coordination of cross-sector activities. 35. The elevated level of readiness, combined with the establishment of cross-sectoral coordinating mechanisms, created synergies between sectors and between levels of service delivery (community, health facilities, schools) that supported the fast speed of implementation and established robust processes that are now being scaled up. For example, the tools created for planning and budgeting for the project’s activities are now being used by the Ministry of Health, the Ministry of Education, and the National Nutrition Office to cost out implementation of the neglected tropical diseases strategy in the country. 36. The project’s efficiency was maximized by directing money to regions and districts rather than going through national levels, thus incurring lower transaction costs. This arrangement was also instrumental for the high speed of disbursement because resources could be allocated directly to the implementing partners without having to add an administrative layer. Time to Effectiveness, Disbursement, and Staff Turnover 37. The time from approval to effectiveness (4.9 months) and from effectiveness to first disbursement (1.3 months) was below the average in other projects implemented in the country and generally in Africa (Table 14). The total time from concept to first disbursement was 10.4 months, compared to 24-26 months on average. In part, the shorter time reflected the fact that this was an emergency project, and as such, an expedited process was used. In fact, OP/BP 8.00 specifies that for simple new emergency recovery loans, the task team should aim to prepare the operation (from concept to approval) within 2.5 months. Given the complexity and the scope of the operation, the time for the preparation of this operation was reasonable (4.2 months). The time for the consecutive phases of the project (to approval and to first disbursement) was significantly shorter than the average. Page 63 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Table 14: Time for Project approval, effectiveness, and first disbursement compared to Bank’s average time 38. The project had an exceptionally fast disbursement rate, disbursing US$68 million in just 3.5 years, including the five-month extension of the Original Credit closing date and the twelve-month extension of the Additional Financing credit closing date. This rate of disbursement is very high compared to other Bank projects, and reflected the high speed of implementation achieved by having in place a thorough design and implementation mechanism. Data on disbursement over time also showed a very smooth process and a continually increasing amount of resources disbursed, reflecting the fact that once disbursement started, no major challenge was encountered that slowed down implementation on the ground. 39. Staff turnover was limited in the PIUs over the duration of the project, and the government largely addressed this in a timely manner. However, there were delays in the nomination of the replacement of the Head of the Health PIU and also in the nomination of the Director of the Nutrition PIU. From the Bank’s side the turnover was low, with the two TTLs being on the project throughout most of its lifecycle, thus reducing to a minimum the learning costs for inexperienced staff to get up-to-speed with the project. Operating Costs 40. The total operating costs from the Bank’s side for preparing and supervising the project amounted to US$897,774 (see Annex 2), which represents 1.3 percent of the total project’s costs. In addition to that, the in-country operating costs to implement the project were about US$0.7 million for the education PIU, US$5.2 million for the health PIU, and US$2.5 million for the nutrition PIU, for a total of US$8.4 million. When compared to the total Project’s costs of almost US$68.2 million, the in-country operating costs made up 12 percent of the total costs (Table A.15). Given the multi-sectoral nature of the project, the challenging environment due to the emergency situation in the country, and the problems arising from the political crisis in the country, the operating costs remain in line with the expected costs for the Bank’s operations (5-10 percent). Table A.15: Operating costs on the ground (PIUs) Project Sector Costs (US$ M) Education PIU 0.7 Health PIU 5.2 Nutrition PIU 2.5 Subtotal Operating Costs 8.4 Operating costs as % of Total Project Costs 12% Page 64 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Financial Analysis Macroeconomic Context43 41. Madagascar is an island country with a per capita GDP of US$401 and a total population of 25.6 million people, 77.8 percent of whom are living in poverty. This poverty rate makes it one of the poorest countries in the Sub-Saharan Africa (SSA). The bottom 40 percent of the population holds 26.8 percent of the total income. 42. Its economy is based on the agriculture sector (which employs almost 80 percent of the population) and on the informal sector. The county is exposed to natural disasters which significantly affect the population and the country’s economy. As a predominantly agrarian economy, Madagascar’s main export products are non-ferrous metals, cash crops (e.g. coffee, tea, cocoa) and apparel/clothing. Most manufactured goods and processed foods need to be imported (e.g. petroleum products, fabric/textile), making the country sensitive to price volatility. While the official unemployment rate is low (3.4 percent in the urban areas, and virtually nonexistent in the rural areas), the rate of underemployment is striking at 80 percent of workers. Agriculture productivity (value added per worker in the agriculture sector) has declined from 2001 to 2010. As a consequence, poverty is mainly a rural phenomenon, as shown by the significantly lower urban poverty rate (about 59 percent). Figure A.2 shows the high variation in poverty rate by region in 2012. Figure A.2: Poverty Rate by Region (2012) Source: 2012 ENSOMD, IMF 2017. 43 World Bank Group 2015b; WDI. Page 65 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 43. Over the past 50 years Madagascar suffered from multiple political crises, as reflected by the high variation in GDP growth over time compared to other low-income countries (LIC) and other countries in the SSA region (Figure A.3). The first political crisis in 2003 halted a trend of GDP growth that had been about 4 percent per year, and resulted in the GDP dropping by 12 percent. The second political crisis in 2009 resulted from the un-constitutional transfer of power and had devastating consequences on the country’s social and economic system. The political crisis ended in 2014 with the new democratic elections. GDP growth dropped from 7 percent in 2008 to negative 4 percent in 2009. Beginning in 2011, GDP started growing again and reached a few years of stable economic growth rate around 3-4 percent that exceeded the demographic growth rate (2.8 percent). The increase in inflation rate has slowed down from 7.4 percent in 2015 to 6.7 percent in 2016. Figure A.3: GDP Growth (%) from 2000 to 2016 15 10 5 0 2000 2002 2004 2006 2008 2010 2012 2014 2016 -5 -10 -15 LIC Madagascar Sub-Saharan Africa (excluding high income) Source: Authors, using WDI. 44. The country is currently facing two main risks to macroeconomic stability: low capacity to generate fiscal revenue; and low international reserves. However, the new economic and social development strategy, National Development Plan adopted by Madagascar after the democratic elections, has been bearing fruits in the last few years (IMF 2017). 45. The capacity to generate resources from tax revenue is one of the lowest in SSA and has been decreasing since 2008. However, since 2014 the tax burden has been gradually increasing. Fiscal revenue, already low before the crisis at 13 percent of GDP in 2008 compared to an average 15.6 percent in low income countries, constituted 9.9 percent of GDP in 2014, 10.1 percent in 2015, and 10.9 percent in 2016, and is forecasted to be 11.2 percent and 11.6 percent in 2017 and 2018, respectively. Fiscal deficit declined from 3.7 percent in 2015 to 1.9 percent in 2016. 46. International reserves are low as a consequence of the dwindling external aid inflows that occurred after the political crisis, and the government’s policy of subsidizing imported fuel. The scarce international reserves make Madagascar more exposed to economic downturn due to less resources being available to serve as a cushion to external shocks. However, the country made some progress in this respect: international reserves represented 4.0 months of imports in 2016, up from 2.9 months in 2015. 47. Other significant political milestones included the preparation of an inclusive National Reconciliation Plan and the adaption of the National Reconciliation Law, the promulgation of the 2016 anti-corruption law and planned creation of institutional structures to fight corruption, and the proposal of a law on special economic zones to promote inclusive, sustainable economic growth, especially through stimulus of the private sector. Page 66 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 48. However, risks to the outlook remain elevated according to the last IMF Country Report (IMF 2017). The success of the new National Development Plan hinges on domestic political and macroeconomic stability as a precondition to increased private investment and foreign capital inflow. The biggest risks identified by the IMF are the remaining risk of political instability and the institutional weaknesses (e.g. human, financial, and technical resources), which are prerequisites to successfully implement the new strategy. Therefore, continued support by the development community is paramount for the country to exploit the economic potential. Health Financing 49. Madagascar spends significantly less on health than other low-income countries and the same is true when compared to other fragile low-income countries in the SSA region (“Peers”) (Figure A.4). Total health expenditure (HE) was 3 percent in 2014, compared to 6 percent in low-income countries and in these peer countries. The per capita HE is as little as US$13.7 (current US$), far below the average in low-income countries of US$40 per person. The low capacity of the government to generate fiscal resources translates into low health spending despite higher than the average public HE as percentage of total HE (48.8 percent versus 42 percent) and comparable public spending as a percentage of total government spending (around 10 percent). Figure A.4: Health Financing Indicators in Madagascar Compared to Similar Countries (2014-16) 60 50 48.8 40 44 42 38 40 30 20 10 13.7 6 6 10.2 10 10 0 3 HE (%GDP) Per Capita HE Public HE Public HE (% Total HE) (% government expenditure) Madagascar Peers Low-income countries Source: Authors, using WDI. 50. In addition to low HE, Madagascar has shown unfavorable trends in health financing indicators since 2010 (Figure A.5), with 2013 being an outlier year. Total HE, already low before the crisis at 4-5 percent of GDP, dropped further since 2010 and reached 3 percent of GDP in 2014. The per capita HE (in current US$) has been decreasing since 2008, and is now close to the 2005 levels at US$13.7 per person. Both health financing indicators are far below the minimum of $86 per person and 5 percent of GDP on health recommended by the Chatham House (Chatham House 2014). Also, the percentage of HE over total HE financed with government resources has been decreasing from 57.5 percent in 2011-12 to 48.4 percent in 2014. The share of out-of-pocket expenditures had ranged between 35 and 40 percent since 2000. It decreased significantly from 2010 to 2013 (likely as a consequence of safety concerns and a reduced demand for health services), and peaked at 41.4 percent in 2014. Finally, public HE as percentage of total government spending has also been decreasing from 18.6 percent in 2010 to 10.8 percent in 2014. Page 67 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Figure A.5: Trends in Health Financing Indicators in Madagascar, 2000-14 Madagascar Health expenditure per capita (current US$) Madagascar Health expenditure, public (% of total health expenditure) Madagascar Health expenditure, public (% of government expenditure) Madagascar Out-of-pocket health expenditure (% of total expenditure on health) 70 Madagascar Health expenditure, total (% of GDP) 61.7 60 57.5 54.3 55.1 55.0 53.8 53.6 51.8 52.1 52.5 50.6 51.2 49.9 48.4 50 47.0 40.7 41.4 39.2 39.1 39.9 37.8 38.4 38.2 40 36.9 35.9 36.0 36.8 34.8 34.9 30.7 30 21.2 20.1 18.9 19.6 18.6 19.3 19.2 18.1 17.1 17.3 17.2 20 15.3 14.4 15.3 15.0 14.7 15.4 14.0 13.9 13.9 13.2 13.5 13.5 13.7 12.5 11.7 12.0 12.3 9.7 10.2 10 5.1 5.2 5.3 4.8 4.9 5.0 5.1 5.0 4.5 4.7 4.9 4.2 4.1 3.5 3.0 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Source: Authors using Find my Friends tool based on WDI. 51. Trends in health financing indicators are mainly driven by changes in external aid, which funds almost 80 percent of the public health expenditure in Madagascar. The over-reliance on external funding exposes Madagascar to the funding volatility of aid common to fragile countries and poses serious concerns in terms of sustainability, ownership, and efficiency of existing resources. Trends in levels of externally funded health expenditure are shown in Figure A.6. Even though internally-financed HE nearly doubled in real terms between 2006 and 2013, this was not enough to compensate for the drop in external aid from 229 billion of 2013 Ar. to 158 billion of 2013 Ar (World Bank Group 2014h). 52. Fiscal space for health in Madagascar critically depends on the sustainability of external funding and the extent to which additional government and other domestic resources can be used to finance health services. Project expenses for the health sector of about US$7 million per year (US$25 for 3.5 years) made up 1.37 percent of the total health budget, based on the most recent NHA data (US$520 million, 2015). In light of the increasing fiscal revenue and the coming back of external aid, the project is likely to be sustainable. However, while this project addressed pressing needs arising from the emergency, significant progress remains to be made to achieve the SDGs. The new strategies put in place by the Government, the catalytic role of IDA financing (including the new Bank’s Project on stunting (P160848)), Page 68 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) and the growing commitment of development partners will be critical to support the government of Madagascar in achieving the SDGs. Figure A.6: Internally and Externally Funded Expenditure of the Ministry of Health, 2006-13 Note: Financing from internal resources (RPI) includes budget support to finance recurrent expenditure. All externally funded expenditures are in the investment budget. Source: World Bank Group 2014h. 53. The nutrition sector has experienced severe cuts to the National Nutrition Office budget, with the allocation declining from US$4.6 million in 2013 to US$1.7 million in 2016, as a consequence of the political crisis. In 2017, the National Nutrition Office budget almost doubled, but most of the domestic resource envelope continues to be spent on salaries, leaving limited resources for operational activities. The latter is also true in the health sector, where regular wages have increased as a percentage of the internally financed MoH expenditures from 53.4 percent in 2006 to 85.2 percent in 2013 (World Bank Group 2014h). Education Financing44 54. Public expenditure for education followed a similar pattern as spending for health (Figure A.7). Government education expenditures (GEE) were increasing from 2002 to 2005, both in terms of levels of spending (billions 2013 Ar.) and as a percentage of GDP. In real terms, public expenditures reached a peak in 2008, followed by a sharp drop coinciding with the onset of the political crisis. Since 2010, public expenditure has remained stable at about 600 billion 2013 Ar. despite the challenges posed by the crisis, but spending decreased to 2.6 percent of GDP. Over the same period (2000-12), public spending increased in comparable countries. For example, in SSA (developing countries only) public expenditure increased from 3.73 percent to 4.31 percent of GDP, and in low-income countries from 3.42 percent to 4.31 percent. 44 World Bank Group 2015g. Page 69 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Figure A.7: Government Education Expenditure (GEE), 2000-13 Source: World Bank Group 2015g. 55. The decrease in public spending on education since the inception of the political crisis was mainly driven by a contraction in external aid by 45 percent from 2011 to 2013 (from 250 2013 Ar. Billion to 138), while internal funding remained stable in constant terms over time (Figure A.8). The contraction in external aid increased even further the already high percentage of government spending as a percentage of total spending on education. Figure A.8: Composition of Public Expenditure in Internal Funding versus Foreign Aid Source: World Bank Group 2015g. 56. Despite the decreasing total public expenditure, the government’s commitment to the education sector seems to have increased after the crisis. In fact, the share of government expenditures for education increased from 15 percent in 2006 to almost 25 percent in 2013, while overall government spending contracted. These figures confirm the fact that, despite lower revenue, the government continued to finance education at the same levels as before the crisis, thus committing a higher percentage of its budget. 57. Nevertheless, the decrease in overall public spending led to a significant decrease in the per capita expenditure over the period 2009-13 compared to the period before the crisis, i.e. 2006-08. Figure A.9 shows the reduction in per capita spending on education by region together with the poverty ratio levels in 2005 and 2010. The 2010 Household Survey found that financial problems are the key drivers for school dropouts, especially direct costs which shifted from the government to the private households. One of the key challenges faced by the government in terms of sustainability are the rapidly growing number of teachers and their increasing salaries, which make up a steadily growing percentage of the total public spending on education. Page 70 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) 58. The project’s annual investment made up a sizable proportion of the government budget for education, so careful monitoring and continued support from external aid will be needed to ensure its financial sustainability. An annual investment of US$6.3 million per year for the education sector made up 2.8 percent of the total education budget (US$222.3 million based on most recent data in the PER, i.e. 2013) but 3.8 percent of government spending on education (US$165 million). Figure A.9: Per Capita Expenditure in Education Before (2006-08) and during the crisis (2009-13) Source: World Bank Group 2015g. Page 71 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The ICR was shared with the client. The Coordinator of the CCT: • Confirmed that the introduction of additional innovative activities focused on quality at mid-term, for which there was a strong government ownership, had a limited impact on the results chain; • Concurred with the assessment of achievement of each outcome; • Agreed with the discussions on the key factors that affected project implementation and outcome; • Agreed that the lessons learnt were incorporated in the follow-up operations; and • Confirmed that the flexible nature of the operation was crucial to its successful implementation. The Head of the Education PIU confirmed that they reviewed the ICR and did not have comments. The Head of the Nutrition PIU provided (i) further information about the gradual expansion of Component 3 (nutrition), which were incorporated in the final version of the ICR; (ii) revised dates regarding the turnover of the monitoring and evaluation specialist position in the Nutrition PIU; and (iii) editorial suggestions. These comments and suggestions were incorporated in the final version of the ICR. Page 72 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 6. SUPPORTING DOCUMENTS Adris. 2014. Controle de l'eligibilite du paiement des subventions au salaire des enseignants non- fonctionnaires dans le cadre du projet d'appui d'appui d'urgence aux services essentiels de l'education, nutrition et sante (PAUSENS). Antananarivo, Madagascar: Adris. Baird, S, Hicks, JH, Kremer, M, and Edward, M. 2015. Worms at work: the long-run impacts of a child health investment. The Quarterly Journal of Economics, vol. 131(4), pages 1637-1680. Black, R., Allen, L., Bhutta, Z., et al. 2008. Maternal and child undernutrition: global and regional exposures and health consequences. Lancet; 371: 243–60. International Labor Affairs (ILAB). 2016. Child Labor and Forced Labor Reports. Report can be found at https://www.dol.gov/agencies/ilab/resources/reports/child-labor/madagascar (visited in December 2017). Cabinet ECR. 2015. Projet d'appui d'urgence aux services essentiels d'education, de nutrition et de sante--Credit 5186-MG : Evaluation des prestataires de service de paiement dans le cadre de la composante education. Antananarivo, Madagascar: Cabinet ECR. Chatham House. 2014. Shared Responsibilities for Health a Coherent Global Framework for Health Financing. Faculte de medecine d'Antananarivo. 2017. Prevalence de la filariose lymphatique, des schistosomiases et des geohelminthiases dans les cinq regions concernees par le projet PAUSENS a Madagascar. Antananarivo, Madagascar: Faculte de medecine d'Antananarivo. Garchitorena, A., A. Miller, L. Cordier, R. Ramananjato, V. Rabeza, M. Murray, A. Cripps, L. Hall, P. Farmer, M. Rich, A. Orlan, A. Rabemampionona, G. Rakotozafy, D. Randriantsimaniry, D. Gikic, and M. Bonds. 2017. "In Madagascar, Use of Health Care Services Increased When Fees Were Removed: Lessons for Universal Health Coverage." Health Affairs 36, no 8: 1143-1451 (https://www.healthaffairs.org/doi/pdf/10.1377/hlthaff.2016.1419). Government of Madagascar. N.d. Plan National de Developpement 2015-2016. Antananarivo, Madagascar: Government of Madagascar. Government of Madagascar. 2014. Plan national de gestion des dechets medicaux a Madagascar 2014-2018. Antananarivo, Madagascar: Government of Madagascar. Graham, W., Bell, J. and Bullough, C. 2001. “Can skilled attendance at delivery reduce maternal mortality in developing countries?” pp 97-130. In: Safe Motherhood Strategies: A Review of the Evidence. Studies in Health Services Organisation & Policy, 17, 2001. Ed. W. Van Lerberghe, G. Kegels, V. De Brouwere. Antwerp: ITGPress. Institut Pasteur de Madagascar. 2015. Mise a jour des donnees de base sur la prevalence de la filariose lymphatique, des schistosomiases et des geohelminthiases dans les regions Amoron'i Mania, Page 73 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Matsiatra Ambony, Androy, Vatovavy Fitovinany et Atsimo Atsinanana. Antananarivo, Madagascar: Institut Pasteur de Madagascar. International Monetary Fund. 2017. Country report 17/225. Kakietek, J. 2016. Scaling Up Nutrition in Madagascar: Estimating cost and cost-effectiveness, of the expansion of high impact nutrition interventions in Madagascar. Presented on Nov 1, 2016. Lübker, M. 2007. Labour Shares. Geneva: Policy Brief, Policy Integration Department, International Labour Office. McGuire, JW. 2006. Basic Health Care Provision and Under-Five Mortality: A Cross-National Study of Developing Countries. World Development 34(3): 405-425. Mahomby. 2016. Controle de l'effectivite et de l'eligibilite du paiement des subventions au salaire des ENF et des SAE dans le cadre du PAUSENS. Antananarivo, Madagascar: Mahomby. Montenegro, CN, and Patrinos, HA. 2014. Comparable estimates of returns to schooling around the world. Policy Research working paper; no. WPS 7020. Washington, DC: World Bank Group. NICE International. 2014. Methods for Economic Evaluation Project: Final Report. Available from https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-International/projects/MEEP- report.pdf. Republic of Madagascar and International Development Association. 2012. Financing Agreement (Emergency Support to Critical Education, Health and Nutrition Services Project) dated December 3, 2012. World Bank Group. 2012a. Emergency Project Paper on a Proposed Credit in the Amount of SDR 42.2 million (US$65 million equivalent) to the Republic of Madagascar for an Emergency Support to Critical Education, Health and Nutrition Services Project. Washington, D.C.: World Bank Group. World Bank Group. 2012b. Core Sector Indicators and Definitions--Updated April 2012: Health. Washington, D.C.: World Bank Group. World Bank Group. 2013a. Aide-memoire--Republique de Madagascar--Mission d’appui à la mise en oeuvre du Projet d’Appui d’Urgence aux Services Essentiels d’Education, de Santé et de Nutrition (PAUSENS) (Crédit 5186-MG) du 13 au 24 mai 2013. World Bank Group. 2013b. Aide-memoire--Republique de Madagascar--Projet d’Appui d’Urgence aux Services Essentiels d’Education, de Santé et de Nutrition (PAUSENS)--Crédit 5186-MG--Mission d’appui à la mise en oeuvre--10 au 24 novembre 2013. World Bank Group. 2013c. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 1. Page 74 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) World Bank Group. 2014a. Project Paper on a Proposed Additional Credit in the Amount of SDR 6.5 million (US$10 million equivalent) to the Republic of Madagascar for an Emergency Support to Critical Education, Health and Nutrition Services Project. Washington, D.C.: World Bank Group. World Bank Group. 2014b. Aide-memoire--Republique de Madagascar--Mission d’appui à la mise en oeuvre du Projet d’Appui d’Urgence aux Services Essentiels d’Education, de Santé et de Nutrition (PAUSENS-P131945) et du Projet d'Appui d'Urgence a l'Education pour Tous (PAUET-P132616) du 10 au 28 fevrier 2014. World Bank Group. 2014c. Emergency Support to Critical Education, Health and Nutrition Services Project (PAUSENS)--Credit 5186-MG--Implementation Support Mission--May 5-25, 2014. World Bank Group. 2014d. Republique de Madagascar--Projet PAUSENS--Aide-memoire: Mission de revue a mi-parcours et d’appui à la mise en oeuvre du projet du 27 octobre au 7 novembre 2014. World Bank Group. 2014e. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 2. World Bank Group. 2014f. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 3. World Bank Group. 2014g. Implementation Completion and Results Report (IDA-41040 and IDA- 51240) - Report No: ICR00003307. World Bank Group. 2014h. Health Public Expenditure Review. World Bank Group. 2015a. Restructuring Data Sheet. Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945)--Report No: RES17364. Washington, D.C.: World Bank Group. World Bank Group. 2015b. Madagascar Systematic Country Diagnostic. Washington, D.C.: World Bank Group. World Bank Group. 2015c. Republique de Madagascar--Projet PAUSENS (P131945)--Aide-memoire-- Mission d'appui a la mise en oeuvre du projet du 15 au 31 juillet 2015. World Bank Group. 2015d. Republique de Madagascar--Projet PAUSENS (P131945)--Aide-memoire-- Mission d'appui a la mise en oeuvre du projet du 30 novembre au 12 decembre 2015. World Bank Group. 2015e. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 4. World Bank Group. 2015f. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 5. World Bank Group. 2015g. Education Public Expenditure Review. Page 75 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) World Bank Group. 2016a. Restructuring Paper on a Proposed Project Restructuring of Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project to the Republic of Madagascar--June 14, 2016. Washington, D.C.: World Bank Group. World Bank Group. 2016b. Restructuring Data Sheet. Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945)--Report No: RES24512. Washington, D.C.: World Bank Group. World Bank Group. 2016c. Republique de Madagascar--Projet PAUSENS--Aide-memoire--Mission d’appui à la mise en oeuvre du projet du 13 au 17 juin 2016. World Bank Group. 2016d. Republique de Madagascar--Projet PAUSENS--Aide-memoire--Mission d’appui à la mise en oeuvre du projet du 26 octobre au 4 novembre 2016. World Bank Group. 2016e. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 7. World Bank Group. 2016f. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 6. World Bank Group. 2017a. Country Partnership Framework for the Republic of Madagascar for the period FY17-FY21. Washington, D.C.: World Bank Group. World Bank Group. 2017b. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 9. World Bank Group. 2017c. Madagascar--Aide-Memoire--Mission de Preparation du Programme de Prevention de Retard de Croissance (P160848) et de Preparation de la Cloture du Financement Additionnel du Projet d'Appui d'Urgence aux Services Essentiels de l'Education, de Sante et de Nutrition (P131945) du 29 mai au 15 juin 2017. World Bank Group. 2017d. Prestation de services d’éducation à Madagascar : Résultats de l’enquête sur les indicateurs de prestation de service 2016. Washington, D.C.: World Bank Group. World Bank Group. 2017e. Implementation Status & Results Report: Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project (P131945) - Seq No 8. Page 76 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) ANNEX 7. SUMMARY OF CLIENT’S IMPLEMENTATION COMPLETION & RESULTS REPORTS (ICRs) Each PIU recruited a consultant to prepare the ICR by component. The three ICRs were then summarized by the CCP. There are several caveats regarding the results framework of this summary: • It was prepared before the closing date of the Additional Financing and, as such, does not take into account the nutrition-related results achieved during 2017; • Even the data for components 2 and 3, which closed in December 2016, was collected during the last year of implementation—the author inaccurately indicated December 2016 while the sector ICRs mentioned accurate dates during 2016 for data collection; and • The author did not reflect accurately the data provided in the Health ICR, which had been agreed upon with the Bank team during the mission held in October-November 2016 and reflected in the mission aide- memoire and the subsequent Implementation Status & Results Report (Government of Madagascar 2017, World Bank Group 2016d, World Bank Group 2017e). Page 77 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 78 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 79 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 80 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 81 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 82 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 83 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 84 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 85 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 86 of 87 The World Bank Madagascar Emergency Support to Critical Education, Health and Nutrition Services Project ( P131945 ) Page 87 of 87