Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004308 IMPLEMENTATION COMPLETION AND RESULTS REPORT ON GRANTS IN THE AMOUNT OF SDR 68.1 MILLION (US$84.8 MILLION EQUIVALENT) TO THE REPUBLIC OF BURUNDI FOR A HEALTH SECTOR DEVELOPMENT SUPPORT ( P101160 ) February 21, 2018 Health, Nutrition & Population Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective October 30, 2017) Currency Unit = FBU FBU 1,753 =US$1 US$ 1.40 = SDR 1 FISCAL YEAR Jan 1 - Dec 30 Regional Vice President: Makhtar Diop Country Director: Bella Deborah Mary Bird Senior Global Practice Director: Timothy Grant Evans Practice Manager: Magnus Lindelow Task Team Leader(s): Laurence Elisabeth Marie-Paule Lannes ICR Main Contributor: Aly B. Sy, Laurence Lannes ABBREVIATIONS AND ACRONYMS ABC Community-Based Batwa Association AF2 Second Additional Financing ASC Community Health Agent BCAI Bureau de coordination des Aides Internationales BCR Benefit Cost Ratio BDS District Health Office BPS Provincial Health Office CABINET Minister’s Office CAM Medical Assistance Card CAS Country Assistance Strategy CBA Cost-benefit analysis CHSP Community Health Strategic Plan COGE Management Committee COSA Health Committee CPF Country Partnership Framework CT-FBP Technical Cell-PBF CPVV Provincial Verification and Validation Committee DBA Budget and Procurement Directorate DGR General directorate of Resources DGP General Planning Directorate DGS General directorate of Health Services DGS General directorate of Health Services and Fight Against AIDS DISE Health Infrastructure and Equipment Directorate DODS Health Care Supply and Demand Directorate DPSE Planning and Monitoring & Evaluation Directorate DPPS Health Programs and Projects Directorate DPSHA Directorate of Health Promotion, Hygiene and Sanitation DRH Human Resources Directorate DSNIS National Health Information System Directorate FM Financial Management GASC Group of Community Health Agents GAVI The Global Alliance for Vaccines and Immunization, also known as the GAVI Alliance GDP Gross Domestic Product HD District Hospital HN National Hospital ICR Implementation Completion and Results Report IDA International Development Association IEC Information, Education and Communication Service IGSSLS General Inspection for Health Services and for the Fight against AIDS IPP Indigenous Peoples Plan IRR Internal Rate of Return KIRA Burundi Health System Support Project MBB Marginal Budgeting for Bottlenecks MCH Maternal and Child Health M&E Monitoring and Evaluation MSPLS Ministry of Public Health and Fight Against AIDS MTN National integrated Program for the fight against neglected tropical diseases and blindness NGO Non-Governmental Organization NPV Net Present Value PAD Project Appraisal Document PADSS Health Sector Development Support Project PBF Performance-Based Financing PDO Project Development Objective PEV Expanded Immunization Program PIU Project Implementation Unit PNC Prenatal Care PNDS National Health Sector Development Plan PNILP National Integrated Program for the Fight against Malaria PNILMCNT National Deficiency-related and Non-Communicable Diseases Integrated Program PNLS Programme National de Lutte contre le SIDA POLNC National Contractualisation Policy (i.e. Politique Nationale de Contractualisation) PRSP Poverty Reduction Strategy Paper QER Quality Enhancement Review SARA Service Availability and Readiness Assessment SDR Special Drawing Rights SP Permanent Secretary TA Technical Assistance TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 11 II. OUTCOME .................................................................................................................... 14 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 15 C. EFFICIENCY ........................................................................................................................... 18 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION ................................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 23 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 27 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 30 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 33 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 42 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 44 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 45 ANNEX 5. BORROWER COMMENTS ...................................................................................... 57 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 71 ANNEX 7: REVISED RATINGS OF PROJECT PERFORMANCE IN ISRS ......................................... 73 ANNEX 8: ACHIEVEMENT RATE FOR SELECTED INDICATORS ON THE BATWA POPULATION ... 74 ANNEX 9. OVERVIEW OF BURUNDI PERFORMANCE-BASED FINANCING ................................ 75 The World Bank Health Sector Development Support ( P101160 ) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P101160 HEALTH SECTOR DEVELOPMENT SUPPORT ( P101160 ) Country Financing Instrument Burundi Specific Investment Loan Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Additional Financing P131919-Second 23-Oct-2012 IBRD/IDA Additional Financing Burundi Health Sector Development Support Project Supplement P126742-Health Sector 21-Jun-2012 Recipient Executed Activities Development Support - Additional Financing Organizations Borrower Implementing Agency Ministry of Finance Ministere de la Sante Publique et de Lutte contre le Sida Project Development Objective (PDO) Original PDO To increase the use of a defined package of health services by pregnant women and children under the age of five. 1 The World Bank Health Sector Development Support ( P101160 ) Revised PDO To increase the use of a defined package of health services by pregnant women, children under the age of five and couplesofreproductive age. FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing P101160 IDA-H4880 25,000,000 24,985,236 26,064,542 P101160 TF-12526 14,800,000 14,787,286 14,787,286 P101160 IDA-H8080 25,000,000 25,000,000 22,986,161 P101160 TF-13043 20,000,000 20,000,000 20,000,000 Total 84,800,000 84,772,522 83,837,989 Non-World Bank Financing Borrower 0 0 0 Total 0 0 0 Total Project Cost 84,800,000 84,772,522 83,837,988 KEY DATES Project Approval Effectiveness MTR Review Original Closing Actual Closing P101160 09-Jun-2009 30-Sep-2009 01-Dec-2014 30-Dec-2012 30-Jun-2017 P126742 21-Jun-2012 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 30-Jan-2015 29.98 Reallocation between Disbursement Categories 26-Mar-2015 29.98 Change in Loan Closing Date(s) Reallocation between Disbursement Categories 02-Feb-2017 45.19 Reallocation between Disbursement Categories 2 The World Bank Health Sector Development Support ( P101160 ) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 22-Oct-2009 Satisfactory Satisfactory .46 02 07-Dec-2009 Satisfactory Satisfactory .46 03 18-Jun-2010 Satisfactory Moderately Satisfactory .85 04 12-Mar-2011 Satisfactory Moderately Satisfactory 6.60 05 12-Oct-2011 Highly Satisfactory Moderately Satisfactory 9.43 06 03-Mar-2012 Highly Satisfactory Satisfactory 12.83 07 28-Oct-2012 Highly Satisfactory Satisfactory 14.83 08 25-May-2013 Satisfactory Satisfactory 24.09 09 05-Oct-2013 Highly Satisfactory Satisfactory 24.29 10 26-Mar-2014 Highly Satisfactory Moderately Satisfactory 26.50 11 08-Nov-2014 Satisfactory Moderately Satisfactory 27.68 12 12-May-2015 Satisfactory Moderately Satisfactory 31.22 13 20-Nov-2015 Satisfactory Moderately Satisfactory 34.05 14 12-May-2016 Satisfactory Moderately Satisfactory 36.66 15 11-Nov-2016 Satisfactory Moderately Satisfactory 40.11 16 30-Jun-2017 Satisfactory Satisfactory 48.57 3 The World Bank Health Sector Development Support ( P101160 ) SECTORS AND THEMES Sectors Major Sector/Sector (%) Health 100 Public Administration - Health 21 Health 79 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 8 Public Finance Management 4 Public Expenditure Management 4 Public Administration 4 Transparency, Accountability and Good 4 Governance Human Development and Gender 92 Disease Control 21 Malaria 21 Health Systems and Policies 71 Health System Strengthening 29 Reproductive and Maternal Health 21 Child Health 21 ADM STAFF Role At Approval At ICR Regional Vice President: Obiageli Katryn Ezekwesili Makhtar Diop Country Director: Bella Deborah Mary Bird Senior Global Practice Director: Timothy Grant Evans Practice Manager: Lynne D. Sherburne-Benz Magnus Lindelow Laurence Elisabeth Marie-Paule Task Team Leader(s): Montserrat Meiro-Lorenzo Lannes 4 The World Bank Health Sector Development Support ( P101160 ) ICR Contributing Author: Aly B. Sy 5 The World Bank Health Sector Development Support ( P101160 ) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. The project was prepared during a period of recovery from 10 years of open conflict, almost three years of economic embargo and a four-year post-emergency period. The 2000 Arusha peace agreement brought some stability and recovery. The subsequent ceasefire agreements and the 2005 elections continued this process, but the overall social and political situation remained fragile, despite the economic progress achieved. In 2009, Burundi had one of the lowest GDP per capita (190.4 current US$ in 2009) in the world and ranked 174th out of 182 countries on the UN Human Development Index. At the same time, Burundi was experiencing rapid population growth of 3.3 % with a population of 8.2 million people in 2009 and one of the highest fertility rates in the world (6.3 children per women). 2. At the time of appraisal, Burundi was making the transition from a post-conflict to a stable and growing economy. The GDP grew by 4-5 % since 2004, the deficit was reduced from 5 % to 3.4 % over just three years (2004 and 2006), and government’s execution rates increased significantly from 79.2 % to 98.3 %. Some of these improvements were linked to successful reforms made in public financial management through external budget support. Total spending for health experienced a significant increase between 2000 and 2013, mainly driven by a 10-fold increase in constant 2015 USD in external assistance. However, the per capita spending on health remained around US$ 30. 3. Effects of the conflict included a substantial shortage of qualified personnel in all fields, as well as weak institutions and poor health outcomes as a result of low social service coverage levels well below those in countries with similar income level. In 2010 mortality levels were extremely high, with a maternal mortality ratio at 824 deaths per 100,000 live births, under five mortality at 98.7 per 1,000 live births and chronic malnutrition rampant (58% among children under five)1. The main reasons for poor health outcomes were underuse of health services as well as scarce, poorly distributed and unmotivated human resources for health staff. 4. In 2006, the President of the Republic declared a policy of free health care (FHC) for pregnant women and children less than five years of age. The selective FHC policy has had some notable successes such as increasing the portion of women delivering in health facilities from 22.9% in 2005 to 56.3% in 2008 but funding was inadequate and reimbursement of health facilities experienced significant delays. Performance-Based Financing (PBF) appeared as a complementary strategy to ensure resources reached health facilities to cover the cost of the FHC policy. In 2006, PBF pilot programs were introduced in two provinces (Bubanza and Cankuzo) by the international Non-Governmental Organization (NGO) Cordaid with financing from the Dutch government and the European Union. A second NGO, Health Net International-TPO and the Swiss Development Cooperation started pilot projects in Gitega and Ngozi provinces. Due to positive results and funding by the European Union, these pilot experiments were extended to six new Provinces. By December 2009, 9 out of 17 provinces had ongoing 1 World Development Indicators. 6 The World Bank Health Sector Development Support ( P101160 ) pilot PBF projects. An evaluation in 2009 showed better performance in the provinces under PBF, hence the Ministry of Health and Fight against HIV/AIDS (MSPLS)and development partners decided to scale up PBF from 2010 onwards. 5. In April 2010, the national PBF system was launched, a provider payment reform initiative which merged selective FHC with PBF. Development partners and the Government of Burundi (GoB) signed a memorandum of understanding to support the two strategies and ensure alignment of development partners to the national PBF system. The MSPLS requested development partners to convert their programming into output based financing mechanisms. For instance, a large classical health strengthening program, along input lines, financed by the European union was converted into PBF. The Health Sector Development Support Project (PADSS) was therefore also designed and implemented in a way to be fully aligned to the GoB’s strategy and to support its efforts to implement FHC and PBF in the country. Annex 9 provides further details on the FHC/PBF reform and PBF mechanism (payment, verification, adaptation to change, etc.). 6. The combined FHC/PBF program is financed by different actors but the largest contributor is the Government of Burundi through its national budget. In all health policy documents (National Health Development Plans 2006–10 and 2011–15 and National Health Policy 2016–22), the FHC-RBF is a high priority for the government and has been at the center of the MSP’s implementation priorities. Therefore, even during the peak of the political crisis, the Government has maintained its financial commitment. Two other important contributors are IDA (World Bank) and the European Union. The program has also attracted several other technical and financial partners, each focusing on a subset of the country’s provinces and/or on selected indicators only. They include the Belgian Cooperation, the Dutch Cooperation through CORDAID and HealthNet TPO, Global Alliance for Vaccines and Immunization (GAVI), and the United States Agency for International Development (USAID) through the NGO FHI360. 7. The Health Sector Development Support Project (PADSS) was successful and achieved more than 99% disbursement rate. The project was nationwide in scope and both development partners and the Government contributed financially to FHC/PBF. The GoB was the main contributor (44% of total program costs in 2011, followed by the World Bank (WB) with 42% and other partners with 14%). The initial funding of US$ 25 million (with a closing date in December 2012, extended to December 2014) was complemented by two Additional Financings in 2012 (from IDA and the Health Results Innovation Trust Fund), bringing the total project amount to US$ 84.8 million allowing the project to reach higher objectives (through revised and more ambitious targets) as well as new activities to reflect changes in the FHC/PBF policy. The Additional Financings were justified by the impressive results of the Health Sector Development Support project, its highly satisfactory rating, and the need to expand the scope and duration of the program to help address two of the country’s key developm ent challenges: fertility rates and child malnutrition rates that were among the world’s highest. The project closing date was first extended to December 2018 to accommodate for the two additional financings, but then revised to June 2017 as performance of the project was higher than expected. 8. Three external shocks affected the FHC/PBF program during implementation of PADSS and demonstrated the health sector’s resilience: the civil unrest between May 2015 and December 2015 which hampered access to health care services; the role of religious leaders in the slow uptake of contraceptives; and the malaria outbreak in 2016-2017 which resulted in high mortality among pregnant women due to poor access to drugs. Nevertheless, 7 The World Bank Health Sector Development Support ( P101160 ) health outcomes, which dropped during the crisis, quickly recovered, showing the resilience of the health system. Resilience was indeed part of the PBF design: health facilities were obliged to have at least two months’ worth of operating costs as reserves in the Bank and they were held accountable for at least 2 months’ stock of tracer drugs. 9. The 2015 political crisis significantly affected the macroeconomic-, fiscal-, and health system performance, among others. Government’s revenue decreased significantly as consequence of the decline in GDP (by 3.9 % in 2015 and 0.6 % in 2016) and in external aid (from 13 % of GDP to 9.4 %), while current expenditure increased due to the heightened need for security in the country. Reduced revenue and increased expenditure resulted in increased domestic borrowing and fiscal deficit (up from 3.1% to 6.6 % of GDP). One of the features of PADSS, which required the GoB to contribute to the program a minimum of 1.4 % of its general state budget, ensure GOB’s commitment to support this program and reinforced its sustainability, even during the crisis. It is evident that continued external support will be needed to not jeopardize the significant improvements made under the FHC/PBF in MCH. The Bank therefore decided to support a “second generation of FHC/PBF” (KIRA). This also raises the need to reflect on modalities and right timing for external funding to phase out of the program. 10. Project design links to higher level objectives: At appraisal, the project was aligned with the 2006-2011 Poverty Reduction Strategy Paper (PRSP). Human development was identified as a key pillar to pull the country out of poverty in the PRSP. It fully supported the FHC/PBF strategy of the GoB and supported achievement of the targets of the 2006-2010 National Health Sector Development Plan (PNDS): i) reducing maternal and neonatal mortality; ii) reducing infant and juvenile mortality; iii) reducing the prevalence of communicable and non- communicable diseases; and, iv) strengthening the national health system performance. 11. Rationale for Bank engagement: The Health Sector Development Support Project (PADSS) was complementing two projects: The Second Multisectoral HIV/AIDS Control Project and the Community and Social Development Project. The rationale for World Bank engagement was premised on its flexibility in financing, its convening power to bring together most stakeholders around a common vision in line with the PNDS, and its support to broader public financial management reforms. Theory of Change (Results Chain) 12. The theory of change presented in Figure 1 illustrates how the Project’s activities and their attached outputs aimed to create an enabling environment leading to the expected objectives/outcomes. Three activities were identified by the project to achieve the PDO: i) the transfer of funds to health facilities to pay for the provision of a defined package of free health services; ii) the transfer of resources to build the capacity of community health actors, and; iii) capacity building of the MSPLS and entities involved in the PBF. The first activity relates to PBF and provision of incentives on both supply and demand side after verification of data submitted by healthcare facilities. This verification mechanism allows to ensure that outputs are being delivered and the path to PDO has been followed. 13. Outputs and outcomes: The first output is the provision of quality free healthcare services which are paid after services have been delivered. The expected related outcome is the increase in the use of a defined package of free health services by pregnant women and children under the age of five. The second output is the provision 8 The World Bank Health Sector Development Support ( P101160 ) of training to facilitating agencies. Linking this output to the PDO assumes that training activities will lead to better awareness of providers and beneficiaries, which will translate into increased use of healthcare services by pregnant women and children under five. Similarly, the third activity aiming to build the capacity of the MSPLS and entities involved in the PBF through: i) provision of training to officials; ii) provision of Technical Assistance (TA) for improved health systems; and iii) provision of training and TA for PBF strengthening assumes that training TA would strengthen the health system for the delivery of health services, thus leading to increased use of free health services by pregnant women and children under the age of five. 14. Impact: Through increased access to maternal and child health services, the project is expected to contribute to the reduction in maternal mortality and under five mortality. This was in line with the GoB’s policy objectives and broader international commitments of the Millennium Development Goals. Figure 1: Theory of change at appraisal • Activity 1A: Transfer of resources to health facilities to pay for the Free package of Health Services (FPS) on the basis of RBF • Activity 1B: Funding for capacity building of community health actors Activities • Activity 2: Transfer to MSPLS and entities involved in the RBF • 1A: Provision of Quality Free health care services • 1B: Provision of Training; Technical Assistance Outputs • 2: Training of Officials; Technical assistance for improved systems; Training and Technical Assistance for RBF strengthening • Increased use of a defined package of health services by pregnant women Expected • Increased use of a defined package of health services by children under the age of 5 outcomes • Reduced maternal mortality ratio Expected • Reduced under five mortality rate project impact Project Development Objectives (PDOs) 15. The objective of the project at appraisal was to “increase the use of a defined package of health services by pregnant women and children under the age of five”. Key Expected Outcomes and Outcome Indicators 16. The above compound PDO statement with multiple outcomes was “unpacked” to assess separately each promised outcome (following recommendations of the ICR revised guidelines). For each population group targeted (namely pregnant women and children under five) key PDO indicators to be used to assess the achievement are presented in Table 1. 9 The World Bank Health Sector Development Support ( P101160 ) Table 1: Key Expected Outcomes and Outcome Indicators Outcome Outcome Indicator Increase the use of a Coverage of prenatal care (PNC) services for pregnant women defined package of Contraceptive prevalence rate (modern methods) health services by Percentage of births assisted by skilled personnel pregnant women Increase the use of a Average number of visits to a health provider by children under the defined package of age of 5 health services by Coverage of DPT3/Pentavalent vaccine in children immunized children under the before reaching the age of 1 age of five Percentage of children under 5 who slept under an insecticide treated bed net Components Initially, the Project had two main components: 17. Component 1: Increased financing for a redefined free package of services ($20 million). At closing and due to several additional financings, the total component cost was $64.5 million. This component represented 80 % of the grant and aimed to stimulate the supply and demand of cost-effective services targeting pregnant women and children under the age of five, through two subcomponents: i. Transfer of resources to health facilities to pay for the Free Package of Health Services (FPS) on the basis of PBF ($18.5 million). Health facilities were reimbursed quarterly for delivering selected health services to pregnant women and children under five, using standard prices at health center and district hospital level. Five different unit fees were provided to account for remoteness and underperformance. Performance payments were also provided to managerial units involved in verification of PBF claims. ii. Building the capacity of community health actors ($1.5 million). This sub-component aimed at boosting the demand for health services by enlisting strengthened community health actors who would organize themselves and sign contracts with health facilities to increase demand for health services. 18. Component 2: Strengthening the capacity of the MSPLS and entities involved in PBF ($5 million). The total component cost at closing amounted to $20.1 million. The objective was to build and reinforce capacities of MSPLS and other entities involved in PBF implementation and Monitoring and Evaluation (M&E), procurement, resource management and strategic purchasing. It also aimed to facilitate running of the national PBF system, including capital and training costs for provincial health offices and the creation of a technical unit to support PBF policies. The component also supported costs of external verification. 10 The World Bank Health Sector Development Support ( P101160 ) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 19. In December 20122, the Project Development Objective was revised to add couples of reproductive age as a third group of beneficiaries. The project objective changed to “Increase the use of a defined package of health services by pregnant women and children under the age of five and couples of reproductive age” (Figure 2). This reflected the greater emphasis put on family planning services in the FHC/PBF program which resulted in higher fees being paid for family planning services to address the high fertility rate in the country. At the same time, and to address the challenge of malnutrition and provide appropriate care for malnourished children, the MSP, with support from the World Bank, also decided to introduce nutrition-related indicators into existing PBF. The objective was to strengthen the national strategy for the integration of nutritional services and make it effective in all regions of Burundi. The PBF Nutrition interventions, both preventive and curative, were added at hospital, health center and community levels, with a focus on children under 5 years. The Project benefited from several increases in financing to cope with the expanded scope of the Project and increases in coverage targets. The additional financing and changes in Project scope strengthened the theory of change rather than dilute it by focusing on essential interventions that were necessary to achieve the stipulated outcomes and expected impact. The interventions were scaled-up progressively to allow evaluation using a rigorous impact evaluation design. Figure 2: Revised theory of change • Activity 1A: Transfer of resources to health facilities to pay for the Free package of Health Services (FPS) on the basis of RBF • Activity 1B: Funding for capacity building of community health actors Activities • Activity 2: Transfer to MSPLS and entities involved in the RBF • 1A: Provision of Quality Free health care services • 1B: Provision of Training; Technical Assistance Outputs • 2: Training of Officials; Technical assistance for improved systems; Training and Technical Assistance for RBF strengthening • Increased use of a defined package of health services by pregnant women • Increased use of a defined package of health services by children under the age of 5 Expected • Increased use of a defined package of health services by couples of reproductive age outcomes • Reduced maternal mortality ratio Expected • Reduced under five mortality rate project impact 2This revision was approved on December 10, 2012 during the Second Additional Financing (AF2) funded through a grant (SDR 16.5 million) and a Trust Fund (US$20 million). 11 The World Bank Health Sector Development Support ( P101160 ) Revised PDO Indicators 20. The change in PDO led to a revision of PDO and intermediate level indicators. PDO indicators were changed twice, although the PDO was revised only once (Table 2). Table 2: Revised PDO indicators Date when the Date when Date when the Target indicator has been the indicator indicator has change added has been been reinstated dropped Outcome Outcome Indicator Number of Pregnant women living Apr-10 2894 (2012) with HIV who received antiretrovirals 3600 (2017) to reduce the risk of mother to child transmission Increase the use of a defined Coverage of prenatal care (PNC) Apr-10 package of health services by services for pregnant women (at least pregnant women 3 visits) Percentage of births assisted by Apr-12 68 (2012) - skilled personnel 86.2(2017) Pregnant women receiving antenatal Dec-12 care during a visit to a health provider (number) Female Beneficiaries Mar-14 Increase the use of a defined Average number of visits to a health Apr-10 2.2(2012)- package of health services by provider by children under the age of 2.3(2017) children under the age of five 5 Children between the age of 6 and 59 Mar-14 (AF2) months receiving Vitamin A supplementation (number) Children under age five treated for Mar-14 (AF2) moderate or severe acute malnutrition (number) Children immunized under 12 months Apr-10 against DTP3 (number) Children fully immunized Dec-12 Percentage of children under 5 who Apr-10 Oct-12 slept under an insecticide treated bed net Increase the use of a defined Contraceptive Prevalence Rate Apr-10 Dec-12 Mar-2014 22(2012) package of health services by (modern methods) -34(2017) couples of reproductive ages Number of couple years of Mar-14 (AF2). contraceptive prevalence protection Intermediate using modern methods” which was Outcome Indicator added in AF-2 12 The World Bank Health Sector Development Support ( P101160 ) Revised Components 21. The two components remained mostly unchanged despite the 2 Additional Financings. The bulk of the AF- 1 and 2 was provided for RBF payments under component 1 (sub-component 1A). The only significant change was triggered by the December 2012 Additional Financing in which RBF payments for additional contraceptive and nutrition services to health facilities ($4 million) were introduced under sub-component 1A. In addition, under component 2, enhanced institutional support ($350,000) for the provision of RBF services was included. This was mainly to provide financing of selected critical goods (nutritional products and basic medical equipment) for selected health facilities that needed to be centrally procured. Also, for the first time under AF-2, financing was given for small scale low-cost incinerators to be provided to hospitals (Table 3). Table 3: Allocations Under Original Project, First AF and Second AF, and Activities Financed Original Project (IDA First AF (HRITF Second AF only, US$ 25 million only, US$ 14.8 New IDA Grant, US$25 New HRITF Grant, equivalent) million) million eq. US$20 million Subcomponent RBF payments to health RBF payments to RBF payments to health RBF payments to 1A: Ongoing facilities for the ongoing health facilities facilities for the ongoing health facilities activities basic package of services for the ongoing basic package of services for the ongoing and for selected basic package of and for selected basic package of supervisory units services (US$11.9 supervisory units services involved in the RBF million eq.) involved in the RBF (US$11.65 million program (US$18.5 million program (US$18.5 million eq.) eq.) eq.) Subcomponent RBF payments to health RBF (US$2.5 1A: New facilities for additional million eq.) activities services (US$1.5 million eq.) Subcomponent Capacity building for No financing Capacity building for No financing 1B: Ongoing community health actors, provided community health actors, provided activities etc. (US$ 1.5 million eq.) etc. (US$ 1.5 million eq.) Component 2: Verification activities Verification Verification activities Verification Ongoing Institutional activities (US$ 2.9 Institutional activities (US$ activities strengthening, cap. million) strengthening, cap. 5.85 million) Building, etc. (US$ 5 Building, etc. (US$ 3.15 million eq.) million eq.) Component 2: Enhanced support for No financing New activities service provision (US$ provided 0.35 million eq.) Other Changes 22. The main changes relate to the size of the envelope of the project, the allocation of funds among different components or categories, the inclusion of nutrition in the package of services, the increased emphasis on family planning services and the list of indicators and the closing date. The initial funding of US$ 25 million was increased through two Additional Financing bringing the total amount of the project to US$ 84.8 million. The main project restructuring occurred following the mid-term review of December 1, 2014. The closing date of the project which had been extended to December 31, 2018 as part of the second Additional Financing was advanced 13 The World Bank Health Sector Development Support ( P101160 ) to June 30, 2017 due to the high pace of disbursements under the project which resulted from a high uptake in services, combined with a reduction in contributions to the national FHC/PBF program by partners and by Government, the latter due to an overall reduced budget impacting on the size of its counterpart allocation. There was a reallocation as well to take into account the reduced period of the project. There were two other restructurings to reallocate funds between categories: on August 13, 2013 and on February 7, 2017 in which there was also a revision of the cofinancing percentages of the IDA grant (H-8080) and the HRITF grant (TF013043) to take into account the exchange rate variations of the SDR against the US$. The appreciation of SDR for the initial grant led to an increase in project funds of $862,760. Rationale for Changes and Their Implication on the Original Theory of Change 23. Changes were informed by new evidence generated by surveys and routine data from the Health Management Information System (HMIS). They were also driven by the new focus on nutrition and family planning. Proposed changes strengthen the rationale of the theory of change by expanding the scope of interventions that could positively impact the expected outcomes. The project became overall more ambitious with the added indicators for nutrition and family planning; for this reason, even though the project funding increased and the PDO was changed, a split rating was not done to evaluate the Project outcome. Instead, the project was assessed based on revised outcomes and outcome targets. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 24. At appraisal, the project was aligned with the Bank’s 2008 Country Assistance Strategy (CAS) and with the government’s priorities set out in the Burundi PRSP. During the project, the Bank’s implementation support was responsive to changing needs and the operation remained important to achieving the WB’s and country’s development objectives. The 2015 crisis did not jeopardize the relevance of the PDO as the need for basic health services became even more acute. 25. The relevance of PDO is rated High. The Country Partnership Framework (CPF) 2018-2022 under preparation3 highlights challenges and opportunities linked to the health sector and will maintain a strong focus on health. The PDO is in line with Result Area 2 (Improving Access to and quality of social services) of the second pillar of the CAS 2013-2016 on Increasing Resilience by Consolidating Social Stability. The CAS states that the Bank “will continue to provide support for results-based financing in the health sector with a view to strengthening the health system to improve health outcomes for pregnant women and children under five and improving the delivery of reproductive health services essential to reducing the fertility rate”. The performance 3Consultations with stakeholders have already pinpointed challenges in the health sector such as the “unequal access to and distribution of services, insufficient qualified medical staff, no initiatives to combat malnutrition, lack of essential medicines and cholera/malaria epidemics”. Opportunities, such as the existence of free care for children under five and pregnant women and of the PBF have also been highlighted in the sector. 14 The World Bank Health Sector Development Support ( P101160 ) and learning review of the CAS (2015) also encouraged “continued support to health sector programs that have already led to encouraging results in reducing maternal mortality. The PDO is also highly relevant to the policy on free care and PBF4 put forward by the GoB. The PDO focused on increasing the quantity of services which was relevant at appraisal given the poor coverage of basic health services. With the poor quality of services in the country and the still high mortality levels despite increased coverage, one may have also considered to include quality of healthcare services in the PDO. The PDO of the follow-up project (KIRA) addresses this concern as the project aims to increase the use of quality services. B. ACHIEVEMENT OF PDOs (EFFICACY) 26. The assessment of achievement of PDOs is organized around each outcome indicated in the PDO statement. The accelerated decline in the under-five mortality rate and the maternal mortality ratio in 2010 (beginning of the project) and steady decrease throughout the duration of the project suggests the positive impact of FHC and PBF on health outcomes and supports the soundness of the project’s theory of change (Figure 3). Overall large improvements were observed over time in coverage of basic health services and quality of care (Figure 4 and Figure 5). The below paragraphs present for each outcome, the evidence of achievement. Detailed progress in the achievement of PDOs is also summarized in Table 4. Figure 3: Trend in under 5 mortality rate and maternal mortality ratio before and after the project 1400 1200 1000 Mortality 800 600 400 200 0 1990 1995 2000 2005 2008 2010 2013 2015 2016 Mortality rate, under five 171 166 160 155 152 59 53 49 47 Maternal Mortality ratio 1220 1122 1023 925 866 500 446 410 392 27. PDO 1 (Increase use of a defined package of health services by pregnant women) achievement is rated Highly Satisfactory: all 5 indicators surpassed or almost met (minimum of 94%) their targets leading to an average achievement rate of 100.8%5. The indicators focus on pregnant women and are closely linked to the achievement of the PDO. iii. Number of pregnant women living with HIV who received antiretrovirals to reduce the risk of Mother to Child transmission (Number, Custom). This indicator has exceeded its target (139.61%) (2016 PNLS Report). 4 See the Government of Burundi’s Health Policy 2016-25. 5 Assuming all indicators have the same weight. 15 The World Bank Health Sector Development Support ( P101160 ) iv. Coverage of prenatal care services for pregnant women (at least 3 visits) (Percentage, Custom). Preliminary data for 2016 shows a drop in this indicator from 66.5 % in 2015 down to 62%, bringing the achievement rate to 94.66%. Thus, the target has almost been met. v. Percentage of births assisted by skilled personnel (Percentage, Custom). At the end of the project, 85.1% of births were assisted by skilled personnel; the target has almost been met (98.72%) (2016-17 DHS). vi. Pregnant women receiving antenatal care during a visit to a health provider (Number, Custom). The target was almost met (95.21%). vii. Female beneficiaries. The target was met (100.08%). It was estimated based on the assumption that pregnant women represent 5% of the total population. 28. PDO 2 (Increase use of a defined package of health services by children under the age of five) achievement is rated Satisfactory: 3 out 4 indicators almost met their targets (95%); the average achievement rate is 90.96%. All four indicators target children under five and contribute to the achievement of the PDO. viii. Average number of visits to a health provider by children under the age of 5 (Number, Custom). Target almost met (95.65%) with 2.2 visits at the end of the project compared to 1.68 at baseline. ix. Children immunized (Number, Custom). Target almost met (98.78%). x. Children between the age of 6 and 59 months receiving Vitamin A supplementation (Number, Custom). Target almost met for this nutrition indicator (99.23%). xi. Children under age five treated for moderate or severe acute malnutrition (Number, Custom). Not achieved. Number of children treated at achievement was below that of the baseline. The target was not met. Figure 4 : Assisted deliveries and visits for children (2011-2016) Figure 5 : Improvements in quality (2010-2012) 100 6 83 76.5 78.7 5.2 5 80 72.9 64.4 68 4.6 4.3 4.1 3.8 4 60 3 40 2 20 1 0.6 0 0 2011 2012 2013 2014 2015 2016 Assisted deliveries (%) (left) Average number of visits to a health provider(U5) (right) 29. PDO 3 (Increase use of a defined package of health services by couples of reproductive age) is rated Satisfactory. xii. Contraceptive prevalence rate. Data on contraceptive prevalence rate vary in Burundi depending on the sources. Although the target is met if one uses HMIS (121.95%), preliminary data from DHS 2016-17 report a contraceptive prevalence rate of 23% compared to 19.8% at baseline (66% achievement rate). 16 The World Bank Health Sector Development Support ( P101160 ) The drop in contraceptive prevalence in recent years is probably due to the electoral process and political crisis. xiii. Number of couple-years of contraceptive protection using modern methods. This is the only intermediate indicator linked to this PDO. Its achievement rate was 75.7% (2016 Health Statistical Yearbook); although the target was not met, significant progress was achieved (doubling from baseline). Table 4 : Achievement of PDOs (outcome indicators) Baseline Endline Target Increase Endline (2009) (actual) (C) (B-A)/A vs target (A) (B) (B-C)/C Outcome Outcome Indicator Number of pregnant 1582 5026 3000 218% 68% women living with HIV who received antiretrovirals to reduce the risk of mother to child transmission Increase the Coverage of prenatal 59.4 66.5 65.5 12% 2% use of a care (PNC) services for defined pregnant women (at package of least 3 visits) health Percentage of births 64.4 85.1 86.2 22% -1% services by assisted by skilled pregnant personnel women Pregnant women 415799 529662 549564 27% -4% receiving antenatal care during a visit to a health provider (number) Female Beneficiaries 0 1394317 1393176 - 0.1% Increase the Average number of 1.68 2.2 2.3 31% -4% use of a visits to a health defined provider by children package of under the age of 5 health Children between the 1265275 1651266 1665878 31% -1% services by age of 6 and 59 children months receiving under the Vitamin A age of five supplementation (number) Children under age 87170 58307 83081 -33% - five treated for 30% moderate or severe acute malnutrition (number) Children immunized 297780 322560 341030 8% -5% under 12 months 17 The World Bank Health Sector Development Support ( P101160 ) against DTP3 (number) Children fully 297780 355551 359930 10% -1% immunized Increase the Contraceptive 19.8 37.46 34.85 89% 7% use of a Prevalence Rate defined (modern methods) package of health Number of couple- 228082 555702 734520 144% - services by years of contraceptive 24% couples of protection using reproductive modern methods age Justification of Overall Efficacy Rating 30. The achievement of PDO1 is rated Highly Satisfactory and that of PDO2 and PDO3 Satisfactory. Overall, 9 out of 10 PDO indicators were met (or nearly) and 4 out of 5 intermediate results indicators (IO) were met (or nearly) (Table 5). Only one intermediate outcome indicator (Number of couple-years of contraceptive protection using modern methods) was not achieved (75.66% achievement). The overall achievement rate of indicators is 87%. Thus, the overall efficacy of the project is rated Substantial (Table 6). Table 5: Achievement of PDO and intermediate results indicators Met Almost met Not met Total PDO1 2 3 0 5 PDO2 0 3 1 4 PDO3 1 0 0 1 TOTAL PDO 3 6 1 10 TOTAL IO 2 2 1 5 GRAND TOTAL 5 8 2 15 (PDO+IO) Table 6: Overall Efficacy rating PDO1 PDO2 PDO3 Overall Efficacy rating Highly satisfactory Satisfactory Satisfactory Substantial C. EFFICIENCY Economic analysis 31. Comparison to cost benefit analysis (CBA) at appraisal: The CBA conducted at appraisal used the Marginal Budgeting for Bottlenecks tool to translate the expected increase in health service utilization into lives saved and 6 Preliminary DHS data for 2016-17 indicate a drop in contraceptive prevalence to 23% of women of reproductive age. 18 The World Bank Health Sector Development Support ( P101160 ) aimed to assess the impact of the PADSS project in the provinces where there was no PBF. The MBB tool is not used anymore for economic evaluations. Therefore, the CBA in the ICR could not replicate the CBA done at appraisal. Moreover, the assumptions under which the CBA was conducted at appraisal did not hold true over project implementation, making a direct comparison of the results from the CBA at appraisal and from the ICR virtually impossible. Firstly, the CBA conducted at appraisal assumed that the Bank would support PBF only in regions where there was no existing RBF (supported by other organizations/donors). However, the role of the Bank evolved significantly since then, taking the lead on the scaling up process (nationwide), including providing instrumental support to strengthen the financial management system necessary for successful implementation of the PBF. Thus, simply allocating the benefits of the Bank’s involvement based on the percentage contribution of costs would not reflect the critical role it played. Therefore, it was not possible to isolate the contribution of the World Bank-support to the PBF program from that of other partners. Secondly, the impact of PBF payments on maternal and child health (MCH) outcomes cannot be attributed to supply side incentives only (as assumed at appraisal), as boosting demand of health services and strengthening the health care system are also expected to play a significant role in the program’s impact. On the positive side, the two CBA used similar approaches to monetize the benefits in dollar value (see Annex 4). However, the CBA conducted at appraisal overestimated the reduction in mortality by about 10 percentage points across all groups using DHS data from 2016-17 (37.8 % compared to 25 % reduction in maternal mortality, 24.6 % instead of 15 % for neonatal mortality, and 15.8 % instead of 6 % for child mortality). The estimated benefit cost ratio was 1.79, which is relatively close to the benefit cost ratio estimated in the CBA conducted for the ICR (1.67). 32. Economic analysis for the ICR: A new CBA was conducted over the entire program’s duration to assess the returns on investment of the FHC/PBF programs under the new circumstances - see Annex 4 for details. The analysis used standard methods to estimate the economic benefits that can be generated through the increased use of MCH services to assess whether the investment is justified on economic grounds. To this end, the impact of the FHC/PBF programs on maternal and child mortality was estimated (number of lives saved). Since the FHC/PBF programs were implemented nationwide and covered the entire MCH sector, a sectorial approach was used. All costs related to the FHC/PBF programs were included in the analysis, i.e. the entire IDA costs, the contributions by other partners, and the government’s contributions. This impact was then translated into economic benefits in terms of average lost future earnings of an individual (GDP per capita). Benefits were assumed to accrue over the working lifetime of pregnant women (28-60) and children (15-60). We assumed a long-term economic growth of 2.24% and used a conservative discount rate of 11% to reflect the opportunity costs of capital in Burundi. This rate was used to discount costs and benefits occurring in the future. Finally, the overall FHC/PBF costs were compared to the benefits, and the internal rate of return and benefit-cost ratio (BCR) were calculated. The analysis showed a total investment by the Bank, government, and other partners for the FHC/PBF programs of about US$213 million over 7 years (2010-2017). The estimated number of maternal and U5-child lives saved was 86,013 and 89,755, respectively. The reduction in maternal and child mortality yield an internal rate of return (IRR) of 13% and generated economic benefits with a net present value of US$43.9 million (assuming a 10.7 % discount rate). With an IRR higher than the cost of capital in Burundi (which reflects the return on investment from alternative projects), the project yields higher benefits than investments in the financial market. The investment has an attractive 1.67 benefit-cost ratio, suggesting that each US$ invested in PBF yields an economic return of US$1.67 for the population of Burundi (67%). Sensitivity analysis was conducted to show the impact of changes in key assumptions on the CBA. The analysis demonstrated that a conservative approach was taken when estimating the returns on investment of the FHC/PBF programs; and that even under a very unlikely scenario of very slow long-term economic growth (0.5%), the benefits outweighed the costs. 19 The World Bank Health Sector Development Support ( P101160 ) Operational efficiency 33. Average processing time was shorter than the Bank’s average for other projects implemented in Africa, and for other projects implemented in Burundi. For example, the time between approval and effectiveness was about 4 months, compared to 5 months on average. Similarly, the time between effectiveness and first disbursement was just above 1 month, compared to 4 months on average across the Bank’s projects (Table 7). Table 7 : Average processing time (in months) Concept- Approval- Effectiveness- Concept-first Approval Effectiveness first disbursement disbursement PADSS 10.8 3.8 1.1 15.6 Burundi 13 5 2 21 Africa 14 5 3 24 Bank 14 5 4 24 34. The staff’s turnover rate was low, as most staff remained over the entire time of the project, except for the General directorate of Health Services (DGS) and the Director of the General Directorate of Resources (DGR). The TTL on the Bank’s side was changed only once. A low staff turnover rate ensures efficiency in project implementation, as it minimizes learning and onboarding costs for new staff. 35. The inclusion of family planning and nutrition-sensitive interventions as PDO indicators strengthened the theory of change to achieve the desired outcomes, i.e. reduction in maternal and U5 mortality, and reflected changes in the government’s FHC policy priorities. In fact, studies have shown that nutrition-related illnesses contribute to 21% of the U5 mortality alone, and acute malnutrition had increased severely in some areas of the country (from 4.7 % in 2008 to 7.4 % in 2011). Moreover, family planning brings many benefits by spacing or delaying deliveries that also contribute to reduce maternal and child mortality. These indicators were included in the PDO only at a later stage because these services were either underfunded (family planning) or not covered (nutrition services) in the FHC/PBF at project’s appraisal. However, indicators related to family planning could have been better designed to accurately monitor implementation progress and enable timely correction of the project’s activities, if needed. 36. The Project was provided with continued and scaled-up financing from 2009 to 2017 because of its high performance. The first additional financing was deemed held necessary due to the higher than expected increase in health service utilization resulting from the PBF program, and therefore the need for more resources to contribute to the PBF funds pool. The second additional financing allowed to expand the package of services under the FHC/PBF programs to family planning and nutrition-sensitive interventions, both aiming at strengthening the theory of change, and to continue the project for a longer period, given its very positive achievements. It is important to highlight that the targeted PDO indicators were adjusted (increased) to reflect the extended Project time horizon, again reflecting the fact that this was not a project delay, but rather a desire to continue a very impactful program. 37. The project experienced some delays in disbursement due to changes in its scope (from selected provinces to all provinces) and due to the political crisis. However, disbursements quickly caught up when 20 The World Bank Health Sector Development Support ( P101160 ) the PBF went into full force and the disbursements resumed a few months after the crisis. In the end, almost all the funds were disbursed (i.e. disbursement rate of 99%). 38. The Project was embedded within the government’s structures, thus minimizing operational costs. No dedicated PIU or autonomous structure/agency was appointed to manage the project’s implementation. Therefore, operational costs mainly covered salaries and related expenses for experts to provide technical assistance on key functions, such as financial management, procurement, M&E, community and environmental health, communication and ICT. 39. The Bank’s administrative efficiency is also satisfactory, as actuals and commitments for activities led by the Bank (supervision and grant supporting activities) were slightly higher than budgeted, and reached a burn rate of about 113% (Table 8). The driving factor for the difference between budgeted and actual expenses was supervision, which was deemed as necessary to ensure effective project implementation. Table 8: Original versus actual costs for Bank’s led administrative activities (cumulative amount). Original Final Actuals + Burn Commitments Rate IDA 1,406,568 1,200,378 1,637,381 136% Trust Funds 280,648 2,166,225 2,183,306 101% Total 1,687,216 3,366,603 3,820,687 113% Assessment of efficiency and rating 40. Considering the economic and operational efficiency, the rationale for working with the public sector and motivation for the Bank’s involvement, the overall efficiency of the Project is rated as Satisfactory. Justification of Overall Outcome Rating 41. As shown in Table 9 below, the project overall outcome is rated “satisfactory”. Table 9: Overall outcome rating Relevance Efficacy Efficiency Overall Outcome Rating High Substantial Substantial Satisfactory E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 42. Pregnant women were one of the two ultimate beneficiaries of the project. Although the project did not specifically focus on raising awareness of relevant gaps between males and females and on contributing to increase women’s assets, income or employment opportunities by measures like vouchers, etc., it did contribute to the improvement of reproductive and maternal health. In so doing, it opened more possibilities for women to be in good health, to properly care for their pregnancies, to have babies in good health and to be able to earn income in the future and to avoid the possible adverse effects linked to poorly managed pregnancies and maternal health situations. In one specific group, the Batwa, the project made great strides for women by integrating them in the community structures like the Health Committees and community Health 21 The World Bank Health Sector Development Support ( P101160 ) workers, thus reducing the gender gap in the decision-making process geared to solving problems in these communities. Institutional Strengthening 43. Institutional strengthening was at the heart of the Project as one of its two components specifically focused on it. Over the course of the project, many institutions benefitted from capacity building in many different fields. The General Directorate of Resources (DGR), the General Directorate of Services (DGS), the Directorate of Health Promotion, Hygiene and Sanitation (DPSHA) of the MSPLS as well as provincial and district health offices. The training covered the PBF Manual and its revised version, the index tool and the use of the web-based PBF database for the Provincial Verification and Validation Committees (CPVV)7. The Project implementation unit was composed of MSPLS staff thus strengthening of MSPLS fiduciary capacity facilitated management of PADSS as well as of other projects. 44. PBF contributed to health facilities strengthening as it allowed them to manage their own resources to cover running costs. 15 hospitals also acquired incinerators and subsequently were trained in the management of potentially harmful biomedical waste. 45. At the community level, local NGOs, Health Committees (COSAs), Community Health Agents (ASCs) and Groups of Community Health Agents (GASC) were strengthened through sensitization in health-related themes with the help of facilitating and coaching agencies. National coaching agencies replaced the international agencies who initially coached community organizations as national agencies demonstrated the needed competencies to carry out the job. Poverty Reduction and Shared Prosperity 46. The project benefitted people from the lower income quintiles through its focus on basic health services and lowest level of care, which are mostly used by the poor and rural populations. The free healthcare policy benefitted the most those who could not access care because of financial barriers and PBF was implemented mostly in rural areas, where most of the poor live. Indeed, a study on equity revealed an improvement in utilization of and financial access to health services between 2009 and 2012 as a result of FHC. However, the 2017 Public Expenditure Review for the health sector noted that inequities were aggravated by the 2015 crisis. Other Unintended Outcomes and Impacts 47. There is some general agreement that the FHC policy and PBF supported resilience of the system during and after the 2015 crisis. As noted above the PBF system also contained structural elements of resilience in its design and the financial agreement between the GoB and the WB ensured that the Government contributed annually to the program. Despite the difficulties, health facilities could maintain a minimum level of services thus avoiding major impacts on health outcomes. The health system was also able to quickly recover after the crisis. The strong leadership of the Government of Burundi to promote free healthcare and PBF also facilitated donors’ alignment and mobilization of resources for the health sector, that is quite unprecedented for a low-income and fragile country as Burundi. 7 http://www.fbpsanteburundi.bi/ accessed 5 Dec 2017 22 The World Bank Health Sector Development Support ( P101160 ) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 48. Two key factors guided the project set up: the “Etats Généraux de la Santé” (which led to a National “Contracting out” Policy (Politique Nationale de contractualisation or POLNC); and the Government’s FHC policy. The POLNC led to the launch by international NGOs of PBF pilots as early as 2006 in three provinces in partnership with the MSP, to address supply side constraints (see Annex 9 for more details). The pilots were evaluated and demonstrated positive results: Service utilization increased in all pilot areas; complete child vaccination coverage increased, as did the rate of assisted births and the number of women who used contraception. B. KEY FACTORS DURING IMPLEMENTATION 49. The 2015 conflict and instability, leading to insecurity and a major macroeconomic slowdown was the most important factor affecting project implementation. This resulted in: i) Development partners stopping or curtailing their funding of projects in Burundi; ii) shrinking Government budget and therefore of funds available to the health sector; iii) development partners closing their accounts in private banks and opening accounts at the Central Bank; iv) travel restrictions throughout the country; v) and slowdown of implementation support from WB team as staff relocated to Nairobi from May to September 2015. Project supervision and verification/counter verification activities for PBF became more difficult, generating a backlog in payments made to health facilities. The same issue happened to agencies hired by the project to support community-based NGOs. Overall, this resulted in important delays in project execution and translated into a decline of some indicators that had already reached their targets. Health care provision was discontinued during the crisis, especially in the capital city Bujumbura. The nutrition impact evaluation revealed that the crisis affected availability of basic inputs (e.g. flour), thus hampering service delivery for nutrition interventions. As already pointed out, project implementation was also impacted by the malaria outbreak in 2016 and the role of religious leaders that were not supporting family planning services. 50. Although it showed Government’s commitment to FHC/PBF, a feature of the project design related to counterpart funding affected project implementation. The way arrangements were designed generated delays during project implementation as the agreement did not allow the Bank to spend any funds on PBF until Government funds had been fully utilized. On the positive side though, this guaranteed Government’s fulfillment of its obligation to fund the program. 51. Factors under the control of the World Bank were properly handled during project implementation. When faced with potentially problematic issues like the one related to “counterpart funding”, the Bank was pro-active and flexible. The Bank also provided appropriate advice as needed and ensured that program transition arrangements were made at the end of the project by appraising a follow-up project before the PADSS closing date. 52. Factors under Government and implementing agency’s control also positively affected the project. Qualified and high performing staff were hired and supported the Project throughout implementation. A technical cell within the MSPLS was created and strengthened. An online database was set up to better monitor 23 The World Bank Health Sector Development Support ( P101160 ) PBF indicators. Enlisting in the project partners like Cordaid and Health Net International-TPO, who had piloted PBF projects in some provinces also proved to be highly valuable. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 53. The M&E design was thorough and had a broad focus. The PDO was clear and indicators chosen adequately reflected the PDO. Indicators were sufficiently specific and measurable and data collection methods were appropriate and aligned to Government’s system. In addition, baseline data was available for all indicators. DHS data was also available for baseline (2010) and end line (2016-17) allowing to compare data sources and to use DHS data when appropriate. Due to the nature of the RBF process which requires health facilities to report their activities and get paid based on the quantity and quality of said activities, an independent third party was needed to verify reported activities and invoices presented by these health facilities. This was done at the onset of the project. Indicators were selected to monitor the progress made to the PDO and project components. On the fiduciary side, financial management and procurement specialists were hired by the implementing unit to monitor these issues and an external independent financial auditor was also hired to certify the proper use of funds by health facilities and by the implementing unit over the course of the project. A Monitoring and Evaluation specialist was also hired by the implementation unit to monitor activities. 54. Only one minor weakness in the M&E Design was detected by the ICR Team. The indicator on contraceptive prevalence was not well captured by the results framework. It results from the PDO revision in 2012 as the indicator associated to the PDO for couples was not specific to couples but more so to women of reproductive age. This may be due to the close link between this group and pregnant women or to the fact that the PDO packed all three beneficiary targets in the project legal documents. Thus, the indicator (on contraceptive prevalence) was not well captured by the results framework. However, the associated IO indicator, also added in 2012 (number of couple years of contraceptive protection using modern methods) could be tracked over time. 55. The theory of change was clear as project activities created an enabling environment that could lead to the expected outcomes/objectives. By creating incentives for health facilities to provide increased quality services in exchange for a financial stimulus and for facilitating agencies to train NGOs to sensitize the target population to increase demand for health service and by strengthening the MSPLS and entities involved in the PBF, the project was accurately set up to contribute to the expected outcomes. PDO indicators were closely related to the stated objectives and clear arrangements were planned to collect and analyze those indicators. M&E Implementation 56. M&E Implementation is rated Substantial. The M&E design facilitated project implementation and ensuring adequacy of the theory of change. Changes to the M&E framework were made to reflect changes in the scope of the project (e.g. new focus on nutrition and family planning); they were made in a proactive way to improve measurement accuracy and better monitoring of progress. 57. M&E data was collected through several channels. To monitor PDO indicators, four channels were used: i) routine data collected by the Health Management System (HMIS) from facilities; ii) regular and focused 24 The World Bank Health Sector Development Support ( P101160 ) independent surveys to provide data on facilities8 and on households9; iii) irregular surveys10 like a Demographic and Health Survey (DHS), and the Service Availability and Readiness Assessment (SARA)11 survey, and; iv) the online database set up by the CT-FBP. The surveys were done by institutions following internationally accepted methodologies. Although these four channels did generally provide data needed to monitor PDO indicators and other aspects of the project, one shortcoming lies in the lack of clear integration of their different collection methodologies and indicator definitions. This did sometimes lead to contradicting figures being produced for the “same” indicator. An impact evaluation on RBF for nutrition was done by the Institute of Tropical Medicine of Antwerp during PADSS implementation and was funded by an HRITF grant. Finally, two studies were conducted to document health financing and equity issues in Burundi. M&E Utilization 58. Monitoring and evaluation were strong elements of the project and were adequately used during the project’s implementation. Results provided by the M&E system were used during the entire implementation period to inform project management and decision making. These results were regularly compared to targets and the project was re-tuned several times to consider new data collected through surveys or it was adjusted when the scope or closing date of the project was changed. Policy making and policy influence and advocacy at all levels of the project was informed by these results. For instance, following a conclusion from a costing study that costs had been underestimated, fees linked to the PBF were raised for national hospitals. Furthermore, the Government and the Bank project staff built on these same results to start a follow up project (KIRA). For these reasons, M&E Utilization is rated High. Justification of Overall Rating of Quality of M&E 59. The Overall Quality of M&E is rated Substantial. This is in view of the Substantial ratings attributed to both the M&E Design and the M&E Implementation and the High rating given to the M&E Utilization. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental Compliance 60. The safeguard policy 4.01 pertaining to biomedical waste management was triggered. As shown in the PAD, the Government did update its National Biomedical Waste Management Plan (NBWMP). A set of activities were planned to mitigate the risks associated to this safeguard policy. Between 2013 and 2016, the Project funded the installation of incinerators in 14 district hospitals and in the Musaga health center and, from 2016 until 2017, the installation of mechanical grinders or glass breakers and metal cases for crushed glass in 14 of those health facilities. Disposal equipment (such as garbage bags, wheelbarrows, terry towels, toilet soap) and individual protection equipment was also provided to staff in charge of disposal of biomedical waste in these health facilities. Finally, training on how to handle biomedical waste was provided to many people involved in the health sector, 8 Two health facility surveys were done: one in 2010 called “Evaluation Rapide /Restreinte sur les Formations Sanitaires du Burundi” and one in 2013 called “Evaluation de la Qualité des Prestations dans les Formations Sanitaires du Burundi”. 9 These household surveys are: the “Enquêtes ménages pour le suivi et l’évaluation de l’impact de l’appui au système de remboursement du Paquet Minimum des Services de Santé (PMS 2009)” and the “Enquêtes ménages pour le suivi et l’évaluation de l’impact de l’appui au système de remboursement du Paquet Minimum des Services de Santé (Edition 2012)”. 10 The Project did contribute to the funding of the DHS and SARA Surveys, but was not the sole financier in both cases. 11 SARA measures the availability and operational capacity of health services. It was used as a continuation of the health facilities surveys done in 2010 and 2013. 25 The World Bank Health Sector Development Support ( P101160 ) starting by MSPLS high-level staff and heads of health facilities all the way down to cleaning personnel in charge of maintaining and using the incinerators. The Project Specialist in charge of Environment and Community Health benefitted from training abroad in Strategic Planning and results-based management and from a study tour in Uganda (together with the Director of DPSHA) 61. Closely linked to the environmental safeguard, community health was also developed by the project. A procedure manual and a training manual for the community health agent were produced and a Community Health Strategic Plan (CHSP) was published. Six coaching agencies were hired by the Project to implement community health and safeguard activities. Community relays made of 12,000 community health agents (grouped in 639 GASCs) and 6384 health committee members were elected. Among them, 557 Batwas were included in the COSAs and 851 in the ASCs to be part of the decision-making process. Training on the ASC Manual (produced through Project funding) was also provided to 492 participants, including 443 ASCs. Thus overall, the Project fully complied with this safeguard. Social Compliance 62. The project also triggered the Bank Operational Policy on Indigenous Peoples (OP 4.10) which is specifically applicable to the Batwa people of Burundi. In fulfillment of the policy requirements, the project prepared a “Comprehensive Social Assessment with an abbreviated Indigenous People Plan (SA-IIP)”, which was endorsed by the Batwa and approved by the Government. The objectives of the Plan were to build the capacity of the Batwa people so that they could take charge of their own health and would participate in decision-making forums and in the management of health programs. The project approached the policy requirements first through the DPSHA of the Ministry of health, from 2010-2014, and eventually through a third party, from 2014-2017. At the close of the project, the Ministry commissioned an evaluation of the implementation of the Indigenous People Plan. 63. The Ministry carried out a number of activities to lay the foundation to facilitate the access of the Batwa to public health services: i) sensitization of medical directors at provincial and district levels to encourage them to co-opt Batwas as health workers in the public health service; ii) sensitization of Batwas to make them aware of their rights as Burundian citizens, especially with respect to rights to public health services; iii) undertaking the administrative processes to co-opt Batwa health workers into public health services; iv) training Batwas health workers; v) distribution of training materials to Batwa health workers in order to facilitate their outreach to their community; vi) recruitment of agents to plead the cause of vulnerable groups, including the Batwa, with local authorities to obtain the necessary identification papers necessary to gain access to public health services. From 2014, the Ministry of Health recruited a non-governmental organization, UNIPROBA (Unissons- nous pour la promotion des Batwas) to complete the implementation of the Indigenous People Plan. This involved: i) sensitize Batwa women to influence behavior change and improve their rate of attendance at health centers when ill; ii) mobilize the Batwa and their grassroots organizations for better participation in the management of their health problems; iii) mobilize Batwa households to systematically use health care structures, especially the national FHC package; iv) conduct sensitization sessions in the Batwa community for the registration of marriages, births in civil registry services, acquisition of the medical assistance card (CAM). 64. At the end of the Project, a clear change in behavior of the Batwa beneficiaries was noticed. There is a marked increase in the number of Batwa women who receive antenatal care, deliver and immunize their babies and children at health facilities. This has led to a noticeable reduction in infant mortality but specific figures are yet to be collected. The acquisition of health insurance (assistance cards) by 6000 Batwa improved 26 The World Bank Health Sector Development Support ( P101160 ) community access to public health services. This in turn led to an increase in self-esteem amongst individual community members. The familiar presence of Batwa workers at community health centers also means that the community’s voice can be heard, attention paid to their specific health concerns and they are part of the decision- making process. Finally, personal hygiene has improved considerably through awareness-building and advocacy sessions. Financial Management Compliance 65. Financial management complied with Bank procedures and policies and was always rated Moderately Satisfactory or Satisfactory. To mitigate the risks identified in the PAD, the Project set up an accounting system configured to interface with the accounting system of the DGR and produced a PBF Procedures Manual. It also built capacity by providing training to the Financial Management Specialist and to the accountants who had been hired. Changing the scope of the Project from covering “selected provinces” to “all provinces” had a negative effect on disbursements as this led to a one year delay in implementation. However, disbursements quickly caught up when the PBF went into full force and, in the end, almost all the funds were disbursed (i.e. disbursement rate of almost 99%). 66. Interim unaudited financial reports were regularly provided bi-annually and the frequency of these reports was even accelerated to quarterly for the Additional Financing signed in December 2012. The Project was regularly audited by external accounting firms who certified without reservation year after year that proper accounting procedures had been followed and that the financial statements truly reflected the project financial situation and that no ineligible expenses were funded. Procurement compliance 67. At the time of appraisal, the overall procurement risk was moderate as the procurement of goods was limited in number (there would be no works or international bidding) and, for consultants, an international consultant was hired to assist the DGR. Procurement delays in the Project originated both from internal and external sources. Internally, an initial lack of mastery of procedures slowed down the recruitment of coaching agencies for instance. Externally, the lack of availability of some products (for instance the provision of bricks by the contractor hired to build the incinerators) and problems with the institutions in charge of doing some of the surveys forced the Project to delay some of its activities. However, due to the small relative share of procurement activities in the Project, this did not have a big impact. C. BANK PERFORMANCE Quality at Entry 68. The Bank performance on ensuring Quality at Entry is rated Satisfactory. The background analysis done by the Bank Team was thorough. Not only were lessons from the previous IDA-funded health project considered (particularly when it comes to the potential negative effects of having a Project Implementation Unit) but lessons from other post-conflict settings were incorporated into this Project. A thorough investigation of Results Based Financing (RBF) approaches was also undertaken by the Bank team to see what successful lessons could be learned from countries where RBF had been applied (such as Rwanda, Afghanistan, Cambodia and Haiti). 69. A $600,000 Project Preparation Facility Advance was provided to help prepare the Project. It was to be 27 The World Bank Health Sector Development Support ( P101160 ) used to fund: i) the carrying out of studies and the provision of technical advisory services for the preparation of the project and; ii) capacity building activities to strengthen the capacity of the DGR. In the end, only US$172,241.99 of this amount was used before the start of the Project. 70. The Bank team held a Quality Enhancement Review (QER)12 to agree on issues to be raised in the PAD and on recommended actions. As a result of this background analysis, institutional arrangements were properly set up, safeguards properly assessed and mitigation measures properly defined. Risk assessment linked to the institutional arrangements was also properly done and powerful mitigation measures were defined including the necessity of strengthening the implementing agency, the MSPLS, through capacity building. Quality of Supervision 71. The Quality of Supervision done by the Bank is rated Satisfactory. Supervision missions were done regularly (at least twice every year and sometimes more) and, most of the time, involved fiduciary specialists. For instance, these specialists were team members on all 5 missions during the first year of the project. They were involved in every Additional Financing mission and in the Mid-Term review. They were also members of the last 3 mission teams before the end of the project. As the project dealt mainly with PBF, a PBF specialist was part of the first missions (in fact, a 4-member team was involved in the September 2009 mission) and in the 13 subsequent missions starting in September 2011, a PBF specialist was always included in the supervision teams. At the beginning of the project (i.e. first 5 missions), M&E specialists were team members on every mission to design a strong and performing M&E system. On top of these supervision missions, Bank specialists and hired consultants from other fields were brought in when needed (like, for instance, specialists on environment safeguards). The Bank posted a Financial Management Specialist in Bujumbura from 2014 to 2017 to support the Project financial management unit. Even when political unrest made it impossible to travel to Burundi, a supervision mission was held by the Bank in Nairobi even though it meant transporting the whole project team to that city. 72. During implementation, the Bank showed a high level of flexibility and adaptability which was very helpful for the Project. A case in point was the situation caused by the Government inability to provide the funds it was expected to bring for the PBF. Although no formal counterpart funding was included in the Project, one of the clauses included in the Legal Agreement stipulated that the Government was to set aside (in a Special Treasury Account) 1.4% of its total annual budget for health expenditures and, that the Project could start funding PBF- related activities only after Government funds had been totally exhausted. When the crisis occurred in 2015, the Government was unable to honor its part of the agreement and requested a special waiver to get the Project to fund the PBF and get refunded later when conditions improved. In order not to slow down the PBF, the Project obliged and, as it turned out, the Government did reimburse (before the end of the year) the full amount that had been provided as an advance by the Project. This should be seen in the context of a crisis when most development partners had stopped or reduced their funding to the country. Thus, the Bank’s flexibility in accepting the waiver was instrumental for supporting the Government during the crisis. Flexibility and adaptability were also demonstrated through the revisions and restructuring of the project which aimed to strengthen the theory of change by putting greater emphasis on the country’s main developmental challenges and strengthening the way 12 The following issues were addressed during the QER: 1) Appropriateness of Project Development Objectives and Results Framework; 2) Design issues regarding nature and size of payments; 3) Financial Management and Flow of Funds issues; 4) Use or not of an external agency (with highly paid staff) to verify activities and pay, rather than using the government structure; 5) Cohabitation of two systems (Cordaid and Government’s); 6) Need to ensure appropriate checks regarding reported utilization figures; 7) Support for Management Capacity of the Ministry of Health; 8) Monitoring and Evaluation (M&E), and; 9) Measurement and reimbursement of community level services. 28 The World Bank Health Sector Development Support ( P101160 ) services were delivered. 73. Implementation Status and Results (ISR) Reports were successfully used to monitor project implementation and raise questions to management. A total of 16 ISRs were filed during the project (Annex 7). Management comments were particularly useful in the event of exchange rate variation to allow proactive management and changes to accommodate for higher/lower costs. Besides, missions’ aide-memoires were used both by the Government and the Bank as key documents to follow-up progress and actions to be completed after the mission. Each mission started from a review of the past aide-memoire to make sure there were no pending issue to address. 74. Finally, staffing stability helped avoid disruptions in implementation and make collaboration easy to achieve. During implementation, the Task Team Leader on the Bank side was changed only one time whereas, on the Government side, only the DGS was replaced a few times. Project staff remained the same for the most part. The only exceptions were the Director of the DGR who was changed 4 times since the beginning of the Project and one or two specialists who left the Project and were replaced. Justification of Overall Rating of Bank Performance 75. Considering the thorough research and attention to details provided during project design (including a QER) and the quality of supervision (characterized by strong and regular supervisions throughout Project implementation) and the Satisfactory rating given to both design and supervision, the Bank Overall Performance is rated Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 76. The main risks to development outcomes are the same as identified during project preparation, namely political instability and security issues and their consequences. These risks did materialize in 2015 during project implementation and had a negative impact on health care access and financial access. If the political/security situation risks were to deteriorate again, the financial consequences would be a shrinking budget and reduced external aid that could negatively impact on funding available to the health sector. This could limit activities that rely on logistics (through the reduced availability of cars and gasoline) and necessary to the proper functioning of the PBF system. This could create a situation where the Government would not be able to cover its share of the funding needed for the PBF, thus not fulfilling its contractual responsibilities (i.e. 1.4% and more of the total budget devoted to the Health Sector). Security issues reduce the ability of the staff to do supervision and, even worse, may affect the PBF process at its core by limiting the possibility of verification and counter- verification agencies to move around to properly monitor health facilities. Facilitating agencies may also face difficulties in accomplishing their sensitization mission towards NGOs that are in contact with beneficiaries. 77. The project has learned to deal with these risks and has developed mechanisms to circumvent most of the negative features that had an impact on access to health care. As further developed in the lessons learned section, these risks have been factored in the follow-up project (KIRA) and strong mitigating measures have been included in its design. The KIRA project, effective since September 2017, places greater emphasis on quality of health care services provided to ensure increases in service utilization to translate into better health outcomes. Its focus on community health workers also aims to better address bottlenecks in service delivery for family planning and nutrition services. Furthermore, to address the high ‘political and governance’ and ‘security’ risks, the KIRA project continues to be anchored within the MSPLS under the national PBF technical unit as this 29 The World Bank Health Sector Development Support ( P101160 ) arrangement has shown to work well even in times of crisis. In the long-term, this institutional arrangement is expected to safeguard the sustainability of the health system and its ability to deal with any emerging health crises. 78. Another potentially major risk that the achievements will not be maintained relates to financial sustainability. Considering the effect that reduced funding from partners created after the crisis, a lack of funding could jeopardize the observed achievements. During the implementation of PADSS, the Government has always respected its commitment to allocate 1.4 % of its annual general budget to the FHC/PBF program. Despite the fragile status of Burundi and limited revenues generated by the country compared to other countries in the region, the GoB’s commitment to the PBF and free care programs has been higher than in many other places. To reinforce its commitment to support FHC/RBF and to ensure greater financial sustainability of the program, the GoB is committed, under the KIRA project, to increase its contribution by 0.1 % annually to reach 1.5 % in 2017, 1.6 % in 2018, 1.7 % in 2019, 1.8 % in 2020, and 1.9 % in 2021. V. LESSONS AND RECOMMENDATIONS 79. Main success factors of the Burundi PBF: i. Successful pilot: Health Reforms such as PBF based reforms benefit from a period of piloting well- designed and well-implemented schemes at a small scale prior to scaling up, as was done in Rwanda, and in Burundi; ii. Counterpart funds: Linking PBF reforms to country commitment to finance health through similar methods was successful. The design of the Financial Agreement between the GoB and the WB for this project can be considered good practice. It stated: “Although no formal counterpart funding was included in the Project, one of the clauses included in the Legal Agreement stipulated that the Government was to set aside (in a Special Treasury Account) 1.4% of its total annual budget for health expenditures and, that the Project could start funding RBF-related activities only after Government funds had been totally exhausted.” This greatly contributed to ownership of the program and its financial sustainability, even throughout the crisis; iii. Government ownership and donor alignment around the PBF vision: Strong consistent health leadership and ownership by the Government is necessary to make such health reforms stick. Local ownership of PBF reforms was expressed by large domestic financing for these reforms. Linking funds for selective FHC for pregnant women and children under five years of age and funds available for PBF based health reforms was also a winning formula; iv. Local community ownership: Community support to the program was critical during the crisis. The ownership of the program at the local level supported resilience and rapid recovery after the crisis; v. Institutional strengthening: Institutional capacity strengthening was essential to ensure resilience of the program even after withdrawal of some external partners. Use of the country systems is a factor of resilience even in times of crisis: the project showed that when there is a good dialogue and good accountability, development partners should support the use of governments’ systems. Building local capacity for PBF was also crucial to enable local champions to stand up and advocate for these health reforms. Finally, public finance management was key in ensuring the success of the program and the role of PADSS and the WB was critical in that respect. PFM rules imposed by the WB set the tone for eligible expenditures under the FHC/PBF program; 30 The World Bank Health Sector Development Support ( P101160 ) vi. Growing economy and increase in external assistance: The FHC/RBF program scale-up occurred at a time of strong economic growth and increased development assistance to the health sector. Creating domestic fiscal space for PBF is crucial prior to engaging in nationwide scale-up of PBF, or any provider payment reform. In situations of restricted domestic fiscal space and abundant development partners’ financing for health, efforts should be made to leverage domestic financing with development partners’ financing. The WB long-term engagement was indeed critical to the Program’s sustainability and continues beyond PADSS. This also raises the need to reflect on modalities and right timing for external funding to phase out of the program. 80. Design features of Burundi PBF that contributed to its success: vii. Rigid civil service rules and regulations demand innovative approaches to retaining competent public servants to work and lead PBF reforms within the Ministry. Using performance contracting for the technical support unit, which contained consultants financed through the WB but also development partners, but which was led by civil servants, has worked very well. Key civil servants received up to 400% more (performance-based) bonus than their base salary; this was a key factor for the stability of this approach, and the engine behind retaining key civil servant champions for continuing to champion this approach. Public health doctors in districts, key actors in the internal verification mechanisms for the national PBF approach, received an estimated 300% (performance-based) bonus over and above their civil service salaries enabling their full attention to the FHC/PBF program; viii. Solid third -party counter-verification of internal ex-ante verification mechanisms is required and has been championed by this project; ix. Systematic community client satisfaction surveys, like done in this project, provide good health intelligence and can be made operational to include in the quality index for performance measures; x. Publicly accessible performance data, like in the case of the Burundian PBF system, enhance the Governance and credibility in the system, and this is crucial when many development partners co- finance this approach; xi. Decentralizing health financing to the frontlines, while enhancing autonomy through well-funded PBF approaches over a significant period, and especially when baselines are very low, can have dramatic positive results on health system performance; xii. Autonomy in the management of human resources and breaking monopolies in the procurement of drugs and medical consumables are crucial preconditions for PBF to work; xiii. Success of the Burundian health reforms were greatly due to successful public-private partnerships and systematic application of internal contracting (public health facilities and health administration) and contracting-in approaches (Government financing NGOs both local and international, and community-based organizations for strengthening verification and counter-verification functions). 81. Recommendations from PADSS implementation are the following: xiv. Prior to starting any PBF based health reforms ensure that sufficient or as large as possible domestic funding is secured and apply methods using available tools (e.g. financial agreement) to ensure that this funding is likely released. The Burundi financial agreement is a good practice of how to possibly nudge Government to adhere to its commitments; 31 The World Bank Health Sector Development Support ( P101160 ) xv. Link PBF health reforms to desirable local funding priorities. Selective FHC was an explicit presidential decree issued in 2006. There was political commitment, and therefore, an opportunity to seize to use this domestic commitment with available funding to achieve the same objectives, but with a revisited design; xvi. Push back if reforms seem to derail. For instance, the GoB wanted internal verification without any separation of functions or involvement of external non-governmental actors, while development partners wanted external verification only. A compromise solution was found, reasonably workable in the Burundian context. Another example is that the GoB at the onset wanted to create a parallel system to reimburse selective FHC and have PBF funded by its development partners. In that case, it was critical to take sufficient time to get design right and ensure good implementation of this design while being ready to adjust if things do not work well; xvii. The WB needs to study better how Burundi scaled up PBF and ensure that the ‘how to’ will be considered as a ‘good practice’ to be followed by similar WB financed health reform initiatives. Especially now, as the focus has turned on Universal Health Coverage, one needs recognition that these PBF reforms are essentially reforms to work towards Universal Health Coverage, a stated SDG goal, and a key method for the WB to work towards its twin goals. xviii. Multisectoral interventions are needed to address malnutrition in Burundi. The results of the impact evaluation suggest that human resources (at health facilities and at community levels) do not have the necessary skills to deliver quality nutrition services; availability of inputs, equipment, and protocols as well as the quality of supervision are also major weaknesses. Nutrition specific and nutrition sensitive activities must be supported to fight against malnutrition, beyond supply side incentives provided by PBF. The nutrition agenda must be taken up by other projects from the WB and other development partners. From the WB’s perspective, nutrition is tackled in the KIRA project, as well as the JSDF funded pilot on nutrition, the Social safety net project, the agriculture project, the education project and the Investing in Early Years project under preparation. . 32 The World Bank Health Sector Development Support ( P101160 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increase the use of a defined package of health services by pregnant women Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of pregnant women Number 1582.00 2894.00 3600.00 5026.00 living with HIV who received antiretrovirals to reduce the 31-Dec-2009 31-Dec-2014 30-Jun-2017 30-Jun-2017 risk of Mother To Child Transmission Comments (achievements against targets): Target has been achieved and surpassed (139.61%). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Coverage of prenatal care Percentage 59.40 70.00 65.50 66.50 services for pregnant women (at least 3 visits) 31-Dec-2009 30-Dec-2012 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target has been achieved (101.53%) if the 2015 figure shown on the 2016 HMIS Yearbook is taken. However, preliminary figures presented in the same Yearbook show a slightly lower percentage, 62. If confirmed, this figure would show an achievement rate of 94.66%. 33 The World Bank Health Sector Development Support ( P101160 ) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Pregnant women receiving Number 415799.00 501262.00 549564.00 529666.00 antenatal care during a visit to a health provider (number) 31-Dec-2009 31-Aug-2014 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target was almost achieved (95.21%). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of births assisted by Percentage 64.40 50.00 86.20 85.10 skilled personnel 31-Dec-2009 30-Dec-2012 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target has almost been achieved (98.72%). The latest data comes from the 2017 Demographic and Health Survey (DHS 2017). Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct project beneficiaries Number 1.00 2273188.00 2210111.00 2263154.00 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 Female beneficiaries Percentage 1.00 1368232.00 1393176.00 1394317.00 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 34 The World Bank Health Sector Development Support ( P101160 ) Comments (achievements against targets): Target has been achieved (102.4%). The target for the Custom Supplement indicator below has also been achieved (100.08%). The basis behind these numbers is that pregnant women are estimated to represent 5% of the total population while children under five are estimated to represent 17.7% of that population. Objective/Outcome: Increase the use of a defined package of health services by children under the age of five Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Average number of visits to a Number 1.68 2.80 2.30 2.20 health provider by children under the age of 5 31-Dec-2009 30-Dec-2012 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target almost achieved (95.65%). This figure comes from a household survey (PMS 2) and was to be updated in the 2016-17 DHS. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Children immunized (number) Number 297780.00 341030.00 359930.00 355551.00 31-Dec-2009 31-Aug-2014 30-Jun-2017 30-Jun-2017 Children immunized - under Number 297780.00 341030.00 341030.00 322560.00 12 months against DTP3 (number) 31-Dec-2009 31-Aug-2014 31-Aug-2014 30-Jun-2017 Comments (achievements against targets): Target almost achieved (98.78%). The figure comes from the 2016 Health Management Information system. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion 35 The World Bank Health Sector Development Support ( P101160 ) Pregnant/lactating women, Number 0.00 0.00 0.00 0.00 adolescent girls and/or children under age five-reached by basic 31-Dec-2009 30-Dec-2012 31-Dec-2009 30-Jun-2017 nutrition services (number) Children between the age of 6 Number 1265275.00 1709804.00 1665878.00 1651266.00 and 59 months receiving Vitamin A supplementation 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 (number) Children under age five Number 87170.00 93500.00 83081.00 58307.00 treated for moderate or severe acute malnutrition 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 (number) Comments (achievements against targets): As the figures show, the custom indicator was never really measured and no baseline or targets were provided. However, the 2 breakdown indicators are related to PDO 2; therefore the link was made with that PDO. The first breakdown indicator, related to "Vitamin A supplementation" almost met its target with an achievement rate of 99.23%. The second breakdown indicator could be considered as an outlier as it is the only PDO indicator which, at 70.18%, fell below 90%. Objective/Outcome: Increase the use of a defined package of health services by couples of reproductive age Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Contraceptive prevalence rate Percentage 19.80 15.00 34.85 37.40 31-Dec-2009 30-Dec-2012 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Although the target seems to have been achieved (121.95%), new preliminary figures provided by the 2017 DHS show 23% for 2016 (which would mean an achievement rate of 66%). This large discrepancy is being investigated by the MSP and, at the time of writing the ICR Report, no clear explanation had been provided to the ICR team regarding which of these two figures should be considered right. 36 The World Bank Health Sector Development Support ( P101160 ) A.2 Intermediate Results Indicators Component: Transfer of resources to health facilities to pay for the Free Package of Health Services (FPS) on the basis of the RBF Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percent of public and non- Percentage 50.90 100.00 100.00 96.80 profit health facilities offering the free package of services 31-Mar-2010 31-Dec-2018 30-Jun-2017 30-Jun-2017 through the RBF system Comments (achievements against targets): Target almost achieved (96.8%) according to the latest figure from the RBF database. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Percentage of health facilities Percentage 0.00 0.00 4.90 5.10 that did run out of essential drugs in the previous month 31-Mar-2010 31-Dec-2018 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): The actual formulation for this indicator is "Percentage of health facilities that did not run out of essential drugs in the previous month". Although the target seems to have been met (104.08%), 3 different definitions were used over the course of the Project, therefore making the collected figures not comparable. The definition of "essential drugs" changed over time and the time period used to monitor this indicator varied from 3 months to one month to the day preceding the survey. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion 37 The World Bank Health Sector Development Support ( P101160 ) Percentage of health facilities Percentage 0.00 0.00 60.00 62.00 that have a Head with A2 Level 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target achieved (103.33%). Component: Building the capacity of community health actors Component: Strengthening the capacity of the MSP and entities involved in RBF Unlinked Indicators Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Births (deliveries) attended by Number 188135.00 0.00 364090.00 359411.00 skilled health personnel (number) 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 Comments (achievements against targets): Target almost achieved (98.7%). The target was indeed achieved in 2016 with an actual value of 368,092 (101.10%), but the 2016 Statistical Yearbook shows a decline to 359,411. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of couple-years of Number 228082.00 0.00 734520.00 555702.00 contraceptive protection using modern methods 31-Dec-2009 31-Dec-2018 30-Jun-2017 30-Jun-2017 38 The World Bank Health Sector Development Support ( P101160 ) Comments (achievements against targets): Target not achieved (75.66%). Indeed the actual figure had reached 631,204 (i.e. achievement rate of 85.93%) per the 2015 Statistical Yearbook, but according to the 2016 Statistical Yearbook, there was a decline to the new figure shown here. B. SUMMARY OF ACHIEVEMENT OF THE RESULTS FRAMEWORK INDICATORS Baseline End line Target Achievement Remarks (%) Outcome Outcome Indicator Number of pregnant women living with HIV 1582 5026 3600 140% Surpassed (100%+) Increase the use of a who received antiretrovirals to reduce the risk defined package of health of Mother To Child Transmission services by pregnant Coverage of prenatal care (PNC) services for 59.4 66.5 65.5 102% Surpassed (100%+) women pregnant women (at least 3 visits) Percentage of births assisted by skilled 64.4 85.1 86.2 99% Achieved/Substantially personnel (85+) Pregnant women receiving antenatal care 415799 529662 549564 96% Achieved/Substantially during a visit to a health provider (number) (85+) Female Beneficiaries 0 1394317 1393176 100% Achieved (100 +) Percent of public and non-profit health facilities 50.9 96.8 100 97% Achieved/Substantially offering the free package of services through (85+) the RBF system Percentage of health facilities that did run out 0 5.1 4.9 104% Surpassed (100%+) of essential drugs in the previous month Percentage of health facilities that have a Head 0 62 60 103% Surpassed (100%+) with A2 Level Birth (deliveries) attended by skilled health 188135 359411 364090 99% Achieved/Substantially personal (number) (85+) Increase the use of a Average number of visits to a health provider 1.68 2.2 2.3 96% Achieved/Substantially defined package of health by children under the age of 5 (85+) services by children under Children between the age of 6 and 59 months 1265275 1652052 1665878 99% Achieved/Substantially the age of five receiving Vitamin A supplementation (number) (85+) 39 The World Bank Health Sector Development Support ( P101160 ) Baseline End line Target Achievement Remarks (%) Children under age five treated for moderate or 87170 58307 83081 70% Not achieved severe acute malnutrition (number) Children immunized under 12 months against 297780 322560 341030 95% Achieved/Substantially DTP3 (number) (85+) Children fully immunized 297780 355551 359930 99% Achieved/Substantially (85+) Increase the use of a Contraceptive Prevalence Rate (modern 19.8 37.4 34.85 107% Surpassed (100%+) defined package of health methods) services by couples of reproductive age 40 The World Bank Health Sector Development Support ( P101160 ) C. KEY OUTPUTS BY COMPONENT Components Outputs (linked to the achievement of objectives/outcomes 1, 2 and 3) Component 1A 1. Number of facilities funded through the PBF for providing FPS (Health Centers and Hospitals) at all levels: 804 (including 689 with primary contracts and 115 under secondary contracts). 2. Number of Managerial Units involved in verifying PBF Payments and funded by the PBF (at all levels): 18 3. Amounts paid by the PADSS to Health Facilities: $51,330,143.68 4. Amounts paid by the PADSS to PBF implementing entities: $11,738,401.67 Component 1B 1. Number of facilitating agencies funded: 6 2. Number of CBOs trained by Facilitating Agencies (i.e. NGOs): 670 3. Number of CHWs trained by Facilitating Agencies (i.e. NGOs): 11,845 4. Number of COSAs Facilitating Agencies (i.e. NGOs) helped to set up: 706 Component 2 1. Number of people trained in FM: 4 2. Number of people trained in ICT: 2 3. Number of people trained in Environment and Community Health: 1 4. Number of people trained in Communication: 1 5. Number of controllers hired by the DGR for its de-concentrated units: 30 6. Number of people trained in M&E at the Planning Department of the MSP: 1 7. Number of people recruited in M&E at the Planning Department of the MSP: 1 8. Number of surveys (i.e. PMS, FOSA, DHS, etc.) partially or totally funded through the M&E unit: 7 9. National entity hired to carry out the third party external control exercise: 2 10. Number of people trained in PBF: 545 11. Amount spent on funding the PBF Unit: $5,004,542.5013 12. Recurrent costs paid for BPS verification: $878,548 13 Due to the nature of the project (a Results-based financing project), all the amounts shown in this table, although not an output per se, represent payments for an output already produced and verified. Thus, they can be considered as a reflection or a mirror image of an output instead of an input. 41 The World Bank Health Sector Development Support ( P101160 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Laurence Elisabeth Marie-Paule Lannes Task Team Leader(s) Melance Ndikumasabo Procurement Specialist(s) Christian Simbananiye Financial Management Specialist Evelyn Anna Kennedy Team Member Laura Di Giorgio Team Member Aly B. Sy Team Member Alain-Desire Karibwami Team Member Paul-Jean Feno Environmental Safeguards Specialist Clarette Rwagatore Team Member Moulay Driss Zine Eddine El Idrissi Team Member Ishanlosen Odiaua Social Safeguards Specialist Nneoma Veronica Nwogu Counsel Richard Shugugu Team Member Issa Thiam Team Member Alice Museri Team Member 42 The World Bank Health Sector Development Support ( P101160 ) B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY06 0 2,759.37 FY07 14.939 52,391.49 FY08 21.615 62,784.02 FY09 48.895 262,580.42 Total 85.45 380,515.30 Supervision/ICR FY10 39.833 227,524.81 FY11 26.809 151,176.52 FY12 58.056 347,628.78 FY13 22.852 121,747.25 FY14 15.800 94,417.95 FY15 47.155 154,529.79 FY16 16.390 94,152.53 FY17 14.909 56,317.46 FY18 4.050 27,452.06 Total 245.85 1,274,947.15 43 The World Bank Health Sector Development Support ( P101160 ) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (%) Component 1 - Increased financing 64.5 64.5 99.9 for a redefined free package of services Component 2 - Strengthening capacity of the 20.1 20.1 99.9 Ministry of Public Health and entities involved in PBF Total 84.7 84.6 99.9 44 The World Bank Health Sector Development Support ( P101160 ) ANNEX 4. EFFICIENCY ANALYSIS 1. Project development impact The proposed project aimed to contribute to Burundi’s development through the following pathways : improving mother and child survival; saving unnecessary health care costs and social care costs; increasing productive labor force; promoting equity and shared prosperity; and improving health system performance. The project aimed at improving mother and child survival and reducing mortality related to communicable diseases by increasing the coverage of evidence-based, high-impact, maternal and child health interventions such as assisted deliveries, pre- and post-natal care and integrated management of childhood illnesses; promotion of preventive care and vaccination; family planning; treatment of malnourished children; and improvement of the management and treatment of communicable diseases, in particular HIV/AIDS. Most complications during and following pregnancy and childbirth are preventable or treatable via the selected interventions. For example, the major complications that account for nearly 75% of all maternal deaths are severe bleeding (mostly bleeding after childbirth); infections (usually after childbirth); high blood pressure during pregnancy (pre-eclampsia and eclampsia); complications from delivery and unsafe abortion. All these issues can be prevented or managed through the selected MCH interventions supported by FHC/PBF. The project contributed to maternal and child mortality reduction by addressing barriers that limit access and delivery of quality MCH services, in particular through addressing inequalities in access to quality reproductive, maternal and newborn health care services; ensuring universal health coverage for comprehensive reproductive, maternal, and newborn health care; strengthening the health system to respond to the needs and priorities of women and girls; and ensuring accountability in order to improve quality of care and equity. By reducing child and maternal mortality, the project also contributed to saving health care costs related to disease treatment as the project focused on cost-effective preventive and curative measures, and to saving social economic burden that is related to extra care needed for children who are stunted and who suffer from preventable diseases. At appraisal, the project was expected to generate long-term economic benefits for the country’s economy by increasing active and productive labor force who can potentially contribute to economic growth and poverty elimination. The rationale was that with improved health and nutrition status, more children would survive into adulthood and work more productively as a result of better cognitive development. Women who would be saved from maternal deaths would contribute directly to productive activities or relieve household members who would have had to provide child care without their presence. A recent study on the impact of maternal and child health on economic growth found a bi-directional relationship in Burundi between mortality and changes in GDP meaning that changes in GDP have an impact on under-five and maternal mortality and vice versa. The study further demonstrates that the effect of marginal health investments on health outcomes is higher at low levels of GDP, i.e. in countries where the level of health investments is generally lower, such as Burundi14. A study on WHO AFRO countries also found that maternal mortality of a single person reduces per capita GDP by US$ 0.36 per year15. 14 Amiri, A. and Gerdtham, U.G., 2013. Impact of maternal and child health on economic growth: New evidence based granger causality and DEA analysis. Newborn and Child Health, Study Commissioned by the Partnership for Maternal, Lund University, Sweden. 15 Kirigia, J.M., Oluwole, D., Mwabu, G.M., Gatwiri, D. and Kainyu, L.H., 2006. Effects of maternal mortality on gross domestic product (GDP) in the WHO African region. African journal of health sciences, 12(3), pp.55-64. 45 The World Bank Health Sector Development Support ( P101160 ) The project was expected to contribute to improved technical efficiency in the health service delivery system which freed up resources that can be used to save lives. It supported the Ministry of Health and other entities involved in PBF, including the National Technical Unit in charge of the coordination of the FHC/PBF Program, the Verification and Validation Provincial Committees, the Provincial Health Offices, the District Health Offices and the Department in charge of Resources Management. By strengthening institutional capacity and improving availability and quality of key inputs, it was expected that more facilities would be pushed to the production function frontier, and therefore, would deliver better services to the extent possible at a given cost. The project was also designed to increase allocative efficiency at health facilities and community levels. Indeed, the project focused on primary health care and community-based activities, which are the most cost-effective modalities to provide a defined package of high impact services. It supported the Burundi health system to be more results-focused and to get value from the money invested by supporting the PBF approach. 2. Rationale for working with public sector The rationale for working with the public sector is related to the critical role of the government in regulating the health sector, and by the Project’s economic and social goals. Investments funded through the Project aimed to abolish user fees for the end beneficiaries, strengthen health service delivery, and improve institutional capacity. Public sector intervention was thus critical to promote good health in the target population. Public sector investments are key to provide and promote preventive health services and support equity improvements to access good quality RMNCH services. Since the main objective of PBF payments was to remove user fees for the final beneficiaries, the investment by the private sector was not an option. These interventions are known to have positive externalities and important spillovers (societal returns of investing in women’s and children’s health for economic growth) which face market failures and advocate for public sector intervention. 3. Value added of the World Bank support At appraisal, four main reasons were given to justify the Bank’s engagement. First, the Bank had extensive country and global experience, as well as analytical capacity that could add value. The Bank was one of the leading agencies engaged in Burundi's health sector and had conducted IDA-funded studies that provided the analytical underpinning for future projects. Second, IDA funding had several comparative advantages compared to other donors’ funding that the government highly valued. Burundi has suffered from fragmented, inefficient, and inflexible external assistance, while IDA provided on-budget support that could be flexible adapted to the changing circumstances. Moreover, thanks to its convening power, the government envisioned a positive impact arising from the potential of aligning funds from other donors and building a common vision for the health sector. Third, the project aimed to strengthen the government’s capacity to implement PBF, and this was expected to further contribute to the ongoing discussions with the Ministry of Finance regarding nationwide public sector reforms such as public financial management that were very much needed. Fourth, IDA financing was deemed necessary to fill the gap in the health system investments given that, even with other partners’ contributions, the health expenditures remained well below the resources needed. The government had indicated that IDA continued supported in the health sector was critical. 46 The World Bank Health Sector Development Support ( P101160 ) The motivations provided at appraisal for the Bank’s involvement held true over the project’s implementation. Firstly, the Bank’s knowledge of the country and of the health sector were key inputs for the successful design and implementation of the project. Secondly, the Bank’s involvement and support in the PBF program increased coherence and coordination among partners contributing to a common pool of resources. Secondly, the benefits of IDA’s funding flexibility were demonstrated on multiple occasions, such as the ability to increase funds (additional financing), the change in the scope and size of the project, as well as adjustments in the project timeline, as requested by the government. Thirdly, the project did strengthen the government’s capacity to implement PBF (such as monitoring and evaluation, payment processes, etc.) and this contributed significantly to the rapid and efficient response of the government to the crisis. Finally, with increasing pressure on fiscal space for health and the reduction in external aid from partners after the crisis, IDA’s continued financial support to the health sector has become even more important. 4. Cost-benefit analysis 1.1 Comparison to cost benefit analysis at appraisal The economic evaluation conducted at appraisal was also a CBA and estimated a benefit cost ratio under two scenarios; a low case scenario and a high case scenario. Contrary to the low case scenario, the high case scenario assumed the construction of new health facilities and assumed that demand for health care would increase by 50% as consequence of the increased geographical access. However, this scenario did not materialize. Instead, the construction of health centers and hospitals virtually halted from 2011. The CBA conducted at appraisal used the Marginal Budgeting for Bottlenecks (MBB) tool to translate the expected increase in health service utilization into infant, child, and maternal lives saved. For simplicity, the CBA conducted at appraisal included only component 1A, i.e. the PBF payments to health facilities and assumed that the project would cover only provinces where there was no PBF. The MBB tool is not used anymore, and therefore it is outdated. Consequently, the CBA in the ICR could not replicate the CBA done at appraisal. Moreover, the assumptions under which the CBA was conducted at appraisal did not hold true over project implementation, making a direct comparison of the results from the CBA at appraisal and at ICR virtually impossible. Firstly, the PBF was scaled up nationwide, and coordinated among all partners involved in PBF in the country. Therefore, it was not possible to isolate the contribution of the World Bank-supported PBF payments from those of other partners. Secondly, the impact of PBF payments on MCH outcomes cannot be attributed to supply side incentives only, as boosting demand of health services and strengthening the health care system are also expected to play a significant role in the program’s impact. Thirdly, there were significant changes in the scope (extension of PBF to family planning and nutrition-related interventions) and total size of the envelope which was increased from US$ 25 million to US$ 84.8 million through two Additional Financing. While assumptions regarding the scope and size of the project could have been updated, the differences in the methodological approach made a replication of the analysis conducted at appraisal impossible. However, even though the approach used to translate increase in service utilization into number of lives saved differed, the two CBA used similar approaches to monetize the benefits in dollar value , as shown in Table 10. For example, each life saved was valued one time the per capita GDP and economic growth was assumed to be around 2%. The discount rate differed significantly (3% at appraisal versus 10.7% for the ICR), however, the CBA for the ICR included a scenario with 3% discount rate in the sensitivity analysis. The CBA conducted at appraisal estimated a benefit cost ratio for the low case scenario of 1.79. This benefit cost ratio is very close to the benefit cost ratio estimated by the CBA for the ICR (1.67) at the end of the program’s implementation. 47 The World Bank Health Sector Development Support ( P101160 ) Table 10: Key assumptions made in the CBA at appraisal and at the end of program’s implementation. CBA at appraisal CBA at end of program’ s implementation Time horizon 3 years 7 years (2010-2012) (2010-2017) Discount rate 3% 10.7% Economic growth 2% 2.24% Value of one life saved 1 * GDP p.c. 1 * GDP p.c. Reduction in maternal mortality 37.8% 29% Reduction in neonatal mortality 24.6% Na Reduction in child mortality 15.8% Na Reduction in U5MR Na 25% 1.2 Justification for CBA for ICR Economic analysis aims to assess whether the dollar benefits of a program outweigh its dollar costs. CBA allows comparing the pros and cons of policies and programs to help policymakers identify the most valuable options to pursue. Cost-benefit analysis monetizes all major benefits and all costs associated with a project so that they can be directly compared with each other. For this reason, CBA is often considered the gold standard method for evaluating programs. Conducting a CBA requires data on a variety of inputs on costs and effectiveness. Costs included PBF program costs (i.e. incentives, costs of monitoring and evaluation), costs of consumables, and other costs above the facility level. Effectiveness includes information on improvement in use of selected health services, which must be translated to the number of lives saved. An economic evaluation generally compares an intervention approach (such as PBF or another innovation) with the status quo, i.e. what would have happened without FHC/PBF. The methods used in the CBA are discussed in details below. The CBA conducted at the end of the project is taking a health sector perspective and focuses on the final beneficiaries and outcomes of interest: the reduction in maternal and child deaths. It is fair to assume that improvements made in terms of maternal and child health were achieved through all PADSS activities, including the critical role played by the Bank in strengthening the financial management systems and supporting the scale-up process, and all sources of funding for the PBF payments. In fact, the World Bank and other partners supplemented existing government’s resources to fund the payment of the FHC, which was scaled nationwide. The project’s resources, along with those of other partners’ and government’s own resources were pooled to support these strategies. Therefore, the CBA is not disaggregated by donor/funding agency or health service as it would not make sense to distinguish sources of funds or disentangle the respective contribution of a specific service. Therefore, it was not possible to isolate the impact of PADSS on the overall improvements in maternal and child health. 48 The World Bank Health Sector Development Support ( P101160 ) 1.3 Methodology There are three key outputs from a CBA analysis, which measure the value of a project: the benefit to cost ratio (BCR), the internal rate of return (IRR), and the net present value (NPV) of the project. The benefit cost ratio is defined as the ratio between the monetized benefits and the costs, both expressed in discounted present values. A benefit cost ratio greater than 1 indicates that the net present value of the project benefits outweighs the net present value of the costs and assesses the economic benefits generated for each dollar spent. Another very important output of the CBA is the IRR, i.e. the discount rate at which the NPV of all project cashflows is equal to 0. The IRR is often compared to the costs of capital to assess the project’s profitability. Finally, the NPV of the project is expected to be greater than 0 for a project to be attractive. A conservative approach was taken when conducting the CBA of the PADSS project and PBF. The details of the analysis and assumptions made for each parameter of the CBA are described in details below. 1.3.1 Assessing program costs The first step in a CBA is to estimate the costs of the project or interventions. Incremental costs for the project are used, i.e. the difference in costs between the program and business as usual. The reference is the status quo (no FHC/PBF) and the new program with the addition of FHC/PBF. As mentioned above, the decision was made to take a sectorial approach and included all costs and benefits in the CBA. The incremental costs thus measured the total costs of the FHC/PBF onto the existing health system. Costs included financial expenditures such as PBF bonuses, payment of free health careFHC, training of health workers and managers, support to verification and counter-verification process, and demand side interventions. In other words, the entire project cost and the contributions by other partners and the government were included in the analysis. The annual disbursement by the government and other partners were gathered from PBF database by the Ministry of Public Health and the fight against AIDS in Burundi, while the disbursement from the Bank (including Trust Funds) were gathered from financial reports. Overhead cost from partners were also included based on the indications provided (18%). The cost (PBF disbursement) per year and over the entire FHC/PBF were summarized in Table 11. Table 11: Total cost by year and agency (million US$). Government of World Bank Other partners Total (USD) Burundi (IDA + Trust Funds) 2010 - 1.3 - 1.3 2011 11.7 12.3 3.9 27.9 2012 9.8 14.9 6.4 31.1 2013 19.2 9.2 6.4 34.8 2014 19.2 11.2 2.1 32.5 2015 10.9 9.7 2.3 22.9 2016 11.8 10.6 8.5 30.8 2017 5.4 14.6 11.5 31.4 Total (million USD) 88 84 41 213 1.3.2 Assessing program beneficiaries The second step in the CBA is to quantify the program’s beneficiaries, in this case pregnant mothers and children under the age of five. We assumed that the number of pregnant women was the same as the number of newborns. We calculated the number of newborns using the crude birth rate over the period 49 The World Bank Health Sector Development Support ( P101160 ) 2010-2017 (43.7-41.9 per thousand)16 and applied it to annual data on the total population from the United Nations Population Division (2017). The number of children under the age of five was also derived from the United Nations Population Division (2017) for the years 2010-2015, while it was projected for the years 2016 and 2017 using the proportion of children under 5 over newborn in the past three years (47%). The total number of pregnant women and children under the age of 5 covered under the free package of maternal and child health services over the entire period of the program was 2.9 million and 6.2 million, respectively. Table 12: Beneficiary population covered with the MCH intervention package funded under the PBF/PADSS programs over time (thousand). 2011 2012 2013 2014 2015 2016 2017 Total Pregnant 395 405 416 425 435 446 456 2,988 women Children 823 851 874 896 921 943 965 6,273 under age 5 1.3.3 Assessing program effectiveness The third step in an economic analysis is to estimate effectiveness, which is here quantified in terms of lives saved. The effectiveness of health interventions is often measured as the change in utilization of health services, which is also what is usually tracked by projects through the PDO. However, measuring the effectiveness in terms of lives saved, DALYs or QALYs can allow policy makers to compare a program to other initiatives as this provides a standardized outcome measure. This requires the conversion of utilization of health services into lives saved. The underlying CBA draws from the theory of change outlined in the original PAD and the related PDO while recognizing the greater complexity of the causal chain between interventions and outcomes. The interventions supported by FHC/PBF programs ultimately aimed at reducing maternal and child mortality in Burundi. The inclusion later in 2012 of family planning and nutrition services in the PBF program was also in line with this final objective, as the reduction in fertility rates is known to be an effective way to reduce maternal and child mortality. At the same time, malnourished children are significantly more likely to die from neonatal and child health problems. To convert coverage of maternal and child services into lives saved, it is assumed that the increase in utilization of services will bring about health benefits to target populations through preventing occurrence of diseases (i.e. vaccination); effectively managing illnesses to improve the quality of life, and reducing risks of death. However, this conversion is not straightforward when more than one intervention is implemented, because: 1) changes in service utilization monitored during the program may not fully capture its benefits and 2) the benefits deriving from multiple interventions are correlated and cannot be assumed to be additive. To convert coverage of maternal and child services into lives saved, two approaches were used. First, we assumed that maternal lives saved mainly occurred through the increase in the percentage of assisted deliveries. Data from the Burundi Demographic and Health Survey (DHS) show that the percentage of assisted deliveries increased by 25 % points from 60.3% (2010) to 85.1% (2016/2017) over the program’s horizon. A similar impact was found by an impact evaluation by Bonfrer et al. (2014). We first calculated the incremental number of assisted deliveries by assuming a linear increase in the percentage of assisted deliveries over time. We then estimated the reduction in maternal mortality resulting from the increase in 16 World Development Indicators. 50 The World Bank Health Sector Development Support ( P101160 ) the number of assisted deliveries using findings by Graham (2001)17. The authors estimated a reduction in maternal deaths through the prevention of complications such as obstructed labor, eclampsia, puerperal sepsis and obstetric hemorrhage between 16 to 33 %. To be conservative in our estimates, we applied the lowest estimate (16%) to the incremental number of assisted deliveries in each year. Since purchasing began in April 2010, we assumed that the impact of the program on mortality started in 2011 and that 2008/9 were mainly focused on setting up the program. The estimated total number of maternal lives saved by the FHC/PBF program was 86,013 (Table 13). Second, we converted improvements made in most indicators related to child health into number of U5- lives saved. Improvements in health services related to child health included: the number of antenatal and postnatal care visits, fully immunized children, number of children receiving vitamin A supplementation, number of children under age five treated for moderate or severe acute malnutrition, and number of mothers living with HIV who received antiretroviral to reduce the risk of mother to child transmission. This reflects a set of health interventions that were demonstrated to effectively reduce child mortality by addressing its key drivers. Since the same child likely benefitted from multiple health services, the impact on the outcome, child mortality, cannot be attributed to distinctive health interventions. Therefore, we assumed that reduction in child death was the result of a strengthened MCH sector. However, changes in under-five mortality (U5MR) could have been driven by other factors outside the programs too. Therefore, we felt that changes in U5MR from the DHS could not be solely attributed to the FHC/PBF programs. It is important to note that it was not possible to conduct an impact evaluation of the program, since PBF was scaled up nationwide in a short time. For these reasons, we decided to use a back-of-the-envelope approach. Two main sources of information were used to translate improvements in the MCH sector into child lives saved: evidence on the percentage attribution of U5MR to diseases addressed under the PBF/PADSS programs, and the reduction in U5MR (all-cause) in developing countries over the last decade (GBD 2015 child mortality collaborators, 2016). These two metrics were used to estimate the average impact of standard MCH interventions covered under the PADSS/PBF programs on reducing U5MR. Studies estimated18 that 18% of the U5MR is attributable to pneumonia, 15% to diarrhea, 21% to nutrition-related issues, and 6% is to transmission of HIV/AIDS19. When summed up, we estimated that MCH interventions are responsible for 60% of the U5MR. When multiplied with the reduction in (all-cause) U5MR by 42% achieved in developing countries between 2000 and 201020, we got an estimated reduction in U5MR by 25% attributable to strengthened MCH sector. When applied to the U5MR in Burundi in 2010 (96 per 1,000), it resulted in an U5MR of 71.6 per thousand, which is virtually the same as the predicted U5MR for 201721. This suggests that improvements in U5MR over the last 7 years were mainly driven by improvements in the health sector. However, given the political crisis, it is plausible to assume that without PBF/PADSS programs, U5MR would have increased over the same period. Using this approach, and assuming a linear decrease over time in U5MR from 96 per thousand to 71.6, the estimated number of incremental children U-5 lives saved, compared to the scenario of no FHC/PBF programs, was 89,755 (Table 13). 17 Graham, W, Bell, J, and Bullough, C. 2001. Can skilled attendance at delivery reduce maternal mortality in developing countries?. Safe Motherhood Strategies: A Review of the Evidence. 97-129. 18 Black, R., Allen, L., Bhutta, Z., et al. 2008. Maternal and child undernutrition: global and regional exposures and health consequences. Lancet; 371: 243–60. 19 Black, R, Cousens, S, Johnson, HL, et al. 2010. Global, regional, and national causes of child mortality in 2008: a systematic analysis. The Lancet, 375: 1969-1987. 20 GBD 2015 Child Mortality Collaborators. 2016. Global, regional, national, and selected subnational levels of stillbirths, neonatal, infant, and under-5 mortality, 1980-2015: a systematic analysis for the Global Burden of Disease Study 2015. Lancet, 388: 1725- 1774. 21 UN IGME, 2017. 51 The World Bank Health Sector Development Support ( P101160 ) Table 13: Number of maternal and children U5 lives saved by the FHC/PBF programs. 2010 2011 2012 2013 2014 2015 2016 2017 Total Maternal lives - 3,134 6,436 9,893 13,504 17,266 17,684 18,096 86,013 saved Children U5 saved - 2,869 5,933 9,140 12,493 16,052 19,722 23,546 89,755 1.3.4 Key assumptions to convert the number of child and maternal lives saved into monetary value. The last step of the CBA consists in converting the estimated number of child and maternal deaths averted into (present) monetary value. To this purpose, multiple assumptions were made. Hereafter we summarized the key assumptions for the base-case scenario: • Assign a monetary value to each live saved: in the base-case scenario, each life saved was valued one time the gross domestic product (GDP) per capita in Burundi and was multiplied with the percentage of women employed when converting maternal lives saved (87%)22. This method reflects the average lost future earnings of an individual in Burundi. Some researchers are debating whether the GDP p.c. reflects future earning of the people targeted by development interventions (usually the poorest). However, as of now, no better indicator has been identified. • Time horizon of the CBA: benefits were calculated over the lifetime of children and mothers benefitting from the interventions. While people working in the formal sector may have a predefined retirement age, the poorest beneficiaries of the programs are likely to continue working until alive. Life expectancy at birth in 2015 in Burundi was 57.11 years23. Assuming an increasing life expectancy over time, it was assumed that children would start earning at the age of 15 and continue earning until the age of 60. Instead, for pregnant women we assumed that they would have stopped earning (die) at the age of 28 in the absence of the FHC/PBF programs, i.e. the median age at first birth (25 years) plus half the fertility rate (6.1 children). Therefore, the benefits of the programs for pregnant mothers reflected lost earning from age 28 to age 60. • Economic growth: the base-case scenario used a conservative assumption of economic growth equal to 2.24%, the average annual growth rate since 1997 in Sub-Saharan Africa. This rate was used because of the extended time horizon of the analysis. Projections of GDP growth in Burundi are available until the year 202224, however, the country is coming out of a deep political crisis, and projected economic growth does not reflect its potential, which showed economic growth around 4-5% until 2014. At the same time, assuming that the country would continue growing at the rate before the crisis (4-5%) over the entire horizon of the analysis is also likely to be unrealistic, and would substantially overestimate the benefits of the programs. • Discount rate: this is the rate at which benefits and costs are discounted over time. Most economic evaluations conducted so far assumed a 3% discount rate for cost and benefits from health interventions25. In the base-case scenario of this CBA we assumed a discount rate of 10.7%, Burundi’s average interest rate in the last 10 years (2007-2017), as this reflects the opportunity costs of capital investments in the country. 22 LO/FTF Council. Labour market profile. Burundi 2015. Available from http://www.ulandssekretariatet.dk/sites/default/files/uploads/public/PDF/LMP/LMP2015/lmp_burundi_2015_final_version.pdf. Visited on 11/13/2017. 23 World Development Indicators. 24 World Economic Outlook Database, 2017. 25 NICE International. 2014. Methods for Economic Evaluation Project: Final Report. Available from https://www.nice.org.uk/Media/Default/About/what-we-do/NICE-International/projects/MEEP-report.pdf. Visited on 11/15/2017. 52 The World Bank Health Sector Development Support ( P101160 ) 1.4 Results from the CBA: case-base scenario and sensitivity analyses The CBA showed that the FHC/PBF programs aiming at improving maternal and child health in Burundi brought substantial economic benefits. Under the base-case scenario, the program’s investment of nearly US$ 213 million generated economic benefits with a net present value of US$ 43.9 and an internal rate of return of 13 percent, which is higher than Burundi’s interest rate in the last 10 years (10.7 percent). The programs resulted in a benefit cost ratio of 1.67, indicating that each dollar invested generated 1.67 dollars in economic benefits over the productive lives of women and children whose lives were saved from the program’s interventions. The analysis confirmed the findings of the large body of literature regarding substantial economic benefits resulting from investments in MCH interventions, and suggested that the project had a positive economic impact for beneficiaries and the country’s economy. Due to the uncertainty in the assumptions made in the base-case scenario, principles of economic analysis indicate that sensitivity analyses need to be conducted to assess the sensitivity of the CBA results to changes in key parameters. The value of the key parameters (discount rate and GDP growth over time) were varied in sensitivity analyses and were summarized in Table 14, where the benefit-cost ratio (BCR), the internal rate of return (IRR), and the net present value of economic benefits (NPV) were reported. Because the base-case scenario used an overall conservative approach, most results from the sensitivity analysis reinforced the findings from the base-case scenario, i.e. that investing in MCH is justified on economic grounds. However, even assuming a very slow economic growth of 0.5 percent, the CBA resulted in a benefit cost ratio greater than 1 (1.30), an IRR slightly higher than 10.7 percent, and a positive NPV of 1.4 million. The other scenarios from the sensitivity analysis improved all the CBA indicators significantly. The conservative estimates from the base-case CBA as well as the sensitivity analysis pointed to positive economic benefits resulting from the PBF/PADSS programs. Table 14: Benefit cost ratio, Internal rate of return, net present value (US$ million) associated with PADSS/RFB programs. BCR IRR NPV Base-case scenario 1.67 13% 43.9 Sensitivity analysis Discount rate Alternative discount rate used in economic evaluations (6%) 3.75 13% 329.5 Typical discount rate used in economic evaluations (3%) 7.86 13% 990.6 GDP growth Projected economic growth 2018-2022 (0.5%) 1.30 11% 1.4 Average annual growth rate in the last 10 years (5.48%) 1.38 12% 25.8 BCR: benefit cost ratio. IRR: internal rate of return. NPV: Net present value. 1.5 Limitations The main limitation of the CBA is that the methodology used for the CBA at appraisal could not be replicated in the ICR, and therefore the results are not directly comparable. However, the main assumptions made in the CBA at appraisal and for the ICR were compared and discussed (Table 10). While the method to translate the increase in service utilization into lives saved differed, the economic analysis conducted at appraisal and the economic analysis conducted in the ICR used a similar approach to express the benefits into monetary value. The benefit-to-cost ratios are somewhat comparable. Due to the challenges in translating changes in health service utilization into health outcomes, the current CBA used a second-best approach to assess the impact of the programs on maternal deaths averted. However, the estimated reduction in U5MR at program’s end (2017) is very close to the estimated U5MR 53 The World Bank Health Sector Development Support ( P101160 ) for the same year. It is likely that the real benefit cost ratio of the programs was underestimated in this analysis. Firstly, the CBA used life years saved rather than QALY or DALYs, as life years represent an intuitive and easy to interpret way of measuring population health. Therefore, the benefits resulting from improved quality of life were not considered. Secondly, conservative assumptions were used for the cost benefits analysis in terms of two key parameters: economic growth and discount rate. After recovering from the current crisis, Burundi’s potential economic growth may be closer to its growth before the crisis, i.e. around 4 -5% GDP growth per year, at least for a decade or two. However, a conservative economic growth by 2.24% was assumed. Thirdly, some of the interventions will bring benefits that go beyond those included in the analysis. For example, increased use of modern contraceptives may delay childbearing and reduce fertility rates over time. In turn, delayed childbearing may increase women’s educational attainment as wells as the resources per child, ultimately improving survival rates, health, and productivity for mothers and children. Finally, the improvements in overall health system strengthening will have spill-over positive effects on other health areas, which are also not captured in the benefit analysis. This is particularly important in the case of Burundi, where weak institutions have been identified as one of the key contributors to an inefficient health system and poor health outcomes26. 2. Financial Analysis Macroeconomic situation At the time of rollout of the project, Burundi was making the transition from a post-conflict to a stable and growing economy27. The country had recorded rather high and stable economic growth since 2004 (oscillating between 4 and 5 percent), and had reached an improved fiscal position through a prudent fiscal policy, resulting in a reduction in the deficit from 5 % in 2004-2006 to 3.4 % in 2007-2014. External budget support allowed reforms in public financial management to be pushed through and these led to an increase in execution rates from 79.2 (2004-2006) to 98.3 over the period 2008-2012. The government also made a considerable effort in increasing tax revenue (although external aid was still financing half of the budget until 2015), and stabilized the percentage of domestically-funded health expenditures around 9.9% over the period 2010-2014. Total current revenue collected increased from Burundian Franc (BIF) 234.8 billion (US$ 196.5 million equivalent) in 2007-2009 to BIF 515.6 billion (US$ 351.1 million equivalent) in 2010-2014. This represented a growth rate of 119.6 % in BIF (or 78.7 % in US$). Between 2005 and 2014 Burundi registered lower growth rates than its neighboring countries but the country was close to the SSA average. In per capita terms, however, Burundi’s growth was slightly better than for SSA. Burundi also performed slightly below the low-income countries average but its growth (prior to 2015 crisis) was nevertheless one of the most stable over the decade, as shown by the low standard deviation in (per capita) GDP growth (Table 15). However, its economic growth was outpaced by population growth which explained the declining per capita GDP over time. Table 15: Regional comparison. GDP growth and growth per capita (average 2005-2014). Country name Mean growth SD growth Mean growth per capita SD growth per capita Madagascar 2.75 3.39 (0.04) 3.47 South Africa 2.89 2.05 1.47 2.13 Swaziland 2.94 1.23 1.51 1.41 Mauritius 3.69 1.19 3.39 1.21 Namibia 4.14 1.70 2.83 1.77 26 Health Sector Development Project, Project Appraisal Document, 2009. 27 Burundi Public Expenditure Review, World Bank, 2017. 54 The World Bank Health Sector Development Support ( P101160 ) Country name Mean growth SD growth Mean growth per capita SD growth per capita Burundi 4.28 0.55 2.10 0.57 Lesotho 4.38 0.99 4.13 0.99 Botswana 4.73 4.95 3.50 5.03 Malawi 5.44 2.10 2.82 2.09 Tanzania 6.02 1.10 3.67 1.23 Congo, Dem. Rep. 6.13 1.54 3.31 1.58 Zambia 6.88 1.50 4.11 1.60 Mozambique 7.03 0.92 4.33 0.83 Rwanda 7.33 1.55 5.14 1.58 Low income 4.53 7.49 2.43 8.22 Sub-Saharan Africa 4.29 7.14 1.99 7.56 Following the political election in 2010, Burundi experienced a political crisis that escalated in 2015 and brought significant instability in the country, with consequences on its fiscal performance, macroeconomic stability, and health system performance, among others. The country has faced a deterioration of the political and security situation; a further decline in donor support; a worsening in the terms of trade; a protracted period of slower growth, losses in reforms’ implementation, and social unrest. Burundi’s economy is handicapped by two main weaknesses: limited fiscal space and narrow export base making both fiscal and external positions very vulnerable. Burundi experienced a significant decrease in real GDP as consequence of the political crisis. The real GDP declined by 3.9 % in 2015 and by 0.6 % in 2016 (Figure 6). Decline in exports of goods and services and investments were the main drivers of the economy’s contraction in 2015 (PER, 2017). Estimates of economic growth over the next 5 years (2018-2022) indicate that economic growth will remain well below its potential with values oscillating between 0.15 % and 0.5% % (IMF Outlook Data, 2017). Figure 6: Economic growth before and after the political crisis in 2015 (in %). Source: Burundi Public Expenditure Review, 2017, World Bank. In addition to economic contraction, Burundi has experienced a drastic reduction in external aid and increasing current expenditure related to efforts to meet security needs in the country, undermining the gains made by the government in broadening the tax revenue base (PER, 2017). The tax revenue-to-GDP ratio declined from 12 % in 2014 to 11.4 % in 2015; while donors reduced aid from 13 % of GDP in 2013 to 9.4 % in 2015. Finally, to face the increased expenditure related to the unrest, the government increased 55 The World Bank Health Sector Development Support ( P101160 ) domestic borrowing which led to a fiscal deficit of 6.6. % of GDP in 2015 up from 3.1 % in 2010-2014. The challenges brought by the political crisis also affected the implementation and the results achieved under the PADSS/PBF programs. Service delivery data suggest that maternal and child health services were severely affected by the crisis: pre- and post-natal consultations, use of contraceptive methods, and demand for vaccines by pregnant women declined significantly in 2014 and 2015, slowing down the achievements of the PADSS/PBF programs (PER, 2017). However, the data showed that service delivery was mainly affected in the urban areas (district hospital) and in Bujumbura, the province mostly hit by the political unrest. In contrast, health service utilization increased continuously at the primary health care level, where most of the PADSS/PBF programs focused. Looking forward, the risk of unsustainability of current health financing programs remains elevated. The slow economic growth and the increasing debt ratio paired with continued high population growth are expected to further reduce the fiscal space for health, potentially jeopardizing the government’s ability to reimburse hospitals and health centers for services provided. 56 The World Bank Health Sector Development Support ( P101160 ) ANNEX 5. BORROWER COMMENTS REPUBLIQUE DU BURUNDI MINISTERE DE LA SANTE PUBLIQUE ET DE LA LUTTE CONTRE LE SIDA DIRECTION GENERALE DE LA PLANIFICATION Projet d’Appui au Développement du Secteur de la Santé (PADSS) Rapport d’Achevement I. Dates importantes Don IDA 4880 : 25 M $USD Date de signature : Le 7 Juillet 2009 Date de clôture : Le 30 Août 2014 Don TF 12526 : 14,8 M $USD Date de signature : Le 25 Juillet 2012 Date de clôture : Le 31 Août 2014 Don IDA 8080 : 25 $USD Date de signature : Le 23 Octobre 2012 Date de clôture : Le 30 Juin 2017 Don TF 13043 : 20 M $USD Date de signature : Le 10 Décembre 2012 Date de clôture : Le 30 Juin 2017 Burundi, Mai 2017 Version finale 57 The World Bank Health Sector Development Support ( P101160 ) Cigles et abréviations AEC Agence d'Encadrement Communautaire ASC Agent de Santé Communautaire ASLO Association Locale BDS Bureau du District Sanitaire BPS Bureau de la Province Sanitaire CAM Carte d'Assurance Maladie CDS Centre de Santé COSA Comité de Santé CPVV Comité de Vérification et de Validation CT-FBP Cellule Technique FBP Diriction de la Promotion de l'Hygienne et de DPSHA l'Assainissement EDS Enquête Démographique et de Santé FBP Financement Basé sur la Performance FOSA Formation Sanitaires GASC Groupement d'Agent de Santé Communautaire MSPSL Ministère de la Santé Publique et de la Lutte contre le SIDA OBS Organisations à Base Communautaire ODP Objectif de Développement du Projet PADSS Projet d'Appui au Développement du Secteur de la Santé PCA Paquet Complémentaire de Service PMA Paquet Minimum de Service PNSR Programme National de Santé de la Reproduction Programme National Intégré d'Alimentation et de la PRONIANUT Nutrition PSG Paquet de Service Gratuit PTF Partenaires Techniques et Financiers SARA Service Avalability and Redieness Assesment TPS Technicien de Promotion de la Santé UNIPROBA Unissons nous pour le Promotion des Batwa 58 The World Bank Health Sector Development Support ( P101160 ) Table des matières I. CONTEXTE ET JUSTIFICATION ....................................................................................... 60 II. RÉALISATION DE L’OBJECTIF DE DÉVELOPPEMENT DU PROJET (ODP)............................ 60 III. ANALYSE DU CADRE DES RÉSULTATS (BASELINE, CIBLES, RÉSULTATS) ........................ 63 IV. RÉALISATIONS PAR COMPOSANTE ................................................................................ 65 V. BÉNÉFICIAIRES DU PADSS ............................................................................................ 67 VI. PERFORMANCE DE LA BANQUE ET DE L’EMPRUNTEUR (GOUVERNEMENT) ..................... 67 VII. RESTRUCTURATIONS INTERVENUES.............................................................................. 67 VIII. EVALUATION DES AUTRES ASPECTS ............................................................................. 68 IX. ARRANGEMENTS POUR LA PÉRENNISATION DES RÉSULTATS ........................................ 68 X. LEÇONS APPRISES........................................................................................................ 69 59 The World Bank Health Sector Development Support ( P101160 ) I. Contexte et justification Au début de l’année 2005, les indicateurs de santé du Burundi sont au rouge : le taux de mortalité des enfants de moins d’un an est à 120 pour 1.000 naissances vivantes, le taux de mortalité des enfants de moins de 5 ans est de 176 pour 1.000 naissances vivantes et le ratio de mortalité maternelle est à 615 pour 100.000 naissances vivantes (MICS 2005). Face à cette situation et en vue d’accélérer l’atteinte des Objectifs du Millénaire pour le Développement à l’horizon 2015, le Président de la République du Burundi a décidé le 1 Mai 2006 d’instaurer la gratuité des soins pour les enfants de moins de cinq ans, les accouchements et les césariennes. En Mars 2010, ce paquet de soins gratuit a été étendu aux pathologies liées à la grossesse dès le premier mois de gestation. La mise en œuvre de cette politique de gratuité ciblée des soins a connu des contraintes liées notamment au retard de remboursement des formations sanitaires par l’Etat (4 à 6 mois), au manque d’un système de vérification, à la surcharge de travail du personnel de santé liée à une utilisation accrue des services de santé, à la démotivation du personnel de santé, à une charge administrative élevée (près de 2000 pages de fiches à remplir par formation sanitaire et par mois). Pour faire face aux difficultés observées lors de la mise en œuvre de la politique de gratuité ciblée des soins, le Ministère de la Santé Publique et de la lutte contre le SIDA et les PTF, ont pris l’option d’intégrer les deux approches et de financer le paquet gratuit des soins à travers des mécanismes de financement basé sur la performance. La stratégie de FBP combinée à la gratuité ciblée des soins adoptée par le Burundi vise à atteindre les objectifs suivants : (i) améliorer l’utilisation et la qualité des services de santé offerts à la population ; (ii) améliorer les mécanismes de vérification et de remboursement des prestations du paquet de soins gratuit en faveur de la femme enceinte et des enfants de moins de cinq ans ; (iii) motiver et stabiliser le personnel de santé ; (iv) inciter le personnel de santé à travailler dans les structures de soins périphériques ; (v) renforcer la gestion, l’autonomie et l’organisation des structures de santé ; (vi) prendre en compte le point de vue des bénéficiaires dans la gestion et la résolution des problèmes de santé. C’est dans ce contexte que le Projet d’Appui au Développement du Secteur de la Santé (PADSS) a été préparé, convenu entre la Banque Mondiale et le Gouvernement du Burundi et mise en œuvre par le MSPSL. II. Réalisation de l’Objectif de Développement du Projet (ODP) L'Objectif de Développement du Projet (ODP) est d'accroître l'utilisation du paquet défini de services de santé gratuit par les femmes enceintes et les enfants de moins de 5 ans. Le g graphique suivant montre que l’utilisation du service d’accouchement g ratuit dans les FOSA a connu une évolution positive durant toute la période du projet, passant de 56,9% en 2009 à 83% en 2016 28. 28 Billan PNSR 2016 60 The World Bank Health Sector Development Support ( P101160 ) EVOLUTION DU TAUX D'ACCOUCHEMENT 2000-2016. 90 80 70 60 50 40 30 20 10 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Le projet appuyait principalement le financement de la stratégie FBP couplé avec la gratuité des soins pour les femmes enceintes et les enfants de moins de cinq ans. Cette stratégie visait aussi l’amélioration de la qualité des services de santé offerts dans les FOSA. Le graphique suivant montre que, grâce à l’amélioration de cette qualité, les décès maternels dans les FOSA ont beaucoup diminué durant toute la période du projet sauf en 2014 et en 2015, à la veille et pendant la crise politique des élections politiques de 2015. Evolution de l’utilisation du paquet de service de santé gratuit pour les enfants de moins de 5 ans29 29 Base de données FBP 61 The World Bank Health Sector Development Support ( P101160 ) 10,000,000 9,001,673 9,000,000 8,000,000 7,311,559 7,008,637 6,896,713 7,000,000 5,914,536 6,000,000 5,000,000 4,000,000 3,000,000 2,000,000 1,643,470 1,682,913 1,723,303 1,530,600 1,567,335 1,604,951 988,965 1,000,000 0 2011 2012 2013 2014 2015 2016 Nbre consultations curatives 988,965 5,914,536 7,311,559 7,008,637 6,896,713 9,001,673 Nbre d'enfants de moins de 5 ans 1,530,600 1,567,335 1,604,951 1,643,470 1,682,913 1,723,303 Nbre moyen de vistes d'une FOSA par 0.6 3.8 4.6 4.3 4.1 5.2 enfant de 5 moins de 5 ans A partir de la généralisation du FBP en 2010, le nombre d’enfants de moins de 5 ans qui utilisent le paquet service de santé gratuit a augmenté durant toute la durée du projet excepté les années de 2014 et 2015. La situation sécuritaire en serait la cause. Le chiffre élevé des consultations curatives de 2016 est probablement dû à l’épidémie du paludisme déclaré par le Gouvernement au mois de mars 2017. 62 The World Bank Health Sector Development Support ( P101160 ) III. Analyse du cadre des résultats (Baseline, cibles, résultats) Indicateurs Baseline (2009) Cibles (Juin 2017) Résultat au Observation 31/12/2016 Indicateurs d’Objectif de Développement du Projet Taux de prévalence de la contraception (méthodes modernes) 19.80% 34,85% (Bilan PNSR 42,5%, (Bilan PNSR Selon le bilan du PNSR, la cible a été 2017) 2017) attient, mais on attend les résultats de l’enquêtre EDS 2016 pour une bonne 18% (EDS 2010) 23% (EDS 2016) comparaison avec la situation de base Taux de couverture en CPN 3 59.40 65,49% 66,5% (2015), 61,9% Selon les données de routine, la CPN3 a (Bilan PNSR 2016, baissé 2016) Nombre de femmes enceintes ayant reçu des soins prénataux 415 799 549 564 529 662 (annuaire On attends l’annuaire statistique 2016 durant leur visite dans une FOSA statistique 2015) Nombre de femmes enceintes VIH sous protocole ARV 1582 3 311 3 885 Pourcentage d’accouchements assistés par un personnel de 64.40% 86,2% 80% (accouchement Les résultats de l’enquête EDS montre que santé qualifié ; en milieu de soin, bilan la cible est presque atteinte 60% (EDS210) PNSR 201 85% (EDS 2016) Nombre d’enfants complément vaccinés 297780 359 930 328 533 La cible était déjà attaint en 2015 Nombre moyen de visites par enfant de moins de cinq ans ; 1.6 2 2, On attend les résultats de l’enquête EDS Nombre d’enfants de 6 à 59 mois ayant reçu la Vitamine A ; 1 265 275 1 664 878 1 652 052, Rapport PRONIANUT 2016 Nombre d’enfant de moins de 5 ans ayant été traité de la 87 170 83 081 58 307 Rapport PRONIANUT 2016, la cible n’a malnutrition aigüe, modérée et sévère. pas été atteint car le PAM et l’UNICEF qui appuient ce domains ont dimiué leurs zones d’intervention Indicateurs de Résultats intermédiaires Pourcentage des FOSA publiques ou confessionnelles qui offrent 50.90% 100% 96,8% On a pas atteint la cible car certaines la gratuité des soins FOSA confessionnelles ont résilié leurs 63 The World Bank Health Sector Development Support ( P101160 ) contrats à cause des retards dans le reimbursement de leurs factures. Nombre de couples année protection qui utilisent les méthodes 228 082 734 520 631204 Billan PNSR 2016, vu la situation de 2015 modernes (528029), il y a lieu d’esperer que la cible pourra être atteint fin 2017 Pourcentage des FOSA qui n’ont pas connu de rupture de stock 53.30 % 5,1% 4.9 % On attend le rapport de l’enquête SARA en médicaments essentiels dans les trois derniers mois qui ont 2017 précédé l’enquête; Pourcentage des CDS ruraux ayant un titulaire de niveau 62% 65% Cible déjà atteint minimum (infirmier A2); Nombre d’accouchements assistés par un personnel qualifié 188 135 364 090 381071 Billan PNSR 2016, cible déjà atteint 64 The World Bank Health Sector Development Support ( P101160 ) IV. Réalisations par composante Composante 1: Accroissement du financement d’un "Paquet de Services gratuits" (PSG) redéfini La sous composante1.A concernait le transfert des ressources aux Formations Sanitaires pour payer un paquet de soins gratuits dur base du système FBR. Le Projet PADSS a contribué au payement mensuel d’un paquet de prestations quantitatives relatif aux soins préventifs et curatifs au niveau des Centres de Santé, des Hôpitaux de Districts et des Hôpitaux Nationaux. Environ 24 indicateurs du niveau Centre de Santé (PMA) et 24 indicateurs du niveau Hôpital (PCA) y compris des indicateurs relatifs à la gratuité des soins pour les enfants de moins de 5 ans, les accouchements, les pathologies liées à la grossesse ont été contractualisés et payés aux formations sanitaires. En plus des indicateurs quantitatifs, le projet PADSS a contribué au payement des prestations qualitatives sous forme de bonus de qualité attribué trimestriellement aux formations sanitaires ayant un score de qualité globale supérieure à 70%. Le décaissement des fonds du PADSS était conditionné par le déboursement annuel complet des fonds du Gouvernement alloués au FBP et correspondant à au moins 1,4% du budget général de l’Etat. Suite à des difficultés de trésorerie au niveau du Gouvernement, des retards de payement des formations sanitaires ont été observés, ce qui a entrainé par voie de conséquence des retards de décaissement des fonds du PADSS, entrainant des difficultés de fonctionnement au niveau des formations sanitaires. Composante 1.B : Renforcement des capacités des acteurs communautaires pour la santé Six AEC ont été recrutées pour couvrir toutes les provinces du pays dans le renforcement des capacités des acteurs du niveau communautaire (TPS, COSA, OBC, ASC) comme prévu par le projet. Ces AEC ont parvenu à organiser les ASC en GASC autour de chaque CDS public et les ont aidés à avoir une reconnaissance juridique au niveau de l’administration locale. En outre ces AEC ont formé près de 12000 ASC pour sensibiliser la population à l’utilisation des services de santé. Selon la politique de santé 2016 - 2025, le GASC est le seul point d’entrée pour toute intervention visant l’amélioration de la santé communautaire. Pour améliorer la demande des services de santé par les groupes vulnérables dont les Batwa, une Convention de trois ans (2014-2017) a été signée entre le Ministère de la Santé Publique et de la Lutte contre le SIDA et l’UNIPROBA afin de sensibiliser les communautés Batwa à l’utilisation des services de santé et à adopter des comportements favorables à la santé. Les phénomènes d’exclusion, de stigmatisation et de discrimination ne s’observent plus dans les formations sanitaires à l’endroit de ces communautés. En effet, les membres de la communauté Batwa sont représentés dans les instances de prise de décision communautaire à savoir les COSA et les ASC. Aujourd’hui les rapports produits par l’UNIPROBA montrent que des progrès dans l’utilisation des services de santé par les Batwa est une réalité. Grâce à la convention, quelques ménages Batwa ont pu bénéficier des CAM et ont donc accès aux soins facilement. Au total 6000 ménages Batwa auront bénéficié de la CAM. Malgré cet appui remarquable du projet à l’amélioration de la santé de la communauté Batwa, on constate toujours une extrême précarité dans laquelle vivent les communautés Batwa pour les autres aspects de la vie humaine. C’est pour cette raison que des mesures tendant à la discrimination positive pour jouir des droits sociaux notamment par la suppression des frais administratifs pour obtenir les documents donnant accès aux soins de santé. 65 The World Bank Health Sector Development Support ( P101160 ) Composante 2 : Renforcement de la capacité du MSPLS et des entités impliquées dans le FBR Le MSPLS a mis en place en septembre 2009 une Cellule Technique Nationale chargée de la mise en œuvre technique et opérationnelle du FBP (CT-FBP). La Banque Mondiale a appuyé la CT-FBP en mettant à sa disposition de 2010 à 2015 un expert permanent spécialisé dans le Financement Basé sur la Performance. De plus, le Projet PADSS a assuré le payement des primes de performance des membres de la CT-FBP et a accordé un budget de fonctionnement à cette cellule. Le Projet PADSS a financé des missions d’échange d’expérience et de renforcement des capacités en matière de FBP à l’endroit des membres de la CT-FBP et des cadres du MSPLS impliqués dans la mise en œuvre du FBR. Le Projet PADSS a contribué au fonctionnement des Comités Provinciaux de Vérification et de Validation (CPVV) : (i) contribution au payement des primes de performance des vérificateurs fonctionnaires des CPVV (ii) payement des salaires d’une partie des vérificateurs contractuels des CPVV ; (iii) contribution au payement des frais de fonctionnement des CPVV ( matériel bureautique et informatique, carburant et location véhicules pour les activités de vérification, location des bureaux, connexion internet) ; (iv) contribution au payement des primes de performance des équipes de validation des CPVV. Le projet PADSS a contribué au financement des évaluations de qualité technique des Centres de Santé (réalisée par les BPS et les BDS puis par des ONGs d’accompagnement dans certaines provinces) et des Hôpitaux (réalisée par les pairs). De plus, le projet PADSS a contribué au recrutement et au payement des Associations Locales (ASLO) chargées de réaliser des évaluations au niveau communautaire. Le projet PADSS a financé des formations sur le manuel des procédures FBP (une version originale et ses versions révisées) et ses outils de mise en œuvre à l’endroit des acteurs du niveau périphérique, intermédiaire et central impliqués dans la mise en œuvre du FBP. Le projet PADSS a financé le recrutement et les prestations des ONGs d’accompagnement chargées d’appuyer et d’encadrer les acteurs de mise en œuvre du FBP dans huit provinces sanitaires. En plus, des entités externes indépendantes ont été recrutées pour la contre vérification des prestations relatives à la mise en œuvre du FBP à tous les niveaux. Le projet a donné un appui institutionnel à la DPSHA, notamment l’élaboration des documents stratégiques à savoir : (i) le document sur les orientations stratégiques de santé communautaire, (ii) le manuel des procédures de santé communautaire, (iii) le manuel intégré de formation de l’ASC, (iv) le plan stratégique de santé communautaire (2014-2017). En outre, le projet aura financé la formation de près de 900 ASC sur le manuel de l’agent de santé communautaires dans les provinces sanitaires de la Mairie de Bujumbura et de Bubanza. Les PTF ont apporté leur contribution dans la formation des ASC des autres provinces. Seules les provinces sanitaires de Ngozi, Rutana et Ruyigi n’ont pas encore trouvé de financement. La faible stabilité des responsables des entités de régulation du niveau central, intermédiaire et périphérique a un effet négatif sur l’amélioration ou au moins le maintien du niveau de permane de c es structures. 66 The World Bank Health Sector Development Support ( P101160 ) V. Bénéficiaires du PADSS Directs: - Enfants de moins de 5 ans et femmes enceintes cibles du paquet de soins gratuits ; - Les formations sanitaires publiques, confessionnelles et certaines privées (Centres de Santé, Hôpitaux de District et Hôpitaux Nationaux) ; - La communauté Batwa. Indirects: - Bureaux Provinciaux de Santé et Bureaux des Districts Sanitaires ; - Les Comités Provinciaux de Vérification et de Validation ; - La Cellule Technique Nationale FBP et l’entité de la Direction Générale des Ressources chargée du suivi des payements FBP ; - La DPSHA ; - Les ONGs d’accompagnement et de contre vérification ; - Les Agences d’Encadrement Communautaire ; - Les Associations locales chargées de la vérification communautaire ; - La Population en général ; - Les intervenants en santé communautaires (ASC et COSA) VI. Performance de la Banque et de l’Emprunteur (Gouvernement) 1°. Performance de la Banque La Banque Mondiale a honoré ses engagements de manière satisfaisante en matière de financement et d’appui technique à la mise en œuvre des activités du projet PADSS. De plus, des financements additionnels au projet initial ont été accordés en 2012. 2°. Performance de l’Emprunteur (Gouvernement) Le Gouvernement a honoré ses engagements de manière satisfaisante et a été le premier bailleur de la mise en œuvre du FBP en contribuant au financement à hauteur de 45% à 50% selon les années. Toutefois, des retards de payement aux Formations Sanitaires ont été observés suite à des difficultés de trésorerie de l’Etat, handicapant parfois le fonctionnement normal des structures de santé et retardant le décaissement des fonds du projet PADSS. De plus, des retards d’exécution de certaines activités ont été obs ervés suite à la lenteur dans le suivi des procédures. VII. Restructurations intervenues Le Projet de Développement du Secteur de la Santé (PADSS) est un projet de développement financé conjointement par la Banque Mondiale et le Trust Funds. Le projet PADSS est entré en vigueur le 30 septembre 2009 sur base des financements du don IDA H 4880 d’un montant de seize Millions huit cent mille de Droits de Tirage Spéciaux (16 800 000 DTS), évalué 67 The World Bank Health Sector Development Support ( P101160 ) à vingt Cinq Millions de dollars américains (25 000 000 USD). En 2012, le Projet a bénéficié de trois dons additionnels, qui sont : • le Don Trust Fund 12526 d’un montant de quatorze millions huit cent mille dollars américains, • le Don IDA H 8080 d’un montant de seize Millions huit cent mille de Droits de Tirage Spéciaux (16 800 000 DTS), évalué à vingt Cinq Millions de dollars américains (25 000 000 USD) • le Don Trust Fund 13043 d’un montant de vingt millions de dollars américains. Il y eu une restructuration en 2015 qui a réduit la durée de vie du Projet. Ainsi, la date de clôture est passée de décembre 2018 à juin 2017. En outre, une révision des taux de cofinancement entre les Dons IDA H 8080 et TF 13043 ainsi qu’une réallocation des fonds du Projet sont intervenues en février 2017, pour tenir compte de la dépréciation du DTS par rapport au dollar américain. Le coût total du Projet est évalué à 84,8 millions de dollars américains dont 50 millions de dollars US sur financements IDA et 34,8 millions de dollars américains sur financement Trust Fund. VIII. Evaluation des autres aspects En matière de sauvegarde environnementale, le Projet a financé la construction de 15 incinérateurs de type Montfort avec 3 fosses dont celles à placentas, à compost et à cendre dans les hôpitaux de Ngozi, Buye, Kirundo, Gashoho, Buhiga, Cankuzo, Ruyigi, Rutana, Bururi, Rutana, Muramvya, Rushubi, Cibitoke, le Centre National de Prise en Chargé des Cas de Tuberculose Multirésistante de Kibumbu ex Sanatorium, la Clinique Prince Louis Rwagasore et le CDS de Musaga. Les mêmes formations sanitaires auront bénéficié des broyeurs mécaniques montés sur des caisses métalliques à verre concassé de 3m3 à l’exception de l’Hôpital de Kirundo qui en disposait déjà. Le projet a financé dans lesdites FOSA l’équipement de protection individuelle à savoir la blouse/tablier de protection, salopettes/combinaison pantalon –veste, gants de manutention en cuir, lunettes de protection, bottes de sécurité, masques réutilisables/masque complet mono filtre/poussière, gaz et vapeur et casques protège oreilles. Il aura aussi assuré le financement de l’achat des équipements de gestion des déchets biomédicaux comprenant des balances, poubelles à pédale inox, containers, boîtes de sécurité pour seringues, brouettes polyvalentes, serviettes éponge de bain, savons de toilette, registres et pétrole. En plus le projet a financé des ateliers de formation sur les directives de gestion des déchets médicaux à l’intention des responsables et des travailleurs de 15 FOSA qui ont bénéficié de la construction des incinérateurs, de l’équipement et du matériel de gestion des déchets biomédicaux. IX. Arrangements pour la pérennisation des résultats Existence d’une ligne dans le budget du Gouvernement dédiée au FBP : depuis 2010, une ligne budgétaire allouée au FBP a été créé, ce qui est un gage d’engagement de l’Etat et un outil de plaidoyer envers les différents partenaires. Augmentation progressive de la contribution de l’Etat pour la mise en œuvre du FBP : depuis 2010, le Gouvernement s’était engagé à financer chaque année le FBP /Gratuité à hauteur de 1,4% de son budget général. Le Gouvernement s’est de nouveau engager à augmenter chaque année cette contribution de 0,1% de son budget général ; soit un financement qui va progresser de 1,5% du budget général de l’Etat en 2017 à 2% en 2022. 68 The World Bank Health Sector Development Support ( P101160 ) Financement des coûts opérationnels de la mise en œuvre du FBP : jusqu’en 2016, le budget de l’Etat était destiné uniquement au paiement des formations sanitaires. A partir de 2017, le budget de l’Etat va contribuer au financement des coûts opérationnels du FBP (location véhicule, carburant pour la vérification, matériel informatique et bureautique, …) Le FBP est une stratégie nationale inscrite dans la Politique Nationale de Santé et le Plan National de Développement Sanitaire Le FBP est intégré dans la stratégie nationale de financement de la santé qui vise à terme la mise en place de la couverture sanitaire universelle ; Pour la pérennisation des résultats en matière Santé communautaire et sauvegarde environnementale Le modèle d’incinérateur type Montfort et accessoires, broyeurs mécaniques montés sur les caisses métalliques est une technologie appropriée car le coût de fonctionnement est abordable par les FOSA, ce qui garantit sa pérennité d’utilisation. Cependant, l’équipement de prot ection individuelle, le matériel et équipement de gestion des déchets biomédicaux ne sont pas disponibles sur le marché local. Pour garantir la fonctionnalité des GASC mise en place par le projet, le MSPLS avec l’appui des PFT va initier bientôt le FBP communautaire comme outils de coordination et de motivation de l’ASC. L’implication permanente des membres de la communauté de Batwa dans les corps des ASC et COSA par cooptation est consignée dans le manuel des procédures de la santé communautaire adopté par le MSPLS. X. Leçons apprises FBP couplé avec la Gratuité ✓ Le conditionnement du décaissement des fonds du PADSS au payement complet des fonds prévus par le Gouvernement a entrainé des retards de payement aux formations sanitaires en cas de difficultés de trésorerie chez ce dernier. Toutefois, cette conditionnalité a permis au Gouvernement de s’acquitter de ses engagements. Pour les projets à venir, il faudrait lever cette conditionnalité au décaissement mais insérer dans les conventions le niveau minimum de contribution annuelle de l’Etat et s’assurer que ce dernier a honoré ses engagements à la fin de chaque année. ✓ Le transfert direct des fonds sur les comptes bancaires des formations sanitaires sans aucun autre intermédiaire leur permet de disposer des fonds en temps utile pour s’approvisionner notamment en médicaments et autres intrants, mais aussi réaliser des investissements visant l’amélioration de l’offre des soins. Santé communautaire et sauvegarde environnementale ✓ Dans les premiers contrats avec les AEC, il est arrivé qu’une AEC couvre 8 provinces sanitaires à la fois. Lors des missions de supervision de la DPSHA et de l’évaluation externe des prestations des 69 The World Bank Health Sector Development Support ( P101160 ) AEC, le constat a été que cette AEC a été la moins perfor mante. Ce qui a été à l’origine de la réattribution des provinces sanitaire lors du deuxième contrat en veillant à ce qu’une AEC ne dépasse pas 4 PS. ✓ Les missions d’appui à la mise en œuvre organisées par les équipes de la Banque Mondiale ont été déterminantes pour le développement de la santé communautaire et sauvegarde. En effet, les recommandations pertinentes consignées dans les aides mémoires ont activé les autorités du MSPLS à prendre des décisions allant dans le sens de mettre en œuvre des activité s en difficulté. Passation des marchés Les rapports des sous-commissions d’analyse des offres ou des propositions ont toujours été validés par la PRPM déléguée en dépit de la recommandation des aide mémoires de mettre en place la commission de passation des marchés, qui par ailleurs prévue par la loi. En outre, les délais surtout de mise en place des commissions ou de traitement d’autres étapes de passation des marchés ont été parfois longs sans pouvoir établir des responsabilités. Pour garantir une passation des marchés plus transparente, une commission de passation des marchés doit être mise en place. Le chapitre sur la passation des marchés, de Sélection et d’Emploi des consultants des emprunteurs de la Banque Mondiale devrait prévoir la mise en place de cette commission. Aussi, il est souhaitable que les délais de traitement des dossiers de passation des marchés par niveau soient mentionnés dans les manuels d’exécution des projets. 70 The World Bank Health Sector Development Support ( P101160 ) ANNEX 6. SUPPORTING DOCUMENTS 1. Barakamfitiye Tharcisse. Evaluation des activités réalisées par l’UNIPROBA dans le cadre de la mise en œuvre du Plan des Peuples Autochtones (Batwa) durant la période, June 2017; 2. DHS. Enquête Démographique et de Santé : République du Burundi, 1987; 3. DHS. Enquête Démographique et de Santé : République du Burundi, 2010; 4. DHS. Troisième Enquête Démographique et de Santé (EDSB-III) 2016-2017. Indicateurs Clés : République du Burundi, Mai 2017; 5. Bregmans Consulting and Research. Rapport de Contre Vérification des données quantitatives et qualitatives pour le financement base sur la performance Second Semestre 2015, Aout 2016; 6. IMF. Joint IDA-IMF Staff Advisory Note on the Poverty Reduction Strategy Paper; First Progress Report, January 6, 2008; 7. IMF. Burundi PRSP, February 2007; 8. IMF. Burundi PRSP – Annual Progress, March 2009; 9. ISTEEBU. Enquête ménages de base pour le suivi et l’évaluation de l’impact de l’appui au système de remboursement du Paquet Minimum des Services de Santé (PMS 2009), Avril 2010; 10. ISTEEBU. Enquête ménages de base pour le suivi et l’évaluation de l’impact de l’appui au système de remboursement du Paquet Minimum des Services de Santé (PMS Edition 2012), Janvier 2014; 11. ISTEEBU. Evaluation des besoins en matière de soins obstétricaux et néonatals d’urgence au Burundi « EB-SONUB », Mars 2011; 12. MSPLS Cellule Technique Nationale-PBF, Burundi. Rapports Annuels de Mise en Œuvre du Financement Base sur la Performance et la Gratuité des Soins (2011, 2012, 2013, 2014, 2015, 2016); 13. MSPLS. Politique Nationale de Contractualisation dans le secteur de la sante au Burundi, 19 Décembre 2006; 14. MSPLS. Les outils de mise en œuvre du Financement base sur la performance, Janvier 2017; 15. MSPLS. Manuel de procédures pour la mise en œuvre du Financement base sur la performance - Seconde Génération, Version Révisée 3, Janvier 2017; 16. MSPLS. Déclaration de Consensus Stratégique sur le Financement de la Gratuite et le Financement base sur la Performance, 16-17 Mars 2009; 17. MSPLS. Evaluation Rapide/Restreinte sur les Formations Sanitaires du Burundi – Rapport Provisoire, Mai 2010; 18. MSPLS. Evaluation de la qualité des Prestations dans les Formations Sanitaires du Burundi (2eme Edition), Janvier 2014; 19. MSPLS. Enquête d’Evaluation de la disponibilité et de la capacité opérationnelle des services de santé, (SARA), Edition 2017; 20. MSPLS. Enquête de Couverture Vaccinale de Routine et après une Campagne de Suivi de Vaccination contre la Rougeole couplée à l’administration de la Vitamine A, de l’Albendazole et du Praziquantel (ENCV 2012) -Rapport Définitif, Avril 2013; 21. MSPLS. Rapport Annuel sur les Activités de Sauvegarde et de Sante Communautaire des Projets financés par la Banque Mondiale - Exercice 2016, Mars 2016; 22. MSPLS. Annuaire Statistique des Données des Centres de Santé et des Hôpitaux. Années 2009,2010,2011,2012,2013,2014, 2015 et 2016; 23. MSPLS. Rapport d’Evaluation du Système d’Information Sanitaire de Routine par l’Approche et les Outils PRISM. Direction Générale de la Planification; 24. République du Burundi. Cadre Stratégique de Lutte contre la Pauvreté – CSLP, Septembre 2006; 25. World Bank. International Development Association and International Finance Corporation: Country 71 The World Bank Health Sector Development Support ( P101160 ) Assistance Strategy for the Republic of Burundi, September 18, 2012; 26. World Bank. Burundi Poverty Assessment, June 2016; 27. World Bank, Public Expenditure Review for Health, 2017. 72 The World Bank Health Sector Development Support ( P101160 ) ANNEX 7: REVISED RATINGS OF PROJECT PERFORMANCE IN ISRS This table corrects the actual disbursements linked to each ISR because, in the table shown in the Data Sheet, Trust Fund amounts were mistakenly left out of the disbursement amounts. No. Date ISR DO Rating IP Rating Actual Archived Disbursements (US$M) 1 22-Oct-2009 Satisfactory Satisfactory 0.46 2 07-Dec-2009 Satisfactory Satisfactory 0.46 3 18-Jun-2010 Satisfactory Moderately Satisfactory 0.85 4 12-Mar-2011 Satisfactory Moderately Satisfactory 6.60 5 12-Oct-2011 Highly Satisfactory Moderately Satisfactory 9.43 6 03-Mar-2012 Highly Satisfactory Satisfactory 12.83 7 28-Oct-2012 Highly Satisfactory Satisfactory 14.83 8 25-May-2013 Satisfactory Satisfactory 30.16 9 05-Oct-2013 Highly Satisfactory Satisfactory 31.37 10 26-Mar-2014 Highly Satisfactory Moderately Satisfactory 38.86 11 08-Nov-2014 Satisfactory Moderately Satisfactory 42.48 12 12-May-2015 Satisfactory Moderately Satisfactory 48.58 13 20-Nov-2015 Satisfactory Moderately Satisfactory 52.82 14 12-May-2016 Satisfactory Moderately Satisfactory 59.30 15 11-Nov-2016 Satisfactory Moderately Satisfactory 65.65 16 30-Jun-2017 Satisfactory Satisfactory 82.81 73 The World Bank Health Sector Development Support ( P101160 ) ANNEX 8: ACHIEVEMENT RATE FOR SELECTED INDICATORS ON THE BATWA POPULATION The achievement rate for the below selected indicators on the Batwa population were monitored as part of the Indigenous Peoples Plan (IPP). Indicator Target Actual % reached Comments Number of Batwas in COSAs30 311 557 179 Target met Number of Batwas in ASCs 311 851 274 Target met and surpassed Number of Batwa children under five who benefited from the FHC Program 12,000 11,368 88 Target not met, but on the right track Number of Batwa women who have been sensitized/trained on health issues and on proper health behavior Target almost met 21,217 20,750 98 Number of Batwa households using mosquito nets 7,843 7637 97 Target almost met Source : « Evaluation des activités réalisées par l’UNIPROBA dans le cadre de la mise en œuvre du Plan des Peuples Autochtones (Batwa) durant la période » by Tharcisse Barakamfitiye, June 2017. 30The first two figures in the column labelled “Actual” being the latest available are higher than what was reported earlier in the UNIPROBA document which shows that only 314 Batwas were members of the COSAs and 424 of the ASCs. 74 The World Bank Health Sector Development Support ( P101160 ) ANNEX 9. OVERVIEW OF BURUNDI PERFORMANCE-BASED FINANCING31 Working definition of Performance-Based Financing (PBF)32: “Performance-Based Financing is a health systems approach with an orientation on results defined as quantity and quality of service outputs. This approach entails making health facilities autonomous agencies that work for the benefit of health- related goals and their staff. It is also characterized by multiple performance frameworks for the regulatory functions, the performance purchasing agency and community empowerment. Performance-Based Financing applies market forces but seeks to correct market failures to attain health gains. PBF at the same time aims at cost-containment and a sustainable mix of revenues from cost-recovery, government and international contributions. PBF is a flexible approach that continuously seeks to improve through empirical research and rigorous impact evaluations which lead to best practices (see footnote).33” Background Burundi is a small country in central Africa, with a 2008 population of 8 million people, and a population density of 311 per square kilometer; the second highest in Sub-Saharan Africa. The annual population growth rate is 2.9%. Since independence in 1962 it has had frequent internal political upheaval with the last war ending around 2007/2008. Protracted war led to brain drain and the collapse of the economy; the 2007 GDP was $110 per capita, one of the lowest in the world, with a human development index ranking it as 174th from 182 countries (2007). Health indicators are also dismal with a child mortality of 176/1,000 and a Maternal Mortality Ratio estimated at 615/100,000. Household surveys document poor quality general health services, poor household hygiene and inadequate living conditions and a high unmet need for family planning (Cordaid, 2009). In 2006, the President of the Republic declared a policy of free health care for pregnant women and children less than five years of age. During that year, Performance-Based Financing (PBF) pilot programs were introduced in two provinces (Bubanza and Cankuzo) by the international Non-Governmental Organization (NGO) Cordaid with financing from the Dutch government and the European Union. A second NGO, Health Net International-TPO and the Swiss Development Cooperation started pilot projects in Gitega and Ngozi provinces. Due to positive results and funded by the European Union, these pilot experiments were extended to 6 new Provinces: Ruyigi, Rutana, Makamba, Bururi, Karusi and Ngozi. By December 2009, 9 out of 17 provinces had ongoing pilot PBF projects. The health sector, although underfunded, is getting more attention from the GOB; between 2005 and 2009, the GOB increased the share from its national budget dedicated to the health sector from 2% to 5%. An evaluation in 2009 showed better performances in the Provinces under PBF, hence the MOH and development partners decided to scale up from 2010 onwards. On 1 April, 2010, a national PBF system 31 This annex is taken from a case study developed by G. Fritsche (WB) (Burundi PBF Drill-Down), October 2014 32 As discussed on the PBF google groups forum, a discussion forum of the African PBF Community of Practice, final consensus working definition as of 17th August 2010. 33 PBF draws from micro-economic, systems analysis, public choice and new institutional economics theories. The effectiveness can be enhanced by demand-side interventions such as equity funds; conditional cash transfer programs, vouchers schemes and obligatory health insurance programs. 75 The World Bank Health Sector Development Support ( P101160 ) was launched, a provider payment reform initiative which merged selective free health care with PBF. Health Finance Total overseas development aid was $62.5 per capita per year (2008). The 2007 national health accounts showed a total health expenditure of $17.4 per capita per year. Households and donors contribute together around 40% (37% out of pocket) while the government finances 17% (of which 6% are through funds from the highly indebted poor country initiative). The public health sector -which encompasses public and faith-based institutions - receives about 50% of these funds (21% for the district hospitals; 24% for the health centers and 5% for the tertiary hospitals) with the remaining going to the private sector. Most funds (40%) to the public sector are used for outpatient curative care while preventive and public health services use 22%. Access to health care was poor: it was estimated that, in 2005, 17% of patients did not have access to care and 81% had to incur debts when accessing care (MOH, 2009). The private for profit sector is small and predominantly located in the capital and the larger urban centers. Providers were reimbursed for selective free health care expenses. However, providers had to incur high transaction costs as supportive documents such as photocopies of the original bills had to be sent to the central MOH. Also, the reimbursements were irregular, incomplete and inequitable (MOH, 2010). For instance, tertiary hospitals in the capital consumed large parts of the free health care budget due to high reimbursements for caesarian sections. Also, clients were bypassing primary health centers to access the tertiary level in the capital, leading to overcrowding, a lower quality of care and high costs to the GOB. In addition, reimbursements were versed in the ‘drug account’ and facilities were not authorized to use these resources for other than drugs expenses, leading to situations of large unutilized resources pooled in such accounts. The selective free health care policy has had some notable successes such as increasing the portion of women delivering in health facilities from 22.9% in 2005 to 56.3% in 2008 (MOH, 2010). Input funding was highly inequitable: a 2007 WB health financing study documented a factor 60 difference between the best financed, and the worst financed province (Mwaro $0.1/c/yr against Ruyigi $6/c/yr). Apart from inequity in input funding, output funding proved also not equitable. A mapping of PBF and free health care inflows for 2009, showed large inequities in resource flows. While the average inflow through output financing was $1.96/c/yr, the difference between the best and the worst financed province was a factor 16 (Buja Marie $0.34/c/yr against $4.45/c/yr in Makamba). It appeared that the PBF provinces were capturing a relatively large part of the free health care budget due to a better output (quantity and quality wise). Established PBF provinces captured on average $1.39/c/yr free health care funds while non-PBF provinces $0.81/c/yr attracted of these funds. The three tertiary hospitals were gaining $2.87/c/yr from the available free health care funds. 76 The World Bank Health Sector Development Support ( P101160 ) Figure 7: a thematic mapping of output financing for 2009, combining PBF and free health care reimbursements in USD per capita per year 7 shows for 2009, using thematic mapping, the estimated combined output budget for PBF and free health care reimbursements. Large inequalities in output financing are obvious. In the new combined PBF/selective free health care system, all available PBF monies for three years were pooled virtually (an estimated $100m); this consisted of IDA; European Commission and the government’s free health care budget. This translates to about $3.55/c/yr PBF budget. This budget estimate is conservative, other sources will be added. For instance, the budget of WB-MAP will be used to purchase HIV services. To ‘protect’ the provinces that were worst off, and to advance the notion of horizontal equity,34 a system of equity adjustments was introduced. The available output budget was allocated to provinces based on five criteria (inhabitants/health center; population density; number of health workers per inhabitant; poverty score and travel time to the capital). The worst-off province receiving 42% more output budget than the best-off province ($2.07/c/yr for Buja Mairie versus $2.97/c/yr for Ruyigi). Provincial PBF output budgets were ring fenced, while within each province the share going to health center and community against first level referral hospitals was 2:1. A set of 24 services are purchased at the health center level, and a set of 24 services at the hospital level (Figure 8 and Figure 9). Services purchased are predominantly those that are known to be of importance for reaching health related MDGs. These are mostly preventative services, and include HIV services at both levels. The complementary package at the first level referral hospital was, at the request of the MOH, also applied for the three tertiary hospitals in the capital. This is an issue which is under review. 34“Horizontal equity means providing equal healthcare to those who are the same in a relevant respect (such as having the same 'need'), A.J. Culyer (1995).”Need — the Idea Won’t So — but We Still Need It" Social Science and Medicine, 40, pp. 727– 730 77 The World Bank Health Sector Development Support ( P101160 ) Figure 8: PBF Basic Package of Health Services Note: US$ 1 = FBU 1,235 (Sept 2010). The unit fees are valid for this specific health center, for the second quarter of 2010. Figure 9: PBF Complementary Package of Health Services Note: US$ 1 = FBU 1,235 (Sept 2010). Unit fees are valid for this hospital, for the second quarter of 2010. 78 The World Bank Health Sector Development Support ( P101160 ) Provider Payment Staff working in health facilities is either a public employee or a contract worker. A large faith-based organization (FBO) service delivery network exists, although they are considered part of the public health system. MOH staff also work in FBO health facilities, but there are less such staff than in ‘pure’ MOH clinics. The private health sector outside the major urban areas is not large, bar in some rural areas along the Tanganyika Lake (Rumonge, Nyanza Lac and Makamba). Health facilities receive subsidies from the state through salaries, minor operating costs and for drugs (at the hospital level, but not at the health center level). Since 2006, health facilities are reimbursed for free health care delivered to pregnant women and children less than five years of age. Contract workers are paid through income generated from out-of-pocket expenses, and from PBF in those provinces where PBF was operational. On 1 April 2010, a provider payment reform initiative combining free health care funds and PBF funds was launched nationwide. Providers sign purchase contracts with semi-autonomous purchasing bodies (‘Comite Provincial de Vérification et Validation’ (CPVV)), in which three monthly an amendment stipulates the new fees for a defined set of services (24 for the basic package of health services, and 24 for the complementary package of health services). The purchasable services have been standardized nationwide; however, the unit fees differ. Each CPVV has been provided a certain annual global prospective budget, which consists of two parts which are each ‘ring fenced’; one for its health centers, the other for its hospitals. The budget for health centers is ring-fenced at about 2/3 of the total provincial output budget. For both levels, a set of five unit fees has been provided to ensure those facilities which are, for example more remote or underperforming, attain higher fees for each service provided. The CPVVs are authorized to determine which health facilities are in each of the categories ‘0%-10%-20%-30%-40%’, in which the middle ‘unit fee set’ of 20% corresponds to a forecasting based on available budget.35 The CPVV’s can therefore ‘push’ more of the output budget to those health facilities within their province which they consider to be the most destitute. The CPVV’s have been trained to apply an Excel based forecasting tool to follow population based targets. A web-enabled application has been designed which allows the CPVVs to enter performance data for their facilities; this will enable them to follow budget consumption and results achieved. The CPVV’s will be held accountable for the level of budget disbursement (they need to disburse available output budget to its maximum possible extent) while they have the authority to change unit fees depending on results achieved. For instance: they can lower or increase unit fees in the purchase contracts, based on results achieved. These results can be of a positive nature such as reaching performance targets, or can be a lack of results in certain areas, possibly due to moral hazard or insufficient effort. Insufficient effort is dealt with through a combination of increased incentives and explicit negotiations on certain targets (‘business plan’ see below). The quarterly amendment to the purchase contract is integrally linked to a ‘business plan’ in which providers must indicate what resources they would commit, and which strategies and level of effort they propose to reach the proposed results. The purchase contracts are negotiated, and ‘contestable’ in the sense that there is a real pressure for following through on the deliverables. There remains a possibility 35The entire budget has been modeled into the fee categories ‘0%-80%’ and each of the 17 provinces have thereafter been put into one of five ‘fee-set’ categories. Each CPVV has been provided a set of five fees (which cover a range of 5-10 % increases). The forecasting model is available from the author, for those who wish to study the model more closely. 79 The World Bank Health Sector Development Support ( P101160 ) to lose the main contract, which can be a signal for the public health administration to take managerial action (an action as for instance replacing the head of the health center).36 Qualitative elements linked to these services are an explicit part of the business plan. Providers are paid through a ‘carrot and carrot’ method: on top of the earnings through pre -established fee schedules for defined services, they can earn up to 25% more each quarter, if they score 100% through the quality checklist and if they score 100% on the community client satisfaction surveys. The community client surveys are weighted 40% on the quality score (see below on the ‘regulation knob’). Providers can use up to 30% of the health facility income (from all sources; out-of-pocket; insurance; PBF etc.) for bonus payments to their staff.37 A special excel tool called the ‘indice’ has been created for this purpose, and providers have been trained in its use. Technical assistance to the CPVVs is provided through embedded technical assistants, and through the central technical support unit (Cellule Technique de mise en oeuvre du FBP (CT-FBP)).38 A web-enabled database will allow the CT-FBP to map the share of each payer to each province. This enables the CT-FBP to maximize and to coordinate the use of all available PBF output budgets in the entire country. Pre- formatted payment orders are extracted and sent to the payer. For instance, the NGO Cordaid is sent its share of the output bills for the provinces it works; the remainder is paid by the CT-FBP/MOH. As a further example: the WB-MAP is sent, directly, all HIV services from the basic and complementary package, from the entire country, for payment. Payments flow directly from the fund holders to the health facility bank accounts. Transparency and accountability are assured; all CPVV’s, the CT-FBP and payers have access to the performance data for the entire country, including the financial data and information related to payments. The GOB has thus created an internal/quasi market for Performance-Based Financing. Decentralized strategic purchasing for important health related MDG services has been made operational. 36 Heads of health facilities are Government employees. 37 On condition that the health facility has on its bank account the equivalent of three months’ worth of operating costs. 38 In addition, the NGOs Cordaid, HealthNet-TPO and Pathfinder International signed contract with the Burundi MOH to provide technical assistance to those provinces that did not have previous PBF pilots. 80 The World Bank Health Sector Development Support ( P101160 ) From 2011 onwards, it was found that services benefiting from the free-health care policy have experienced very high overall growth, leading to an increase in costs which were much higher than those estimated at the outset, thus leading to a risk of overspending. Strategies to control these costs are essential to rationalize the system, making it as efficient as possible. Multiple discussions within the CT-FBP Extended Team were then initiated, leading to the following actions: 1. Downward revision of applied tariffs for nearly all services, subject to upward revision in the event of additional resources. Moreover, the existence of contracts between the SEP / CNLS and certain health units in the framework of the fight against HIV / AIDS entailed a double financing for the same services paid at the same time within the PBF framework. Thus, it was recommended that this SEP / CNLS funding be channeled through PBF to (1) avoid double contracting in the same sector (2) to prioritize verification of benefits essential before payment and thus (3) ensuring the rational management of available resources. 2. Removal of some services targeted through PBF, mainly those that were problematic relative to (1) the provider level at which they are paid (2) the reliability of their audit. For example, services such as the construction of new latrines (which is a Community indicator and for which the health facility plays but a minor role and the distribution of mosquito nets (also a Community service, insofar as the most important aspect to be stimulated would be more use than the mere distribution of mosquito nets). 3. Determination and adherence to maximum envelopes per province: for each province. Considering targets by health facility and available resources, an annual envelope should be set and use in the Database ensuring that the resource envelope is not exceeded. Annual budget would be fixed by province, the (target) objectives may vary from one period to another, the maxima of the budgets could also vary from one period to another, as the health facility would share a fixed envelope per set objectives and the trend of the level of performance of each, which would gradually lead to an enhanced effectiveness of the provincial budget management by the provincial purchaser (the CPVV). Thus, a scale of values for remuneration of benefits based on targets would be established. By this method, the cost of PBF would no longer be increased uncontrolled, parallel to the growth in the use of services, but would follow a development compatible with the desired efficiency of the health system (achievement of health or service objectives) and with available resources (strategic purchasing). Prevention and correction of abuses of free care: it is becoming increasingly clear that the population tends to take advantage of the benefits granted by the free health care policy, especially since the control of eligibility criteria was not strictly practiced. On the other hand, the providers did not have an interest in reducing the number of beneficiaries, rightly or wrongly, because it generated profits. Therefore it was necessary to take corrective actions by the Ministry of Public Health and the Fight against AIDS, in particular: (1) multiplying and disseminating regulatory texts (presidential decrees and ministerial ordinances) (2) organize briefings and formalize the application of these texts and communicate to the general public on compliance with the provisions of the regulatory texts through the media (3) to designate a focal point in each health facility to control the eligibility for free health care. Given the temptation of the beneficiary population to benefit from free health services and the lack of interest of the providers to deter this, it has been proposed that a public official be seconded to the health facilities to ensure regular monitoring of eligibility for free health care. This person would be under a performance contract (modality to be determined) and would report to a higher level administrative structure (CPVV or BPS), (4) reinforce the audit through community client satisfaction surveys and third party counter-verification, (5) put in place a health logbook system and establish a functional counter-reference system to counter the non-respect of referral pathways which overloads hospitals with cases appropriate for a lower level, which increases contributes to the escalating costs of health services. Technical Notes on Strategic Purchasing 2011-2015 (Rigobert Mpendwanzi) 81 The World Bank Health Sector Development Support ( P101160 ) Organization The below figure depicts the set-up of the Burundi PBF administrative system. The institutional structure of the Burundi PBF system is circumscribed by a set of eleven contracts.39 1. Service contract between the MOH and the CPVV (delegation of power) 2. Service contract between the MOH and the Provincial Health Office (BPS) 3. Service contract between the MOH and the District Health Office (BDS) 4. Service contract between the MOH and the tertiary hospital 5. Service contract between the MOH and the central MOH departments 6. Service contract between the MOH and the CT-FBP (PBF central technical support unit) 7. Service contract between the MOH and the paramedical training institutions 8. Purchase contract between the CPVV and the health facility (health center or district hospital) 9. Purchase contract between the health facility and a second-tier provider (whether non-for-profit, or for-profit) 10. ‘Motivation’ contract between the health facility and the individual health worker 11. Contract between CPVV and grass root organization (for community client satisfaction surveys) Figure 10: set-up of the Burundi PBF administrative system Other sources of funding – Government sources of World Bank – Cordaid – funding – possibly Ministry bilateral – Global Fund, etc of Finance in collaboration with MOH $$ $$ National PBF steering committee – coordination role Fund holder Fund holder Regulator at Ministry of Health level: setting standards, Selection and monitoring of Financial department –payment of supervision, coaching, controlling purchaser in coordination with PBF invoices after verification by fund holder donor or government agencies purchaser and steering committee, auditing $$ District or provincial PBF steering committee – coordination role District or provincial level Regulator at provincial and purchaser – insurance district levels – quality company, church group or assurance, coaching, NGO: contract development, training, etc Autonomous Health verification, coaching facilities Community empowerment: Local NGO – verification at household level – under contract with purcahser Patients – demand and purchase health services – user fees 39The manual is available -from the author- in French. The most significant performance frameworks are available in English; a translation of the manual in English is being worked upon. 82 The World Bank Health Sector Development Support ( P101160 ) The rules and regulations of the new PBF system are well described in the above contracts (MOH, 2010). Provincial Verification and Validation Committees (CPVVs) have been established. Their staff is drawn from a mix of Provincial Health Office (BPS) staff, and contracted technicians.40 Their office is in the BPS. This CPVV has an impressive set of tasks. These tasks are basically drawn from the NGO ‘fund holder model’ (Soeters et al, 2006; Soeters et al 2010), while the CPVV is not managed by an NGO, but by an incentivized provincial health administration (see section on ‘regulation’).41 A few tasks have changed though, as compared with the NGO fund holder model. Firstly, and importantly: the CPVV does not physically manage the payment function. This payment function has been delegated to the various payers. However, the CPVV has retained the ‘purchaser role’; it has been given the authority to negotiate and sign the purchase contracts. The CPVV also verifies the reported production of services. This verification, a monthly task, consists of CPVV staff visiting the health facilities and verifying the reported production in the various registers. They essentially re-count the reported production, and triangulate the figures with the figures for these same services reported in the monthly HMIS report, the national health management information system, at the source. The regulatory role related to quality has been conferred to the Provincial Health Office and the District Health Offices. This function ought to be carried out by a different team than the ones that are part of the CPVV, which is done to prevent or decrease potential conflict of interest situations. The quality role is done quarterly, and in collaboration with the District Health Office staff. Each quarter, each health facility is subject to a quality checklist. Its results impact on the service volume payments. The maximum amount health facilities can earn as bonus payment on top of their volume payments is 25% of achieved earnings. A ‘motivation contract’ stipulates the tasks of a health worker, and is written between the health facility management and the employee. It indicates which ‘share’ of the ‘bonus budget’ allocated by the health facility, the employee is entitled to get on top of her regular monthly take home salary, if she performs per requirements. This is an important tool for the management to manage its human resources. A so-called ‘indice’ which can be used as a paper-based or an excel-based tool, is then used to share the ‘bonus budget’ among the health workers. Payment for performance is monthly, so health workers earn a regular significant variable (depending on the overall health facility results and on their personal role- adherence) bonus payment on top of their salaries. Management has an incentive to use existing human resources efficiently, to ensure sufficient earnings; however, they have to negotiate target based deliverables through their ‘business plans’. The ‘indice tool’ is an Excel-based tool, which enables the management to manage all resources, income and expenditures, including income from PBF, in a holistic fashion. The GOB has effectively created an internal/quasi market in which a package of basic and complementary services can be purchased by a multitude of purchasers. There has been a fair amount of attention paid to the development of the indicators (see the topic area of ‘managing change’), which also capture HIV and STD related services. 40 The CPVV is composed of two sub units, one that ensures the verification and making the second validation. The first is itself composed of auditors officials and contractors, while under validation unit is composed of members from BPS, BDS, NGOs and civil society in the province 41 This is a novel arrangement, and the institutional arrangements are monitored closely. 83 The World Bank Health Sector Development Support ( P101160 ) In the dominant PBF pilot project, which informed the national model, a separation of functions was instituted. This was a separation of functions between the purchaser/fund holder (an independent institution), the regulator and the provider. In the national model a semi-autonomous body was created which has taken over all the original fund holder functions, except physically holding the funds (see ‘managing change’). NGO and bilateral agency technical assistants have been mobilized to assist the GOB to roll-out and to make operational the national PBF model. Conceptually this can therefore be described as a ‘contracting- in’ situation, in which technical assistants assist the Government to implement a program.42 Increased autonomy is an important pillar of the Burundian PBF system. Health managers can manage resources in an integral fashion. Although still a contentious issue, when drugs or medical supplies run out in the provincial medical store drug revolving system, health managers do purchase items from the private sector. In fact, health facilities are judged through the quality checklist on a series of performance measures related to drug management, among which the availability of tracer drugs. Managers will therefore ensure using whatever means to their disposition, to score as high as possible on the quality checklists, and therefore, will purchase missing drugs from the private sector if need be.43 CPVVs also select and contract grassroots organizations GROs, and train select members in community client satisfaction surveys. The GROs are paid on a performance base: per client found and correctly interviewed in the community. Apart from verifying their existence (to detract ‘phantom patients’), clients are also asked a host of questions related to the level of out-of-pocket payments and satisfaction with services received. These results are converted into a quantitative score, and contribute each quarter to 40% of the composite quality score of the health facility. Providers can sign sub- contracts with other providers, both non-for-profit and for-profit, for certain parts of the health packages that they have been contracted to provide. Increased health facility autonomy, which allows management to manage resources in a holistic fashion, is part of the reform. This approach in fact incorporates the private for profit sector in the public health system, which allows leverage on quality assurance in the private sector. This is a win-win situation as the private sector will benefit from the subsidies. The PBF central technical support unit in the central MOH, the ‘cellule technique nationale’ (CT-FBP) is staffed with dedicated bureaucrats, and contracted workers. Technical assistants are added progressively by various partners (WB; EC; Cordaid; Belgium). This CT-FBP has an important role in the oversight and coordination of the system. It recently created a ‘CT-FBP élargie’, which is modeled on Rwanda’s ‘extended team’ mechanism. This coordination mechanism is created to bridge the gap between policy and implementation. Finally, the CT-FBP is also the secretariat of the national PBF steering committee (‘plateforme nationale’),44 42 The MLSP contracted these NGOs to assist it in implementing its program. Funding was through IDA. 43 Price ought also to play a role; under conditions of comparable quality, managers should not be obliged to buy from the public distribution center if the public distribution center is more expensive. 44 This PN has meanwhile been dissolved in the CPSD = Consultation Framework Partner for Health and Development to limit 84 The World Bank Health Sector Development Support ( P101160 ) a higher-level policy and strategy making body. An interesting element of the CT-FBP is that it is itself under a performance framework, applied each month, and a larger one each quarter. Result payments for the MOH staff working in this unit are very significant. The first payment that the GOB did to its health facilities was made 20 days faster than the officially indicated time frame in the project management manual. This was way before any one of the other payers paid its share. Managing change The first Burundi PBF pilot project was started in 2006. Actors involved in the successful PBF pilot projects in Rwanda (2002-2005), after scaling up in Rwanda (2006), moved to Burundi and DRC to replicate these interventions. The need for early advocacy and capacity building within the MOH were some of the lessons learned from the scaling up processes in Rwanda. The NGO Cordaid started with early capacity building, and provided funding for a central coordination unit in the MOH with concomitant capacity building of MOH staff linked to this coordination function, and successfully lobbied the MOH for including PBF in national policy documents. Proof of this activity can be found in the 2006-2010 national health strategy document (PNDS), in which PBF was put forward as one of the strategic priorities. The Burundi MOH therefore, was quickly convinced of the utility of PBF for strengthening their health system. Frequent high level missions to neighboring Rwanda, 2007-2009, where the scaling up had been accomplished successfully, also informed the MOH of the nuts and bolts. Various actors, international and local with field experience in Rwanda played a part in the knowledge transmission in Burundi. Human resources from Burundi, working in Rwanda, also contributed their bit in the quick acceptance of PBF as a valuable health system intervention. Various development partners were requested by the MOH to convert their programming into output based financing mechanisms. Such was the case with the Swiss Development Cooperation. The issue with these requests was that such agencies were not financially or technically prepared to do PBF (Bertone and Meessen, 2010). Also, a large classical health strengthening program, along input lines, financed by the EC was converted into PBF. This did not go very smoothly. Reasons for these wobbles can be explained through a combination of a ‘clash of paradigms’, a ‘clash of ideas’ and a ‘clash of characters’. The innate contradiction in input financed programming versus results based financing programming led to this ‘clash of paradigms’. There is a deep divide in planning systems, strategic approaches, and financing methods in both approaches. Having worked an entire career in ‘input financing’ projects, some actors were challenged conceptually, but also in some instances, did not embrace this new financing method emotionally. The ‘clash of ideas’ revolved around conceptions of how to improve the quality; this issue has an interesting resemblance to similar conflicts in the scaling up processes in Rwanda. The largest conflicts in Rwanda also surrounded the issue of how to measure, to evaluate and to pay for quality performance.45 Finally, the ‘clash of the multiplicity of bodies and meetings dialogue partners of the same ... but some of its meetings are often reserved exclusively for PBF ... 45 The lessons learned here are that if such processes are not well-managed, that the danger lies in the phenomenon of ‘you do not like my indicator, so you do not like me’ 85 The World Bank Health Sector Development Support ( P101160 ) characters’ was about individuals strongly engaged in health reform processes, here, also, interesting parallels can be drawn with the Rwandan experience. It might be that the inherent nature of these fundamental reforms leads to these types of conflicts.46 The MOH, although it had embraced PBF as their key strategy to reform their health system, had different ideas on how to organize this than its partners. Most partners proposed to follow the ‘NGO fund holder PBF approach’, in which the purchasing and the funds were held by non-state entities. This vision was fundamentally based on the insurance company model as the purchaser and fund holder, such as in the Netherlands. The MOH did not share this vision at the time. A team of independent mediators was brought in to negotiate a solution to this institutional challenge. A week of negotiations between the various actors led to a consensus agreement, which was ratified during a national workshop in Bujumbura in March 2009. This consensus agreement proposed a hybrid arrangement, a so-called provincial verification and validation committee (CPVV), in which incentivized bureaucrats would work alongside contracted staff and civil society from the province. The WB was involved significantly in preparatory work, for instance in the legal agreement which showed the way how to engage in Performance-Based Financing mechanism through WB procurement mechanisms. An IDA grant to support the GOB’s health reforms worth $25M for three years was signed 8 May, 2009; the financial agreement was signed July 7, 2009. A draft PBF manual was prepared by a consultant. This PBF manual was further elaborated during a retreat with all stakeholders in August 2009. In September 2009, a workshop was held, in which, using a modified Delphi technique, the services to be bought at the health center and hospital levels was determined. Also, their relative weights were decided using the Delphi technique, and a costing was done based on these findings. During this period, intense consultations were held with the MOH/CT-FBP on how to integrate the selective free health care funds in PBF. In September two consultants worked with a group of technicians to design a training curriculum using the PBF manual. Training was done through a snow-ball methodology. Between January and April 2010, the trainings were done in the entire country, and contracts were signed at all levels. In Nov/Dec 2009 a mission looked at the issue of equity, and proposed a way forward (see the health finance knob). The costing was done December 2009-March 2010. A web-enabled application and a website through which to access this application was created between Jan-June 2010. A server location in Bujumbura had been selected also. The new PBF system was launched April 1, 2010, nationwide. Regulation The role of the regulator in PBF belongs to the MOH, and the provincial and district health offices. The CT- FBP/MOH has a clear stewardship role; it manages the contracts with the CPVVs, with the BPS’s and the BDS’s, the paramedical training schools, the four tertiary care institutions, the third-party counter- verification agent and the follow-up on the NGOs recruited to provide technical assistance for PBF. The CT-FBP also coordinates the development partners, functions as the secretariat of the national PBF coordination committee (PN), and organizes (and presides) the ‘CT-FBP élargie’.47 It is also managing the 46 Health Policy, An Introduction to Process and Power, Gill Walt (ed) (1994), London, Zed Books Ltd. 47 The CT-FBP ‘élargie’, is an implementation oriented coordination mechanism, modeled on the Rwandan ‘extended team’ 86 The World Bank Health Sector Development Support ( P101160 ) web-site and the web-enabled database, and is involved in operational research and information dissemination related to PBF. At the onset of the scaling-up of PBF, ten partners financed the new provider payment mechanism through a virtual basket funding mechanism coordinated through the CT-FBP/MOH; the Government, the WB (PADSS), European Union (through Cordaid), SEP-CNLS (WB and GF), Belgian Technical Cooperation, Pathfinder International, Swiss Development Cooperation, HealthNet TPO, Gavi (through HealthNet TPO and Cordaid), GVC and PTRPC. Key functions of the CT-FBP have been incentivized through a performance framework. This framework is applied monthly, while each quarter a larger performance framework is applied. Core MOH staff in leadership and technical positions in this CT-FBP are held accountable for a couple of key deliverables, such as for instance the speed with which the payment orders are approved. The Provincial Health Office, in conjunction with the District Health Offices in its province, is responsible for applying the quality checklists to the health centers in a timely manner. This function is incentivized through a performance framework, which also includes supervisory tasks of the District Health Office, their role in the drug revolving fund (provincial pharmacy), their planning and coordination role, and their role in the health management information system. The District Health Offices, on their turn, are held accountable through a performance framework for a series of tasks also. The incentives linked to these tasks represent a significant sum of money: if the tasks are carried out correctly (100% performance), then each bureaucrat can potentially earn double to triple her (admittedly low) base salary. These performance based bonus payments could potentially elevate the take home earnings to a comparable level with the contracted technicians in the CPVV. A third-party verification agent, HDP, has been hired to check the validity of all performance measures throughout the system, each quarter. From the CT-FBP performance frameworks all the way down to the community client satisfaction surveys. This is done ex-post, that is, after the payments have been made. Ex-ante control is significant, and at all levels of the system also (see ‘organization knob’). Additional information sources: Burundi PBF website. Operational Dashboard for public viewing and repository of documents (see below). http://www.fbpsanteburundi.bi/ (accessed 15 October 2017). Manuel des Procédures pour la mise en œuvre du financement base sur la performance au Burundi. Latest version (2017) is available, including latest tools on the website. Also, rich collection of previous versions and a host of documents. http://www.fbpsanteburundi.bi/documents.html (accessed 15 October 2017). mechanism. It is presided by the CT-FBP, and it incorporates MOH/CT-FBP and technical staff from non-state agencies involved in providing technical assistance to the provinces. 87 The World Bank Health Sector Development Support ( P101160 ) File ‘Master costing Burundi v 22 July 2010.xls’ shows the costing for the PBF system and the output budget allocations per province MOH (2011). Rapport de la mise en œuvre du financement base sur la performance au Burundi Avril 2010 à Mars 2011. Bujumbura, Ministère de la sante publique et de la lutte contre le SIDA. MOH (2012). Rapport annuel de mise en œuvre du financement base sur la performance au Burundi. Bujumbura, Ministère de la sante publique et de la lutte contre le SIDA. Soeters, R., C. Habineza, et al. (2006). "Performance-based financing and changing the district health system: experience from Rwanda." Bulletin of the World Health Organization 84(11). Meessen, B., J.-P. Kashala, et al. (2007). "Output-based payment to boost staff productivity in public health centers: contracting in Kabutare district, Rwanda." Bulletin of the World Health Organization 85(2): 108-115. Meessen, B., L. Musango, et al. (2006). "Reviewing institutions of rural health Centres: the Performance Initiative in Butare, Rwanda." TMIH 11(8): 1303-1317. Pay for Performance for improved health in Burundi, Jean Francois Busogoro et al, http://www.rbfhealth.org/resource/pay-performance-improved-health-burundi (accessed 15 October 2017) Brand New Day: Newly Launched Nationwide PBF Scheme in Burundi Reflects the hopes of a Nation, Lindsay Morgan http://www.rbfhealth.org/resource/brand-new-day-newly-launched-nationwide-pbf- scheme-burundi-reflects-hopes-nation (accessed 15 October 2017) Collectivity: a collaborative platform for Performance-Based Financing and Strategic Purchasing. https://www.thecollectivity.org/ (accessed 15 October 2017) PBF google groups: a discussion forum for the African PBF Community of Practice (need to register to get access). http://groups.google.com/group/performance-based-financing?hl=en SINAHEALTH website. A rich repository of documents, training reports and blogs by the first PBF pioneer. http://www.sina-health.com/ (accessed 15 October 2017) Performance-Based Financing Website with blog and a listing of various tools and other useful websites http://performancebasedfinancing.wordpress.com/ (accessed 15 October 2017) A personal story: seeking the roots of PBF, Robert Soeters http://www.rbfhealth.org/resource/personal- story-seeking-roots-performance-based-financing-pbf (accessed 15 October 2017) RBF website http://www.rbfhealth.org/ (accessed 15 October 2017) 88 The World Bank Health Sector Development Support ( P101160 ) Falisse, J.-B., et al. (2012). "Community participation and voice mechanisms under performance-based financing schemes in Burundi." Tropical Medicine and International Health 17 (5): 674-682. Nimpagaritse, M. and M. P. Bertone (2011). "The sudden removal of user fees: the perspective of a frontline manager in Burundi." Health Policy and Planning 26(Supplement 2): 63-71. Bertone, M. P. and B. Meessen (2013). "Studying the link between institutions and health system performance: a framework and an illustration with the analysis of two performance-based financing schemes in Burundi." Health Policy and Planning 28(8): 847-857 Bonfrer, I., et al. (2014). "The effects of performance incentives on the utilization and quality of maternal and child care in Burundi." Social Science & Medicine 123: 90-104. Bonfrer, I., et al. (2014). "Introduction of Performance-Based Incentives in Burundi was Associated with Improvements in Care and Quality." Health Affairs 33(12): 2179-2187. Falisse, J.-B., et al. (2015). "Performance-based financing in the context of selective free health care: an evaluation of its effects on the use of primary health care services in Burundi using routine data." Health Policy and Planning 30: 1251-1260. Ventevogel, P. (2016). Borderlands of mental health: Explorations in medical anthropology, psychiatric epidemiology and health systems research in Afghanistan and Burundi. Amsterdam, University of Amsterdam. PhD. Rudasingwa, M. and M.-R. Uwizeye (2017). "Physicians' and nurses' attitudes towards performance- based financial incentives in Burundi: a qualitative study in the province of Gitega." Global Health Action 10(1). Fritsche, G., Soeters, R., Meessen, B, et al. (2014). Performance-Based Financing Toolkit. Washington DC, © World Bank. https://openknowledge.worldbank.org/handle/10986/17194 License: CC BY 3.0 IGO. Roberts, M. J., W. Hsiao, et al. (2004). Getting Health Reform Right, a Guide to Improving Performance and Equity. New York, Oxford University Press 89