IMPROVING GRASSROOTS SERVICE DELIVERY USING RESULTS-BASED FINANCING IN VIETNAM An Assessment Of The Nghe An Pilot DISCUSSION PAPER August 2018 Ha Thi Hong Nguyen IMPROVING GRASSROOTS SERVICE DELIVERY USING RESULTS-BASED FINANCING IN VIETNAM An Assessment of the Nghe An Pilot Ha Thi Hong Nguyen August 2018 Health, Nutrition, and Population (HNP) Discussion Paper This series is produced by the Health, Nutrition, and Population Global Practice. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors, or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. For information regarding the HNP Discussion Paper Series, please contact the Editor, Martin Lutalo at mlutalo@worldbank.org or Erika Yanick at eyanick@worldbank.org. RIGHTS AND PERMISSIONS The material in this work is subject to copyright. Because the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. © 2018 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii Health, Nutrition, and Population (HNP) Discussion Paper Improving Grassroots Service Delivery Using Results-Based Financing in Vietnam An Assessment of the Nghe An Pilot Ha Thi Hong Nguyena a Health, Nutrition and Population Global Practice, The World Bank Abstract Over the last decade, results-based financing (RBF) health programs have been implemented in several countries at different levels of income. Due to its requirement of rigorous verification of results as a condition for financing, as well as a number of accompanying measures to help achieve the results, RBF has a promise of value for money. RBF’s potential for improving the performance of the service delivery system has led the government of Vietnam to undertake a pilot of RBF in the Nghe An province as part of a World Bank funded operation. The main objective of the pilot was to experiment an RBF approach in the Vietnam context, where public sector providers have been receiving budget allocation based on inputs rather than performance. A secondary objective was to test the effects of RBF in improving quality of care at the grassroots level and in addressing the challenges of emerging noncommunicable diseases. The intervention included quality improvement at the district level and both quality and quantity of services at commune health station. This paper provides a rapid assessment of the pilot and draws lessons for possible expansion of RBF in the country context. After two years of implementation, the results of the pilot are largely positive. The Balanced Score Cards used to measure quality were seen as an effective tool to help with monitoring and supervising the facility performance. The aggregate quality score increased from 70 percent to over 90 percent in participating district hospitals and from 78 percent to 88 percent in commune health stations. Service provision in the commune health stations, the lowest level of the health system, has increased appreciably for 9 out of 14 indicators. Though the pilot had some features that were built on the existing system, the issue of institutionalization remains a challenge for further RBF expansion in the country. Keywords: Results-Based Financing, Vietnam, Quality of Care, Balanced Score Card Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Correspondence Details: Ha Thi Hong Nguyen, The World Bank, Email: hnguyen19@worldbank.org iii Table of Contents ACKNOWLEDGMENTS ........................................................................................................... V ABBREVIATIONS ..................................................................................................................... VI BACKGROUND ........................................................................................................................... 1 DESCRIPTION OF THE PILOT ............................................................................................... 3 THE PILOT TIMELINE .................................................................................................................... 3 THE RBF SERVICE PACKAGE ....................................................................................................... 3 INSTITUTIONAL ARRANGEMENT ................................................................................................... 7 THE PILOT COST........................................................................................................................... 8 ASSESSMENT OF THE PILOT PERFORMANCE .............................................................. 11 OBJECTIVE 1: TESTING AN RBF MODEL IN VIETNAM ................................................................ 11 The Relevance of the Design ................................................................................................. 11 The Acceptability of the RBF Model...................................................................................... 12 OBJECTIVE 2. IMPROVING QUALITY, COVERAGE, AND EQUITY IN SERVICE DELIVERY .............. 14 Quality ................................................................................................................................... 14 Quantity at CHS..................................................................................................................... 17 Equity ..................................................................................................................................... 19 DISCUSSION .............................................................................................................................. 21 SUMMARY OF FINDINGS ............................................................................................................. 21 IMPORTANT CONDITIONS FOR INSTITUTIONALIZATION .............................................................. 22 WHERE DO WE GO FROM HERE – FROM A PILOT TO THE SYSTEM ........................................... 23 REFERENCES ............................................................................................................................ 25 iv ACKNOWLEDGMENTS This paper was written by Ha Thi Hong Nguyen (Senior Economist, Health, Nutrition, and Population Global Practice) with contributions from Trung Dang Le (consultant for data analysis), Huong Lan Dao (Senior Health Specialist, Health, Nutrition, and Population Global Practice and Task Team Leader of the Vietnam Central North Region Health Support Project of which the Nghe An pilot was a part), and Caryn Bredenkamp (Senior Economist, Health, Nutrition, and Population Global Practice and Task Team Leader for the Investing and Innovating for Grassroots Service Delivery Reform project). The preparation of this paper was financed by the Global Financing Facility (GFF) for Every Women and Every Child to inform the potential for scale-up of results- based financing (RBF) interventions under the aforementioned project. The author would like to thank the Vietnam Ministry of Health’s Project Management Teams, including the Central Project Management Unit and Nghe An Provincial Management Unit of the World Bank-financed Central North Region Health Support Project for their support and collaboration during this assessment. Special thanks go to Dr. Ha Van Thuy, Project Manager, Mr. Nguyen Minh Trung, Project Coordinator, Dr. Hoang Van Hao, Nghe An Project Manager, and staff of the health facilities visited during field trips, for sharing valuable insights into the design and implementation of the Nghe An RBF pilot. The author is grateful to two reviewers—Dinesh Nair (Senior Health Specialist, Health, Nutrition, and Population Global Practice) and Son Nam Nguyen (Lead Health Specialist, Health, Nutrition, and Population Global Practice)—for their substantial and constructive comments on an earlier draft. Administrative support by Nga Thi Anh Hoang is greatly appreciated. This assessment was overseen by Toomas Palu (Practice Manager, Health, Nutrition, and Population Global Practice). The author is grateful to the World Bank for publishing this report as an HNP Discussion Paper. v ABBREVIATIONS ANC Antenatal Checkup BSC Balanced Score Card CHS Commune Health Station CPMU Central Project Management Unit DH District Hospital DHC District Health Center DOH Department of Health EPI Expanded Program on Immunization FP Family Planning GDP Gross Domestic Product HBP High Blood Pressure HMIS Health Management Information System HRITF Health Result Innovation Trust Fund MCH Maternal and Child Health MOH Ministry of Health PHC Primary Health Care PNC Postnatal Care PPMU/PMU Provincial Project Management Unit/Project Management Unit RBF Results-Based Financing VHEA Vietnam Health Economic Association VSS Vietnam Social Security vi BACKGROUND For more than a decade, results-based financing (RBF) has been gaining traction among the global development community. RBF was originally defined as “a cash payment or non-monetary transfer made to a national or subnational government, manager, provider, payer, or consumer of health services after predefined results have been attained and verified. Payment is conditional on measurable actions being undertaken.” Musgrove (2011) developed a typology of health RBF programs, whereby the primary recipients of incentives could be the country, the provider, or the beneficiary (population); and the incentives could be payment based on fee-for-service, other monetary payments, and non-monetary rewards, such as a provider recognition program (Musgrove 2011). More recently and based on the actual experience of implementing RBF in many countries, RBF was characterized as multifaceted interventions that enable an explicit benefit package, necessitate an investment in good data system, and direct the attention to quality, provider autonomy, as well as other pillars of a health system. The key innovation compared to ‘business- as-usual’ is the fact that financing takes effect only if and when results are achieved and are verified by a third party. For this feature, RBF has a promised to deliver value for money. The World Bank, with financial contribution from the British and Norwegian governments, manages a Health Result Innovation Trust Fund (HRITF) that supports 35 RBF programs in 29 countries. With the recent rollover of HRITF to the Global Financing Facility, focus-on-results remains a core principle in the Bank operation. The Nghe An pilot is a very small subcomponent in the USD 65 million Central North Region Health Support Project which aimed to strengthen primary health care (PHC) by tackling both the supply and demand sides of services. With a total executed budget of roughly USD 1 million, the pilot set a rather modest expectation. The primary objective was to experiment an RBF approach— the design, instrument, institutional arrangement, fund flow mechanism, and financial management—and learn from the implementation. This is not a trivial objective in the context of Vietnam, where public sector providers for decades under the control and command system have been receiving budget allocation based on inputs rather than performance. A secondary objective of the pilot was to test the effects of RBF in addressing important shortcomings in the service delivery system, including most importantly low quality of care, under-provision of chronic diseases management and preventive services, low staff motivation, and poor coordination between levels of care. As the Central North Region Health Support Project and the RBF pilot closed in 2016, a comprehensive evaluation of the pilot is very important to generate lessons learned from the design and implementation of the RBF. Because the pilot represented the first experience with RBF in Vietnam, lessons from the pilot will feed into the discussions on instruments to improve quality of and access to care at the grassroots level and options for reforming provider payment for PHC. The current paper seeks to: • Provide a description of the RBF Pilot in Nghe An; • Conduct an assessment of the pilot against its objectives, which include: o Testing an RBF mechanism, assessing the operational implementation of the RBF processes, including developing the service package, verification of the indicators, flow of funds, and others; and 1 o Testing the effects of RBF in changing providers’ behaviors and addressing bottlenecks in the provision of PHC services; • Draw lessons learned and provide recommendations for the Ministry of Health (MOH) and the World Bank, emphasizing the relevance of the pilot for the broader system context and sustainability of the intervention. The report relies on qualitative information obtained from the author’s field visit to Nghe An in February 2017 and an extensive analysis of administrative data. 2 DESCRIPTION OF THE PILOT THE PILOT TIMELINE The idea of testing out an RBF model was clearly elaborated in the Project Appraisal Document in 2011. However, given the novelty of RBF in Vietnam, the preparation took nearly two years. In the preparation phase, a baseline survey was conducted and multiple stakeholder consultations were held to develop the incentivized service package and the operational manual. The pre-pilot activities started in May 2013 in eight facilities in Nam Dan and Thanh Chuong districts. In each district, participating facilities included a district hospital (DH) and district health center (DHC), and two commune health stations (CHSs). After six months of experiment, the pilot rolled out to 54 facilities in 11 districts (11 DHs, 11 DHCs, and 33 CHSs). The process of RBF pilot development is presented in figure 1. Figure 1. The RBF pilot timeline It is noteworthy that while the long duration of preparation was likely necessary, the time for implementation in comparison was too short. For 46 out of 54 participating facilities, the implementation time was only two years, likely not giving enough time for all stakeholders to learn from the experience and to adjust accordingly. Several activities intended for the pilot did not have enough time to materialize—such as developing an online cloud-based data system and adjusting the instruments further. THE RBF SERVICE PACKAGE The package of service and type of participating providers were carefully selected to align with the objectives of the overall project, which were to improve quality of care and accessibility to PHC services. A decision was made to include the lowest level of the health system, the CHS, despite the fact that the project worked mostly at the district level. This allowed for a reinforcement of the integration of care between commune and district levels and a full coverage of both curative and preventive services. Another deliberate decision was to focus heavily on quality of care, a critical gap in the health service delivery in Vietnam. 3 On this basis, the incentivized package at CHS included 14 quantity indicators and 9 groups of quality indicators. The quantity indicators reflected priority preventive and curative services, with the key being family planning (FP) and maternal and child health (MCH), including child growth monitoring. Of note were the indicators for noncommunicable diseases (detection and treatment of high blood pressure - HBP), and the indicators reinforcing referral and counter referral; both aspects were not adequately practiced in Vietnam before. An attempt was made to address equity issues, with the price tag for curative outpatient exam provided to the poor, near poor, insured patient, or children under 6 years being set more than three times that for an uninsured patient. The quality checklist (or Balanced Score Card - BSC) included both management aspects and clinical/treatment aspects. It is noteworthy that while the quality BSC of this nature was never introduced in Vietnam, the quantity indicators relied almost exclusively on the existing routine information system and can be extracted from the A1 Book of the CHS. The total payment to a CHS is a product of total quantity payment and quality performance (measured as percentage of the maximum 200 points). Therefore, total payment for the most part is lower than possible earning from quantity. There is also a cap of USD 3,700 per year for the CHSs; performance beyond that will not be reimbursed. The cap effectively kept under control the total payment of the pilot and made sure it was within the originally allocated financing of USD 1.2 million (which was materialized as USD 1 million at the end of the project due to the appreciation of the USD against the Special Drawing Right). Table 1 shows the list of quantity indicators at the CHS. As shown, the unit price for the quantity indicators at CHS is rather small. A quick comparison could be made with other RBF programs in the HRITF portfolio in their initial period. For example, according to the respective projects’ original operational manuals (before revision and adjustment), a normal delivery in a health center was USD 4.48 in Burundi, USD 5.25 in Cameroon, and USD 9 in Kenya. In Vietnam, the USD equivalent is 3.27. The pattern is rather similar with other indicators as well. 1 Table 1. Quantity indicators for CHS No. Indicators VND USD equivalent 1 Curative outpatient contact, uninsured 3,000 0.14 2 Curative outpatient contact, insured, poor, near poor, U6 1,000 0.05 3 Follow-up exam from upper levels 20,000 0.94 4 Growth monitoring of children under 1 year 7,000 0.33 5 Growth monitoring of children ages 1–2 years 7,000 0.33 6 Growth monitoring of children ages 2–5 years 7,000 0.33 7 HBP screening among adults above 25 years 10,000 0.47 8 Management of HBP patients 7,000 0.33 9 Antenatal checkup (ANC) four times 3,000 0.14 10 ANC fifth time onwards 15,000 0.70 11 Normal delivery 70,000 3.27 1 Source: Operational data database, HRITF 4 No. Indicators VND USD equivalent 12 Postnatal care (PNC) 35,000 1.64 13 Referral of complicated deliveries 30,000 1.40 14 Contraceptive use among women 15–49 years 3,000 0.14 1a Growth monitoring for children <=2 years 7,000 0.33 2a ANC three times 3,000 0.14 3a ANC fourth time onwards 15,000 0.70 4a Positive TB cases screened 25,000 1.17 Note: USD value is estimated using the exchange rate of December 31, 2014. The indicators highlighted in blue were removed and the indicators highlighted in brown were added at the time of indicator revision in April 2015. Table 2 shows the main categories included in the BSC for CHS with relative weight assigned to each of them. The BSC distinguishes two types of quality—for general management and for treatment activities, with a heavier weight assigned to treatment quality (114 out of 200 points). Quality of treatment activities was assessed for five services/disease conditions: curative care for outpatient and temporary patients (who stayed overnight temporarily), nutrition service, prenatal care, delivery care, and management of patients with HBP. Each of these required an assessment of the medical records among other factors. For example, for assessing the quality of outpatient care, verifiers would randomly select 10 cases including 5 children from the A1 Book, review records of diagnosis and treatment to see if the treatment followed clinical guidelines, if drugs prescribed were in line with the diagnosis, and if auxiliary drugs were within 10 percent of each prescription. Similarly, for management of hypertension, the reviewers would randomly select 6 cases from the list of hypertensive patients to check if (a) they were monitored last month, (b) their blood pressure measurement was indicated in the examination book or outpatient record, (c) cases of hypertension from level 2 and above were referred to higher level. The facility also scored additionally if they had staff who were trained in hypertension prevention program and if the staff correctly answered the questions asked by the verifier on how to diagnose and manage HBP. Table 2. Quality indicators for CHS Indicators Max score Management quality , 1 General and data management 24 2 Hygiene and waste management 24 3 Drug management and availability 22 4 Family planning 16 Treatment quality 5 Outpatient curative care and temporary patients 36 6 Nutrition 24 7 Prenatal care 14 8 Delivery care 18 9 Management of patients with HBP 22 Total score 200 5 For district hospitals, only quality of care was incentivized by the RBF scheme. As shown in table 3, besides the general management issues, selected clinical conditions reflected the prevalent burden of disease and conditions typically seen in a DH. They included children with acute pneumonia, appendicitis, complicated delivery including C-section, management of patient with HBP, outpatient care, and referral/counter referral. Among these categories, the highest weight was designated to children with acute pneumonia. Table 3. District hospital indicators No. Indicators Max score Management 1 General and data management 12 2 Drug management and supply to CHS 20 3 Hygiene and waste management 22 Clinical quality of care 4 Treatment of children with acute pneumonia 35 5 Appendicitis treatment 15 6 C-section and management of complicated deliveries 30 7 Management of patients with HBP 22 8 Outpatient care 28 9 Referral to provincial hospitals and to CHS 16 Total score 200 As is the case with the BSC for CHS, the assessment form for the DH contains numerous details which guided the hospitals on how to concretely achieve good scores. Among the general management categories, of note is the indicator on ‘hygiene and waste management’, which looked at how solid and liquid waste were handled, how medical hazardous waste was treated, and if medical instruments were cleaned and disinfected according to standard processes. This is very important as waste management in general is often overlooked in the hospital context in Vietnam. For clinical quality of care, the assessment was based almost exclusively on reviewing a set of randomly selected medical records. For DHC, the BSC dwelled on its core functions to provide coordination of the health activities in the district and preform supervision to the CHSs. The major components of the BSC include management of the health information, inter-sectorial coordination and supervision of the CHSs, nutrition management, training, drug and medical equipment management, expanded program of immunization (EPI), and financial management. There was also a task to perform quality verification of the CHSs. As is the case with CHS, payment for DH was capped at USD 18,000 per year and for DHC at USD 5,000 per year. Copies of the full BSCs are available from the author upon request. 6 Table 4. DHC indicators Indicator Max score 1 Health information management 32 2 Inter-sectorial and CHS supervision 16 3 Supervision and deployment of RBF activities 40 4 Nutrition management 20 5 Training activities 12 6 Drug and equipment management 34 7 Expanded program of immunization 20 8 Financial management 26 Total score 200 The use of the RBF bonus is aligned with the prevailing financial management rules in the country. For DHC and DH, incentive from RBF was included in the facility common budget and was then used fully for staff bonus, with a higher share going to staff directly involved in providing the incentivized services. For CHS, since there is no regulation, the province (or DHC) issued the guidelines which could vary by district. For example, in one CHS visited, 60 percent of the bonus was used for staff directly involved in providing the incentivized services, 15 percent for staff not directly involved in providing such services, 15 percent for equipment, supplies, medicines, and repair for improving quality of care, and 10 percent for other recurrent expenses such as per diem and utility. In another CHS, the breakdown was 45 percent for staff bonus and 55 percent for other expenses. INSTITUTIONAL ARRANGEMENT The pilot was nested within the larger, overall, project which was coordinated by a Central Project Management Unit (CPMU) based in Hanoi and different Provincial PMU (PPMU) in each project province. Specifically in Nghe An, the PPMU was the RBF fund holder and purchaser of services, backed with technical support from the CPMU and the World Bank (figure 2). The PPMU set up an RBF Task Force (or RBF team), staffed with current and former staff of the provincial Department of Health (DOH). The RBF Task Force was in charge of coordinating all pilot activities and organizing first level (internal) verification to the DHs and DHCs. It also performed second level (external) verification to the CHSs. Contracts were signed with DHs and DHCs, while because CHSs did not have their own bank account, their RBF bonus was channeled through the DHCs. 7 Figure 2. Institutional arrangements of Nghe An RBF pilot Internal verification was organized quarterly covering all facilities. External verification of the DH and DHC was performed every quarter on a sample of facilities and by an agency contracted by the project—the Vietnam Health Economic Association (VHEA) based in Hanoi (table 5). Table 5. Verification arrangements Provider Service to Agencies involved in internal Agencies involved in external verify verification verification CHS Quantity DHC RBF Task Force + VHEA CHS Quality DHC (+DH + RBF Task Force) RBF Task Force + VHEA DH Quality RBF Task Force VHEA DHC Quality RBF Task Force VHEA Although CHS quantity data are largely selected from the routine Health Management Information System (HMIS) indicators, because the quality checklists were new, RBF pilot data were maintained separately in an MS Excel database. The project intended to develop an online dashboard for data the same way as in many other RBF programs. However, the implementation time was too short to allow for the work to be materialized. THE PILOT COST Table 6 presents the details of the Nghe An RBF pilot cost. As shown, a total of nearly USD 600,000 was spent on performance-based pay to service providers, more than half of which went to DHs. While administrative costs and cost for internal verification only accounted for 2 percent of the total pilot budget, external verification accounted for 11 percent of the total budget or equated to more than 20 percent of payment to service delivery. This confirmed the view by stakeholders shared during the field visit that external verification was ‘costly’. As it is, Nghe An RBF is in the comparable range of the budget share devoted to service delivery (60 percent). For example, in Zambia’s first RBF project, the total program cost was USD 16.5 8 million, about 50 percent of which was used to pay incentives and to fund health facilities in the control group receiving unconditional additional financing (Zheng et al. 2017). In Zimbabwe, USD 7,045,211 out of USD 10,704,473 was used for subsidy payment (Shepard et al. 2016). Note however that the duration of the Nghe An pilot is much shorter than the other two programs. If the pilot time were to be extended, the share going to service delivery would increase as start-up equipment investment would no longer be needed. Table 6. Breakdown and total expenditure of the pilot (USD) 2013 2014 2015 Total Percent Performance-based pay to service providers 24,968 266,745 308,030 599,743 59 of which CHSs 4,302 72,285 89,866 166,452 16 DHs 17,627 138,118 156,493 312,238 31 DHCs 3,039 56,341 61,672 121,052 12 External verification 55,640 55,640 111,280 11 Administration, incl. internal 4,208 6,546 12,203 22,957 2 verification Study tour to RBF programs in Africa 100,479 100,479 10 Start-up equipment investment to 23,144 120,725 143,869 14 C S Others 36,330 36,330 4 Total 52,320 550,134 412,203 1,014,6 100 Source: CPMU. 8 Note: USD value is estimated using exchange rate of December 31, 2014. To see if RBF brought about a drastic change in the resource envelope for participating providers, a comparison was made on RBF payment versus total facility revenue, and on bonus payment from RBF versus total facility fund for staff salary and allowances. As shown in table 7, in 2015, on average, a DH had a total revenue from all sources of USD 1.9 million, USD 15.4 thousand of which came from RBF. RBF payment accounted for less than 1 percent of total hospital revenue and RBF bonus payment accounted for 2.4 percent of the salary fund. Hence for DH, additional financing from RBF is rather insignificant. The same can be concluded for DHC (2 percent and 2.6 percent, respectively). However, for CHS, RBF represented a rather nontrivial share of facility revenue as well as staff take home (formal) income—8.2 percent and 8.6 percent, respectively. It is also noteworthy that in the last year of implementation 2015, neither DH, DHC, or CHS had maxed out the potential earning capped at USD 18,000, USD 5,000, and USD 3,700 according to the project rule, although the exact gaps need to be viewed carefully given the potential discrepancy in USD/VND exchange at different points in time. 9 Table 7. RBF payment compared to facility revenue and staff earning (2015 data) Total revenue Total RBF RBF payment as % of RBF payment (USD) payment (USD) total facility’s on bonus as % revenue of total salary and bonus fund DH 1,946,610 15,422 0.8 2.4 DHC 221,873 4,439 2.0 2.6 CHS 34,386 2,809 8.2 8.6 Source: Nghe An DOH. Note: Presented figures are averages among participating facilities of the same type. Amounts in USD are estimated based on VND/USD exchange rate on December 31, 2014. 10 ASSESSMENT OF THE PILOT PERFORMANCE The following provides an assessment of the pilot performance in relation to its two main original objectives: (1) to test a model of RBF in Vietnam and (2) to use RBF to improve quality, coverage, and potentially equity in service delivery. OBJECTIVE 1: TESTING AN RBF MODEL IN VIETNAM This section assesses the relevance and feasibility of RBF as an innovative model when it was introduced in Vietnam. It reviews in detail the design to assess whether the project features are relevant to the Vietnam context. It also assesses the acceptability of RBF by the key stakeholders when the model was being implemented. The Relevance of the Design The assessment of the pilot design mirrors the description of the pilot, namely looking at: (a) service package; (b) institutional arrangement; and (c) the cost of the pilot. On the service package, it is clear that the list of indicators was developed in an extensively consultative manner and were deliberately selected to reflect the priorities in the Vietnamese system. Specifically, quality of care is the focus in all three provider types: CHS, DH, and DHC. While low quality of care has been notoriously reported as a problem in Vietnam, there has not been an effective mechanism to address this. Another longstanding issue is the low utilization of CHS services. Although bringing health services closer to the population is generally accepted as highly cost-effective and is in fact a priority for the Vietnamese government, CHSs in Vietnam have not been successful in attracting patients. By helping to improve quality of care and rewarding providers for the volume of services provided at the CHS, the pilot was on the right track to boost health workers’ motivation and increase attractiveness at the lowest level of the system. The pilot also selected the quantity indicators from the routine health information system rather than imposing new indicators, which helped with acceptability and eased the implementation. Because no equivalent quality measuring instruments existed in Vietnam, the decision to generate checklists used for the pilot is a good one. The checklists adopted the popular design and structure used in various World Bank financed RBF programs, with an adaptation to Vietnam’s context. As described above, the BSCs were extensive, which had an advantage of guiding the providers concretely on how to improve quality. On the other hand, the downside of a lengthy BSC (for example, the BSC for the CHS is a 12-page document) is that it would take a long time to go through. It is the view of the author that the checklists could be simplified and tested further so that verification will not impose a major time and effort burden on the implementers. A shorter checklist will likely be more acceptable in the context that does not have financial resources from a Bank project like the pilot in Nghe An. Regarding institutional arrangements, again, the pilot in Vietnam adopted a rather typical model used in many World Bank financed RBF programs that involve a PMU. What was missing, however, was a role of the insurance agency, the Vietnam Social Security (VSS). As was explained to the author, VSS was approached in the beginning but a decision was made later that VSS would not be involved because a significant share of the RBF package was preventive care (insurance in 11 Vietnam only covers curative care). In the author’s view, this is a missed opportunity because VSS is too significant not to involve, given its important and growing role in the health system. As Vietnam vows to achieve universal health coverage by 2020, as of 2015, VSS had covered 75 percent of the population (Vietnam Ministry of Health 2016). Thus, although the institutional setup worked well in the project context, it did not lend itself into a model that the country could pick up right away without creating significant new bodies (in this case the Project Implementation Unit/fund holder and the Vietnam Economic Association /external verifier). On cost, the information presented in table 7 demonstrates that the RBF pilot in Nghe An is a rather lean model. The payment to the facilities is small relative to facility budget and compared to other RBF programs. Some of the high-cost items could be avoided if the pilot were to continue. This finding is important as one of the concerns about RBF in general in the HRITF portfolio is that RBF may have brought in too much money and disturbed the existing system. A lean model imposes minimal intrusion to the system and will have a better chance of being replicated. The RBF model in Vietnam thus resembles the one in Armenia, Estonia, or other more developed systems where RBF only provides marginal incentives to health workers rather than covering a large share of the service cost. This contrasts to the RBF model in some countries in the HRITF portfolio that have weak government system (Zimbabwe, Sierra Leone), where RBF is used as a mechanism to channel funding to frontline facilities for operation. The spending cap imposed by the project on different facility types could certainly help with the budget control. On the other hand, they may have also signaled a cap on performance. Although the paper could not thoroughly address this question, the fact that most facilities did not max out the potential earning suggest that such threshold effects were small if any. In summary, the assessment of the design reveals that the Vietnam model has many features that resemble the popular performance-based financing model in the HRITF portfolio. This is understandable given the opportunity for cross-learning which helped the project to avoid starting from scratch. At the same time, the design demonstrates a conscious effort to adapt to the country context, reflecting priority in improving quality, building on existing HMIS, and staying lean deliberately. All these should facilitate the rolling out of the model in the future. On the other hand, the pilot could have done more to prepare for a post-pilot stage, if the goal is for the country system to eventually adopt RBF. For example, the BSCs could be simplified and tested further; or the institutional setup could somehow involve the statutory third party purchaser—the VSS. Building on the VSS’s network and function as a strategic purchaser will help strengthen VSS and make the pilot more sustainable. The Acceptability of the RBF Model The field visit made in February 2017, more than a year after the pilot ended, revealed a great deal of enthusiasm from provincial health leaders and frontline facilities. According to stakeholders interviewed, RBF is a real game changer—although the amount is small, it changed the mindset and it drove people away from ‘business as usual’. Similar to the pattern observed in many other countries, after some initial resistance, facilities quickly realized the potential benefits in participating in the RBF scheme. In the beginning facilities didn’t want RBF, they were reluctant as they saw that the money was small but efforts could potentially be large, lots of data reporting, 12 paperwork, etc. But only after 2-3 months, they started to say “Ah!”. (DHC Director) Positive aspects were mentioned regarding quality and quantity of services, staff motivation, quality and intensity of supportive supervision from DHC, community outreach, multi-sectorial collaboration, support from local authority, and innovation, among others. For example, a case was cited where CHS staff decided to mobilize their own funding to build a waste treatment spot or to dig up from a big stone behind the facility to build a place for trash burning. A different CHS was reported to receive one ton of cement from the Commune People Committee for paving the road leading to the CHS. Or a doctor was happy to be appointed as head of a CHS that was not in his home commune because that CHS participated in the RBF, and so on. These are inevitably anecdotal examples but they were reflected all the way to the head of provincial DOH who believed there could be many similar stories from the frontline providers. The stories exemplified how powerful the use of incentives can be in promoting entrepreneurship among health workers who have for so many years worked under a control and command system. The most significant aspect of the pilot that came up repeatedly during stakeholder interviews was the BSC and its effects on quality of care. As mentioned, the country has never experienced anything like the BSC. Efforts to improve quality of care are typically manifested in training, distributing clinical guidelines, and upgrading equipment or infrastructure. For CHS, there is a national benchmark with indicators issued by the MOH, and there is also a similar tool for DH. However, as commented by the providers—the benchmark is the target for the facilities to aim toward, it does not say how to get there. RBF’s BSC is a ‘ladder’ to help the facilities to move toward the target one step at a time. It specifies concretely what items need to be in place and how to get there. As a matter of fact, a comment made by a DHC head was that, if a CHS is included in the RBF pilot, it is pretty much guaranteed that it will meet the national benchmark. The following quote from a DH reveals how RBF helped improving quality of care: RBF came to our hospital when we were trying various quality improvement measures. We were trying to find solutions to improve quality. So when RBF came, it helped a lot. With the BSC, it has become a part of our quality improvement program, it changed the way we think and the way we do. Although the incentive is small and concentrated only in a few areas, the RBF approach can be the backbone for us to expand and develop further. (Director DH). Interestingly, although RBF does not incentivize quantity of services, is was seen by the hospital staff as a plus in attracting more patients. A point was made about how RBF does not provide a lot of bonus, the increase in patient load induced by RBF quality improvement helped increase user fees which in turn increased staff allowance and boosted motivation. The other side of the story one should be careful about, however, is if facilities used RBF as an instrument to induce demand unnecessarily for the sake of increasing revenue and staff bonus. An interesting point about verification in the Nghe An setup is that it was viewed not only as method for checking the accuracy of reporting, its value was recognized as providing professional guidance to the front line workers. For example, staff in the DH visited appreciated the fact that the external verification team from Hanoi brought in a famous professor, highly respected in the 13 field. The external verification agency was able to provide professional guidance to the hospital. A similar view was held by staff from CHSs regarding the role of the provincial RBF Task Force and respective DHCs. This deviates from the standard RBF principle where verification is supposed to be totally independent. However, it is not uncommon among the World Bank’s RBF programs, such as the programs in Zimbabwe and Zambia, for the first level verification to also serve the function of ‘supportive supervision’. For DHC, the BSC was seen as an effective tool to help with monitoring and supervising the CHS. The BSC was developed around DHC’s existing functions and responsibilities, and so it helped them to fulfil the task they had to do anyway. It was revealed during the field trip that, only one or two quarters after the pilot was launched, many DHCs started using the BSC in their quarterly monitoring and supervision visit in all CHSs, not only in the ones participating in the pilot. The positive spillover is worth noting for potential efforts to compare RBF and non-RBF communes. DHCs also reported developing short training programs based on the findings from internal verification/supportive supervision. An example was given, when the DHC saw many errors in insurance-related reporting and invited experts from health insurance to speak in a monthly all- commune meeting. The need for continuous learning and adjusting the tool was well recognized. The list of quantity indicators was adjusted several times during the implementation of the pilot. For quality, although no revision was made, the head of the PPMU made it very clear that the intention was to focus on the weaknesses in the system: Whatever the system is weak in, we focus on it. We target it until it gets better, then we move on and target a different weakness (Director, DOH) Seeing the role of the BSC in improving quality of care and of supportive supervision, the provincial DOH decided to continue using the tool even after the World Bank project closed. It was reported that an official letter was sent to participating providers requesting them to continue using the tool. As of the time of the field trip, the author found that the visited DHC had adapted the BSC incorporating their other needs to monitor the implementation of the National Target Programs. However, we cannot comment on whether this is an across-the-board pattern or only the action of the visited DHC. OBJECTIVE 2. IMPROVING QUALITY, COVERAGE, AND EQUITY IN SERVICE DELIVERY Operational data for two years 2014–2015 (or first level verification data used for RBF payment) were analyzed thoroughly. This provides an in-depth picture of the pilot performance in relation to the incentivized package. The analysis does not include the short period of pre-pilot in 2013 that involved only 8 facilities. Detailed results of the analysis are available from the author on request. The following summarizes key trends and observations. Quality Figure 3 shows the change over time in the DH’s overall quality scores and figure 4 shows the scores of the main components for three periods in time, the first quarter (Q1 2014), the last quarter (Q4 2015), and a quarter in the middle of the implementation (Q4 2014). All the figures are 14 averages of the 11 participating hospitals and are presented as percentage of the total maximum score of 200 points. As shown, the aggregate quality score in the DH increased within two years from 70 percent to over 90 percent. This is a rather remarkable increase although one should note that that the baseline score of 70 percent was rather high. When it comes to the specific components, one sees that the areas where quality fell short in the beginning were mainly clinical quality, including appendicitis treatment, C-section and complicated deliveries, and management of patients with HBP. In turn, these are the areas that experienced the most notable change between the examined periods. For example, score for C-section and complicated deliveries increased from 58 percent in Q1 2014 to 87 percent in Q4 2014 and Q4 2015. Although not shown in figures 3 and 4, there is quite a variation among DHs, with some achieving 100 percent very early on and some still falling short even during the last quarter of the pilot. Figure 3. Overall quality score, DH (%) Figure 4. Component quality scores, DH (%) 100 90 General & data mgnt 80 Referral to 100 Drug mgnt & 70 upper & 80 supply to lower levels 60 CHS 60 40 Outpatient Hygiene & 50 20 care waste mgnt - 40 Management 30 Child acute of HBP 20 pneumonia patients C-section & Appendicitis 10 complicated treatment deliveries 0 Q1-2014 Q4-2014 Q4-2015 Figures 5 and 6 present similar information from the BSC for the DHC. As shown, overall score increased from 76 percent to 96 percent after two years of implementation (figure 5). Among the quality components, the ones that started out the lowest and experienced the most notable improvement were management of the EPI, drug and medical equipment, and training activities (all averaged about 60 percent among the DHCs). By the last quarter of the pilot, HMIS reached 100 percent and the majority of the component scores were above 90 percent. 15 Figure 5. Overall quality score, DHC (%) Figure 6. Component quality scores, DHC (%) 100 90 HMIS 80 100 Intersector Financial 80 70 ial and CHS mgnt 60 supervisi… 60 40 50 20 Supervisio 40 EPI - n and deploym… 30 20 Drug & Nutrition equipment 10 mgnt mgnt 0 Training activities Q1-2014 Q4-2014 Q4-2015 The overall and component BSC scores for the CHSs are presented in figures 7 and 8, respectively. Similar to the case of DH and DHC, the overall quality score started out rather high, at 76 percent. After two years, this increased to 88 percent (figure 7). Looking at figure 8, one sees that FP started out as almost 100 percent. The scores for Q4 2014 and Q4 2015 were almost identical, showing little improvement in the second year except for the ‘hygiene and waste management’ component. Figure 7. Overall quality score, CHS (%) Figure 8. Component quality scores, CHS (%) 100 General and data mgnt 90 100 80 Managemen Hygiene & 80 t of HBP… waste mgnt 70 60 60 40 Delivery care 20 Drug mgnt 50 0 40 30 Prenatal care FP 20 10 Inpatient Nutrition care 0 Q1-2014 Q4-2014 Q4-2015 Note that some components in the BSCs not only reflect quality, they can also be seen as indicators of health system strengthening at the grassroot level. For example, BSCs for all three types of facilities contain indicators showing if there was an improvement in the supply chain management (drug management and supply to the CHSs from the DH, and drug management and availability in the CHSs themselves). Similarly, one can see an effect on the HMIS through the across-the- board increase in the component score for data management indicators in all facility types. It is worth noting still that the baseline values for these were quite high and performance reached nearly 100 percent in a short time period, again pointing to the need to adjust the instrument. 16 Quantity at CHS Facility provision of services for incentivized indicators was reviewed in detail using verified operational data. Results for each indicator are available from the author upon request. In general, there is an appreciable variation among CHSs although they all seem to follow similar pattern of ups and downs. There is a clear increasing trend among 9 out of 14 indicators—curative outpatient visits both for uninsured patients and other groups (poor, near poor, insured, and children under 6 years), follow-up with referral from higher level, management of hypertension cases detected, number of pregnant women receiving at least three antenatal checkups (ANC3), the timing of the fourth ANC, normal delivery, PNC, and TB screening. For the remaining five indicators, the pattern is either fluctuating (family planning, growth monitoring among children under 2 years and among children ages 2–5 years), or shows a downward trend (referral of complicated deliveries to higher level, HBP screening). The two nutrition indicators—growth monitoring for children under 24 months and for children ages 2–5 years also experienced a slight increase. However, the pattern is not consistent. Especially for growth monitoring of children ages 2–5 years, figure 9 shows a spike every other quarter, in quarter 2 and quarter 4 of 2014 and 2015. The absolute number of services is much higher in quarter 2 of 2015 compared to the same period one year before that, however, this seems to dwindle in quarter 4 of 2015. The biannual spikes could be explained by the fact that the indicator, which was instituted at the second revision, stated that the kids should be weighed every six months. However, the fact that this did not spread out smoothly throughout the year suggests that the ‘growth monitoring’ was not integrated in the annual checkup of the kids, otherwise, it would not look like a ‘campaign mode’ as depicted in the graph. Figure 9. Growth monitoring of children ages 2–5 years Some trends in the data are worth noting. On hypertension, figures 10 and 11 show contrasting pictures: the number of hypertension cases screened and detected increased sharply in the second quarter of the pilot, quarter 2 of 2014, but then dropped to the same or lower level compared to the 17 first quarter (figure 10). At the same time, the number of hypertensive cases being managed by the CHSs increased drastically and consistently from a baseline of virtually zero (figure 11). Some tentative conclusions can be made from these patterns. First, the CHSs may have reached their maximum capacity to detect more hypertensive cases, either because they exhausted all high-risk cases in their community or that for a small pocket of remaining cases, interventions of different nature may be required, such as much more active population education on the need to have hypertensive screening early. Second, as suggested in figure 11, it could be that RBF had instituted a practice of managing and providing consultation to HBP patients which was pretty much nonexistent before RBF. Figure 10. HBP screening Figure 11. Management of patients with HBP A somewhat similar contrast is observed in the case of delivery. As shown in figure 12, the number of normal deliveries attended in CHSs increased quite appreciably albeit with some sign of seasonality. On the other hand, figure 13 shows a rather steep decrease in the number of complicated deliveries referred to higher level. One reason could be that the CHS became better capable of handling complicated deliveries. However, this hypothesis is not supported with quality data—BSC shows some improvement in delivery care from Q1 to Q4 2014, but the score decreased slightly between Q4 2014 and Q4 2015 (figure 8). On the other hand, incentive payment is much higher for normal deliveries handled in the CHS than for complicated cases referred out (VND 70,000 versus 30,000). More in-depth research will be needed to see if the price difference generated a perverse incentive to keep patients with complicated deliveries and if this has negatively affected health outcomes, or if RBF has made the CHSs more proactive in dealing with complicated cases which they did not do before but could have done so. To answer this question, one would need to assess in detail all medical records of delivery cases, which is beyond the scope of this paper. Note that this is verified data, so presumably the appropriateness of referral (and non-referral) should have been vetted by the verification team. 18 Figure 12. Normal delivery Figure 13. Referral of complicated delivery Equity Within the modest scope and scale of the pilot, the project has made an attempt at improving equity: it paid more than three times for a curative outpatient contact for certain groups of patients compared to regular, uninsured patients. These groups of patients include the poor, the near poor, children under 6 years (all receiving free health insurance from the government), and other people insured from different venues (such as compulsory or voluntary insurance). For lack of a better word, we call these ‘special patients’. Classifying them as ‘disadvantaged’ will likely be imprecise: except for the poor and near poor, children under 6 years come from both disadvantaged and better- off families, and insured people in rural Vietnam for the most part are people on a payroll, retirees from the formal sector, or those who have the money to pay for voluntary insurance, so they are not the most disadvantaged. Nevertheless, the decision to adopt these groups of population as priority in the RBF program is understandable as it is based on an existing system of distinguishing patients. It would be too unrealistic to expect that the pilot embraces the task of precisely identifying the disadvantaged population to target. With all these caveats, figures 14 and 15 compare trends over time for the uninsured and ‘special’ patients’ curative visits. Both show a strong increase over time, and the slope is much steeper for the ‘special’ patients. Thus, the differential price seems to have exerted the expected effort and the pilot has been successful in giving the ‘special’ population group a somewhat more special treatment. 19 Figure 14. Curative outpatient contact among Figure 15. Curative outpatient contact uninsured patients among ‘special’ patients 20 DISCUSSION SUMMARY OF FINDINGS The paper has provided a detailed description of the RBF pilot in Vietnam and made an attempt to assess its performance in relation to the originally stated objectives. The scale of the pilot is rather small, with the total real disbursement slightly higher than USD 1 million. Nevertheless, a significant amount of work has gone into the pilot from all the stakeholders. A pre-pilot had been conducted to test the mechanism, significant amount of time had gone into developing and fine- tuning the BSC and the institutional arrangement. Compared to many RBF programs in the HRITF portfolio, the pilot in Vietnam is a rather lean model. This will eventually help with institutionalization efforts as the model did not impose a big intrusion into the current system. The two broad objectives of the pilot include testing the model and improving service delivery in the PHC setting. On objective 1, the pilot has proved that RBF can work in Vietnam. Its design is relevant to the country’s context, although more could have been done to prepare for a post-pilot stage. Within a rather short period of time, frontline health workers at both CHS and district levels picked up RBF naturally. The mechanism seemed to work smoothly, and the use of funds was fully situated within the existing financial management system. More than one year after the two- year pilot ended, stakeholders on the ground were still enthusiastic about the experience and able to recall its important features. After two years, the machine as it was built was running smoothly. The tool developed for the pilot was used also for the non-participating facilities and reported to remain in use after the pilot ended. It is also reasonable to conclude that RBF has been relatively effective in improving the quality and quantity of PHC service delivery. On average, all types of institutions involved—DH, DHC, and CHS—demonstrated an improvement in various quality dimensions. The improvement is more notable for DH and DHC than CHS. However, the fact that overall quality was already rather high to begin with and has almost reached the maximum level by the end of the two years suggests that further adjustment to the BSC is necessary. The story with quantity of services in the CHS is largely positive, but not fully straightforward and should be taken with a grain of salt. On the one hand, RBF has probably instituted a new practice of managing HBP patients. On the other hand, it did not seem to smooth out the biannual growth monitoring calendar for children ages 2–5 years. This is something to pay attention to if the pilot were to continue or roll out. The nuances in the patterns of hypertension screening and complicated delivery referral make a strong case for constant monitoring of the providers’ behaviors, continuous learning and adjusting the program to avoid potential perverse incentives, and finding innovative ways for improvement. Finally, on ‘equity’, the pilot achieved what it intended, which was an increase in service volume for the poor, near poor, insured, and children under 6 years. However, one can also see this as a mere supply response to a price difference, and the concept of equity is in no way rigorous. Finally, one should note that all these observations are the result of cross-sectional administrative data analysis, not rigorously designed impact evaluation in its true sense. Due to the short duration of the field visit, coupled with the fact that it took place more than one year after the pilot ended, we were not able to solicit detailed feedback from the stakeholders on the potential system effects of RBF beyond what is presented in the operational data. For example, 21 we could not firmly state if greater autonomy was given to RBF compared to non-RBF facilities, or if supervision was more regular. The key to some positive results of the pilot seems to be in the fact that it combines both the carrot and stick to the frontline providers—carrot being the incentive payment, the hands-on support, and the concrete guidance provided by the BSC, and stick being the regulations on roles and responsibilities backed with rigorous verification. In other words, while the existing system appears to focus on the necessary conditions for quality improvement, by imposing benchmarks, regulations on roles, functions, responsibilities, and clinical guidelines, RBF’s added value was to provide sufficient conditions, which are resources and incentives to mobilize the providers to make use of the existing interventions. IMPORTANT CONDITIONS FOR INSTITUTIONALIZATION Despite the positive experience, like many other RBF programs and virtually all externally funded programs, sustainability remains the moot question for the pilot. For RBF to be institutionalized in the Vietnam context, at least two critical issues need to be addressed: • Financial aspect: We have seen that the pilot was rather lean and constituted a relatively insignificant addition to the operation budget, especially of the DH and DHC. With less than USD 600,000 for 54 participating facilities in more than two years, the financial impact is small. Vietnam’s economy is developing very fast and finding additional fiscal space to accommodate RBF payment will not likely impose a major hardship. As the Director of Nghe An DOH stated, if there is a policy issued by ‘the Center’ (the government), the province will be able to follow. It will help persuade the government if (a) RBF can be more cost-efficient, for example, the costly external verification can be replaced by some kind of random check using data; and (b) RBF can deliver good value for money, that is, it can be highly effective in improving service delivery. For (a), different modalities for external verification will need to be tested, and for (b), a rigorous evaluation is required. For example, the RBF program in Zimbabwe has switched from indiscriminate monthly verification of all contracted facilities to a risk-based mechanism, whereby facilities are verified every month, twice a quarter, or every quarter depending on the expected risk for misreporting (Nguyen 2017). Another option to explore is to use the BSC without providing financial incentives (but still keeping the supportive supervision aspect). Experience from the RBF program in the Kyrgyz Republic shows that using the BSC alone combined with quarterly peer review could improve provider’s performance, albeit at a lower rate compared with providing financial reward tied to the BSC scores (Kyrgyz Ministry of Health 2017). • Institutional aspect: RBF will be institutionalized when it is fully absorbed in the system and not managed by a PMU. The pilot in Nghe An already had many features that were built on the existing system: the RBF Task Force partly consisted of DOH staff who had the responsibility to supervise the lower level; the DHC performed the functions (internal verification) that it would perform anyway under the government regulated roles and responsibilities; and the BSCs were developed based on existing guidelines. However, the key remaining question is who will play the role of the purchaser in the place of the PMU. VSS, the agency in charge of social health insurance, seems to be a logical choice. 22 However, as a purchasing agency, VSS can review claims but may not have adequate technical capacity or responsibilities to provide supportive supervision to the hospitals. It is the view of the author that VSS remains the most relevant option for the purchaser but the arrangement could be a blend that involves the participation of the DOHs. WHERE DO WE GO FROM HERE – FROM A PILOT TO THE SYSTEM Although the RBF pilot ended on a positive note, it could be among various successful pilots supported by development partners or originated from local initiatives, that did not get replicated, scaled up, or institutionalized. However, not picking up from where the pilot ended will be a missed opportunity to build on four years of intensive work and valuable lessons generated. On the other hand, as argued before, the implementation time is too short for the project to test and retest different features of RBF to reach a model that can be readily handed over to the government for institutionalization. As stated by one of the stakeholders, it would be good to have some more time to prepare for a rollout of the RBF. The following provides recommendations for what could be a five-year roadmap to gradually build the foundation and prepare the country for institutionalization of RBF. The ultimate goal is that after such an operation, RBF will not remain ‘a pilot’. This plan could be implemented by the government alone or with support from development partners, including the World Bank in its follow-on project. The roadmap to achieve this goal could be: • Using the first two years for fine-tuning the model: testing out options for lower-cost external verification, developing a good data system, exploring options for purchaser and fund flow mechanism, and fine-tuning the incentivized package taking into account lessons from the Nghe An pilot; • Simultaneously, strengthening efforts in capacity building, institutionalizing the tools, and securing stakeholder buys-in. The various BSCs in the pilot were developed with inputs from experts in the field. The process could have benefited from better involvement by key stakeholders/decision makers such as heads of concerned departments in the MOH. This will help secure the acceptance of the tools by the national players. The ultimate goal should be for the BSCs to be issued in official government decree(s), maybe as annexes to the various national benchmarks; • Using the remaining three years to gradually move the pilot into the system, financially and institutionally. The financial institutionalization can be arranged as a matched financing mechanism whereby grant financing for RBF stays at 100 percent in the first two years and gradually reduces as budget financing increases. Grant could be provided from the central to provincial government if there is extra fiscal space. Vietnam has experienced encouraging gross domestic product (GDP) growth in the last few years which provides a favorable condition for financing initiatives that have proved to be cost-effective. A final note of importance is that RBF is not a panacea. There are many system constraints which incentives to the providers cannot solve and which limit the providers from making full use of RBF: human resources, infrastructure, information system, and capacity for strategic purchasing. Therefore, parallel activities for system strengthening should continue to be an important part of 23 the effort. Within the scope of the RBF component itself, continuous learning will always be required to screen out potential adverse effects and develop further the positive aspects of RBF. 24 REFERENCES Kyrgyz Ministry of Health. 2017. “RBF Experience: Payment Based on Results for Service Providers (Territorial Hospitals and Primary Health Care Facilities).” Presentation During the Joint Annual Review, Bishkek, Kyrgyzstan, April 2017. Musgrove, P. 2011. Financial and Other Rewards for Good Performance or Results: A Guided Tour of Concepts and Terms and a Short Glossary. Washington DC: The World Bank. Nguyen, H. 2017. “Reimagining Result-Based Financing.” Presentation at Global Meeting on Strategic Purchasing for UHC: Unlocking the Potential. Geneva, April 2017. Shepard, D. S., W. Zeng, R. Mutasa, et al. 2016. “Cost-effectiveness of Results-based Financing in Zimbabwe: A Controlled Pre-post Study.” Presentation at The RBF Annual Workshop, Harare, Zimbabwe, September 2016. Vietnam Ministry of Health. 2016. “Social Health Insurance Scheme in Vietnam. Achievements and Challenges.” Presentation at Policy Research Institute, Ministry of Finance, Japan. Accessed on March 5, 2018 at https://www.mof.go.jp/pri/research/seminar/fy2016/tff2016_s1_04.pdf. Zeng W., D. S. Shepard, H. T. Nguyen, et al. 2017. “Cost-effectiveness Analysis of Results-based Financing in Zambia: Results from a Cluster Randomized Trial.” Unpublished Manuscript. Washington DC: The World Bank. 25 Over the last decade, results-based financing (RBF) health programs have been implemented in several countries at different levels of income. Due to its requirement of rigorous verification of results as a condition for financing, as well as a number of accompanying measures to help achieve the results, RBF has a promise of value for money. RBF’s potential for improving the performance of the service delivery system has led the government of Vietnam to undertake a pilot of RBF in the Nghe An province as part of a World Bank funded operation. The main objective of the pilot was to experiment an RBF approach in the Vietnam context, where public sector providers have been receiving budget allocation based on inputs rather than performance. A secondary objective was to test the effects of RBF in improving quality of care at the grassroots level and in addressing the challenges of emerging noncommunicable diseases. The intervention included quality improvement at the district level and both quality and quantity of services at commune health station. This paper provides a rapid assessment of the pilot and draws lessons for possible expansion of RBF in the country context. After two years of implementation, the results of the pilot are largely positive. The Balanced Score Cards used to measure quality were seen as an effective tool to help with monitoring and supervising the facility performance. The aggregate quality score increased from 70 percent to over 90 percent in participating district hospitals and from 78 percent to 88 percent in commune health stations. Service provision in the commune health stations, the lowest level of the health system, has increased appreciably for 9 out of 14 indicators. Though the pilot had some features that were built on the existing system, the issue of institutionalization remains a challenge for further RBF expansion in the country. ABOUT THIS SERIES: This series is produced by the Health, Nutrition, and Population Global Practice of the World Bank. The papers in this series aim to provide a vehicle for publishing preliminary results on HNP topics to encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account this provisional character. For free copies of papers in this series please contact the individual author/s whose name appears on the paper. Enquiries about the series and submissions should be made directly to the Editor Martin Lutalo (mlutalo@ worldbank.org) or HNP Advisory Service (askhnp@worldbank.org, tel 202 473-2256). For more information, see also www.worldbank.org/hnppublications. 1818 H Street, NW Washington, DC USA 20433 Telephone: 202 473 1000 Facsimile: 202 477 6391 Internet: www.worldbank.org E-mail: feedback@worldbank.org