96523 Scaling up HIV treatment for MSM in Bangkok: what does it take? —— a modelling and costing study Lei Zhang1, Nittaya Phanuphak2, Klara Henderson1, Siriporn Nonenoy2, Sasiwan Srikaew2, Andrew J Shattock1, Cliff C. Kerr1, Brenda Omune2, Frits van Griensven2, Sutayut Osornprasop3, Robert Oelrichs3, Jintanat Ananworanich2,4, David P Wilson1 1 The Kirby Institute, The University of New South Wales, Australia 2 Thai Red Cross Society AIDS Research Center, Bangkok, Thailand 3 The World Bank Group, Washington, United Sates should be States 4 Current address: U.S. Military HIV Research Program, Walter Reed Army Institute of Research, Silver Spring, MD, USA; Henry M. Jackson Foundation for the Advancement of Military Medicine, Bethesda, MD, USA Scaling up HIV treatment for MSM in Bangkok: what does it take? Table of Contents Executive Summary .............................................................................................................. 6 Background ......................................................................................................................... 10 Methodology ........................................................................................................................ 13 Data collation and synthesis ......................................................................................... 13 Modelling methodology .............................................................................................. 14 Forecasting epidemiological impacts and cost-effectiveness ...................................... 15 Key findings ........................................................................................................................ 17 Finding 1: Low HIV testing and treatment linkages in Bangkok MSM ..................... 17 Finding 2: High HIV testing service and treatment availability in Bangkok ............. 17 medical facilities Finding 3: Spending on linkage and service provision for Bangkok MSM ................... 18 Finding 4: Significant epidemiological benefits in HIV testing and ART ........................ 21 services scale-up Finding 5: Services scale-up are economical and cost-saving ........................................ 23 Discussion ............................................................................................................................ 27 Conclusion ........................................................................................................................... 30 Appendix.............................................................................................................................. 32 References ............................................................................................................................ 59 List of Tables Table 1 Current operational load and capacity availability of Bangkok medical facilities in providing HIV testing and ART services ........................................................................... 18 Table 2 Estimated investments on HIV testing and ART services among MSM in Bangkok, expected epidemiological outcomes and cost benefits during 2013-2022 ........................... 25 Table S1: Assessment of service provision and capacity of HIV testing and ART sites in Bangkok. ..........35 Table S2: Disability weights for HIV related health states from the 2010 Global Burden...........................42 of Disease Study Table S3: Assumed disability-weights for DALY calculations. The disability-weight for HIV-positive .....42 MSM is the maximum of the value for HIV-negative MSM and the corresponding HIV-positive category disability weight. Table S4: Facility characteristics and operational capacity of HIV testing services ....................................43 in 91 Bangkok medical facilities. Site information was obtained from the National Health Security Office (NHSO) database, service load and capacity data from capacity assessment survey, and distance information from site mapping exercise. Table S5: Facility characteristics and operational capacity of ART services ................................................47 in 48 Bangkok medical facilities. Data was obtained from the National Health Security Office (NHSO), Thailand THE WORLD BANK GROUP Table S6: Service load and capacity for (a) HIV testing services in 91 and (b) ART services .....................49 in 48 Bangkok medical clinics in 2011, stratified by facility types and distances to MSM hotspots. Table S7: Strategies and costs in linking high-risk MSM to HIV testing services in Bangkok. ...................50 Table S7a: Cost of linking high-risk MSM to HIV testing services ...................................................50 using community-based outreach through peer-educators Table S7b: Cost of linking high-risk MSM to HIV testing services....................................................51 using mobile point-of-care night clinics Table S7c: Cost of linking high-risk MSM to HIV testing services using Adam's Love website. ......52 Table S8: Cost distribution for HIV testing service provision in 13 Bangkok medical................................53 facilities in 2011. Table S9: Cost of linking high-risk diagnosed HIV+ MSM to ART services using.....................................54 the Case Management Model Table S10: Cost distribution for ART service provision in 13 Bangkok medical facilities in 2011. ...............55 List of Figures Figure 1 Low HIV testing and ART commencement rates among high-risk MSM in Bangkok .............................................................................................................................. 17 Figure 2 Strategies and costs for linkage and services provision of HIV testing and treatment among MSM in Bangkok. .................................................................................................. 19 Figure 3 Cost breakdown for HIV testing and ART services provision in Bangkok facilities .............................................................................................................................. 20 Figure 4 Projected epidemiological outcomes for scenarios of achieving universal ART coverage (80%) among MSM (undiagnosed and diagnosed) in Bangkok in 3, 5 and 10 years. .................................................................................................................................. 22 Figure 5 The most economical strategies to scale-up (a) HIV testing; (b) ART services among MSM in Bangkok. ............................................................................................................... 24 Figure S1: Low HIV testing and ART commencement rates among high-risk MSM in Bangkok. ...............34 Figure S3: HIV Infection Progression. ............................................................................................................37 Figure S4: Calibration of Optima to the HIV epidemic among MSM in Bangkok during ............................39 2000-2012 , (a) HIV prevalence; (b) number of diagnosed HIV+ MSM on ART Figure S5: Projected decline trend in the unit cost for HIV testing and ART provision in ............................56 relation to service capacity in Bangkok medical facilities. The solid dots present current service load and corresponding estimated unit costs. Scaling up HIV treatment for MSM in Bangkok: what does it take? Abbreviations AEM AIDS Epidemic Model ART Antiretroviral therapy BMA Bangkok Metropolitan Administration CEA Cost-effectiveness Analysis CI Confidence interval DALY Disability-adjusted life year FSW Female sex worker MARP Most-at-risk population MOH Ministry of Health MSM Men who have sex with men NHSO National Health Security Office POC Point-of-care PLHIV People living with HIV PWID People who inject drugs STI Sexually transmitted infections TRCARC Thai Red Cross AIDS Research Center UNAIDS Joint United Nations Programme on HIV/AIDS UNPD United Nations Political Declaration on HIV/AIDS VCT Voluntary counseling and testing WB World Bank WHO World Health Organization THE WORLD BANK GROUP THE WORLD BANK GROUP Executive Summary Scaling up HIV treatment for MSM in Bangkok: what does it take? Executive Summary Background: The HIV epidemic amongst men who have sex with men (MSM) in Bangkok is substantial. The population size of MSM in Bangkok is 120,000-250,000, with approximately one-third (33.5 percent) considered high-risk, characterized by their young age, multiple partnerships, frequent unprotected anal intercourse, and sexual activities around MSM hotspots. In metropolitan Bangkok, HIV prevalence among MSM reportedly increased from 21 percent to 28 percent between 2000 and 2012. The Thai Working Group of Estimation and Projection (2013) projected an estimate of 39,000 new HIV infections would occur in Thailand during 2012-2016, based on the AIDS Epidemic Model (AEM). MSM will account for 44 percent of these new HIV cases, and 25-30 percent of these infections will likely to occur in Bangkok. In 2011, the United Nations held a high-level meeting on HIV/AIDS where they adopted the ambitious epidemio- logical targets of the United Nations Political Declaration on HIV/AIDS (UNPD), to be met by 2015. Attaining these specific targets would lead to substantial progress towards ending AIDS. UNAIDS has also been prioritizing the “Getting to Zero” initiative (“Zero new HIV infections. Zero AIDS-related deaths. Zero discrimination.”). The Bangkok Metropolitan Administration (BMA) recently responded with the “Bangkok: Getting to Zero” initiative, which strategizes an increased focus on prevention amongst MSM in the city. The clinical trial, HPTN052, demonstrated a 96 percent reduction of HIV transmission among heterosexual discordant couples who received ART. “Treatment as prevention” has become an increasingly accepted strategy to prevent new infections. A cost-effectiveness analysis comparing current levels of investment in targeted HIV prevention interventions for MSM in Bangkok (including treatment), with scenarios of increased coverage, would provide evidence to shape efficient national and metropolitan strategies. A return-on-investment analysis would provide an economic rationale to finance this strategy in allocating sufficient resources to address the epidemic at the most appropriate scale. Demonstra- tion by the study that a significant reduction in transmission (including potential elimination) are both feasible and cost effective, may galvanize global political support. Study objectives: • To assess the cost, service load, and capacity of Bangkok’s health facilities for linking and providing HIV testing and ART programs to MSM; • To identify the most effective and cost-effective strategy to allocate available and additional resources to in order to achieve universal ART coverage (80 percent of treatment-eligible) and minimize the number of new HIV cases and related deaths among Bangkok MSM. Key findings: • HIV testing and treatment coverage in Bangkok MSM has been low. Approximately 61,975 (40,200-83,750) MSM are considered high-risk (defined as those engaged in unprotected anal intercourse in the past three months), but only 14,387 (23 percent) were tested for HIV in 2011. Among those tested, 75 percent of tests were conducted in a Thai Red Cross Anonymous Clinic or Silom Community Clinic. The National Health Security Office (NSHO) reported 989 MSM initiated ART in 2011, corresponding to 30 percent of those diagnosed and treatment-eligible. • In 2012, the estimated ART coverage was 19.8 percent among undiagnosed and diagnosed treatment-eligible MSM (2013 WHO treatment threshold, 500/ml). • Medical facilities in Bangkok had high spare capacity in HIV testing and ART services. On average, 5.3 percent, 39.1 percent, 72.9 percent, and 89.4 percent of HIV testing capacity 6 THE WORLD BANK GROUP in research clinics, public facilities, BMA health centers, and private hospitals were unfilled respectively. This corresponds to an additional 405,144 HIV tests that can be carried out per year in metropolitan Bangkok by further utilizing available infrastructure. • Existing ART infrastructure in public and private hospitals can be further utilized, with about 48 percent and 36 percent currently used. This means that a total of 94,384 available spaces for further ART expansion. • Existing recruiting strategies require an average of $36 to link an MSM to HIV testing services. Public facilities had the lowest unit cost ($15/test) in providing HIV testing services, followed by research clinics ($33/test), BMA health centers ($34/test) and private hospitals ($40/test). • The peer-driven “Case Management Model” cost $177 to follow one diagnosed and treatment-eligible MSM case to ART initiation. On average, the cost for ART provision per person-year was 23.3 percent lower in public facilities ($1,220) than in private hospitals ($1,587). • Sustaining current rates of HIV testing and ART provision will require $73.8m in investments over the next decade, and lead to ART coverage of 44.2 percent by 2022. However, an additional $55.3m during the same period will increase ART coverage to 80 percent (universal coverage) by 2022. • To achieve universal coverage by 2022, 46,700 (30,300-63,200) MSM would need to be linked to HIV testing, and 197,100 (127,800-266,300) tests would need to be conducted. This enables an extra 12,600 (8,800-16,600) eligible MSM to be linked to ART with 101,000 (70,100-132,400) ART person-years. As a result, by 2022, an estimated 5,100 (3,600-6,700) HIV-related deaths and 3,700 (2,600-4,900) new cases would be averted, corresponding to a 53.0 percent and 35.0 percent reduction respectively, in comparison with the cases in 2012, attaining the UNPD goals (by 2022) in Bangkok MSM. • ART expansion to universal coverage by 2022 is highly cost-effective. The costs for each averted HIV-related death, new infection, and DALY are $10,809 (9,071-13,274), $14,783 (12,398-17,960), and $351 (290-424) respectively. Summary and recommendations: • One-third of MSM is considered high-risk; it is important to prioritize limited health resources to specifically target the highest risk MSM. • Current HIV testing and ART coverage is very low among MSM in Bangkok, rapid scale-up of corresponding services is a high priority. • Overall, Bangkok medical facilities have large spare capacities of HIV testing and ART provision to enable substantial expansion of ART coverage. • Over 75 percent of HIV tests among MSM have been conducted in two research clinics whose remaining available capacity is low. The burden of testing service provisions needs to be shifted to other medical facilities. Public facilities have the lowest unit cost ($15/test) for testing service provisions and sufficient spare testing capacity for shifting of service load. Together with its lower unit cost for ART provision (in comparison with private hospitals), public facilities will play a unique and irreplaceable role for ART scale-up among Bangkok MSM. Drawing from the successful experiences of research clinics will enable the creation of a patient-friendly environment in public facilities. Involving experts from research clinics and MSM community representatives in the implementation and management of services delivery in public facilities will improve the overall service quality. Scaling up HIV treatment for MSM in Bangkok: what does it take? 7 • It is beneficial to employ multiple recruitment methods to link MSM to HIV testing services. However, the currently low ART coverage implies an inefficient link between positive diagnosis and ART initiation. The piloted “Case Management Model,” through peer follow-up, has been shown to be effective in connecting diagnosed individuals to ART initiation and may be a potential strategy for further expansion. • The current level of investment into HIV testing and ART service will not achieve UNPD goals by 2022. However, a moderate increase in investments will enable the achievement of universal ART coverage by 2022 and halve new HIV cases and HIV-related deaths over the same period. In addition, ART expansion to universal coverage by 2022 is highly cost-effective and therefore should be considered as an effective strategy for implementation. • Early achievement of universal ART coverage among MSM in Bangkok will result in even greater epidemiological benefits. • Investment for HIV responses among MSM in Thailand is under-funded. More importantly, due to persistent social stigma, MSM are often afraid of being seen by peers as well as discriminated against by healthcare workers when receiving HIV tests. Stigma and discrimination is the most important barrier preventing scale-up of services for MSM. 8 THE WORLD BANK GROUP THE WORLD BANK GROUP Background Scaling up HIV treatment for MSM in Bangkok: what does it take? Background Thailand is one of the countries in Asia that is most severely affected by the HIV epidemic, but effective intervention programs have contained the epidemics in most of the Thai populations. During 2006-2011, HIV prevalence among female sex workers and people who inject drugs substantially decreased from 5.7 percent and 40.0 percent, to 3.2 percent and 26.6 percent respectively [1], while prevalence in general low-risk population observed a slight reduction from 1.4 percent to 1.3 percent. In contrast, HIV prevalence among MSM remained between 20-30 percent across the country [1]. In metropolitan Bangkok, HIV prevalence among MSM reportedly increased from 21 percent to 28 percent [2-7]. The Thai Working Group of Estimation and Projection (2013) project an estimate of 39,000 new HIV infections will occur in Thailand during 2012-2016 based on AIDS Epidemic Model (AEM). Men who have sex with men (MSM) will account for 40 percent of these new HIV cases, and 25-30 percent of these infections will happen in Bangkok [8]. The National Strategic Plan for HIV for this same period aims to reduce the number of new infections by two-thirds from the estimate, and reduce number of deaths in people living with HIV (PLHIV) by more than 50 percent. In 2011, the United Nations high-level meeting on HIV/AIDS adopted the epidemiological targets of the UN Political Declaration to be met by 2015. The goals include reducing both HIV new infections and HIV-related mortality by 2015, and limiting mother-to-child transmission of HIV to below five percent among breastfeeding and two percent among non-breastfeeding women. Although attaining these specific targets will lead to substantial progress towards ending AIDS, achieving them by 2015 is no longer possible. This study investigates the financial commitment required to halve HIV-related deaths and new infections over the period of 2013-2022. The epidemic of HIV amongst MSM in Bangkok is severe and substantial. The population size of MSM has been estimated between 120,000 and 250,000, accounting for one-fourth of the MSM population in Thailand [9]. Approximately one-third (33.5 percent) of Bangkok MSM has been considered as high-risk, characterized by their young age, multiple partnerships, frequent unprotected condom use, and appearance in MSM hotspots where they practiced high-risk sexual activities. Alarmingly high rates of incidence have been detected among high-risk MSM (7.7 per 100 person-years in 2007 [2]). MSM are encouraged to take up HIV testing twice a year, as public and research clinics provide free HIV testing. Antiretroviral therapy (ART) is procured centrally from the Thai Government Pharmaceutical Organization to all ART sites. According to the National AIDS Program (NAP) database, until March 2013, the National Health Security Office (NHSO) program covered care and treatment for 79.5 percent of PLHIV in Thailand, while 16.9 percent are covered under the Social Security Office program, 2.6 percent under Civil Servant Medical Benefit Schemes, and only one percent under other categories, such as self-pay or research studies. CD4 count threshold for ART eligibility in Thailand during the study period is less than 350 cells/mm3, although the country is moving towards ART regardless of CD4 in fiscal year 2015. Curbing the epidemic in this high-risk subgroup will reduce the overall epidemic trend in the broader population. The long-term costs of this unchecked epidemic will be substantial in treatment, and other health related costs, as well as potential impacts on productivity. Despite this, investment in prevention for MSM has been modest, accounting for only 1.3 percent of the total proposed budget in the 10th National HIV Plan (for years 2007-2011). The Bangkok Met- ropolitan Administration (BMA) has recently responded to this important public health problem with the “Bangkok: Getting to Zero” initiative, which strategizes an increased focus on prevention amongst MSM in the city. A recent study by the World Bank demonstrated that, at a global level, 10 THE WORLD BANK GROUP comprehensive preventative interventions for MSM (including treatment) are both cost-effective and will have a significant impact on diminishing the growth of HIV epidemics within general national populations [10]. A cost-effectiveness analysis comparing current levels of investment in targeted HIV prevention interventions for MSM in Bangkok (including treatment), with scenar- ios of increased coverage, would provide evidence to shape efficient national and metropolitan strategies. A return-on-investment analysis would provide an economic rationale to finance this strategy by allocating sufficient resources to address the epidemic at the most appropriate scale. Demonstration by the study that a significant reduction in transmission (including potential elim- ination) are both feasible and cost effective, may galvanize global political support. Despite these initiatives, societal stigma and discrimination against MSM is still widely prevalent in Thailand and undermines the HIV response effort. HIV-positive MSM may experience stigma due to both their sexual orientation and infection status (8). Among Thai MSM, being stigmatised is cited as one of the major reasons for poor treatment received from primary care (9), poor uptake of vol- untary HIV testing, lack of serostatus disclosure to partners (10), delay in access to health care (11), depression and isolation (12). In 2008, a review of economic evaluation literature relating to Thailand revealed inadequate resources and capacity in conducting domestic assessments. It was found that the majority of cost-outcome studies were subject to bias, as they relied on low-quality evidence [11]. A need for high quality and standardized methodology was identified, and the Health Intervention and Technology Assessment Program (HITAP) published the National Methodological Guidelines for Health Technology Assessment and economic evaluation of health interventions [12]. Options under the scenarios of increased coverage will be developed in line with Treatment 2.0 priority areas, which aim to increase coverage in an environment of fiscal constraint. Responding to the HIV epidemic and political momentum, this study focus particularly on high-risk MSM in Bang- kok the following objectives: (1) assess the cost, service load and capacity of Bangkok’s health facilities for linking and providing HIV testing and ART programs to MSM at high risk of HIV; (2) identify the most effective strategy, based on costs of service provision, to allocate available and additional resources to achieve ‘universal’ ART coverage (80 percent of treatment-eligible) and minimise the number of new HIV cases and HIV-related deaths among MSM in Bangkok. Scaling up HIV treatment for MSM in Bangkok: what does it take? 11 Scaling up HIV treatment for MSM in Bangkok: what does it take? Methodology THE WORLD BANK GROUP Methodology Target population We specifically targeted MSM in metropolitan Bangkok. “Active MSM” is defined as “men who have had sex with other men in the past 12 months.” This study does not distinguish subgroups of MSM, such as transgender, male sex workers, and MSM with other specific risky sexual behaviors. We also assumed that MSM who utilized health facilities have similar HIV infection risk to and are “well-mixed” with those otherwise. Data collation and synthesis We collected the required epidemiological, service load, and capacity and costing data for HIV testing and ART services in Bangkok facilities. The main data sources include an extensive literature review on published and grey literature, database provided by the NHSO Thailand, capacity assessment survey in NHSO-listed medical facilities, and an empirical study on HIV testing and ART cost breakdown in 13 Bangkok medical facilities. Epidemiological data Epidemiological indicators, including population size estimates of MSM, characteristics of the HIV epidemic (e.g., HIV prevalence, annual HIV diagnoses, number of people on ART), risk behaviors of MSM (e.g., number of sexual partners and level of condom usage in sexual acts) were synthesized based on a systematic literature search for all available data in published articles, conference presentations and reports for Bangkok MSM over the period 2000-2012. Healthcare-seeking behaviors were reported by the NHSO database. The NHSO program provided free HIV testing twice a year for Thai citizens, regardless of health care insurance schemes they were registered for, through NHSO-registered HIV testing sites. For Thai PLHIV who use the NHSO scheme (around 80 percent of Thai PLHIV), NHSO also provides free antiretroviral treatment (ART), CD4 count twice a year (regardless of ART status), HIV viral load once a year, resistance testing when clinically indicated, and other key safety lab tests (for those on ART). NHSO-registered sites need to enter test results and clinical information into the NHSO electronic database in order to get cost reimbursement from NHSO. The number of Thai citizens who accessed HIV testing, number of those who tested HIV-positive, number of HIV-positive people who registered into the NHSO system for HIV treatment and care, number of HIV-positive people who started first-line and second-line ART regimens, CD4 count at entry into NHSO system, and the number of HIV-positive people who achieved undetectable HIV viral load after ART can be obtained by year of services from the database. Relevant data were synthesized to obtain best point estimates and uncertainty ranges for all the parameters used in later modelling evaluation. Mapping of MSM hotspots in Bangkok was obtained through previous mapping exercises shared by stakeholders, through an Internet survey via Adam’s Love website, and experts in the field. These hotspots are related to venues frequented by MSM in Bangkok. Venues mainly include saunas, spas, educational institutions, and department stores that are very well known to MSM. In addition, based on the address information provided by the NHSO database, we estimated the distance from the facilities to the nearest MSM hotspot on Google Maps. Service load and capacity data We conducted a telephone/letter survey to assess service load and capacity in providing HIV testing and ART services in 91 NHSO-listed medical facilities across Bangkok in 2011. The survey has Scaling up HIV treatment for MSM in Bangkok: what does it take? 13 been intentionally designed to be concise, with specific questions on the following aspects: (1) type of facility; (2) capacity to provide HIV testing; (3) capacity to provide ART; (4) capacity to provide treatment of HIV-related opportunistic infections and co-infections; (5) capacity of HIV reporting and surveillance. The complete survey is listed in Table S1. For the following modelling and resource allocation exercise, we made the assumption that all spare capacities in these facilities may be prioritized to MSM. This is based on two observations. Firstly, HIV prevalence among FSW and PWID have declined from 11 percent and 44 percent in 2000 to 2.5 percent and 24 percent in 2010 in Thailand, whereas HIV prevalence among MSM increased from 17 percent in 2003 to 20 percent in 2010. Secondly, the population size of MSM (550,571) is triple the sum of FSW (123,530) and PWID (40,300) in Thailand, but the HIV testing rates are far lower (MSM: 29 percent versus FSW: 51 percent and PWID 41 percent) [13, 14]. Based on these two reasons, we argue that the scale-up of HIV programs among MSM has much greater urgency than in other at-risk population, in which slower scale-up may be justifiable. Costing data for service linkage and provision Costing data for service linkage were collected based on a survey of available activities that con- nect eligible MSM to appropriate HIV testing and ART services. This part of data collection was conducted mostly in collaboration with in-country collaborators and local NGOs that conduct the services. More detail description of the services was provided in Table S7 and S9. In brief, we identified three major recruitment methods of MSM to HIV testing services: (1) conventional community-based outreach via peer-educator; (2) mobile point-of-care (POC) night clinics provided by Thai Red Cross AIDS Research Center (TRCARC) and BMA health centers; (3) Adam’s Love websites with innovative follow-up technologies hosted by TRCARC. AIDS Projects Management Group was the organization that piloted the sole linkage program to facilitate diagnosed and eligible HIV+ MSM to connect to ART services. The linkage model has been named “case-management model” (Table S9). Based on internal reports and communication with the responsible organizations, for each of these linkage programs, we collected indicators on program spending (e.g., cost of implementation and operation) and program effects (e.g., the number of individuals connected to HIV services). Out of the 91 NHSO-listed medical facilities, 13 were specifically chosen to collect costing data on service provision. The 13 sites were selected based on the current high number of MSM who accessed HIV testing and/or ART services at these sites, along with potential capacity to increase the scale of the services in the future. Data collection was conducted by a team of data collectors trained by Thai Red Cross in conjunction with key personnel at each of the sites. Overarching data was collected per site, including the characteristics of the sites (e.g., type of site, opening hours), service capacity, service load in the past 12 months, and cost breakdown of service provision. The main categories for HIV testing provision included staffing costs, commodity costs, laboratory costs, and operational costs specifically for HIV testing, whereas the categories for ART provision were monitoring costs, treatment cost, adherence cost, and operational costs specifically for ART. Modelling methodology To assess HIV epidemic trends and project the cost-effectiveness of investment scenarios, We employed a mathematical model to assess HIV epidemic trends and project the cost-effectiveness of investment scenarios. The epidemiological model, known as Optima, uses best-practice HIV 14 THE WORLD BANK GROUP epidemic modelling techniques and incorporates biological transmission processes, infection progression and sexual mixing patterns and other high-risk behaviors (Figure S3); we refer the reader elsewhere for a description of the model [15]. Optima incorporates a model of HIV transmission and progression. The model uses a coupled system of ordinary differential equations to track the movement of people between health states (Figure S1). The overall population is partitioned by group and health state. The model distinguishes people who are undiagnosed, diagnosed, and on effective ART. Diagnosis of HIV-infected individuals occurs based on an HIV testing rate dependent on CD4 count and population type. Similarly, diagnosed individuals begin treatment at a CD4 count dependent rate. The model tracks those on successful first- or second-line treatment (who have an increasing CD4 count) and those with treatment failure. The model is calibrated to match HIV prevalence data, and the uptake of ART from 2000-2012. The UNPD goals require reducing the number of HIV-related deaths and new cases by 50 percent by 2022. We investigate three ART scale-up scenarios, which represent the achievement of universal ART coverage among Bangkok high-risk MSM by 2022, 2017, and 2015 respectively. For each of these scenarios, we employed an optimization routine to identify the most economical way to achieve these targets. The two key optimization indicators are the HIV testing rate () and annual ART commencement rate (). In brief, if these rates increased from currently (0,0) to (+0,+0), the increase in ART coverage is −0=(+0)(+0)−00, and the associated extra cost C can be represented by: where 1 to 4 are the unit costs for each step of service linkage and provision (as illustrated in Figure 1). The optimization route minimizes total cost by finding the best combination of linkage and service provision strategies provided by various type of medical facilities (full details in Appendix). Forecasting epidemiological impacts and cost-effectiveness We project the trajectory of HIV epidemic among Bangkok MSM over the period of 2013-2022. The key epidemiological indicators for impact evaluation include the number of new HIV cases, HIV-related deaths, and number of ART person-years in service provision. We used disability-adjusted life years (DALYs) to measure the overall impact of HIV programs and for basic health economic calculations. The most thorough empirical study of disability weights is the 2010 Global Burden of Disease Study [16]. Based on the forecasted epidemiological outcomes and investment, we calculated costs required for each HIV-related death, new HIV case, and DALY averted. A strategy is considered “cost-saving” if cost per DALY averted is less than one GDP per capita and cost-effective if less than three GDP per capita [17]. All dollars are U.S. dollars in 2011. A discounting rate of 3 percent for costs and DALYs was used. A discounting rate of three percent for expenses and DALY is used. Full technical details are provided in the Appendix. Scaling up HIV treatment for MSM in Bangkok: what does it take? 15 Scaling up HIV treatment for MSM in Bangkok: what does it take? Key findings THE WORLD BANK GROUP Key findings Finding 1: Low HIV testing and treatment linkages in Bangkok MSM Out of 120,000-250,000 MSM in Bangkok, approximately 61,975 (40,200-83,750) MSM were consistently involved in high-risk sexual behaviors and the likely driver of HIV epidemic. Our capacity assessment survey indicated that 14,387 MSM were tested for HIV in 91 Bangkok medi- cal facilities in 2011, corresponding to an overall testing coverage of 23 percent. Estimated 4,028 MSM were diagnosed to be HIV+, and 3,303 (82 percent, [18]) had CD4 cell count level below 500/ml and were treatment-eligible. NHSO database reported that 989 MSM initiated ART in 2011, corresponding to a 30 percent treatment commencement rate. If the 2010 WHO guideline (treatment threshold CD4<350 /ml) was considered, the corresponding number of treatment-eligible MSM would be 2,498, with an annual treatment commencement rate 40 percent. Figure 1 Low HIV testing and ART commencement rates among high-risk MSM in Bangkok 23% 28% 82% 30% Finding 2: High HIV testing service and treatment availability in Bangkok medical facilities A total of 91 Bangkok medical facilities provided HIV testing services information in our capacity assessment survey (Table S4). Among these facilities, 31 were public facilities that provide both HIV testing and ART services, 44 BMA health centers that provide HIV testing services only, 14 private hospitals, and two research clinics (Thai Red Cross Anonymous Clinic hosted by TRCARC and Silom Community Clinic). Public facilities and private hospitals were capable of providing ART services. In terms of HIV testing services, our findings indicated that private hospitals have the largest capacity, on average is 23,752 per site, and most of these capacities (~89.4 percent) remained unfilled (Table 1). Research clinics were capable of providing 15,792 HIV tests per site per year, but only 5.3 percent of its capacity remained. The 31 public providers and 44 BMA health centers were capable in providing 7,047 and 717 tests per site, and 39.1 percent and 72.9 percent of these capacities remained available, respectively. Overall, on top of the current service load, Bangkok medical facilities may still be able to provide an additional 405,144 HIV tests per year. For ART provision, public facilities were able to accommodate a maximum of 4,217 ART patients per site per year, whereas capacity in private hospitals was about half of this (2,274 per site per year). About 52.1 percent and 64.2 percent of these capacities remained available, giving rise to a total of 94,384 available spaces for further expansion (Table 1). Scaling up HIV treatment for MSM in Bangkok: what does it take? 17 A detail mapping of facility locations and MSM hotspots have been conducted (Table S6). We did not find significant association between distance to MSM hotspots and service load in HIV testing and ART provision in Bangkok medical facilities (Spearman Correlation, HIV testing: r=-0.0449, p= 0.7341; ART: r=-0.2110, p=0.4163, Table S6). Table 1 Current operational load and capacity availability of Bangkok medical facilities in providing HIV testing and ART services Finding 3: Spending on linkage and service provision for Bangkok MSM We reviewed available strategies in linking MSM to HIV testing and ART services over the period of 2011-2013. We estimated their effectiveness by comparing the investment on these strategies and the number of MSM recruited. New web-based technology, such as Adam’s Love website (hosted by TRCARC) is the most effective means to recruiting MSM for HIV testing ($24/per- son, Figure S7a). Followed by this is mobile night clinic, which provided POC HIV testing in MSM hotspots, and costs $26 per recruitment (Figure S7b). The conventional community-based outreach by peer-educator was found to be the most expensive mean, costing $72 to link an MSM to HIV testing services (Figure S7c). For ART linkage, “Case Management Model” (piloted by AIDS Projects Management Group, APMG) provided an effective mechanism in following-up diagnosed MSM with ART and adhering to treatment. This program cost $178 to link one extra diagnosed case to initiate ART (Figure 1). Public facilities had the lowest unit cost ($15/test) in providing HIV testing services, followed by research clinics ($33/test), BMA health centers ($34/test) and private hospitals ($40/test). Running a public facility for HIV testing services required $338,736 per year, whereas the corresponding costs for private hospitals and research clinics were $331,448 and $495,260 per year. BMA health centers were usually much smaller in size and cost less ($34,619 per annum) to run (Figure 3). In public facilities, BMA health centers and research clinics, the proportion of spending on staff salaries were similar (30-40 percent). In contrast, the percentage of spending on staff salaries in private hospitals is slightly greater (46 percent) whereas public and private hospitals spent 18 THE WORLD BANK GROUP a majority of resources on conducting HIV testing (67 percent and 52 percent respectively). Around 30 percent of spending on BMA centers was dedicated for HIV testing (Table S8). Notably, the same staff members in BMA centers were also responsible for conducting tests for sexually transmitted infections (STIs) and a typical ratio of workload split between HIV and STI testing is 1:2 (personal communication with BMA centers). The total spending on ART services was $1,299,636 and $571,143 per year in public facilities and private hospitals respectively. On average, cost for ART provision per person-year was 23.1 percent lower in public facilities ($1,220) than in private hospitals ($1,587). Figure 2 Strategies and costs for linkage and services provision of HIV testing and treatment among MSM in Bangkok. Scaling up HIV treatment for MSM in Bangkok: what does it take? 19 Figure 3 Cost breakdown for HIV testing and ART services provision in Bangkok facilities Figure 3a. Distribution of cost for the provision of HIV testing services in public facilities, private hospitals, research clinics and BMA centers. The costs per test provision were calculated by dividing the average running cost for HIV testing provision per facility by the average number of HIV tests conducted per facility per year (public facility = 22,243; private hospitals = 7,774; research clinics = 14,969; BMA centers =1,021). The calculation was based on the 13 selected Bangkok medical facilities. 20 THE WORLD BANK GROUP Figure 3b. Distribution of cost for the provision of ART services in public facilities and private hospitals. The ART costs per person-year were calculated by dividing the average running cost for ART provision per facility by the average number of ART patients per facility (1,066 per public facility and 361 per private hospital). The calculation was based on the 13 selected Bangkok medical facilities. Finding 4: Significant epidemiological benefits in HIV testing and ART services scale-up If the current investment on HIV services is maintained over the next decade, by the end of 2022, it is expected that around 43,000 (27,900-58,000) and 5,100 (3,500-6,700) MSM at high risk will be linked to HIV testing and ART services, respectively (Table 2), increasing ART coverage among high-risk MSM from the current level of 20 percent to 44 percent in 2022. However, if universal ART coverage (defined as 80 percent coverage among treatment-eligible people) is to be achieved in 10 years, an additional 46,700 (30,300-63,200) MSM would need to be linked to HIV testing, enabling a further 12,600 (8,800-16,600) eligible high-risk MSM to initiate ART (Table 2). Our model indicates that this would lead to the aversion of 5,100 (3,600-6,700) HIV-related deaths and 3,700 (2,600-4,900) new cases by 2022, corresponding to 53 percent and 35 percent reductions compared with 2012 levels. Achieving universal coverage in the next five years will avert 6,900 (4,800-9,100) deaths and 5,700 (4,000-7,500) new infections, corresponding to 68 percent and 46 percent reduction over the next decade, whereas achieving this goal in three years would prevent 8,100 (5,600-10,600) deaths and 7,600 (5,300-10,000) new cases, corresponding to 75 percent and 54 percent reductions (Figure 4). Scaling up HIV treatment for MSM in Bangkok: what does it take? 21 Figure 4 Projected epidemiological outcomes for scenarios of achieving universal ART coverage (80%) among MSM (undiagnosed and diagnosed) in Bangkok in 3, 5 and 10 years. 22 THE WORLD BANK GROUP Finding 5: Services scale-up are economical and cost-saving It is important that HIV testing and ART services are scaled-up in the most economical way. The co-existing recruitment strategies led to an average of $36 for each MSM linked to services. The “Case Management Model” is the sole strategy being implemented in the Bangkok gay community and costs an average of $178 for ART linkage. Available resources can be optimised for service expansion. The most efficient way in Bangkok is to first use the 83,253 available testing spots in public facilities, given their lowest unit cost for conducting HIV testing. This is then followed by 1,662 available tests that can be conducted at research clinics, then 23,008 in BMA centers and finally 297,220 in private hospitals if necessary (Figure 5a). However, the available capacity in public facilities, alone is already sufficient to provide the required number of extra tests for all scale-up scenarios. Currently available clinical infrastructure for supporting ART (94,384 persons-years of ART support) would suffice to provide necessary services in all scale-up scenarios. The most cost-efficient strategy would involve prioritising ART first in public facilities given its lower provision cost (Figure 5b). Achieving universal ART coverage by 2022 among MSM at high risk will require a total of $129.1m over the next decade, $55.3m more than the spending of sustaining the status quo ($73.8m). The costs for each averted HIV-related death, infection and DALY would be $10,809 (9,071-13,274), $14,783 (12,389-17,960) and $351 (290-424) respectively. Achieving universal coverage in earlier will result in slightly greater cost-effectiveness ratios (Table 2). In contrast, ART scale-up from scenario that achieves universal coverage in 10 year to in 5 and 3 years would yield in higher incremental cost-effective ratios of $440 (360-529) and $518 (427-621) per DALY averted, respectively. Scaling up HIV treatment for MSM in Bangkok: what does it take? 23 Figure 5 The most economical strategies to scale-up (a) HIV testing; (b) ART services among MSM in Bangkok. 24 THE WORLD BANK GROUP Table 2 Estimated investments on HIV testing and ART services among MSM in Bangkok, expected epidemiological outcomes and cost benefits during 2013-2022 Scaling up HIV treatment for MSM in Bangkok: what does it take? 25 Scaling up HIV treatment for MSM in Bangkok: what does it take? Discussion THE WORLD BANK GROUP Discussion Our findings indicate that there is currently limited utilization of HIV testing services among MSM at high risk in Bangkok but there is sufficient infrastructure available to support scale-up of testing and treatment towards universal coverage. Since only one-fifth of treatment-eligible MSM were on ART in 2012, there is an urgent need for treatment expansion. Current service utilization and associated costs differ substantially between venue types. HIV testing has been most economically provided by major public hospitals. However, research clinics have been the most effective in reaching MSM at high risk in Bangkok, conducting over 75 percent of HIV tests. However, research clinics cannot cope with the increasing demand of HIV tests required for ART expansion. One of the most important factors associated with any larger-scale expansion of testing and treatment is to remove the social barriers from the most economic venue types. The successful experiences of research clinics, such as their efficient client-management system, friendly approach and quick turn-over of testing outcomes can be drawn upon for other settings which do have substantial capacity for expanded use. Our analysis indicated that public hospi- tals, together with small contributions from research clinics and BMA centers, may be sufficient to provide the necessary capacity for HIV testing in all ART scale-up scenarios for achieving universal coverage. In contrast, the current unit cost for private hospitals is high with a greater proportion of its investment being spent on staffing costs. We found that private hospitals have the largest capacity for increasing HIV testing yet their usage is currently very low. Stigma and discrimination are massive impediments to utilization of available services. If stigma and discrimination barriers can be reduced such that available infrastructure is utilized, then there is capacity to service all need. We have investigated an optimised investment pathway to scale-up ART to universal coverage and estimated that it can be effective in reducing new HIV cases and related deaths. In comparison with sustaining investment ($73.8m over 2013-2022), an additional $55.3m investment over the same period will reach the 80 percent coverage mark among MSM in Bangkok at high risk and expected to lead to one-third the number of new infec- tions and one-half the number of HIV-related deaths among MSM at high risk by 2022 compared with 2012 levels. This is consistent with previous modelling in Thailand (18) and elsewhere (19, 20). The cost-effectiveness of ART scale-up scenarios ($351-414/DALY averted) is deemed to be comparable to funded ART programs in other settings, including in lower-resourced settings (20). Investment for HIV responses among MSM in Thailand is under-funded. More importantly, due to persistent social stigma, MSM are often afraid of being seen by peers as well as discriminated against by healthcare workers when receiving HIV tests. Stigma and discrimination is the most important barrier preventing scale-up of services for MSM. Inconvenient operating hours of testing sites,concerns over confidentiality and lack of friendliness of medical personnel are symptoms of the barriers to HIV testing (21). The two research clinics which currently provide HIV testing services to the majority of Bangkok MSM have operating hours in the evening and during weekends. Internet sites have proven to be efficient for recruiting MSM into HIV testing services compared with conventional community-based outreach; this approach has also become highly effective in other settings (22). However, internet technology cannot be a replacement of community-based outreach, particularly for retaining MSM in care. Employing multiple recruitment strategies for linking MSM to services is required to access all people. In addition to the common barriers for receiving HIV testing, HIV-positive MSM may experience discrimination due to their infection status as well as their homosexuality (23). So far, the ‘case-management model’ is possibly the only piloted model in Thailand to bridge many of the barriers faced by HIV-positive MSM, offering peer-based follow up, workshops and community support from diagnosis to achieving stable adherence on ART. Scaling up HIV treatment for MSM in Bangkok: what does it take? 27 We note numerous limitations of our study. First, we investigated four HIV service linkage mod- els. Other service delivery models, such as mass media, peer-driven interventions, use of mobile phone applications and online HIV test registration may also have significant impacts. Second, although the NHSO database is the most comprehensive database available for Thailand, it does not cover all medical facilities in both public and private sectors. Due to accessibility issues, stigma and other structural barriers, MSM may choose to access healthcare outside of the NHSO system. This may potentially underestimate the overall available service capacity and access to services in Bangkok. Our analysis did not distinguish general and specialized hospitals either, which may differ in their service provision approaches. Third, Bangkok MSM have heterogeneous sexual identity (homosexual men, straight-acting bisexual men, transgender, etc.) and diversified risk behavioral mixing and HIV risk-related activity patterns. This was not factored into modelled estimates of potential impact. We only modelled transmission of HIV through unprotected anal intercourse among MSM at relatively high risk, but not transmission due to injecting drug use and commercial sex. Fourth, MSM at high risk may be more reachable. Costs will likely be greater to reach MSM at lower risk. Fifth, MSM tested in the public facilities may not be willing to openly self-identify as MSM due to stigma and discrimination, leading to underestimation of its current use. Assumptions on uniform demographics, sexual behavior and preference for healthcare services in our studied population may also lead to bias in our calculation. Sixth, we have not incorporated structural barriers into our epidemiological model. This is difficult to do in a rigorous manner. Seventh, there is low ART coverage among MSM in Bangkok. But since there is no system to track individuals in the cascade of care from HIV testing to linkage to care and receipt of ART, the actual numbers linked to care and on ART may differ from what we used in this study. Worldwide, there is a focus on increasing HIV testing, linkage to care, initiation of treatment and regular monitoring. This is particularly pertinent for marginalized populations at high risk of HIV infection. In many settings, there has been disproportionately less attention given to prevention, testing, and treatment among some of these key affected populations. MSM in Bangkok is one such example. The current study has addressed this problem to determine what is actually required in infrastructure and cost to scale up testing and treatment. The recommendations from this study to create a patient-friendly environment in public facilities and to employ multiple methods to link MSM to HIV testing services and ART initiation support many strategies being discussed or recently initiated in Bangkok. Examples of improving accessibility to public facilities includes initiatives of BMA health centers starting weekly night mobile clinics to venues frequented by Bangkok MSM along with the opening of BMA’s Gay Bangkok website which aims to promote safe sex and HIV testing among MSM. Drop-in centers located in major MSM hotspots in Bang- kok have also recently been established. This study’s approach is likely to be applicable to other settings. Hopefully, other settings can also determine how services can best be utilized, the most cost-effective pathway for expansion of service utilization, and implement such strategies to yield large health and economic benefits. 28 THE WORLD BANK GROUP THE WORLD BANK GROUP Conclusion Scaling up HIV treatment for MSM in Bangkok: what does it take? Conclusion The scale up of ART is currently probably the highest priority in response to HIV epidemics around the world. Modelling has shown that if coverage of diagnosis and treatment increases then it has the potential to have a large epidemiological benefits and various expansion and targeted strategies are cost-effective. However, our study takes such modelling and health economic studies an important step further for pragmatic actual program planning by assessing current and possible infrastructure of HIV testing / ART services and their geographical accessibility. We apply this approach to planning for scaling up ART for MSM in Bangkok and assess the most cost-effective expansion pathway of service utilization to attain universal coverage of ART. We find that currently available spare capacity in Bangkok’s medical facilities can be utilized to expand ART access for MSM and potentially achieve large epidemiological benefits. The expansion is likely cost-effective, firstly starting with public facilities which will need to be made more accessible to Bangkok’s MSM and then to research and Bangkok Metropolitan Administration clinics, followed by private hospitals. Expansion will require moderate-to-large increases in funding to be directed to the currently severely underfunded MSM services. 30 THE WORLD BANK GROUP THE WORLD BANK GROUP Appendix Scaling up HIV treatment for MSM in Bangkok: what does it take? Appendix: Data collection and technical details TECHNICAL DETAILS The technical details associated with this study are contained in three main sections: (i) Data collation and synthesis – where we outline the broad methodology used to collate, assess and synthesize available country data; (ii) Modelling and investment optimization methodology – where we describe the analytical approach in detail; and (iii) effectiveness and cost-effectiveness analysis. DATA COLLATION AND SYNTHESIS To assess HIV testing and ART programmes among MSM in Bangkok, we collected a large amount of data describing the HIV epidemiology, population demographics, acquisition-related behavior, clinical characteristics, and program and health costs. To ensure the most up-to-date and accurate evaluation, we collated and synthesized all available data as described in the following sections. The main data sources include (1) an extensive literature review; (2) database provided by the National Health Security Office (NHSO), Thailand (3) capacity assessment survey in NHSO-listed medical facilities and (4) an empirical study on HIV testing and ART cost breakdown in 13 Bangkok medical facilities. 1. Epidemiological and behavioral data To evaluate the epidemic among MSM in Bangkok, we collected the following indicators from both published and grey literature. 1) Estimated population sizes for men who have sex with men (MSM) 2) The epidemiological characteristics of the HIV epidemic. • HIV prevalence; • Annual HIV diagnoses; • Number of people on ART; 3) Descriptions of risk behaviors, HIV transmission patterns, and health-care seeking behavior in Bangkok. We used this data to understand modes of HIV transmission between population groups and the risk of HIV acquisition. Specific data collected includes: • Level of male circumcision; • Sexual behaviors (e.g. number of sexual partners and level of condom usage in sexual acts); • Rates at which MSM in Bangkok/Thailand test for HIV. To ensure all available data was collected, we also performed a systematic literature search for all available data in published articles, conference presentations and reports for MSM in Bangkok. We conducted this by searching PubMed and Medline. Independent searches were conducted for HIV epidemiology, sexual behavior and HIV clinical factors. Data for HIV biology, HIV infection progression, and HIV mortality are generally independent of population groups and countries. Therefore, we obtained data for these factors from available international literature and meta-anal- yses reporting the results of rigorous scientific studies. A primary source of data is from grey literature available from publically accessible websites or through communication with in-country contacts and stakeholders. Some of the key data sources obtained include results from sentinel surveillance sites; integrated biological and/or behavioral surveys conducted among MSM. 32 THE WORLD BANK GROUP 4) HIV testing and ART data. The National Health Security Office (NHSO) program provided free HIV testing twice a year for Thai citizen, regardless of health care insurance schemes they have registered, through NHSO-registered HIV testing sites. For Thai people living with HIV who use NHSO scheme (around 70 percent of Thai people living with HIV), NHSO also provides free antiretroviral treatment (ART), CD4 count twice a year (regardless of ART status), HIV viral load once a year, resistance testing when clinically indicated and other key safety lab tests (for those on ART). NHSO-registered sites need to enter test results and clinical information into the NHSO electronic database in order to get cost reimbursement from NHSO. Number of Thai citizen who accessed HIV testing, number of those who tested HIV-positive, number of HIV-positive people who registered into NHSO system for HIV treatment and care, number of HIV-positive people who started first-line and second-line ART regimens, CD4 count at entry into NHSO system, number of HIV-positive people who achieved undetectable HIV viral load after ART can be obtained by year of services from the database. These indicators can be extracted by service delivery sites in each province. From epidemiological data collected, we synthesized relevant data to obtain best point estimates and uncertainty ranges for all the parameters used in our modelling evaluation. We collated data over the period 2000 to 2012. Where possible, statistical methods were used to merge multiple sources. Usually, due to limited data, we used a simple weighted average of data from individual studies. However, for many indicators we only collected a single datum value, which we assessed for quality and specified an assumed uncertainty range (usually ±25 percent). We incorporated all model parameters informed by our data synthesis into a detailed Excel “Optima” spreadsheet. The Optima spreadsheets contain instructions for data entry with specific worksheets for general population characteristics, demographics, HIV and STI epidemiology, HIV testing and treatment sexual behavior data, drug use data, biological constants, and health utilities. Where data are available, we have entered a point estimate and an estimated or assumed uncertainty range for each year between 2000 and 2012. For biological constants, we use the same point estimate and range over time. Comments attached to cells provide justifications, calculations, and references for the estimated values based on the collated and synthesized data. A textbox in the first sheet provides further general notes on where we obtained the data, any colour-coding used to classify entries, and key references. Upon receipt of data from sites, data was cleaned, validated and entered into a master spreadsheet to facilitate calculations and analysis of baseline and alternative strategies. Based on collected epidemiological data, we found that out of 120,000-250,000 MSM in Bangkok, approximately 61,975 (40,200-83,750) MSM were consistently involved in high-risk sexual behaviors and the likely driver of HIV epidemic. Our capacity assessment survey indicated that 14,387 MSM were tested for HIV in 91 Bangkok medical facilities in 2011, corresponding to an overall testing coverage of 27 percent. Estimated 4,028 MSM were diagnosed to be HIV+, and 3,303 (82 percent) had CD4 cell count level below 500/ml and were treatment-eligible. NHSO database reported that 989 MSM initiated ART in 2011, corresponding to a 30 percent treatment commencement rate. Scaling up HIV treatment for MSM in Bangkok: what does it take? 33 Figure S1: Low HIV testing and ART commencement rates among high-risk MSM in Bangkok 23% 28% 82% 30% 2. Mapping of MSM hotspots and medical facilities in Bangkok Mapping of MSM hotspots in Bangkok was obtained through previous mapping exercises shared by stakeholders, through internet survey via Adam’s Love website, and experts in the field. These hotspots are related to venues frequented by MSM in Bangkok. Venues mainly include saunas, spas, educational institutions and department stores which are very well known to MSM. In addition, based on the address information provided by NHSO database, we estimated the distance from the facilities to the nearest MSM hotspot on Google Maps. Figure S2. Mapping of MSM hotspots and 91 medical facilities in Bangkok *Legend: Research clinic: red circle; BMA centers that provide HIV testing only: green circle; Public facilities that provide HIV testing only: yellow circle; Private hospitals that provide HIV testing only: blue circle; Public facilities that provide HIV testing and ART: yellow triangle and Private hospitals that provide HIV testing and ART: blue triangle 34 THE WORLD BANK GROUP 3. Service load and capacity in medical facilities in Bangkok We conducted a telephone/letter survey to assess service load and capacity in providing HIV testing and ART services in 91 NHSO-listed medical facilities across Bangkok in 2011. The survey has been intentionally designed to be concise with specific questions on the following aspects: (1) type of the facility; (2) capacity to provide HIV testing; (3) capacity to provide ART; (4) capacity to provide treatment to HIV-related opportunistic infections and co-infections; (5) capacity of HIV reporting and surveillance. The complete survey is listed in Table S1. Table S1: Assessment of service provision and capacity of HIV testing and ART sites in Bangkok. 4. Costing data for service linkage and provision Costing data for service linkage were collected based on a survey of available activities that connect eligible MSM to appropriate HIV testing and ART services. This part of data collection was conducted mostly in collaboration with in-country collaborators and local NGOs that conduct the services. More detail description of the services was provided in Table S7 and S9. In brief, we identified three major recruitment methods of MSM to HIV testing services: (1) conventional community-based outreach via peer-educator; (2) mobile point-of-care (POC) night clinics provided by Thai Red Cross (TRC) and BMA health centers; (3) Adam’s Love website with innovative Scaling up HIV treatment for MSM in Bangkok: what does it take? 35 follow-up technologies hosted by TRC. AIDS Projects Management Group was the organization that piloted the sole linkage program to facilitate diagnosed and eligible HIV+ MSM to connect to ART services. The linkage model has been named ‘case-management model’ (Table S9). Based on internal reports and communication with the responsible organisations, for each of these linkage programs, we collected indicators on program spending (e.g. cost of implementation and operation) and program effects (e.g. the number of individuals connected to HIV services). Out of the 91 NHSO-listed medical facilities, 13 were specifically chosen to collect costing data on service provision. The 13 sites were selected based on the current number of MSM who accessed HIV testing and/or ART services at these sites, along with potential capacity to increase the scale of the services in the future. These sites represented public hospitals operated under the Bangkok Metropolitan Administration (BMA) (5 hospitals: Klang, Rajvithi, Taksin, Charoenkrung Pracharak and Vajira Hospitals), private hospitals (2 hospitals: Phyathai 2 and Mongkutwattana General Hospitals), public clinics operated under the BMA (3 clinics: BMA Primary Health Centers 3, 4 and 28), and clinics specialized in MSM and sexual health services (3 clinics: Thai Red Cross Anonymous Clinic, Silom Community Clinic and Bangrak Clinic). All 13 sites offer HIV testing, and 7 sites offer ART. Of the 7 that offer ART, 2 are private hospitals and 5 are public hospitals. Input data (the costs) and output data (e.g. the number of tests, the number of MSM on treatment) was collected from 13 sites via an input and output costing template. The template was used by a team of data collectors trained by Thai Red Cross in conjunction with key personnel at each of the sites. Overarching data was collected per site, including: - type of site - hours of operation - registration for reimbursement - spare capacity (to what extent the site can increase volume of clients with existing infrastructure) Input and output data collected included for testing: - number of tests (segmented by MSM if possible) - number tested positive - number and type of staff and their salaries - costs of commodities (e.g. test kits, needles) - operational costs (e.g. telephone, electricity costs) - [Note: data was not available for rental costs of sites] Input and output data collected included for ART: - number of people on 1st, 2nd, 3rd line ART (segmented by MSM if possible) - cost of monitoring based on: • staff time with patients (split by initial 12 months; after 12 months) • cost and frequency of monitoring tests - cost of ART drugs 1st, 2nd line ART 36 THE WORLD BANK GROUP MODELLING METHODOLOGY Mathematical Model - Optima To assess HIV epidemic trends and project the cost-effectiveness of investment scenarios, we employed a well-developed mathematical model of HIV transmission and disease progression, called Optima. We used this model to calculate the change in HIV incidence, the number of HIV/ AIDS deaths due to changes in funding and the cost-effectiveness of various investment scenarios for HIV testing and ART services. Optima uses best-practice HIV epidemic modelling techniques and incorporates realistic biological transmission processes, detailed infection progression and sexual mixing patterns and other high-risk behaviors. Optima incorporates a model of HIV transmission and progression. The model uses a coupled system of ordinary differential equations to track the movement of people between health states (Figure S). The overall population partitioned by group and health state. Individuals are assigned to a given population based on their dominant risk; however, to capture important cross-modal types of transmission and relevant behavioral parameters. The model distinguishes people who are undiagnosed, diagnosed, and on effective anti-retroviral therapy (ART). Diagnosis of HIV-infected individuals occurs based on a HIV testing rate dependent on CD4 count and population type. Similarly, diagnosed individuals begin treatment at a CD4 count dependent rate. The model tracks those on successful first- or second-line treatment (who have an increasing CD4 count) and those with treatment failure. Figure S3: HIV Infection Progression The force-of-infection for a population determines rate at which uninfected individuals within the population become infected. This depends on the number of risk events individuals are exposed to in a given period and the infection probability of each event. Sexual transmission risk depends on: • The number of people in each HIV-infected stage (that is, the prevalence of HIV infection in partner populations) • The average number of casual, regular, and commercial homosexual and heterosexual partnerships per person • The average frequency of sexual acts per partnership • The proportion of these acts in which condoms are used Scaling up HIV treatment for MSM in Bangkok: what does it take? 37 • The efficacy of condoms • The extent of male circumcision • The prevalence of ulcerative STIs (which increase transmission probability) The stage of infection (chronic, AIDS-related illness/late stage, or on treatment) for the HIV-positive partner in a serodiscordant couple also influences transmission risk—due to different levels of infectiousness in each infection stage. Mathematically, we calculate the force-of-infection using: =1−(1−) where is the force-of-infection, is the transmission probability of each event, and n is the effective number of at-risk events (thus n gives the average number interaction events with infected people where HIV transmission may occur). The value of the transmission probability is inversely related to CD4 count, is related to the mode of transmission. The number of events n not only incorporates the total number of events, but also other factors that moderate the possibility that these events are capable of transmitting infection. There is one force-of-infection term for each type of interaction (such as, regular, casual and commercial partnerships), and the overall force-of-infection is the sum of overall interaction types. In addition to the force-of-infection rate, in which individuals move from uninfected to infected states, individuals may move between health states via seven other pathways: • Individuals may die, either due to the background death rate (which affects all populations equally), due to injecting behavior, or due to HIV/AIDS (which depends on CD4 count) • In the absence of intervention, individuals progress from higher to lower CD4 counts • Individuals can move from undiagnosed to diagnosed states based on their HIV testing rate, which is a function of CD4 count (for example, people with AIDS symptoms have a higher testing rate) and population type (for example, IDUs usually get tested more frequently than low-risk males). • Diagnosed individuals may move onto treatment, at a rate dependent on CD4 count • Individuals may move from treatment to treatment failure, and • From treatment failure onto second-line treatment • Finally, while on successful first- or second-line treatment, individuals may progress from lower to higher CD4 count. Calibration to HIV epidemics We calibrated Optima to match HIV prevalence data, and the uptake of ART from 2000-2012. While primarily calibrated to match epidemiological data, Optima also optimizes input parameters to match available demographic, behavioral, biological and clinical data. Given the challenges inherent in quantifying all known constraints on an epidemic, we calibrated the model manually, with oversight by and collaboration with in-country stakeholders where possible. The values of each parameter in 2012 represent current conditions for each simulation. 38 THE WORLD BANK GROUP Figure S4: Calibration of Optima to the HIV epidemic among MSM in Bangkok during 2000-2012 , (a) HIV prevalence; (b) number of diagnosed HIV+ MSM on ART Uncertainty analyses Optima uses a Markov chain Monte Carlo (MCMC) algorithm for performing automatic calibration and for computing uncertainties in the model fit to epidemiological data. With this algorithm, the model is run a large number of times (~10000) to generate a range of epidemic projections; their differences represent uncertainty in the expected epidemiological trajectories. The calibration and optimization processes incorporate uncertainties in all parameters. In particular, this includes uncertainties in demographic indicators (e.g. population size), epidemiological data (e.g. HIV prevalence), behavioral indicators (e.g. the number of sexual partners, the frequency of sexual acts and the percentage of condom usage) and costing data (e.g. unit price of providing HIV testing and antiretroviral treatment). All available historical spending data and achieved outcomes Scaling up HIV treatment for MSM in Bangkok: what does it take? 39 of spending, data from comparable settings, experience, and extensive discussion with stakeholders in Thailand to inform these ranges. All parameters within these ranges are then allowable and are incorporated into uncertainty analyses of Optima. These fitting of the model are thus reconciled with the epidemiological, behavioral, and biological data in a Bayesian-optimal way, thereby allowing the calculation of unified uncertainty estimates. Estimation of unit cost of service provision The total cost of a service is calculated as sum of all expenses incurred during the provision of the services. The breakdown of items for HIV testing and ART provision is listed in Table S8 and Table 10. Fixed costs were defined as those expenses that remain constant during a relevant period regardless of the number of people served. These may include cost for program manage- ment (planning, administration, and supervision), training, travel, purchase and operation of mo- bile vans, durable goods, and equipment. In contrast, variable costs were those for recruitment, counselling and testing, and nondurable goods and commodities, such as testing kits, materials for screening and confirmatory testings (Shrestha RK Public Health Rep. 2008 Nov-Dec;123 Suppl 3:94-100.). The unit cost of service provision is hence the ratio of total service costs and the number of person-time the service has been provided. We adapted a similar definition of cost function for the calculation of unit cost in our study (Gesine Meyer-Rath, Mead Over, PloS One. 2012 Jul; 9(7):1-10.). Namely, Where U denotes unit cost, F is the total amount of fixed costs, u is the unit variable cost (e.g. commodity cost of providing one HIV test) and the N is the number of person-times the service is provided (e.g. number of tests conducted among MSM in the past 12 months). The index ‘i’ denotes the type of medical facility where the service has been provided. The same formula applies to both HIV testing and ART services. The relationship between U and N is illustrated in Figure S5. Optimization of resources for services scale-up In this study, we aim to investigate the most economical way of scaling-up ART coverage among Bangkok MSM and forecast their subsequent epidemiological impacts and value for money. The three ART scale-up scenarios we investigated are to achieve universal ART coverage among Bangkok high-risk MSM by 2022, 2017 and 2015 respectively. The differences in epidemiolog- ical outcomes between these scenarios and the projected status quo provided the basis for the evaluation of the effectiveness and cost-effectiveness of the scenarios. The two key indicators for optimizing service sources are the HIV testing rate (X) and annual ART commencement rate (Y). If these rates increased from currently (0,0) to (+0,+0), extra number (T) of MSM living with HIV will initiate ART, with the expression: =∙∙∙[(+0)(+0)−00] , Where is the population size of MSM, is HIV prevalence and is the proportion of HIV+ MSM who are eligible for treatment (with treatment threshold CD4<500/ml). On the other hand, the intended scaled-up ART coverage (R) has the expression: =(+0)⁄(+0). 40 THE WORLD BANK GROUP Therefore, =(−0)⁄(+0) where E is the number treatment-eligible individuals that are yet on ART, and 0 and 0 are cur- rent number of individuals on ART and ART coverage respectively. Since both and +0 represent the number of individuals should be on ART, the expression of coverage, as a function of HIV testing rate and ART commencement rate can be simplified as: −0=(+0)(+0)−00. The intended coverage increase will require extra cost C: where 1 to 4 are the unit costs to each step of service linkage and provision. In fact, we assume the unit cost for HIV testing linkage 1 is a constant $36 per person and unit cost for ART service linkage 3is $178 (Table S7, S9), whereas unit cost for HIV testing and ART provision (2 and 4) could be minimised according to the type of facility for service provision. The objective function for optimising HIV testing services is: Where, is the average unit material cost of a HIV test (averaged over the costs of screening, confirmatory tests and needle and syringes); is the cost of establishing a new HIV testing site of equal size in the currently existing facility (assuming space available); = is the number of new facilities required, in which represents the current availability for HIV testing in the facilities and ̅ represents the maximum capacity for HIV testing in the same facility. The optimization is subjected to the constraint Σ=, where is the number of tests conducted at the facility with type ( can be Public HIV testing and ART providers, BMA clinics, private hospitals and research clinics); is the total number of HIV tests required. The objective function unit cost for ART provision 4 follows a similar expression. Forecasting epidemiological impacts and cost-effectiveness We project the trajectory of HIV epidemic among Bangkok MSM over the period of 2013-2022. The key epidemiological indicators include the number of new HIV cases, HIV-related deaths and number of people on ART. We compare these indicators across the scenarios. We used disability-adjusted life years (DALYs) to measure the overall impact of HIV programs and for basic health economic calculations. The most thorough empirical study of disability weights is the 2010 Global Burden of Disease Study (Salomon, et.al., The Lancet, 2013). This study reports DALYs for people with symptomatic HIV, AIDS, and HIV but on effective HIV treatment (Table S2). Scaling up HIV treatment for MSM in Bangkok: what does it take? 41 Table S2: Disability weights for HIV related health states from the 2010 Global Burden of Disease Study. Based on the results from the Global Burden of Disease Study (Salomon, et.al., The Lancet, 2013) we assigned a disability-weight for HIV-positive people in each CD4 count category (Table S3). Table S3: Assumed disability-weights for DALY calculations. The disability-weight for HIV-positive MSM is the maximum of the value for HIV-negative MSM and the corresponding HIV-positive cat- egory disability weight. Based on the forecasted epidemiological outcomes, we calculated costs required for each HIV-related death, new HIV case and DALY averted. The strategy is considered to cost-saving if cost per DALY averted is less than one GDP per capita and cost-effective if less than three GDP per capita. 42 THE WORLD BANK GROUP Table S4: Facility characteristics and operational capacity of HIV testing services in 91 Bangkok medical facilities. Site information was obtained from the National Health Security Office (NHSO) database, service load and capacity data from capacity assessment survey, and distance information from site mapping exercise. Scaling up HIV treatment for MSM in Bangkok: what does it take? 43 44 THE WORLD BANK GROUP Scaling up HIV treatment for MSM in Bangkok: what does it take? 45 46 THE WORLD BANK GROUP Table S5: Facility characteristics and operational capacity of ART services in 48 Bangkok medical facilities. Data was obtained from the National Health Security Office (NHSO), Thailand Scaling up HIV treatment for MSM in Bangkok: what does it take? 47 48 THE WORLD BANK GROUP Table S6: Service load and capacity for (a) HIV testing services in 91 and (b) ART services in 48 Bangkok medical clinics in 2011, stratified by facility types and distances to MSM hotspots. Association between distances to MSM hotspots and servicing loads are not statistically significant for both services (Spearman Correlation, HIV testing: r =-0.0449, p = 0.7341; ART: r =-0.2110, p =0.4163). Scaling up HIV treatment for MSM in Bangkok: what does it take? 49 Table S7: Strategies and costs in linking high-risk MSM to HIV testing services in Bangkok Table S7a: Cost of linking high-risk MSM to HIV testing services using community-based outreach through peer-educators 1 Community-based outreach through peer-educators This is a conventional HIV prevention strategy. Peer-educators go into the MSM community to reach potential MSM for testing. It is often conducted in combination with other services, such as condom distribution, HIV health education and peer counselling. Source Investment ($) The number of peer-educators in Bangkok 60-70 Provided by TRC The annual salary of a peer-educator in Bangkok* ($) $768 Provided by TRC (*Peer-educator works an average 10 days per month) Total Investment per year ($) $49,920 Effects The number of MSM received HIV testing per year 9,935 Estimated from NHSO database Percentage of MSM received HIV testing as a result 7% The Thai GF Round of peer-educator outreach 8 report* The number of MSM received HIV testing per year 695 as a result of peer-educator outreach Cost required to link one MSM to HIV testing services $72 * Wolf, Cameron, Thailand Global Fund Round 8 External Evaluation: Men Who Have Sex with Men, Aug 2012, Bangkok 50 THE WORLD BANK GROUP Table S7b: Cost of linking high-risk MSM to HIV testing services using mobile point-of-care night clinics 2 Mobile night clinics (point-of-care POC HIV testing) During 2011-2012, the Thai Red Cross conducted 75 times 'mobile night clinics' outreach in MSM hotspots across Bangkok. Qualified health care personnel provided point-of-care rapid HIV tests, using three rapid test kits to confirm HIV diagnosis, to MSM who voluntarily participated. Individuals with positive HIV status were referred to clinics of their choice for CD4 count and further HIV care and treatment. Source Investment ($) Overall cost in 2011: (31 mobile clinic times) $9,486 Provided by TRC Overall cost in 2012: (44 mobile clinic times) $14,638 Provided by TRC Total Investment ($) $24,124 Effects The number of MSM reached per service 10-15 Provided by TRC Total number of mobile clinic times 75 Provided by TRC The number of MSM received HIV testing per year 938 as a result of POC mobile clinic Cost required to link one MSM to HIV testing services $26 Scaling up HIV treatment for MSM in Bangkok: what does it take? 51 Table S7c: Cost of linking high-risk MSM to HIV testing services using Adam's Love website 3 New technology: Promotion of HIV testing on Adam's Love Website Adam’s Love is a pilot project, which is designed after completing an extensive research on the MSM community in Thailand based on various behavior, lifestyle, age group, and educational background. The website’s core objectives: (1) to inspire MSM in Thailand to get HIV tested every 3 months and practice safe sex; (2) to make resources such as information about HIV/AIDS, medication, support and care provided at the Anonymous Clinic or Men’s Health Clinic at TRCARC proactively available to MSM in Thailand by utilizing digital communication mechanisms; (3) to remove stigma and discrimination related to MSM and HIV prevailing in the community. The website hosts over 150 videos and articles on expert advice “Ask Our Expert” section, and more than 50 edu-tainment videos. It offering privileges and souvenirs to MSM who come to get tested, the membership application at Anonymous Clinic and linked private hospitals. It provides web-board for HIV/STI inquiries, answered by experts from TRCARC and also AIDS related queries submission through hotline and service channels i.e. e-mail, Facebook message, and phone. Source Investment ($) Operational cost of Adam's Love website in 2011 Salary for web developer $19,200 Provided by TRC Web site development cost $3,049 Provided by TRC Video content Production $7,014 Provided by TRC Adam's Love promotional Material $694 Provided by TRC Press Conference in Bangkok $1,568 Provided by TRC Total Investment ($) $31,526 Effects The total number of MSM tested in TRC 5,357 Provided by TRC Percentage of MSM received HIV testing 25% Provided by TRC as a result of Adam's Love promotion The number of MSM received HIV testing 1,339 per year as a result of Adam’s Love website Cost required to link one MSM to HIV testing services $24 52 THE WORLD BANK GROUP Table S8: Cost distribution for HIV testing service provision in 13 Bangkok medical facilities in 2011. *Staffing and operation costs were regarded fixed costs †Testing and IEC costs were regarded as variable costs Scaling up HIV treatment for MSM in Bangkok: what does it take? 53 Table S9: Cost of linking high-risk diagnosed HIV+ MSM to ART services using the Case Management Model 1 Case Management model A case management approach for newly-diagnosed MSM and TG people with HIV is an important innovation because the Thai public health system can be difficult to navigate and newly-diagnosed PLHIV often be lost to follow up soon after diagnosis. For more affluent MSM and TG people who can afford to pay their own way, maintaining a health monitoring and treatment regime is often relatively straightforward. For those relying on the government subsidized system, maintaining healthcare is more difficult. Care is rarely coordinated by a single health worker and there are often multiple changes in the personnel providing care and prescribing treatment at government health services. The Case Management Model establishes cross-organization case management systems in Bangkok. These systems allow for highly complex and high-need clients with HIV to get the support they need across multiple organizations. The model also provides self-management workshops to improve treatment adherence among MSM and TG people. It aims to impart the knowledge, skills and social support needed to ensure adherence to ARV treatment. Source Investment ($) Operational cost of Case Management Model Case managers salary per month $8,640 Provided by APMG Coordinator (one for outreach, one for case-management) $7,680 Provided by APMG salary per month Rent per month $3,840 Provided by APMG Administrator salary per month $3,360 Provided by APMG Data input salary per month $1,440 Provided by APMG Total Investment ($) $24,960 Effects Number of diagnosed HIV+ MSM linked to received HIV 141 Provided by APMG testing per year as a result of case management program Cost required to link one diagnosed MSM to ART services $177 54 THE WORLD BANK GROUP Table S10: Cost distribution for ART service provision in 13 Bangkok medical facilities in 2011 *Monitoring and treatment costs were regarded fixed costs †Adherence and operation costs were regarded as variable costs Scaling up HIV treatment for MSM in Bangkok: what does it take? 55 Figure S5: Projected decline trend in the unit cost for HIV testing and ART provision in relation to service capacity in Bangkok medical facilities. The solid dots present current service load and corresponding estimated unit costs. 56 THE WORLD BANK GROUP Scaling up HIV treatment for MSM in Bangkok: what does it take? 57 Scaling up HIV treatment for MSM in Bangkok: what does it take? References THE WORLD BANK GROUP References 1. Thailand_Bureau of AIDS TB and STIs, Surveillance of HIV infection in Thailand, round 28. 2011, Thai Ministry of Public Health: Bangkok. 2. Van Griensven, F., et al., Evidence of an explosive epidemic of HIV infection in a cohort of men who have sex with men in Thailand. AIDS, 2013. 27(5): p. 825-32. 3. Wimonsate, W., et al., Factors associated with HIV testing history and returning for HIV test results among men who have sex with men in Thailand. AIDS Behav, 2011. 15(4): p. 693-701. 4. Van Griensven, F. and J.W. de Lind van Wijngaarden, A review of the epidemiology of HIV infection and prevention responses among MSM in Asia. AIDS, 2010. 24 Suppl 3: p. S30-40. 5. Toledo, C.A., et al., Examining HIV infection among male sex workers in Bangkok, Thailand: a comparison of participants recruited at entertainment and street venues. AIDS Educ Prev, 2010. 22(4): p. 299-311. 6. Van Griensven, F., et al., Trends in HIV Prevalence, Estimated HIV Incidence, and Risk Behavior Among Men Who Have Sex With Men in Bangkok, Thailand, 2003-2007. J Acquir Immune Defic Syndr, 2009. 7. Van Griensven, F., et al., Evidence of a previously undocumented epidemic of HIV infection among men who have sex with men in Bangkok, Thailand. AIDS, 2005. 19(5): p. 521-6. 8. Ding, X., et al., [A high risk behaviors analysis and KABP investigation on STD/AIDS among 489 female attendants of the entertainment establishments in Chongqing]. Chin j Dis Control Prew, 2006. 10(04): p. 365-368. 9. Van Griensven, F., et al., Evidence of an explosive epidemic of HIV infection in a cohort of men who have sex with men in Bangkok, Thailand. AIDS, 2012. 10. Sifakis, F., et al., Combination HIV Prevention Interventions for MSM, in The Global HIV Epidemics among Men Who Have Sex with Men (MSM). p. 149-215. 11. Kapol, N., et al., Making health technology assessment information available for decision making: the development of a Thai database. J Med Assoc Thai, 2008. 91 Suppl 2: p. S8-10. 12. Teerawattananon, Y. and U. Chaikledkaew, Thai health technology assessment guideline development. J Med Assoc Thai, 2008. 91 Suppl 2: p. S11-5. Scaling up HIV treatment for MSM in Bangkok: what does it take? 59 13. Thai Red Cross, Consultation on Strategic Use of ARVs, 9-10 August 2012. 2012: Bangkok, Thailand. 14. Thai Red Cross, The Size Estimation Workshop. 2014: Bangkok, Thailand. 15. C.C. Kerr, et al., Optima: a model for HIV epidemic analysis, program prioritization and resource optimization. Journal of acquired immune deficiency syndromes, 2014. (in press). 16. Salomon, J.A., et al., Common values in assessing health outcomes from disease and injury: disability weights measurement study for the Global Burden of Disease Study 2010. The Lancet, 2013. 380(9859): p. 2129-2143. 17. Walensky, R., et al., Cost-effectiveness of HIV treatment as prevention in serodiscordant couples. New England Journal of Medicine, 2013. 31(369): p. 1715-25. 18. Phanuphak, N., et al., Active voluntary counseling and testing with integrated CD4 count service can enhance early HIV testing and early CD4 count measurement: experiences from the Thai Red Cross Anonymous Clinic in Bangkok, Thailand. J Acquir Immune Defic Syndr, 2011. 56(3): p. 244-52. 60 THE WORLD BANK GROUP