Document of The World Bank Report No: ICR00003323 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75850) ON A LOAN IN THE AMOUNT OF USD$35 MILLION TO THE GOVERNMENT OF BARBADOS FOR A SECOND HIV/AIDS PROJECT May 28, 2015 Health, Nutrition and Population Global Practice Latin America and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective May 28, 2015) Currency Unit = Barbadian Dollar 1.00 = US$ 0.50 US$ 1.00 = 2.01 FISCAL YEAR (April 1, 2014 – March 31, 2015) ABBREVIATIONS AND ACRONYMS ART Antiretroviral Treatment ARV Antiretroviral BCC Behavior Change Communication BSS Behavioral Seroprevalence Survey CARICOM Caribbean Community and Common Market CARPHA Caribbean Public Health Agency CDC Centers for Disease Control and Prevention CD4 Cluster of differentiation 4 CHAA Caribbean HIV/AIDS Alliance CSOs Civil Society Organizations DEBI Diffusion of Effective Behavioral Interventions DLI Disbursement-Linked Indicator DU Drug Users EEP Eligible Expenditure Program FBO Faith-Based Organizations FSW Female Sex Workers GIS Geographic Information System GOB Government of Barbados HIS Health Information System HIV/AIDS Human Immunodeficiency Virus / Acquired Immunodeficiency Syndrome ICR Implementation Completion and Results Report IOI Intermediate Outcome Indicator IP Implementation Progress IRR Internal Rate of Return ISR Implementation Status and Results Report KABP Knowledge, Attitudes, Behaviors, and Practices KAP Knowledge, Attitudes and Practices KPI Key Performance Indicator LGBT Lesbian, Gay, Bisexual and Transgender LMIS Logistics Management and Information System LRU Ladymeade Reference Unit M&E Monitoring and Evaluation MFYS Ministry of Family, Youth and Sports MHNI Ministry of Health, National Insurance and Social Security ii MOF Ministry of Finance MOH Ministry of Health MS Moderately Satisfactory MSM Men Who Have Sex with Men MTCT Mother to Child Transmission MTR Mid-Term Review MU Moderately Unsatisfactory NAP National AIDS Program NHAC National HIV/AIDS Commission NSP National Strategic Plan for HIV Prevention and Control 2008-2013 OI Opportunistic Infections PAD Project Appraisal Document PAHO Pan American Health Organization PANCAP Pan Caribbean Partnership Against HIV/AIDS PDO Project Development Objectives PEPFAR The U.S. President's Emergency Plan for AIDS Relief PLHIV People Living with HIV PMTCT Prevention of Mother to Child Transmission QAG Quality Assurance Group QEH Queen Elizabeth Hospital STI Sexually Transmitted Infections SW Sex Workers SWAp Sector Wide Approach UGLAAB United Gays and Lesbians Against AIDS, Barbados VHRU Virtual Health Research Unit WHO World Health Organization HNP GP Senior Director: Timothy Evans HNP GP Director: Olusoji Adeyi Practice Manager: Enis Barış Project Team Leader: Carmen Carpio ICR Team Leader: Neesha Harnam ICR Author: Neesha Harnam iii Barbados Second HIV/AIDS Project TABLE OF CONTENTS Data Sheet ........................................................................................................................... v A. Basic Information ...................................................................................................... v B. Key Dates................................................................................................................... v C. Ratings Summary....................................................................................................... v D. Sector and Theme Codes .......................................................................................... vi E. Bank Staff ................................................................................................................. vi F. Results Framework Analysis..................................................................................... vi G. Ratings of Project Performance in ISRs .................................................................. xii H. Restructuring ........................................................................................................... xii I. Disbursement Profile............................................................................................... xiii 1. Project Context, Development Objectives and Design ........................................... 1 2. Key Factors Affecting Implementation and Outcomes .......................................... 5 3. Assessment of Outcomes ...................................................................................... 12 4. Assessment of Risk to Development Outcomes ................................................... 18 5. Assessment of Bank and Borrower Performance ................................................. 18 6. Lessons Learned.................................................................................................... 20 7. Comments on Issues Raised by Borrower………………………………………..21 Annex 1. Project Costs and Financing ......................................................................... 22 Annex 2. Outputs by Component ................................................................................. 23 Annex 3. Economic and Financial Analysis ................................................................ 29 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 35 Annex 5. Beneficiary Survey Results........................................................................... 37 Annex 6. Stakeholder Workshop Report and Results .................................................. 38 Annex 7. Summary of Borrower's ICR and / or Comments on Draft ICR................... 39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ...................... 55 Annex 9. List of Supporting Documents ...................................................................... 56 Annex 10. Map ............................................................................................................. 57 Annex 11. Summary of KPI Modifications at Restructuring……………………….…58 Annex 12. Detailed Project Components………………………………………………59 Annex 13. Project Expenditures………………………………………..……………...62 iv ICR Data Sheet A. Basic Information Barbados Second Country: Barbados Project Name: HIV/AIDS Project Project ID: P106623 L/C/TF Number(s): IBRD-75850 ICR Date: 04/24/2015 ICR Type: Core ICR Specific Investment Government of Lending Instrument: Borrower: Loan (SIL) Barbados (GOB) Original Total US$35.00M Disbursed Amount: US$35.00M Commitment: Revised Amount: US$35.00M Environmental Category: C Implementing Agencies: The National HIV/AIDS Commission (previously under the Ministry of Family, Youth and Sports (MFYS) and now under the Ministry of Social Care, Constituency Empowerment, and Community Development) and the Ministry of Health Cofinanciers and Other External Partners: None B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 10/18/2007 Effectiveness: 01/06/2009 01/06/2009 07/19/2011 Appraisal: 02/26/2008 Restructuring(s): 03/05/2013 11/25/2013 Approval: 08/07/2008 Mid-term Review: 12/05/2011 12/05/2011 Closing: 11/30/2013 11/30/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Moderately Satisfactory Agency/Agencies: v Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time No None (QEA): (Yes/No): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 15 15 Public administration- Health 85 85 Theme Code (as % of total Bank financing) HIV/AIDS 70 70 Health system performance 10 10 Population and reproductive health 20 20 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Sophie Sirtaine Yvonne M. Tsikata Practice Enis Barış Keith E. Hansen Manager/Manager: Project Team Leader: Carmen Carpio Joana Godinho ICR Team Leader: Neesha Harnam ICR Primary Author: Neesha Harnam F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project would support the implementation of the 2008-2013 Barbados National HIV/AIDS Strategic Plan, specifically to increase: vi - Adoption of safe behaviors, in particular amongst key populations at higher risk. - Access to prevention, treatment and social care, in particular for key populations at higher risk. - Capacity of organizational and institutional structures that govern the National AIDS Program (NAP). - Use of quality data for problem identification, strategy definition and measuring results. Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Percentage of young women and men aged 15-24 indicating sexual relations as Indicator 1 : a way of transmitting HIV. Value Maintain at 90.0% quantitative or 94.3% 99.6% or higher Qualitative) Date achieved 12/31/2006 11/30/2013 10/29/2014 Comments Surpassed. The knowledge rate is 99.6% in this key population as of 2014; (incl. % rates above 90% were seen in surveys conducted in 2009, 2011 and 2013. achievement) Percentage of female sex workers reporting the use of a condom with their most Indicator 2 : recent client. Value Data not available quantitative or 80% 95% 85% because survey not Qualitative) carried out Date achieved 12/31/2006 11/30/2013 11/30/2013 Unknown. Data was not available for the KPIs relating to FSWs due to a Comments combination of factors, namely the delayed commencement of the survey (due (incl. % to additional requirements from the Centers for Disease Control and Prevention achievement) (CDC)) and requested revisions to the protocol submitted to the Ethical Review Board. Percentage of men reporting the use of a condom the last time they had sex with Indicator 3 : a male partner. Value quantitative or 65% 75% 70% 56.9% Qualitative) Date achieved 12/31/2007 11/30/2013 11/30/2013 10/29/2014 Not Achieved. This indicator is being informed by the men who have sex with men (MSM) Behavioral Seroprevalence Survey (BSS); baseline data was Comments obtained from the Barbados Men’s Lifestyle Survey and is considered proxy (incl. % data. While data from the Adult KABP survey places condom usage at 80% achievement) among MSM, the sample size from that survey was considered too small to be conclusive. vii Percentage of young men and women aged 15-24 years reporting the use of a Indicator 4 : condom the last time they had sex with non-marital non-cohabitating sexual partner. Value quantitative or 21% 31% 60% 71.70% Qualitative) Date achieved 12/31/2006 11/30/2013 11/30/2013 10/29/2014 Comments (incl. % Surpassed. achievement) Percentage of most at-risk populations (female sex workers and men who have Indicator 5 : sex with men) reached with HIV prevention services. Value MSM: 3.5% MSM: 54.4% quantitative or SW: 16.5% SW: 44.5% Qualitative) Date achieved 12/31/2009 10/29/2014 Comments Achieved. While targets were not set using CHAA data, available evidence (incl. % from the CHAA survey for sex workers (men and women) and MSM is used achievement) here and showed a big improvement. Percentage of identified HIV-positive pregnant women who gave birth in the Indicator 6 : last 12 months receiving a complete course of antiretroviral therapy to reduce the risk of mother to child transmission. Maintain at Maintain at more Value more than than 95.00% over quantitative or 96.00% 95.00% over 94.08% the 2008-2013 Qualitative) the 2008-2014 period period Date achieved 12/31/2008 11/30/2013 11/30/2013 10/29/2014 Achieved. These rates were 87.5% (2009), 92.0% (2010), 100% (2011), 100% Comments (2012), and 90.9% 1 (2013), averaging 94.08% over the five-year period. The (incl. % small denominator for this indicator results in large fluctuations in terms of achievement) percentages – the achievement of 90.9% in 2013 represents 2 missed cases out of a total of 24 HIV-positive pregnant women. Percentage of people living with HIV on first line antiretroviral regimen Indicator 7 : achieving virologic success within the first 6 months of treatment. Maintain at Maintain at more more than Value than 70% in the 70% within quantitative or 60% 92.50% last 12 months of the first 6 Qualitative) treatment. months of treatment. Date achieved 12/31/2008 11/30/2013 11/30/2013 10/29/2014 1 A detailed review was conducted by the Project team of the missed cases which revealed several systemic issues: (i) the rapid HIV tests performed in labor did not provide the result prior to delivery; (ii) the results of previous HIV tests in other facilities within the health system were not recorded or available to the ANC staff; and (iii) patients reported stigma as a barrier to early access to care. viii Comments Surpassed. This indicator measures viral load (VL) suppression, specifically, (incl. % persons who reached VL suppression where suppression is calculated as <1000 achievement) copies/mL within 12 mos. (Corresponding to WHO Early Warning Indicator 8). Amount of funds spent by civil society organizations under results-based Indicator 8 : agreements. Value quantitative or No Baseline $150,000 $186,848.00 Qualitative) Date achieved 12/31/2008 11/30/2013 10/29/2014 Surpassed. Improvements in in capacity were also seen. 27 of 34 HIV/AIDS- related projects that received funding were conducted by CSOs new to working Comments on HIV/AIDS. The number of CSOs working with most-at-risk-populations (incl. % increased from 2 in 2007 to 10 in 2014. An assessment of the CSO Grant achievement) System also showed improved proposal writing skills among the 19 CSOs who received training on proposal development. Indicator 9 : An evidence-based Strategic Plan for 2014-2018 prepared by June 2014. Value 2014-2018 NSP 2014-2018 NSP quantitative or Strategic Plan 2008-2013 developed by June finalized and being Qualitative) 2013 disseminated. Date achieved 12/31/2008 06/30/2013 10/29/2014 Achieved. The Strategic Plan incorporated information from a wide Comments consultative process, epidemiological data, KABP reports, the previous strategic (incl. % plan, and results from the stigma and discrimination survey, two M&E achievement) assessments, the CSO grant system consultant report, and the Virtual Health Research Unit (VHRU). Additional Outcome-Level Indicators. These additional indicators are from the NAP framework, and have been included in the ICR to round out overall information available on HIV/AIDS. Indicator 10: Percent of infants born to HIV-infected mothers who are infected. Value Maintain below quantitative or 2.6% <1% 5% Qualitative) Date achieved 12/31/2006 11/30/2013 12/31/2013 Comments Surpassed. The rates were 0% for 2009-2012 and 4% in 2013 for confirmed (incl. % cases, resulting in an average <1%. achievement) Percent of young women and men aged 15-24 who have had sexual intercourse Indicator 11 : before the age of 15. Value quantitative or 19.6% 15% 12% Qualitative) Date achieved 12/31/2006 12/31/2012 06/30/2014 Comments (incl. % Achieved. A reduction in sexual activity was observed in this group. achievement) Percentage of MSM who received an HIV test in the last 12 months and who Indicator 12 : know their results. ix Value 25% (baseline N/A quantitative or 50.0% when target was 100% Qualitative) set) Date achieved 12/31/2009 11/30/2013 6/30/2014 Surpassed. An improvement in safe behaviors was demonstrated through the Comments increase in the percentage of MSM that received an HIV test in the last 12 (incl. % months and who knew their results from 50% in 2009 to 100% in 2013 and achievement) 2014. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1: Number of targeted intervention programs for key populations at higher risk. Value quantitative or 2 11 27 Qualitative) Date achieved 12/31/2008 12/31/2013 6/30/2014 Surpassed. Annual targets were set with an increase of 2 each year to a Comments maximum of 9 in 2013. These were exceeded, increasing to as high as 52 in (incl. % 2012. Interventions included risk reduction counselling, behavior change achievement) communication, and condom use programming. Number of health care providers trained in HIV testing and counseling according Indicator 2: to national standards. Value quantitative or 115 250 408 Qualitative) Date achieved 12/31/2006 11/30/2013 6/30/2014 Comments (incl. % Surpassed. achievement) Indicator 3: Number of PLHIV and/or families accessing the food bank. Value quantitative or 250 375 474 Qualitative) Date achieved 12/31/2008 11/30/2013 12/31/2013 Comments (incl. % Achieved. Annual targets were exceeded for 2012 and 2013. achievement) Indicator 4: Number of CSOs working with most-at-risk populations. Value quantitative or 2 7 16 Qualitative) Date achieved 12/31/2007 11/30/2013 6/30/2014 x Comments Surpassed. The number of CSOs working with most-at-risk populations (incl. % increased to a high of 21 in 2013. achievement) Indicator 5: Number of people trained in M&E at different levels. Value quantitative or 39 125 additional 683 additional Qualitative) Date achieved 12/31/2007 11/30/2013 6/30/2014 Comments Surpassed. Training included introduction to M&E, basic M&E, program (incl. % evaluation and research/evaluation methods. achievement) xi G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (US$millions) 1 12/12/2008 Satisfactory Satisfactory 0.00 2 05/21/2009 Satisfactory Satisfactory 1.50 3 08/14/2009 Satisfactory Satisfactory 1.50 4 03/23/2010 Satisfactory Satisfactory 3.72 5 11/21/2010 Satisfactory Moderately Satisfactory 6.16 6 08/03/2011 Moderately Satisfactory Moderately Satisfactory 7.77 Moderately 7 03/22/2012 Moderately Satisfactory 11.01 Unsatisfactory Moderately 8 11/09/2012 Moderately Satisfactory 11.01 Unsatisfactory 9 06/26/2013 Moderately Satisfactory Moderately Satisfactory 14.70 10 12/20/2013 Moderately Satisfactory Moderately Satisfactory 26.16 11 07/12/2014 Moderately Satisfactory Moderately Satisfactory 29.16 12 11/17/2014 Satisfactory Satisfactory 34.91 H. Restructuring ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in US$ millions Indicators were rephrased to distinguish the indicator from the target. Additional clarity was also provided on some indicators. The use of Project 07/19/2011 N S MS 7.77 funds to develop a Health Information System (HIS) was permitted, and Bank financing of Component 2 expenditures was increased from 79% to 100%. Key changes included modification of institutional arrangements for the Project and extension of target dates to 03/05/2013 N MS MU 14.70 2013 for three indicators, the target for one indicator, and the increase of the percentage of Bank financing for EEPs. The Project Closing Date was extended by one year from 11/25/2013 N MS MS 26.16 November 30, 2013 to November 30, 2014. xii I. Disbursement Profile xiii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal Country/Sector Background and Rationale for Bank Assistance 1. Barbados is the easternmost country in the Caribbean and had a population of 277,000 at the time of Project appraisal in 2008. At that time, the Caribbean Region had the highest Human Immunodeficiency Virus (HIV) prevalence among adults outside Sub-Saharan Africa. The estimated prevalence rate for Barbados increased from 1.3% in 2001 to 1.5% in 2005, and HIV/ AIDS was the greatest burden of infectious diseases in Barbados for the 15-49 year age group. In comparison, estimated HIV/AIDS prevalence in 2005 according to UNAIDS was 1.1% in the Dominican Republic, 1.5% in Jamaica, and 3.3% in the Bahamas. The National Strategic Plan of Barbados 2005-2025: Global Excellence, Barbadian Traditions, identified HIV/AIDS as one of the major threats to its overall success as it endangered the country's human capital, putting productive capacity and economic growth in peril. The potential for escalation in the burden on welfare services and increasing treatment costs was significant. Such a negative impact on the productive population combined with increases in the aged population could create a cost to the country that was monetarily and socially unsustainable. 2. The World Bank (hereafter the Bank) has been providing technical and financial assistance in the area of HIV/AIDS to Barbados since 2001. The Caribbean HIV/AIDS I-Barbados Project for US$15 million (P075220; Loan: IBRD #7066), which closed in 2007, achieved high levels of awareness, treatment, and care, but was less successful in inducing the level of sustained behavior change required to impact the epidemic's prevention and control. Given the success of the First Project and the remaining unfinished work, the Government of Barbados (GOB) requested a follow-on project to contribute to tackling the outstanding challenges. The US$35 million World Bank Loan was based on the estimated total Project cost of US$94.39 million. US$89 million was for investment costs and operational costs, and US$4.5 million was for technical assistance; the balance included a front-end fee and price contingencies. The GOB requested the Bank to provide budget support for 35% of investment and operational costs and to finance 80% of technical assistance costs. The GOB requested the loan in part to ensure sustainability and commitment of funding to the National HIV/AIDS Program while improving accountability and multisectoral coordination of the Program. The Bank’s assistance was justified in the context of the public goods nature of support to HIV/AIDS prevention and control. At the time of Appraisal, the Bank was the only source of external funding as other sources such as the Global Fund against AIDS, TB and Malaria targeted lower-income countries. However, after the first year of Bank financing through the Second HIV/AIDS Project, the U.S. President's Emergency Plan for AIDS Relief (PEPFAR) and Centers for Disease Control and Prevention (CDC) funding also came about, mainly focused on providing additional technical assistance in support of the National HIV/AIDS Program.2 2 CDC funding from September 30, 2009-September 29, 2014 totaled US$1.62 million. PEPFAR funding was also provided for the Caribbean HIV/AIDS Regional Training program; however, information on the specific amount was not available at the time of this ICR. 1 1.2 Original Project Development Objectives (PDOs) and Key Indicators 3. The Project Development Objectives (PDOs), as listed in the Loan Agreement dated September 25, 2008 and in the Project Appraisal Document (PAD) 3 was as follows: The Project would support the implementation of the National Strategic Plan for HIV Prevention and Control 2008-2013 (NSP), specifically to increase: - Adoption of safe behaviors, in particular amongst key populations at higher risk. - Access to prevention, treatment and social care, in particular for key populations at higher risk. - Capacity of organizational and institutional structures that govern the NAP. - Use of quality data for problem identification, strategy definition and measuring results. 4. Key performance indicators (KPIs) in the Loan Agreement (Section II) were consistent with those in the PAD. Behavior change • Maintain the percentage of young people 15-24 years spontaneously indicating sexual relations as a way of transmitting HIV at least at 90% from 2008 to 2013. • Increase the percentage of: o Sex workers who report the use of a condom with their most recent client from 80% in 2008 to 95% in 2013. o Men who have sex with men (MSM) who report the use of a condom the last time they had sex from 65% in 2008 4 to 75% in 2013. o Young people 15-24 years reporting the use of a condom the last time they had sex with a non-marital, non-cohabitating partner from 21% in 2008 to 31% in 2013. Access to prevention, treatment and social care • Increase in the number of people from key populations at higher risk accessing preventive services from 250 in 2009 to 500 in 2013. • Maintain the percentage of HIV-positive pregnant women receiving a complete course of antiretroviral (ARV) prophylaxis to reduce the risk of mother to child transmission (MTCT) above 95% in the period from 2008 to 2013. • Maintain the percentage of People Living with HIV (PLHIV) on Antiretroviral Treatment (ART) achieving virologic success in the last 12 months above 70% in the period from 2008 to 2013. NAP Capacity • Increase in the funds spent by Civil Society Organizations (CSOs) under results-based agreements with the National HIV/AIDS Commission (NHAC) to facilitate the implementation of program interventions for key populations at higher risk in the period from 2008 to 2013. 3 The wording of the PDO in the loan agreement dated September 25, 2008 is slightly different than that in the PAD but does not translate to any difference in substance. Similarly, there is a slight difference in the PDOs listed in the Data Sheet of the PAD and in the main body of the document, but this does not translate to a difference in substance. 4 This baseline figure is listed as 64% in the PAD. 2 M&E • An evidence-based Strategic Plan for the period of 2013-2018 prepared before Project closing, taking into account surveillance and Monitoring and Evaluation (M&E) data available under the Project. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 5. The PDO was not revised. However, the KPIs were modified in the First Restructuring (July 19, 2011) as follows: (i) all nine KPIs were rephrased to distinguish the target from the indicator, (ii) indicators were refined to improve clarity of the definitions, and (iii) targets were revised for indicators for which updated information was available. Overall, the changes were minor with the exception of the following: (i) KPI #4 (increase the percentage of young people 15- 24 years reporting the use of a condom the last time they had sex with a non-marital, non- cohabitating partner) – the target for 2013 was revised from 31% to 65%; (ii) KPI #5 (increase in the number of people from key populations at higher risk accessing preventive services) was changed to specify and set targets for key groups from a target of 500 in 2013 for key populations at higher risk to 75% for MSM by 2012 and 70% for Female Sex Workers (FSW) by 2012; however, no comparable data was provided for the baseline; (iii) KPI #8 – there was a reduction in the amount of funds available to be spent by CSOs under results-based agreements from $502,000 to $150,000 by 2013 given the limited number of CSOs operating in the country and the slow start to the CSO grant system. 6. The Second Restructuring (March 5, 2013) involved changing the final target year for survey completion from 2012 to 2013 for three indicators (KPI #2, #3 and #5) and the target for one indicator (KPI #4). 7. The Third Restructuring (November 25, 2013) extended the KPI target dates for all indicators (and Project closing date) by one year to November 30, 2014. 8. Intermediate performance indicators taken from the National AIDS Program’s framework were listed in the PAD, but were not reported in the system after the first Implementation Status and Results Report (ISR). The exclusion of the intermediate outcome indicators (IOIs) is also seen in the revised Results Framework presented in the First Restructuring Paper which includes only the nine KPIs. The Bank team formally monitored only those indicators agreed on with the Government and included in the Loan Agreement. 1.4 Main Beneficiaries 9. The primary target group defined in the PAD were key populations at higher risk. This included male and female Sex Workers (SW); MSM; prisoners and drug users (DU); PLHIV and with disabilities; youth, especially out-of-school; and unemployed women; further details on each of these groups is provided in a beneficiary assessment in the PAD (Annex 6 of the PAD). Main key populations at higher risk for the Project are considered to be MSM, FSW, and youth based on the KPIs and discussions during the ICR mission. Figures in the PAD show that youth was the 3 largest key population targeted at roughly 12.3% of the total population, while the size of the MSM and SW populations were 1% and 0.6-2.24% respectively. 1.5 Original Components 10. The Project had two components which were closely linked to the PDOs. A summary of these components is provided below; a detailed description is available in Annex 12: Component 1: Prevention and Care (US$89.65 million, US$31.4 million loan) This component aimed to contribute to the implementation of the NSP, specifically where the following three Eligible Expenditure Programs (EEPs) are concerned: Sub-component 1: National Program Coordination and Institutional Strengthening. 5 This program aimed at strengthening the ability of the public sector, private sector and civil society partners to coordinate, monitor and evaluate their activities and use data to continually increase the quality of their programs. Sub-component 2: Scaling up Prevention Efforts. This program aimed to increase access to preventive services, especially among key populations at higher risk, and including Behavior Change Communication (BCC) and provision of condoms. Sub-component 3: Improving Diagnosis, Treatment & Care. The goal of this program was to increase the length and quality of life of PLHIV. The program aimed at increasing PLHIV access to diagnostic services, treatment services (ART and treatment for opportunistic infections (OIs)) and social care and support (counseling, support groups, drug addiction therapy, and home care). Component 2. Institutional Strengthening5 (US$4.47 million, US$3.6 million loan) The objective of Component 2 was to provide institutional strengthening via training and technical assistance that was not funded under the Sector Wide Approach (SWAp) 6 component. While routine surveillance, seroprevalence and behavioral surveys, and quality audits were carried out under Component 1, the second component included non-routine training and technical assistance to review the surveillance system, putting sero- and behavior surveillance and quality audits in place, and assisting with standardization of data collection methodologies, particularly in the case of behavior surveillance. This component financed training and technical assistance on M&E, management, surveillance, prevention, diagnosis, treatment and care of HIV/AIDS and other Sexually Transmitted Infections (STIs), to support the implementation of the Strategic Plan. 5 There was some overlap in the naming of components and subcomponents. Institutional strengthening under Component 1 refers to institutional strengthening as implemented under the NSP; support was provided in the form of budget support. Comparatively, Component 2 on institutional strengthening refers to non-routine training and technical assistance activities; support here was provided in the form of technical assistance. 6 The World Bank defines a SWAp as “…an approach to a locally-owned program for a coherent sector in a comprehensive and coordinated manner, moving toward the use of country systems. SWAps represent a …shift in the focus, relationship and behavior of donors and governments. They involve high levels of donor and country coordination for the achievement of program goals, and can be financed through parallel financing, pooled financing, general budget support, or a combination.” In this Project, the Bank and GOB funds were pooled. 4 1.6 Revised Components 11. Component 2 was revised to include the development of a health information system (HIS) as part of the First Restructuring due to the need to improve the capacity for evidence-based planning. The HIS cost US$2.8 million, which was financed by the Government and the World Bank, and did not require reducing investment from other activities as it fell within the scope of technical assistance. 1.7 Other Significant Changes 12. In addition to the changes listed above, the Project was restructured three times: • First Restructuring: This entailed an increase in Bank financing of Component 2 expenditures for goods and consultants’ services from 79% to 100%, due to the difficulty by the Government of Barbados in providing counterpart funding and due to the impact of the global economic and financial crisis. This was done in response to a letter dated July 29, 2010 by the GOB which requested this increase. • Second Restructuring: Changes aimed to: (i) strengthen leadership and capacity of the Project by further engaging the Ministry of Finance (MOF) to support the NHAC in coordinating across non-health line ministries to accelerate the level of disbursements and procurement, monitoring and evaluation and supervision functions to the Ministry of Health (MOH); (ii) increase the percentage of Bank financing of Eligible Expenditures Program (EEPs) under Component 1 from 35% to 65% to accelerate the disbursement in order to fully utilize all of the funds to achieve the PDOs. • Third Restructuring: The Project Closing Date was extended by one year to November 30, 2014 in order to give the Borrower time to complete outstanding activities, including the first stage of implementation of the HIS, Behavioral Surveillance Survey (BSS) results for FSW and MSM, and organization of consultative sessions for the NSP. The extension of the Closing Date was also expected to allow for full utilization of the undisbursed amount of US$8.75 million. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 13. Soundness of the background analysis: This Project was prepared as a follow-on operation to the Caribbean HIV/AIDS I-Barbados Project. Background analysis included in-depth beneficiary and institutional assessments, and incorporated lessons learned from the previous Project (general lessons and those from the results of a Quality Assurance Group (QAG) review of the First Project in November 2006), as well as from HIV/AIDS projects that tackled the epidemic head-on such as through the provision of condoms and needle exchange programs (Brazil, Thailand and Uganda) and those that focused on the political and social environments and on policies and legislation (Jamaica and other countries in the Caribbean). In addition, the Project’s 5 emphasis on key populations at higher risk was appropriate as the epidemic was considered to be focused among these populations. In general, the background analysis was found to be sound. 14. Assessment of the project design: The PDOs and design of the Project were in line with the goals of the National Strategic Plan, the Pan Caribbean Partnership Against HIV/AIDS (PANCAP) and the Caribbean Community and Common Market (CARICOM)’s Caribbean Regional Strategic Framework. The project design took a comprehensive, multisectoral approach to addressing HIV/AIDS, and took the needs of the various high-risk groups into consideration. The implementing agency, the NHAC (established in 2001), had previous experience in working with the Bank (under the First Project) and worked in partnership with many different Ministries, CSOs, and community and religious groups. 15. Implementation arrangements, as specified in the PAD, could have benefited from improvements in two key areas. First, the PAD notes that the Project would not require a Project Implementation Unit (PIU) since the National AIDS Response was considered integrated according to the World Bank’s project implementation arrangements classification criteria. This was because the majority of activities would be carried out by civil servants and preexisting departments and units, with major ministries reporting to NHAC. However, the PAD also notes the existence of few dedicated posts of HIV Coordinators, and that many government employees were assigned responsibility for HIV coordination in addition to their regular duties. In light of this, the Project would have likely benefited from a PIU, ideally one that included representation from the NHAC and MOH. In addition, the implementation arrangements state that the institutional and implementation arrangements for the Second Project would be similar to those for the First Project, without adequate consideration of the change in size and scope from the First to the Second Project, and of the transfer of the NHAC from the supervision of the Prime Minister’s Office to the MFYS. Greater consideration could have been given to the changes in the size and scope of the Second Project. Inconsistencies in the Project documents such as the PAD and the Project Operations Manual on expenditures eligible for reimbursement also led to some confusion early on in the Project. The GOB sought a formal Legal Opinion on the interpretation of eligible expenditure. 16. Second, the Project was financed by an innovative lending approach – a SIL of US$35 million with disbursement-linked indicators (DLIs) where a SWAp was followed for Component 1 activities and traditional investment lending for Component 2. 7 This approach, while innovative and supportive of government programs, also presented potential challenges in terms of implementation due to difficulties in understanding and executing reimbursable expenditures. Eligible expenditures were not initially submitted for reimbursement as there was a lack of clarity in some line ministries on which expenditures qualified for reimbursement and the process by which reimbursement occurred. The Project design and the co-financing mechanism (where the Bank co-financed a percentage of eligible expenditures of the government program), while 7 For Component 1, the use of a SWAp approach emphasized country ownership and capacity building; the simultaneous use of DLIs provided an emphasis on the achievement of results. DLIs ranged in value from $1.5-$11.5 million, and provided flexibility in cases of partial compliance while requiring the GOB to submit a time-bound action plan for achievement of results going forward. Loan funding was pooled with government funding to finance the NAP in Component 1, while loan funding was disbursed against government contracts in Component 2. For Component 2, traditional investment lending was acceptable given the focus on technical assistance. 6 challenging, was necessary to preserve fiscal space for health, which was a concern at the time of Project preparation. 17. High-level government commitment: The GOB was highly committed to the Project. The request for this Project came from the Prime Minister, and was approved by Cabinet. In addition, the institutional assessment section of the PAD notes that the national response to the HIV/AIDS epidemic in Barbados is mainly led by the GOB, once again highlighting the commitment of the government. 18. Assessment of risks and mitigation measures: Several risks were identified in the PAD, but the overall risk rating was low. One of the risks identified as low, namely the vulnerability of the economy to external shocks, materialized during the course of the Project and the economic downturn resulted in Project restructuring (First Restructuring). In retrospect, the vulnerability of the economy to external shocks should have been rated as a moderate risk given the economic climate at the time. Speed of implementation in the early stages was affected by insufficient implementation capacity – the mitigation measures highlighted that the Bank would provide funding, technical assistance, and training for planning and M&E purposes. However, in at least one instance, training on Financial Management was provided only at the later stages of the Project when it should have been provided much earlier. 2.2 Implementation 19. Implementation Challenges (Pre-Midterm Review): Implementation capacity was insufficient at the beginning of the Project with implementation arrangements continued from the First Project despite the change in PDOs and substantially larger scope of the Second Project. This lack of capacity was compounded by a lack of technical expertise among key staff necessary for successful implementation. Finally, as noted by the PAD, despite the existence of a multisectoral framework to implement the National HIV/AIDS Project, the reality was quite different on the ground. 20. The Project continued to face capacity challenges until the Second Restructuring of the Project, which occurred after the midterm review (MTR). The MTR, held December 5-9, 2011 identified three major weaknesses: (a) lack of effective leadership and weak capacity of the NHAC; (b) lack of coordination between the NHAC and the MOH, the main implementing agency of the Project; and (c) delayed implementation and underutilization of available loan funds. The Second Restructuring resulted in a turning point for the Project which had previously been facing challenges and was rated Moderately Satisfactory for PDO and Moderately Unsatisfactory for Implementation Progress (IP) after the MTR. These issues posed the biggest challenges to Project implementation, and are discussed in further detail below: a. Lack of mandate, weak capacity, and poor coordination: The Project design required the NHAC to take the lead on coordinating the national response across agencies and on monitoring Project implementation. However, other agencies often had competing priorities and did not always report eligible expenditures on a regular basis. In addition, the NHAC had previously been under the Prime Minister’s office but was transferred under the MFYS as the Project started. This transfer may have resulted in the perception of a 7 reduction in the mandate of the NHAC, and may have hindered their ability to lead a successful, robust national response. The possibility of the chair of the National HIV/AIDS Commission stepping down prior to the Second Project commencing was raised as a concern, and occurred during the course of the Project. This concern was heightened considering a successor would be new to the role, not carrying the mandate from the Prime Minister’s office, be tasked with implementing cross-Ministry coordination and reporting without the previous mandate, with less visibility and authority, and implementing a new, larger-scale project with a more elaborate design. In addition, the NHAC had insufficient capacity as Project implementation began. Staff with certain areas of technical expertise were needed at the NHAC, and the MOH was theoretically able to provide this expertise, but ended up not doing so due to a lack of coordination between the two agencies as a result of poorly-defined roles prior to the Second Restructuring of the Project. Such was the case in the M&E and procurement functions. As the Project progressed, the NHAC staff proved to be strongly committed and took on the task of self-teaching where possible and drawing applicable expertise from related areas of their background. The Bank also provided technical assistance. Nonetheless, improved coordination between the NHAC and the MOH at the initial stages of the Project, and perhaps a more proactive effort by the GOB to seek assistance on issues that were unclear in the early stages of the Project, would have been beneficial. It would also have been beneficial if the Bank had provided additional clarification where Project arrangements were concerned, particularly with regard to reimbursement of eligible expenditures. b. Delayed implementation and underutilization of available loan funds: More than 40 officials received financial management and procurement training in several line ministries at Project Launch. However, the training provided appeared to be inadequate; in addition, some staff left and others received no training until late in implementation. A repeated theme during the ICR mission was the initial lack of clarity and subsequently, different interpretations, on items that qualified for reimbursement, and how the reimbursement mechanism worked. The mechanism of incurring costs first to receive reimbursement later was thought to be difficult by some as it meant they needed to find money to spend in the first place, which was challenging as the country was going into a recession. Officials were thus getting mixed messages on the need to conserve spending in general, while at the same time being encouraged to spend for reimbursable items. As the cost-cutting exercise also began in government at that time, there were fewer people available to do the needed work. This was further complicated by inconsistencies in the PAD and the Project Operations Manual which provided conflicting information on whether capital works were eligible for reimbursement. In addition, the location of the Bank’s loan center in Brazil and its financial staff in DC resulted in slower and more challenging communication where financial management and loan disbursement was concerned. 21. Implementation Strengths (Post-Midterm Review): The MTR proved to be a pivotal turning point for the Project, and provided excellent inputs on where improvements could be made 8 by the Bank and GOB. Due to the candor of the review, those involved were forced to face the shortcomings of the Project. The Permanent Secretary of Investment in the Ministry of Finance and Economic Affairs at the time became personally involved and held a meeting with representatives from the different ministries aimed at identifying reimbursable expenses and providing clarity on the reimbursements. Subsequently, implementation improved significantly, particularly in the two areas summarized below: Area 1. Capacity strengthening and strengthening of the multisectoral response: The increased coordination role of MOF in support of disbursement and financial management functions; of procurement, monitoring and evaluation and supervision functions to the MOH; and the establishment of a Project Director housed at the MOH resulted in a substantial, rapid improvement in implementation. The multisectoral coordination between the different agencies improved and roles and responsibilities of the different ministries were clarified. In addition, the establishment of the Project Director post housed at the MOH also made it easier to have direct contact with the Bank for assistance when needed. The appointment of the Project Director also meant there was now a central focal point for relevant MOH activities. As a result of these different factors, the Project picked up speed to the point where it was able to compensate for the implementation challenges faced in the first half of the Project. Area 2. Strengthening of M&E culture: The strong emphasis on identifying and reporting expenses for reimbursement and on the achievement of DLIs by the Permanent Secretary after the MTR highlighted the importance of M&E efforts. By the time of the MTR in 2011, training in M&E had begun to take effect, and the momentum generated by the Permanent Secretary further highlighted the importance of these activities. These improvements are demonstrated in the achievement of PDO #4. 2.3 M&E Design, Implementation and Utilization 22. Design: M&E for the Project used existing indicators from the NAP’s framework. These indicators were developed using the UNAIDS core indicators as guidance, with input from the Bank team. There appeared to be some confusion over the list of 39 indicators tracked by the NAP as some within the MOH felt their input was not taken into consideration, while others within the NHAC did not feel they received feedback and support on which indicators were relevant for the country. The Project design focused on regular reporting of nine KPIs as the Project’s Results Framework. 23. In terms of targets, the ICR mission found that initially, there was a strong feeling among the GOB that some targets had been set too high. Had this been taken into consideration, however, it may have led to lower targets being set, and the Project may not have had such a catalytic impact in terms of behavior change. In general, indicators were found to be appropriate for monitoring the key populations described in Section 1.4 of this ICR. Given the size of some target populations (e.g. as was the case with the PMTCT indicator) it may have been preferable to measure progress in terms of raw numbers instead of percentages. 9 24. Implementation: Even though the Project did not regularly report on the IOIs, the Bank monitored the achievement of the indicators listed in the entire NAP’s framework through its Aide Memoires and in discussions with the GOB. Information from these additional indicators, where relevant, has been used in assessing the achievement of the PDOs for the purpose of this ICR. 25. Implementation of M&E activities was difficult at the beginning of the Project. Baseline information was not available for the most-at-risk populations (MSM and FSW), and tracking progress on some indicators proved to be problematic, partly because of the conservative society in which they were measured and because comprehensive key population programming did not exist at the time that Project implementation began. Identification and subsequent survey of the MSM subpopulation may have also been hampered by a former member of the NHAC that later made media statements indicating a lack of tolerance for MSM. The BSS for MSM, at the time of Project completion, only managed to recruit 126 of an intended 400 participants. 26. While the MOH had the M&E skills that may have helped overcome some of these difficulties, the NHAC was unable to tap into these, likely as a consequence of poor communication. As a result, the designated officer at the NHAC (who did not have an M&E background) had a steep learning curve coupled with the responsibility of monitoring the 39 indicators listed in the NAP’s framework. Indicators were initially difficult to monitor, but there was strengthened capacity and improved M&E performance during the Project, particularly following the MTR restructuring when the MOH took over the M&E function and coordinated with NHAC for indicators related to the non-health sector response. 27. There were several M&E systems that the Project aimed to strengthen, namely the MHNI- MIS, HIV/AIDS and STI Surveillance, Sero-prevalence Surveys, Knowledge, Attitudes, and Practices (KAP) Survey, Health Facility Survey, and an Environmental Health Assessment. Given the results-based Project design, technical reviews were also conducted by an independent agency twice. Altogether, these activities strengthened the M&E capacity in the country. The ICR mission found that M&E began to be seen as important by the various groups involved as the Project progressed, and an M&E culture began to develop. 28. NHAC placed strong emphasis on monitoring and evaluating its outputs, developed an M&E training plan, and conducted biennial meetings to which other organizations were invited to participate. The NHAC also strengthened capacity to the extent that it was able to take over administration of the KABP from the MFYS in 2013; previously it had relied on the MFYS to provide financing and manpower for the survey interviewers. Over time, there was an improvement in M&E efforts, and surveys began to be conducted on a regular basis instead of ad- hoc. However, a gap still exists in key epidemiological and behavioral data where MSM and FSW are concerned. 29. Utilization: Despite the lack of a clear data dissemination and communication plan, utilization of M&E took place in several ways. Dissemination took place in the form of annual reports, while monthly coordinator meetings were held to provide a forum for discussions. Biennial research symposiums were also conducted where research and M&E information were presented. This helped the NHAC’s partners to identify information needed for their specific programs; data 10 was also used to plan programs within the various ministries. Survey results were also used internally by partners of the NHAC to help modify their programs. 30. Utilization of M&E data was also key to the development of the evidence-based Strategic Plan for 2014-2018. This is discussed further in Section 3.2 of this ICR under PDO #4. 2.4 Safeguard and Fiduciary Compliance 31. No safeguard policies were triggered by the Project. 32. Fiduciary compliance: In addition to the difficulties in understanding the Project design (see Section 2.2 of this ICR) and the delay or lack of financial management training to some key Project officials, the design of the Project coupled with the difficult role played by the NHAC as coordinator under the MFYS meant that there were some difficulties in monitoring Project expenditures. No reporting system specifically identified HIV/AIDS-related expenses, and line ministries with reimbursable expenditures did not always list these expenditures in an easily- identifiable manner. Accountants for the ministries also had competing priorities and were not always able to provide a comprehensive list of expenses related to the Project. Auditors for the Project had to visit the different ministries and on occasion, audit reports were delayed because they had to pull information from many different places. While challenging in terms of execution, fiduciary compliance of the Project was moderately satisfactory. Further details on Project Expenditures, including details of the Special Audit, are provided in Annex 13. 33. Procurement: There were some difficulties faced with procurement at the initial stages of the Project, which may have been compounded by the initial lack of clarity over its reimbursable design. In particular, there was some missing information, and a lack of ownership of the procurement plan at NHAC. The Bank provided additional training in response, suggested improvements in coordination between procurement officers at the NHAC and the MOH’s HIV/AIDS unit, and transferred procurement to the MOH where there was a procurement officer familiar with the Bank’s procurement processes. It was also difficult to obtain funds for the initial payments of goods and services, even if they were later reimbursable through the Project. In addition to difficulties faced as a result of the financial crisis, this was due to the line item budgeting for each Ministry defined only at one point in the year, and which required seeking formal lengthy exceptions to access funds outside of that time period. The largest procurement occurred for the HIS, which involved a competitive process and went smoothly. 2.5 Post-completion Operation/Next Phase 34. The National Strategic Plan of Barbados 2005-2025 has as Goal 3 the continued reduction in the spread of HIV/AIDS and minimization of its negative impact, improved information systems, research, and M&E to support decision-making with respect to HIV/AIDS and the creation of a national multi-sector HIV/AIDS Program. Thus, HIV/AIDS is seen as an important issue in the long-term development of Barbados. 35. The NHAC is operating after Project completion, and an evidence-based Strategic Plan has been developed for 2014-2018. A large part of the Project involved building institutional and M&E 11 capacity for the country, both within and beyond the HIV/AIDS program. This is expected to be sustainable given that coordination between the different agencies and M&E capacity has improved during the course of the Project. 36. The rollout of additional modules of the HIS continues to take place. The HIS collects information beyond HIV/AIDS and comes with a 5-year warranty which will be useful for troubleshooting and refinement as it is being deployed around the island. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 37. Relevance of Objectives, Design and Implementation is thus rated as Substantial. The Project’s objectives remain highly relevant and aligned with Barbados’ efforts to address HIV/AIDS. While the Bank does not have a Country Assistance Strategy for Barbados, there is a Regional Partnership Strategy for the Organization of Eastern Caribbean States. The issue of HIV/AIDS continues to be highlighted as one of importance in the Regional Partnership Strategy. In addition, the continued relevance of the Project (which was aimed at supporting the 2008-2013 National HIV/AIDS Strategic Plan) and of HIV/AIDS in Barbados, is seen in the development of an evidence-based Strategic Plan for 2014-2018. HIV/AIDS continues to be seen as a significant economic issue, and there continues to be room for improvement in the area of discrimination and stigma. As of the February 2015 ICR mission, the NHAC has been invited by Cabinet to present an expanded role for itself that would allow it to build on its success in addressing HIV/AIDS and focus on BCC in the context of other areas of health, such as Non-Communicable Diseases (NCDs). 38. The design and implementation of the Project continue to be highly relevant. The focus on key populations continues to be appropriate given the concentrated nature of the epidemic, and the strategies employed to address the epidemic within these key populations continue to be seen as suitable. Successful implementation of the Project required close collaboration between the NHAC and the MOH, which was not included as part of the original Project design, but will continue to be important going forward. 3.2 Achievement of Project Development Objectives 39. Barbados has seen substantial progress in its HIV/AIDS program from the time this Project began. While baseline indicators for some KPIs were already impressive at the beginning of the Project, they continued to show further improvement. As the Bank was the primary agency that provided funds for the NAP during the course of the Project, achievement of the PDOs can be considered attributable mostly due to efforts made by the GOB and the Bank. 40. Achievement of the PDOs are based on the original nine KPIs included in the PAD and legal agreement, three new KPIs included at the time of the ICR as well as IOIs included in the PAD. A summary of the indicators is presented below to provide an overview of Project achievements. 12 Rating KPIs Intermediate Outcome Indicators Surpassed 6 4 Achieved 4 1 Not Achieved 1 0 Unknown 1 0 Total 12 5 % surpassed and achieved 83% 100% 41. Results against each PDO sub-objective are described below, with additional evidence to demonstrate achievement of such sub-objectives included where relevant. PDO sub-objective 1: To increase adoption of safe behaviors, in particular amongst key populations at higher risk. Rating: Substantial. • KPI #1: Percentage of young women and men 15-24 years indicating sexual relations as a way of transmitting HIV. Surpassed • KPI #2: Percentage of female sex workers reporting the use of a condom with their most recent client. Unknown • KPI #3: Percentage of men reporting the use of a condom the last time they had sex with a male partner. Not Achieved • KPI #4: Percentage of young men and women aged 15-24 years reporting the use of a condom the last time they had sex with non-marital non-cohabitating sexual partner. Surpassed • KPI #11: Percentage of young women and men aged 15-24 who have had sexual intercourse before the age of 15. Achieved • KPI #12: Percentage of MSM who received an HIV test in the last 12 months and who know their results. Surpassed • IOI #1: Number of targeted intervention programs for key populations at higher risk. Surpassed 42. Adoption of safe behaviors was observed where high-risk sex was concerned. Improvements in condom use when having sex with a non-marital non-cohabitating sexual partner (KPI #4), and a decline in the proportion having sex with non-regular partners (from 29.4% in 2009 to 12.8% in 2013) was seen. A decline in the proportion of youth 15-24 having sex before the age of 15 (KPI #11) was also observed. The adult KABP survey also revealed an increase in condom use among those with 2+ partners from 52.4% in 2009 to 72.6% in 2013; among those with non-regular partners, these figures were 80% to 86.2% for the same time frame. Consistent condom use was found to increase with non-regular partners from 48.6% in 2009 to 54.5% in 2013. 43. An increase in knowledge was also observed among youth and MSM. Almost all youth surveyed identified sexual relations as a way of transmitting HIV (KPI #1), exceeding the target. In addition, all MSM surveyed in 2013 and 2014 who received an HIV test in the last 12 months knew their results, an impressive achievement from a baseline of 50% in 2009. 44. An increase in targeted intervention programs was seen over the course of the Project, which likely contributed to the increase in safe behaviors in the youth and MSM populations. The 13 number of targeted intervention programs for key populations at higher risk increased from 2 in 2008 to 52 in just four years (IOI #1). 45. Of the three key populations measured by the KPIs, namely youth, MSM, and FSW, the Project appears to have had a particularly strong impact on youth. Estimates from the PAD place the youth population at 12.3% of the total population, while the MSM and FSW populations are estimated at 1% and 0.6-2.24%, respectively. Specific PDO Target #2: To increase access to prevention, treatment and social care, in particular for key populations at higher risk. Rating: Substantial. • KPI#5: Percent of most at-risk populations (SW and MSM) reached with HIV prevention services. Achieved • KPI#6: Percentage of identified HIV-positive pregnant women who gave birth in the last 12 months receiving a complete course of ARV to reduce MTCT. Achieved • KPI #7: Percentage of people living with HIV on first line antiretroviral regimen achieving virologic success within the first 6 months of treatment. Surpassed • KPI #10: Percent of infants born to HIV-infected mothers who are infected. Surpassed • IOI #3: Number of PLHIV and/or families accessing the Food Bank. Surpassed • IOI #4: Number of CSOs working with most-at-risk populations. Surpassed 46. A large increase was seen in the percentage of MSM and SW reached by HIV prevention services (KPI #5). In addition, KPI #6 rates were 87.5% (2009), 92.0% (2010), 100% (2011), 100% (2012), and 90.9% 8 (2013). Achievement of the PMTCT coverage at 100% for two years of the Project and of having zero HIV-positive births for the first four years of the Project (KPI #10) was a notable accomplishment that would have set an example in the region if had it been maintained. Given the small denominator, it may have been better to monitor PMTCT coverage through raw figures, and not through percentages given that the occurrence of a single case could deviate the numbers substantially. As a result of the use of percentages, the 2013 outcomes for KPI #6 appear to decline by almost 10%, caused by two cases of identified HIV-positive pregnant women failing to complete ARVs. 47. The Project was successful in improving treatment outcomes among PLHIV (KPI #7). Using a definition of <50 copies/ml for virologic success, 70.1% of this group achieved treatment success. When the criteria was relaxed to <1000 copies/ml within 12 months of treatment, in line with the World Health Organization (WHO) Early Warning Indicator #8, virologic success was achieved by 92.5% of the group. As the focus was on maintenance, it is also important to note that achievement of virologic success based on the WHO Early Warning Indicator #8 (in line with international standards), was obtained among >80% of the group throughout the duration of the Project. Close to US$10 million was spent on ARVs throughout the duration of the Project, helping to ensure free treatment for those in need. The improvement in treatment outcomes reflects the result of this investment. 8 A detailed review was conducted by the Project team of the missed cases which revealed several systemic issues: (i) the rapid HIV tests performed in labor did not provide the result prior to delivery; (ii) the results of previous HIV tests in other facilities within the health system were not recorded or available to the ANC staff; and (iii) patients reported stigma as a barrier to early access to care. 14 48. Evidence of increased access to social care is available using several measures. An increase in the number of CSOs working with key populations (IOI #4) was seen over the course of the Project. 21 CSOs were working with key populations in 2013, and 16 in 2014 – both substantial increases from a baseline of 2. Of the 34 projects for which CSOs received funding, 27 projects were conducted by CSOs new to working on HIV/AIDS. 25 projects were fully implemented; these projects targeted 7 population groups, such as youth (15 projects), persons with disabilities (2 projects), men who have sex with men/sex workers (6 projects), the general population (2 projects), and ex-offenders, PLHIV, and the elderly, all of which had one project each. Some of these CSOs, which received funds through the CSO grant system, such as the Family Care Foundation used funds to provide 46 PLHIV with training that improved their life skills, while Jabez House, which had not been involved in HIV/AIDS work prior to the Project, trained more than 75 female sex workers in alternative employment skills, such as hairdressing and business development. The total number of persons reached by the CSOs were 535. In addition, the number of PLHIV and/or their families accessing the Food Bank (IOI #3) went from 250 at baseline to a total of 474 in 2013 - 313 receiving hampers and 161 receiving nutritional counseling. Specific PDO Target #3: To increase capacity of organizational and institutional structures that govern the NAP. Rating: High • KPI #8: Amount of funds spent by CSOs under results-based agreements 9 (baseline: $0 as of 2007; target: US$150,000 in 2014). Surpassed. • IOI #2: Number of health care providers trained in HIV testing and counseling according to national standards (baseline: 115 in 2006; target: 250 by 2013). Surpassed 49. An increase in capacity was observed through the activities of CSOs in the country and in the increased number of health providers trained according to national standards. Based on a desk review of the Grant Monitoring Reports conducted by NHAC, a total of US$186,848.00 was spent by CSOs under results-based agreements by 2014 (KPI #8). As an increase in spending does not necessarily translate to an increase in capacity, additional data is provided to support this PDO. First, an increase in the number of CSOs in general and in the number of CSOs working with key populations was seen as discussed under PDO #3. More than fifty strategic partners were trained in HIV work planning and program development, and over half the line ministries submitted work plans in line with national guidelines. The number of institutions that incorporated M&E components into their work plans increased from 8 in 2006 to 27 in 2014. In addition, the NHAC was able to take over the administration of the KABP survey from the MFYS as a result of their improved capacity. Finally, an assessment of the CSO Grant System showed improved proposal writing skills among the 19 CSOs who received training on proposal development. 50. Improvements in data quality demonstrating increase in capacity were also observed and are discussed further under PDO #4. Specific PDO Target #4: To increase use of quality data for problem identification, strategy definition and measuring results. Rating: High • KPI #9: Evidence-based Strategic Plan 2014-2018 (baseline: develop a Strategic Plan for 2013-2018; target: develop a Strategic Plan for 2014-2018). Achieved 9 Grants were given to CSOs on the basis of achieving results in specific areas that were agreed upon with the NHAC. 15 • IOI #5: Number of people trained in M&E at different levels (baseline: 39 in 2007; target: 125 additional people to be trained by 2013). Surpassed 51. A National Strategic Plan for 2014-2018 (KPI #9), which is strongly evidence-based, has been finalized and is currently being disseminated. This Strategic Plan incorporates information from a wide consultative process, epidemiological data, KABP reports, the previous strategic plan, results from the stigma and discrimination survey, the two M&E assessments (conducted in 2008 and 2013), the CSO grant system consultant report, and the Virtual Health Research Unit (VHRU). 52. The evidence base used to generate the Strategic Plan was possible due to the quality data produced through the Project. For example, the Project supported development of the VHRU, which is accessible to researchers, students, and nongovernmental organizations, and comprises more than 5,000 scientific publications and reports on HIV in Barbados and the Caribbean. Project funds were used to conduct M&E training for 683 people (IOI #5), through which the importance of M&E became clear and data entry began to increase as did the number of institutions incorporating M&E components into their work plans (PDO #3). Project funds were also used for the development of a HIS at the national level. This system includes a module on HIV/AIDS, but also includes other modules relevant to the healthcare system of Barbados. As a result, it is expected that it will be widely used going forward, resulting in an increase in quality data. 53. Given the Substantial rating for PDO#1, Substantial rating for PDO #2 and the High rating for PDO#3 and #4, the overall Efficacy of the Project is rated as Substantial. 3.3 Efficiency 54. Investments in prevention and treatment of HIV/AIDS have been established to be highly cost-effective. The ICR includes an economic and financial analysis comparing the with- and without-Project scenarios. This analysis focuses on cases of HIV/AIDS averted, and not on the beneficial effects of providing treatment or social care to PLHIV, or on improvements in the health system which the Project also supported. Without the Project, it is estimated that the incidence rate in 2014 would be 0.64 per 1000 population. However, with the Project, the incidence rate for 2014, calculated using the reported number of new cases until 2012, is 0.49 per 1000 population. Cost- benefit calculations assuming the average annual cost of treatment and care as listed in the PAD of $7,510 and a conservative life expectancy after diagnosis of 10 years reveal that the dollar cost for each HIV infection averted is $167,027.68 and $4,912.58 for each year of life saved. The total benefits of the Project based on reductions in incidence until 2014 is estimated to be US$123 million. Using figures from 2013, based on data until 2008 (provided by the GOB) for the average annual cost of treatment of $4,054 results in total benefits that are marginally lower at US$116 million. Calculations based on the cost of ARVs and average number of PLHIV over the duration of the Project indicate that the cost of ARVs alone per person per year was $800. 10 55. The comprehensive approach used by the Project to address HIV/AIDS also resulted in an overall strengthening of the health system which is vital to the sustainability of preventing, diagnosing and treating conditions such as HIV/AIDS over time and which will benefit the entire 10 Total amount spent on ARVs was US$9,565,115 over ~six years (duration of the Project) for the treatment of ~2,000 PLHIV. This is equivalent to ~$800/person/year or $66/person/month. 16 population over the long run as well. A comprehensive list of the outputs generated through the Project is provided in Annex 2, most of which involved improving M&E capacity, strengthening primary health care, changing behaviors, and strengthening institutions. The project was also fully disbursed, having been extended only once for 1 year. Therefore, on balance, investments in prevention and treatment, as well as in the health sector were efficient. 56. Despite the inherent substantial efficiency of interventions carried out under the project, it must be noted that given the size of the loan (US$35 million) and the size of the Barbadian population (277,000), achievements made under the project could have been much more significant. In particular, funds could have been used to finance: (i) improved monitoring of pregnant women to achieve a sustained rate of 100% PMTCT; and (ii) additional surveys among the high-risk populations which would have yielded a deeper understanding of the epidemic (such as the demographics and transmission information about new cases of HIV/AIDS) to enable an even more successful program and Project outcome. With this untapped potential in mind, overall Efficiency of the Project is considered to be Modest. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory Overall, the Project is rated Moderately Satisfactory based on its Substantial ratings for Relevance and Efficacy, and a modest rating for Efficiency. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development While the Project did not result in measurable improvements in terms of poverty impacts and gender aspects, it did result in social development in several areas. First, the Project resulted in the provision of additional social care to key populations as detailed in Section 3.2, PDO #2. Second, comparative analysis of data from nationally representative surveys conducted in 2011 and 2014 on Knowledge, Attitudes and Practices on HIV-related Stigma and Discrimination among Barbadians ages 18 to 64 towards PLHIV, persons in same sex relationships and sex workers (SW) revealed slightly greater acceptance of these groups in 2014. Some highlights include an increase in persons disagreeing with the idea they would be ashamed if someone in their family had HIV (74.1% in 2011; 77.3% in 2014), an increase in the proportion of persons who would be willing to associate with friends who were sex workers (51.1% in 2011; 57.9% in 2014) and a decline in those who believe the sexual promiscuity of male homosexuals is the reason for HIV (23.7% in 2011; 18.7% in 2014). These results, however, should be interpreted with caution given the lack of information on whether the improvements observed from 2011 to 2014 were statistically significant. (b) Institutional Change/Strengthening The Project had a substantial impact on the institutional capacity and multisectoral coordination capacity of the NHAC and on building capacity within CSOs. These achievements are discussed in Sections 2.2, 2.5, and 3.2. (c) Other Unintended Outcomes and Impacts N/A 17 4. Assessment of Risk to Development Outcomes Rating: Negligible to Low 57. The risk to development outcome for this Project is negligible to low. The NHAC is expected to continue its work, and an evidence-based Strategic Plan for 2014-2018 has been developed. The Project strengthened overall capacity, improved the multisectoral response and advanced monitoring and evaluation efforts, increasing the likelihood of technical sustainability. Given these improvements as well as the achievements of the Project, the risk to development outcome is low. As of February 2015, the NHAC has been invited by Cabinet to present an expanded role for itself that would allow it to focus on BCC in areas beyond HIV/AIDS, such as NCDs, indicating political and institutional sustainability. Financial sustainability is also promising - the NHAC recently presented its budget to the Cabinet, which is thought to be sufficient to continue prevention, treatment, and care efforts that were made through the Project. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory 58. The PAD was very detailed, and the Project used an innovative financing mechanism7 and a comprehensive approach to achieving the PDOs. The Bank provided useful support where achievement of the PDOs were concerned, and managed to add value in several key areas highlighted in Section 2.2 of this document. Project preparation went smoothly, and the Project scope managed to further Goal 3 of National Strategic Plan 2005-2025. The First HIV/AIDS Project in Barbados was rated as Moderately Satisfactory by the Bank’s Independent Evaluation Group, and institutional and implementation arrangements that were in place for the First Project were transferred over to the Second Project. However, the size and scope of the Second Project were very different than that of the First Project. Consequently, the Project faced challenges where implementation was concerned. Despite detailed attention to Project design, implementation arrangements did not fully address capacity gaps that were acknowledged by the Bank (see Implementation Challenges highlighted in Section 2.2 for further detail on these difficulties). Improvement in capacity was indeed one of the Project PDOs, but the lack of capacity at entry and in Project design was only openly addressed in the restructuring conducted after the MTR. The reimbursable expenditures design of the Project proved to be a challenge to implement due to a lack of understanding over its complex design, and due to difficulties in identifying funding as detailed in Section 2.4 of this document. 59. Nonetheless, the Bank did provide assistance that added value to the Project, and restructured the Project to address these gaps in capacity after the MTR. These changes resulted in a substantial improvement to the Project, and also managed to result in building capacity in line ministries (NHAC and MOH) and in technical aspects such as M&E. 18 (b) Quality of Supervision Rating: Satisfactory 60. The concerns about gaps in capacity were echoed in the ISRs. In addition to the gaps in capacity described above, the complex Project design where reimbursement was concerned was not fully understood by many at the beginning of the Project. This was despite the fiduciary workshop and financial management training that was attended by over 40 people. As a result, disbursements were slow in the beginning. Some eligible expenditures were difficult to find funding for, even if reimbursable. Difficulties in finding funding became more challenging as the financial crisis hit. However, the Bank was responsive and the Project was restructured in order to allow for the Bank to assume a greater proportion of the overall costs of the Project, thereby reducing the contribution from the GOB. In total the project was restructured three times to update the financing mechanism in response to the financial crisis, provide adjustments in implementation structure in response to the MTR, and extend the Project in order to achieve the PDOs. 61. In terms of the Project team, the task team leaders changed three times over the course of the Project. Nonetheless, the final task team leader had been with the Project since the beginning, which created continuity. The ICR mission found that the GOB was satisfied with the quality of the technical assistance received during the course of the Project. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 62. Quality at entry is rated Moderately Satisfactory while quality of supervision is rated Satisfactory. Overall Bank performance is thus rated as Moderately Satisfactory, given that there were minor shortcomings in Quality at entry as discussed above. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 63. The government performance in the first half of the Project was mainly through the NHAC. Given their lack of capacity and the complexity of the Project, it was understandable that they faced some challenges. Implementation functions were split between the NHAC and the MOH after the MTR and the MOF took on an increased coordination role. It was then that the different parts of the government came together and the strengths of each agency became apparent. The appointment of the Project Director assisted greatly with multisectoral coordination efforts, and with improving understanding of the Project among the different agencies. Similarly, the role of the Permanent Secretary for Investment (under the Ministry of Finance and Economic Affairs) in ensuring that the relevant officers who needed training on the reimbursement strategy of the Project received training, and in scheduling meetings between the government representatives up to a year in advance is laudable. The ICR mission also found that within each agency, there were highly dedicated staff - those who were in roles beyond their areas of expertise taught themselves the necessary skills (e.g. the M&E Specialist) or drew from their other areas of experience (the 19 BCC Specialist) though they could have reached out to their counterparts in the MOH who had the requisite skills. 64. In general, therefore, the performance on the government’s side is considered Moderately Satisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 65. Up until the first half of the Project, the main implementing agency was the NHAC. The NHAC lacked the capacity and technical expertise to hit the ground running, given the increased scale and scope of the Project compared to the First HIV/AIDS Project and the lack of a transition plan. After the midterm review, the MOF and MOH took over some responsibilities – MOF took on an increased coordination role to support disbursement and financial management functions and the MOH took over the procurement, monitoring and evaluation and supervision functions. This separation of responsibilities resulted in an improvement in implementation. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 66. Government performance was Moderately Satisfactory, and the Moderately Satisfactory rating of the implementing agency is partly due to a lack of transitional arrangements from the First Project. 6. Lessons Learned • Greater attention needs to be paid to implementation capacity at project outset, beyond the successful completion of a previous project. Implementation capacity assessments should also include an evaluation of project staff relative to project scope, and a skills assessment of staff needed to conduct key tasks. • A better training process for financial management and procurement is necessary in the context of complex project design. It would be good to provide training in a timely fashion, and refresher training sessions when there are significant changes to project staff. Task teams should provide frequent and varied training opportunities to address these challenges. • Projects where countries are reimbursed for eligible expenses should look into difficulties with the project design in the event of slow disbursement at the beginning of the project. Slow disbursement may be a result of a lack of understanding of eligible expenses or project design, or of financial difficulties in raising funds to make payments, even if they are reimbursed at a later date. • Greater cultural consideration should be given when designing project activities and targets for indicators. Marginalization and stigma may also impede implementation, such as de- emphasis of certain key populations among other more acceptable groups, reluctance for surveys of these groups to take place, and poor outcomes due to late presentation to treatment and care. 20 • When developing indicators focused on small populations, it may be better to monitor achievements based on actual figures instead of/or in addition to percentages, as a change in n=1 could have a substantial impact in percentage terms. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/Implementing Agencies The Project assessment conducted by the GOB and comments on the Bank’s ICR is provided in Annex 7a and 7b of this report. These comments have been taken into consideration as communicated to the Government by the Bank team. (b) Cofinanciers The project had no cofinanciers. (c) Other Partners and Stakeholders N/A 21 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (US$millions) Appraisal (US$millions) Component 1 31.4000 31.4125 99.96% Component 2 3.6000 3.5000 97.22% Front-end Fee 0.0875 Total Project Costs 35.0000 35.0000 100.00% (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (US$millions) (US$millions) Borrower 59.39 41.17 69.00% International Bank for Reconstruction 35.00 35.00 100.00% and Development 22 Annex 2. Outputs by Component The following outputs (listed by component and subcomponent) were produced through the Barbados Second HIV/AIDS Project: Component/Sub- Outputs component Component 1: Prevention and Care Sub-component 1: Development of Program Guidance Documents National Program • 2009 Policy Document on Prevention of Mother to Child Coordination and Transmission in Barbados Institutional Strengthening • 2010 National HIV Prevention Plan • Policy on HIV Testing in Barbados September 2012 • 2012 National HIV Research Agenda • The Health Sector Response to HIV & STIs in Barbados: A Strategic Framework 2012-2015 • 2014-2018 National Strategic Plan for HIV 2014-2018: Investing for Results Maintained and expanded the multi-sectoral National AIDS Program • Management of multi-sectoral approach to housing People Living with HIV • PLHIV Housing Needs Assessment conducted annually • Development and routine updating of a priority housing list for PLHIV • Development and implementation of the Public Sector-Civil Society Grant System o 33 CSO projects funded o 25 CSO projects fully implemented o 7 population groups targeted representing projects targeting youth (15), persons with disabilities (2), men who have sex with men/sex workers (6), ex-offenders (1), PLHIV (1), people over 50 (1) and general population (2) • Representation of Lesbian, Gay, Bisexual, Transgender, Questioning and Intersex community on the Board of the National HIV/AIDS Commission • Trained at least fifty (50) strategic partners in HIV work planning and programme development • Over half of the line ministries submitted work plans in accordance with national guidelines Strengthened Research Capacity • 3 national knowledge, attitudes, beliefs and sexual practices surveys conducted – 2 among youth and 1 among adults • Implementation of the MSM Behavioral Surveillance Survey 23 Component/Sub- Outputs component • Conducted a Formative Assessment for the Female Sex Workers’ Behavioral Surveillance Survey • Conducted 2 national KAP surveys on HIV-related stigma and discrimination in relation to PLHIV, persons in same sex relationships and sex workers • Conducted key populations needs assessment • Developed and collected data for a LGBT Stigma and Discrimination register • Funds allocated annually for National AIDS Program activities • Attitudes to Homosexuals Survey (NHAC/CADRES) conducted • Biennial Research Symposium held and used to disseminate research to a broad cross-section of stakeholders Improved M&E Capacity • Development and implementation of the Monitoring and Evaluation Training Plan - an average of 123 trained annually in M&E • Greater involvement and participation in monitoring and evaluation by strategic partners • Released two (2) comprehensive HIV Surveillance Reports during the life of the Project • Completion of a multisectoral evaluation of the National AIDS Program (NAP) through Caribbean Public Health Agency (CARPHA) • Number of CSOs reporting on their activities increased from 2 pre- project to 32 during the project • 48 agencies submitted work plans with M&E components compared with 8 at project start • Number of strategic partners reporting on their HIV activities ranged from 19 to 43 over the life of the project • Evaluation of major HIV initiatives – Civil Society Grant System; Annual audits; audit of Grant System; NHAC BCC initiatives etc. Strengthening Primary Care through capacity building, acquisition of medical technology and equipment, supporting privacy and confidentiality, strengthening of specific clinical services and improving surveillance for specific disease conditions. • Training of dental staff from all polyclinics in Infection Control, Oral Radiology, Medico-Legal and Ethical Aspects of Record Keeping, and Medical Emergencies in the Dental Setting conducted • Training of Public Health Nurses from all polyclinics in family planning conducted. • Participation of 2 dental health staff members in a mini-residency in development dentistry in Kentucky, USA completed • Training in Mass Casualty Management for multiple stakeholders from government agencies, the private sector and civil society conducted 24 Component/Sub- Outputs component • Training in Disaster Management (chemical, biological, radiological and nuclear hazards) for multiply stakeholders from government agencies conducted • Participation and training of doctors, nurses, pharmacists and medical records clerks in the Howard Preceptorship Training Program completed • Orientation of doctors from polyclinics to Ladymeade Reference Unit (LRU) completed • Training of doctors and nurses from all polyclinics and the Queen Elizabeth Hospital (QEH) in Urgent Care, Triaging and the Canadian Triage Acuity Score conducted • Training of doctors and nurses from all polyclinic and LRU in Adolescence Health, Chronic Disease Nephology, Pediatrics, Ophthalmology as well as Obstetrics and Gynecology conducted • Training of doctors from all polyclinics, LRU as well as the Psychiatric, Geriatric and District Hospitals in Radiology and Medical Imaging conducted • Training of Environmental Health Officers as data collectors for the Geographical Information System conducted • Training of doctors and nurses from QEH in the Safe and Effective use of Medical Equipment conducted • Equipment, devices, tools and instruments for polyclinics, LRU, Public Health Laboratory and the Medical Intensive care Unit at QEH, procured and commissioned • Procurement of 30 Samsung Galaxy 7 Plus for the Geographical Information System completed • Consultancy on strengthening adolescence health services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted • Consultancy on strengthening obstetrics and gynecology services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted • Consultancy on strengthening ophthalmology services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted • Consultancy on strengthening chronic disease nephology services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted • Consultancy on strengthening pediatric services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted 25 Component/Sub- Outputs component • Consultancy on strengthening radiology and medical imaging services in primary health care (situation analysis, needs assessment, policy review and formulation, clinical guidelines as well as monitoring and evaluation) conducted • Physical improvements as well as preventive and corrective maintenance (providing adequate space, privacy barriers, infection control, power/electrical upgrade and waste management) for institutional strengthening at the Maurice Byer and Randal Phillips Polyclinics for Decentralization of HIV Services completed • Geographical Information Systems software upgraded • Database Development (Design, data acquisition, capture and conversion, QA/QC) completed • Spatial Data Analysis & Map Creation completed • 50 cm Resolution orthorectified Geo Eye 1 Satellite Imagery completed • Ground Control points for orthorectification to enhance accuracy completed • Consultancy for total implementation of system customization of software (form development and system deployment) completed Sub-component 2: Increased BCC Programming Scaling up Prevention • Development of Behaviour Change Communication (BCC) Training Efforts Program – (1) Social BCC Manual and workshops and (2) HIV Intervention for Development Manual and workshops with at least 2 training workshops conducted annually • 5 agencies with active BCC programmes – NHAC, Youth Department, International Business Division, Jabez House and Barbados Defense Force • Partnered with EQUALS (Lesbian, Gay, Bisexual and Transgender (LGBT) CSO) to implement MPowerment DEBI • Substantial increase in targeted interventions from 2 pre-project (2006) to an annual average of 30 realized over the life of the project • Number of CSOs working with key populations increased from 2 pre-project (CARE, UGLAAB) to 38 during the Project. • Distributed 2.99 million condoms from April 2008 to November 2014 Behavior Change A combined analysis of data from three (3) KABP surveys conducted in 2009, 2011 and 2013 revealed: • Sustained high levels of HIV-related knowledge rising from 96.7% in 2009 to 100% in 2013 among youth ages 15-24 • Reduction in stigma and discrimination with there being increases in the proportion of 15-24 year olds (1) being willing to care for a HIV+ family member (76.7% in 2009 to 85.8% in 2013); (2) supporting the provision of HIV to HIV+ students (78.9% in 2009 to 85.9% in 2013) and teachers (76.2% in 2009 to 86.0% in 2013) 26 Component/Sub- Outputs component • Decline in proportion of 15-24 year olds having sex before the age of 15 from 20.1% in 2009 to 12.1% in 2013 • Increased condom use among youth (15-24) engaging in high-risk sex: (1) condom use among person with 2+ partners rose from 52.4% in 2009 to 72.6% in 2013; (2) condom use at last sex with non- regular partners 80.0% in 2009 to 86.2% in 2013; (3) consistent condom use with non-regular partners rose from 48.6% in 2009 to 54.5% in 2013; (4) proportion of youth having unprotected sex with a higher risk partner i.e. non-regular and commercial partners declined from 13.9% in 2009 to 5.1% in 2013 • Reduction in the number of non-regular partners among youth 15- 24: 29.4% in 2009 to 12.8/% in 2013 Sub-component 3: • Implementation of a strong Prevention of Mother to Child Improving Diagnosis, Transmission (PMTCT) Program that has reduced rates of mother to Treatment & Care child transmission in keeping with international best practices • Expansion of the HIV Testing and Counselling Program including the introduction of Provider Initiated Testing and Counselling (PITC) at all Polyclinics as well as the introduction of rapid HIV testing at two (2) polyclinic sites, the Ladymeade Reference Unit and select community testing events • Achievement of Millennium Development Goal 6a, to halt and reverse the spread of HIV as evidenced by a decline in new HIV infections in Barbados • Decline in AIDS cases and mortality rates • Achievement of the targets set for the PAHO Elimination Initiative as evidenced by a mother to child HIV transmission rate of less than 2% and no reported cases of congenital syphilis for more than 4 years • Initiation of decentralization of HIV treatment services at two (2) pilot sites • Provision of ART to over 80.0% to persons with advanced HIV infection Component 2. • Development and implementation of a National Health Information Institutional System Strengthening o Completion of data dictionary for the National Health Information System o Consultancy to assess and support infrastructural requirements for the National Health Information System completed o Review of the current legislation and conduct of gap analysis relating to the Health Information Management completed o Completion of a draft Health Information Systems Policy for Barbados • GIS Consultancy & Project Management to manage project activities, ensure GIS Policy and standards, and provide relevant hand holding completed • Institutional strengthening of the Barbados Community College • Development and implementation of the Virtual HIV Research Unit 27 Component/Sub- Outputs component • Training of Ministry of Health staff in Clinical HIV Management through the Howard Preceptorship Program • Training in Leadership and Strategic Communication by Johns Hopkins • Study Tours completed; - Observation of (1) Behavior Change Communication Program in Jamaica, and (2) Brazil National HIV Program Prepared 2015-01-23 (updated 2015-01-30) 28 Annex 3. Economic and Financial Analysis 1. Total financing for the Second HIV/AIDS Project was originally estimated at US$94.39 million, of which US$35 million was Bank financed. At the end of the Project in November 2014, the full US$35 million financed by the Bank had been disbursed, with some of the amount used to cover up the shortfall in government financing experienced as a result of the financial crisis. The final government contribution was $41.17 million, about 30% less than originally planned. Total Project expenditures were 20% lower than originally planned. 2. The Project had four PDOs and two components. The PDOs focused on supporting the implementation of the National Strategic Plan for HIV Prevention and Control 2008-2013, and aimed at increasing: • Adoption of safe behaviors, in particular amongst key populations at higher risk. • Access to prevention, treatment and social care, in particular for key populations at higher risk. • Capacity of organizational and institutional structures that govern the NAP. • Use of quality data for problem identification, strategy definition and measuring results. 3. The first component, which received slightly over $30 million of the $35 million budget, focused on HIV/AIDS prevention, treatment, and care delivered through the healthcare system, while the remaining budget of $3.6 million was spent on the second component which focused on institutional strengthening. While it is commonly recommended when evaluating project benefits that cost-benefit analyses use with- and without-project scenario when calculating key HIV indicators. Doing so in the context of the without-project scenario is difficult as it is impossible to measure the number of people who may have contracted the disease had the Project not been implemented or if other efforts had been put into place. In addition, the small population size of Barbados means that HIV incidence data is difficult to collect among key populations of interest. Reported surveillance data in these populations may not be representative, and as a result, may not reflect the true number of cases. Further, such an evaluation may underestimate the global and regional public good nature of the Project, which was a theme consistently emphasized during Project preparation. 4. At the time of Project preparation, there were concerns about a growing epidemic in the Caribbean, which had a prevalence rate of 1.5% in 2005. The MTR of the First Bank-financed Project indicated that Barbados may see the number of PLHIV grow to 2,900 in 2010 and 4,000 in 2015. With the adoption of generic compounds that were available on the market for patients initiating antiretroviral treatment, the annual cost of drugs may have increased from US$2 million in 2007 to US$8 million in 2015. 5. Comparatively, the Global AIDS Response Progress Report 2012 reported the number of PLHIV at the end of 2010 at 1,918, approximately 1,000 less than anticipated at the time of the MTR of the First Project which was cited in the PAD. In addition, a report on the Economic Impact Analysis of the HIV Epidemic in Barbados for 1984-2008, completed in 2013 and submitted to the Ministry of Health, reports that HIV prevalence rate in the general population has been under 29 1% since 1999. However, projections made for 2009-2015 indicate an increase in prevalence above 1% in the population aged 15-49 years old. 6. Figure 1 shows the annual number of reported HIV and AIDS cases and deaths among PLHIV prior to the implementation of the Second HIV/AIDS Project (1984-2008). The number of HIV and AIDS cases remained high, though mortality among PLHIV appeared to be decreasing. Data in Figure 2 shows the first four years of the Project (2009-2012) – a decrease is seen particularly in HIV and AIDS cases in the first year, and this decrease seems to have been maintained until 2012. Incidence data on 2013 and 2014 was not available at the time of this ICR. Figure 1. Number of Annually Reported Cases of HIV, AIDS, and AIDS-related Deaths, Barbados 1984-2008 200 180 160 140 120 100 80 60 40 20 0 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 HIV cases AIDS cases Deaths among PLHIV Source: Global AIDS Response. Country Progress Report, Barbados. 2014. Figure 2. Number of Annually Reported Cases of HIV, AIDS, and AIDS-related Deaths, Barbados 2008-2012 180 160 140 120 100 80 60 40 20 0 2008 2009 2010 2011 2012 HIV cases AIDS cases Deaths among PLHIV Source: Global AIDS Response. Country Progress Report, Barbados. 2014. 30 7. Cost-effectiveness calculations for the direct benefits of the Project are provided in the following sections. Where data was not available, simulations were made. The assumptions and simulations are as follows: • Population figures until 2012 come from the World Bank’s World Development Indicators • HIV case data until 2012 for the with-project scenario comes from the Global AIDS Response, Country Progress Report Barbados 2014; data for 2013 and 2014 are linearly projected • HIV case data for the without-project scenario are linearly extrapolated for 2009-2014 using HIV rates from 2000 onward • An averted infection provides an individual with thirty-four more years of productive life which is simply the pension age (66) minus average age of HIV infection – latest available data from Global AIDS Response Progress Report 2014 for 2012 suggests that the median age is 32 • Average working individual productivity is US$15,116 per year using GDP per capita for 2008 as a proxy for productivity (in 2000 constant USD) • An infected patient will live for 10 years 11 after infection with adequate care • The average annual cost of care per patient after infection with adequate care is assumed at US$7,510 (based on a cost study cited in the PAD), and includes pharmaceuticals, labor, infrastructure, overhead and other materials • Benefits accrue five years after the infection has been averted for the disease to progress and translate into productivity losses and health care costs for the individual 8. The analysis below is for the entire population as age-specific incidence rates are not available for multiple years. Table 1 shows the estimated number of new infections in the context of a without-project scenario. Based on these figures, we see that the incidence rate would have been 0.55-0.66 per 1000 from 2009-2014 in the absence of the Project. 11 Exact figures for Barbados are not available, but figures from one study (The ART Cohort Collaboration. The Antiretroviral Cohort Collaboration. Life expectancy of individuals on combination therapy in high-income countries: a collaborative analysis of 14 cohort studies. The Lancet 372: 293 – 299, 2008) suggests that the average number of years remaining to be lived at age 20 years was about two-thirds of that in the general population. Life expectancy for Barbados at the time of Project initiation was approximately 75 years, meaning that those receiving treatment at age 20 could expect to live for 30 years. However, we use the conservative figure of 10 years here to account for the fact that ~55% of those diagnosed in 2012 (latest estimates) are diagnosed with low CD4 counts (<350 cells/mm3). 31 Table 1. Without-Project Scenario 2008-2014 12 Year Population Incidence Number Rate per newly 1000 infected with HIV 2008 277,634 0.56 156 2009 279,006 0.65 172 2010 280,396 0.66 168 2011 281,804 0.60 166 2012 283,221 0.56 165 2013 277,618 0.55 164 2014 278,948 0.64 164 9. With-project scenario indicators are listed in Table 2. With the project, we see that the incidence rate per 1000 is 0.50 or lower. Thus, the incidence rate observed is reduced by about a third in the with-project scenario. Table 2. With-Project Scenario 2008-2014 13 Year Population Incidence Number Rate per newly 1000 infected with HIV 2008 277,634 0.56 156 2009 279,006 0.40 112 2010 280,396 0.48 134 2011 281,804 0.47 132 2012 283,221 0.49 138 2013 277,618 0.50 138 2014 278,948 0.49 136 10. Averted HIV infections and years of life saved are summarized in Table 3. This is the difference between the newly infected cases in the with- and without-project scenarios. Years of life saved is the number of infections averted multiplied by the increased productive life caused by the Project, which is assumed to be 34 years. Project benefits are calculated based on infections averted. Benefits include averted productivity losses as well as the cost of treatment of HIV/AIDS and related illnesses, assumed to be US$7,510 per patient per year. 12 Incidence rates and the number of newly infected for 2009-2014 are projections; population size for 2013 and 2014 were also projected. 13 The number of newly infected for 2009-2012 are based on actual figures (Global AIDS Response, Country Progress Report Barbados 2014). Incidence rates and the number of newly infected for 2013-2014 are projections; population size for 2013 and 2014 were also projected. 32 Table 3. Project Cost and Benefits Years Productivity Averted cost Infections of life losses averted of care 14 Total benefits Year Averted saved (US$) (US$) (US$) 2008 0.00 0 0 0 0 2009 59.69 2,029 0 0 0 2010 34.05 1,158 0 0 0 2011 34.11 1,160 0 0 0 2012 27.02 919 0 0 0 2013 26.78 910 0 0 0 2014 27.90 949 30,676,746 4,482,636 35,159,382 2015 0.00 0 17,497,353 2,556,798 20,054,152 2016 0.00 0 17,532,183 2,561,888 20,094,070 2017 0.00 0 13,888,332 2,029,431 15,917,762 2018 0.00 0 13,761,490 2,010,896 15,772,387 2019 0.00 0 14,338,860 2,095,264 16,434,124 Total 210 5,658 107,694,964 15,736,913 123,431,877 11. Annual costs in Table 4 were based on total Project disbursement. While it may have been preferable to focus on the impact of specific components, attribution of results by component is difficult to make, and would likely have resulted in double counting. Table 4. Project Stream of Cost and Benefit Year Costs Benefits Net Benefits 2008 0 0 0 2009 3,807,500 0 (3,807,500) 2010 2,440,000 0 (2,440,000) 2011 4,848,955 0 (4,848,955) 2012 3,690,000 0 (3,690,000) 2013 11,460,000 0 (11,460,000) 2014 8,753,545 35,159,382 26,405,837 2015 0 20,054,152 20,054,152 2016 0 20,094,070 20,094,070 2017 0 15,917,762 15,917,762 2018 0 15,772,387 15,772,387 2019 0 16,434,124 16,434,124 Note: Costs based on Project disbursement record 12. Figures listed in Table 3 only include infections averted until 2014, and do not take indirect benefits into consideration. The economic and financial analysis in the PAD estimate that 283 cases of AIDS would be prevented by 2014; here the figure is 210 using more recent data, where available. Further, the total Project expenditures ended up 20% lower than originally planned, thus a 25% reduction in the number of prevented cases can be considered acceptable. It is important to 14 Total averted cost of care is calculated to be US$8,494,999 if the cost of treatment per patient per year is calculated at US$4,054, the most recent figure available from the GOB. This results in a reduction in total benefits of the Project to US$116,189,964. 33 also note that total benefits are likely to be an underestimate given that averted cost of care is calculated based on average life expectancy of 10 years after diagnosis.11 13. An Internal Rate of Return (IRR) of 38.8% was calculated for the Project. The net present value (NPV) focusing on HIV infections averted is US$31.28 million assuming a 10% discount rate. Cost-effectiveness of the Project is based on the dollar cost of each infection averted and each year of life saved. Calculations using the data above find that the dollar cost for each HIV infection averted is US$167,027.68 and US$4,912,58 for each year of life saved. Table 5. Key Project Cost and Benefit Indicator Indicator Value Description IRR 38.8% Internal rate of return Net present value in US 2008 NPV $31.28 million dollars Cost-effectiveness Dollar cost of each infection calculation 1 $167,027.68 averted Cost-effectiveness Dollar cost of each year of life calculation 2 $4,912.58 saved 14. The NPV increases as the average life expectancy after diagnosis increases, and decreases as the cost of treatment declines. Assuming a 20 year life expectancy, the NPV increases to $38.01 million. When a 20 year life expectancy is assumed and the average cost of treatment/person/year is reduced to $4,054, the NPV is US$31.81 million. 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Joana Godinho Practice Manager GHNDR Task Team Leader Rolande Simone Pryce Senior Operations Officer GEEDR Lawyer Sr Financial Management LCSFM - Financial Emmanuel N. Njomo Specialist HIS Management Senior Environmental Environmental Gunars H. Platais GENDR Economist Specialist Reynaldo F. Pastor Chief Counsel LEGLE Lawyer Maria Lourdes Noel Senior Program Assistant GENDR Administrative HDNGA - Monitoring & Marcelo H. Castrillo Consultant HIS Evaluation Judith C. Morroy Consultant GHNDR Procurement Supervision / ICR Joana Godinho Practice Manager GHNDR Task Team Leader Shiyan Chao Senior Economist (Health) GHNDR Task Team Leader Carmen Carpio Senior Operations Officer GHNDR Task Team Leader Neesha Harnam Young Professional GHNDR ICR Author Robert Oelrichs Senior Public Health Specialist GHNDR Public Health Rianna Mohammed Senior Health Specialist GHNDR Health Systems Elizabeth Mziray Operations Officer GHNDR Health Systems Anna Wielogorska Senior Procurement Specialist GGODR Procurement IT Yingwei Wu Senior Procurement Specialist GGODR Procurement Judith C. Morroy Procurement Specialist GGODR Procurement Senior Financial Management Financial Mozammal Hoque GGODR Specialist Management Tatiana Cristina de Abreu Finance Officer WFALN Disbursements Victor Ordonez Senior Finance Officer WFALN Disbursements Julius Martin Thaler Senior Counsel LEGEN Lawyer Alessandro Legrotaglie Senior Country Officer LCC3C Country Relations Rolande Simone Pryce Senior Country Officer LCC3C Country Relations Elyssa Finkel Consultant GHNDR Health Systems Stefan Baral Consultant GHNDR HIV Thomas Novotny Consultant GHNDR Public Health / HIV Aristides Barbosa Consultant GHNDR HIV Viviana Gonzalez Program Assistant GHNDR Operations 35 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$ Thousands (including No. of staff weeks travel and consultant costs) Lending FY08 17.74 136.79 FY09 6.95 66.54 Total: 24.69 203.33 Supervision/ICR FY10 7.83 52.74 FY11 16.62 98.15 FY12 26.24 127.45 FY13 27.48 173.59 FY14 24.43 131.87 FY15 21.28 115.31 Total: 123.88 699.11 36 Annex 5. Beneficiary Survey Results Not applicable for Core ICRs. 37 Annex 6. Stakeholder Workshop Report and Results Not applicable for Core ICRs. 38 Annex 7a. Summary of Borrower's ICR 1. Background For the period 2008 to 2013, the aim of Barbados’ National AIDS Programme (NAP) was the mitigation of the social and economic impact of HIV and AIDS on the population. This goal was outlined in its National Strategic Plan for HIV Prevention and Control, 2008 – 2013, and the associated programmes and activities were estimated to cost USD $94.39 million (BDS$188.78 million)over the five-year period. The implementation of the NAP required multi-sectoral involvement and the National HIV/AIDS Commission (NHAC) was identified as the coordinating body. In 2008, the Government of Barbados negotiated a sector investment loan of USD $35.0 million (BDS $70 million) from the World Bank to execute the Second HIV/AIDS Project which would finance a percentage of the eligible expenditures under the NAP and provide technical assistance and training to support the implementation of the Programme. 2. Project Description This was the second Project with the World Bank and it sought to build on the successes in access to HIV testing, treatment and care which had been accomplished under the first Barbados HIV/AIDS Project (2001 – 2007). Specific priorities included strengthening HIV prevention programming and introducing behavior change communication. The second Project commenced on September 25, 2008 (date of loan signature) and was originally scheduled to close on November 30, 2013. (i) Project Objectives The Project aimed specifically to increase: • Adoption of safe behaviors, in particular amongst key populations at higher risk; • Access to prevention, treatment and social care, in particular for key populations at higher risk; • Capacity of organizational and institutional structures that govern the NAP; and • Use of quality data for problem identification, strategy definition and measuring results. (ii) Project Components The project comprised two (2) components as follows: Component 1 – Prevention and Care (USD $31.412 million/BDS$62.824 million) Loan funding under this component was pooled with government funding to finance national programme activities. In its entirety, it was estimated to cost USD $89.65 million (BDS $179.30 million) to fund the national programme over the period of the Project. The Project loan would finance USD $31.412 million (BDS $62.824 million) of this category of expenditure. The Eligible Expenditure Programmes were as follows: 39 • National Programme Coordination and Monitoring. This aimed to strengthen the ability of the public and private sectors and civil society to coordinate, monitor and evaluate their activities and use data to continually increase the quality of their programmes. The Public Sector – Civil Society Organization (CSO) Grant System was to be developed under the Project. • Scaling up Prevention Efforts. This aimed at increasing access to prevention services, particularly Behaviour Change Communication, HIV and Sexual Transmitted Infections (STIs) prevention and treatment, and condom marketing and distribution, with a special focus on key populations at higher risk. • Improving Diagnosis, Treatment and Care. This programme sought to increase the length and quality of life of PLHIV by increasing their access to diagnostic services, treatment services, as well as social care and support. Component 2 – Institutional Strengthening (USD $3.5 million /BDS $ 7.0 million) This component focused on institutional strengthening and loan funding was disbursed against approved contracts. The Project loan would finance USD $3.5 million (BDS $7.0 million) of this category of expenditure. Under the NAP, this component was estimated to cost USD $4.47 million(BDS $8.94 million) and aimed to strengthen agencies and CSOs through training and technical assistance in monitoring and evaluation (M&E), management, surveillance, prevention, diagnosis, as well as treatment and care of HIV and other STIs. 3. Project Restructuring The Project has been restructured twice throughout the five year Project period; July 19, 2011 and March 5, 2013. The first restructuring was done to allow the project funds to finance the development and implementation of a HIS, amend the performance indicators by modifying the wording of all nine performance indicators to make the targets easily identifiable, technically refining indicators to improve clarity of the definitions and revising the targets, and to allow for increased percentage reimbursement of Component 2 expenditures for goods and consultants’ services from 79% to 100%. The second restructuring was done to modify the institutional arrangements to strengthen leadership and capacity, amend four of the performance indicators by changing the final target year for three indicators from 2012 to 2013 and by changing the target for another, and to allow for an increase in percentage reimbursement of Category 1 Eligible Programme Expenditures (EPE) to be financed by the loan from 35% to 65%. 4. Extension of the Terminal Disbursement Date An extension of the terminal disbursement date for the Loan was granted by one year until November 30, 2014, to allow time to achieve the Project Development Objectives (PDOs) and complete agreed activities. 40 5. Assessment of the Project As the Project comes to an end it is important to determine if the project has achieved the project objectives, if the project design was appropriate to achieve the desired effect, if the implementation of the project was successful and to describe the operational experience. Three types of indicators were used for monitoring and evaluation: (i) project development indicators (input, process, and output) to measure progress towards achievement of the Project Development Objectives (PDO); (ii) project results indicators that would trigger disbursements; and (iii) project monitoring indicators to measure progress in project implementation. These were considered in the Government’s assessment of the Project. (i) Project Objectives Below is a table showing the Project Development Indicators which were used to monitor the achievement of the PDOs, the results which were achieved and comments. Project Development Indicators Results/Comments & Targets Behavior Change Maintain the percentage of young The biennial survey conducted by the Division of Youth people 15-24 years indicating Affairs (2009, 2011) and NHAC (2013) revealed 92.3%, sexual relations as a way of 96.7% and 100% respectively. This data reveals that transmitting HIV at least at 90% knowledge levels have remained consistently high over from 2008 to 2013. the Project period 2009-2014. Increase the percentage of sex Using data from targeted testing initiatives being workers who report the use of a conducted at hotspots, consistent condom use (always) condom with their most recent among sex workers ranges from 9.8% to 40.7% for the client from 80% in 2008 to 95% period 2009-2014. However those using condoms at any in 2013. time, ranges from 78.3% to 91.7% for the same period. Increase the percentage of MSM Two surveys were conducted in 2014, one among the who report the use of a condom general population which revealed condom use among the last time they had sex from MSM to be 80% and another conducted among MSM 64% in 2008 to 75% in 2013. which revealed condom use in this population to be 58%. It is recognized in the former the number of respondents was small, and it is noted that additional work is required with this key population. Increase the percentage of young The biennial survey conducted by the Division of Youth people 15 – 24 years reporting the Affairs (2009, 2011) and NHAC (2013) revealed 50.5%, use of a condom the last time they 64.5% and 72.3% respectively. This data reveals that had sex with a non-marital, non- there has been a steady increase in the use of condoms cohabitating partner from 21% in over the Project period 2009-2014. 2008 to 60% in 2013. Access to prevention, treatment and social care Increase in the number of people Information provided by the Caribbean HIV &AIDS from key populations at higher Alliance based on its outreach programme in Barbados 41 Project Development Indicators Results/Comments & Targets risk accessing preventive has shown that with regard to the MSM population, the services. (MSM). number of persons provided with prevention services during the period 2009-2013 varied from as low as 3.5% to 55.6%. Additional information has been provided by the recently completed BSS MSM which indicates 27.8% of the study population accessed preventive services in the last 12 months. Increase in the number of people Information provided by the Caribbean HIV &AIDS from key populations at higher Alliance based on its outreach programme in Barbados risk accessing preventive services has shown that with regard to the FSW population, the (FSW). number of persons provided with prevention services during the period 2009-2013 varied from as low as 16.5% to 54%. Maintain the percentage of HIV- With regard to HIV infected pregnant women in Barbados positive pregnant women accessing antenatal care and receiving ARVs during the receiving a complete course of five year period 2009-2013, the country achieved 100% ARV prophylaxis to reduce the coverage for two years. For the other three years in the risk of mother to child period coverage ranged between 87.5% and 90.9%. transmission (MTCT) above 95% in the period 2008 to 2013. Maintain the percentage of Over the period from 2009-2014 the percentage ranged PLHIV on ART achieving from a high of 88.6% to a low of 79.5% which exceeds virologic success in the last 12 the 70% rate. months above 70% in the period from 2008 to 2013. NAP capacity Increase in the funds spent by The overall target for CSO grant spend was USD$220,000 CSOs under results-based (BDS $440, 000). From the inception of the CSO Grant agreements with the NHAC to Scheme in September 2011, to the end of fiscal year facilitate the implementation of 2013/2014 the total spend has been USD$ 283,548 (BDS Programme interventions for key $567, 096). populations at higher risk in the period 2008 to 2013. Monitoring and Evaluation An evidence-based Strategic Plan NSP 2014-2018 has been completed for the period 2013 – 2018 prepared before project closing, taking into account surveillance and M&E data available under the project. 42 (ii) Project Design The project design could have focused more on specific definitions to avoid ambiguity. For example the Project Development Objectives were broad and left open to interpretation making it difficult to determine which expenditures were eligible for reimbursement and which were not. As a result of the original loan structure, the interpretation of the expenditure eligible for reimbursement, and the low reimbursement percentage (35% for Component 1 and 79% for Component 2) it would not have been possible to have loan funds fully disbursed by the original terminal disbursement date of November 30, 2013. It was therefore necessary to restructure the Loan twice to meet required expenditure. The original scope of the Project was overly ambitious and an assessment revealed that it would not have been possible to implement all of the activities in a five year period. For example the original Project included a National HIV Seroprevalence Study, for which USD$1.123 million (BDS $2.246 million) was earmarked. This study was later found not to be feasible based on the following: • Barbados’ epidemic did not meet the criteria for such a study (i.e. HIV prevalence <5%) • The sample size required was large and logistically impossible (≈50, 000 persons) Projections of the number of PLHIV were over inflated and this resulted in an overestimation of funds required for the purchase of ARVs for the duration of the Project. The funds were subsequently reallocated. (iii) Project Implementation & Operational Experience Project Implementation was hindered because of the governance structure. A mixed model existed, preventing rationale use of resources and reducing cost efficiency. The Coordinating Unit was also not adequately staffed to fulfil its function. 6. Project Achievements of the Government of Barbados A major achievement of the Project was its ability to draw down on the full amount of the loan (US$35 Million, BD$70 Million) despite low spend in the early stages of the Project and the prevailing economic environment throughout the Project. Following is a summary of achievements over the life of the Project. Behavior Change (i) Successful completion of the 2011 survey on knowledge, attitudes, beliefs and practices of the general population relating to Stigma & Discrimination. Data collected showed a decrease in the Stigma and Discrimination towards key populations. (ii) Successful completion of 2013-2014 of Adult Knowledge, Attitudes, Beliefs and Practices Survey. Data collected was used to report on the indicators (see assessment of project objectives above) and identified programme gaps. 43 (iii) The Behavioral Surveillance Survey among MSM provided valuable behavioral and biological baseline information on this hard to reach population. The information acquired will inform planning for prevention and care services. (iv) International Best Practice has been achieved with the successful development and implementation of a Behavior Change Communication Programme in the Barbados Defense Force. (v) An effective collaborative effort by stakeholders to produce and implement the 2010 National HIV Prevention Plan. (vi) The future work in Barbados to address issues and sustain the National HIV Programme will have effective guidelines, policies and procedures which have been articulated in the 2014-2018 National Strategic Plan for HIV 2014-2018: Investing for Results. Access to Prevention, Treatment and Social Care (i) Successful development & implementation of stigma and discrimination reduction campaigns. (ii) Increase in anti-stigma and discrimination programming. (iii) Implementation of a strong Prevention of Mother to Child Transmission (PMTCT) Programme that has reduced rates of mother to child transmission in keeping with international best practices. (iv) Expansion of the HIV Testing and Counselling Programme including the introduction of Provider Initiated Testing and Counselling (PITC) at all Polyclinics as well as the introduction of rapid HIV testing at two (2) polyclinic sites, the Ladymeade Reference Unit and select community testing events. (v) Accessed members of the MSM population who would not ordinarily receive HTC services, and increased access to care through the conduct of the Survey among MSM. (vi) Achievement of Millennium Development Goal 6a, to halt and reverse the spread of HIV as evidenced by a decline in new HIV infections in Barbados. (vii) Achievement of Millennium Development Goal 6b, universal access to treatment for HIV, with more than 80% antiretroviral treatment coverage throughout the life of the Project. (viii) Decline in AIDS cases (104 in 2008 to 78 in 2012) and mortality rates (2.4% in 2008 to 1.5 % in 2012). (ix) Achievement of the targets set for the PAHO Elimination Initiative as evidenced by a mother to child HIV transmission rate of less than 2% and no reported cases of congenital syphilis for more than 4 years. (x) Initiation of decentralization of HIV treatment services at two (2) pilot sites. (xi) Strengthening Support for People Living with HIV (multi-sectoral approach) a. Conduct of Housing Needs Assessment b. Prioritisation of PLHIV Housing Needs National AIDS Programme Capacity (i) Development and implementation of the Public Sector-Civil Society Grant System. (ii) Development of greater partnerships between government and civil society partners. (iii) Expanded relationships with key populations e.g. MSM and FSW. 44 (iv) Representation of Lesbian, Gay, Bisexual, Transgender, Questioning and Intersex community on the Board of the National HIV/AIDS Commission. Monitoring and Evaluation (i) Development and implementation of the Monitoring and Evaluation Training Plan. (ii) Greater involvement and participation in monitoring and evaluation by strategic partners. (iii) Release of two (2) comprehensive HIV Surveillance Reports during the life of the Project. 7. Performance of the World Bank It is the opinion of the Government of Barbados that the performance of the World Bank was as follows: (i) Performance during project preparation Bank personnel were readily available for consultation as required. (ii) Performance during project implementation • The implementation support was generally good. • The online platform required to be used for the upload and approval of withdrawal applications (Client Connection) was not user friendly and technical support was limited. • The switch of Financial Management from Washington DC to Brazil impacted negatively on Barbados’ ability to have financial matters concluded. Difficulties were also experienced as a result of a language barrier. • There was a lack of appreciation of the cultural context in which one could successfully implement programmes for MSM and FSWs in Barbados. Numerous concerns were also expressed about the attempt to bring ‘wholesale’ to Barbados methodologies used in other cultures without any attempt to make the required adjustments. • There was a lack of clarity with regard to the procedures which the Government of Barbados should follow in executing contracts as it related to procurement under Component 2. Procedures for seeking reimbursement under Component 2 were also unclear. Requests for direction from the World Bank on these matters received varied responses, during the life of the Project. 8. Challenges The amounts indicated in the documents of the World Bank with regards to the Loan for inclusion of the Estimates of Expenditure for Barbados for each Financial Year were too high and could not be attained by the Government of Barbados. It was therefore necessary to include smaller projections for each Financial Year. There was a delayed start to Project implementation, although approved in August 2008 the Loan did not become effective until January 2009. In addition, the process of the first restructuring was somewhat protracted which hampered implementation of some activities. 45 Originally, there were too many indicators defined for the loan (45). These indicators were not proportionally linked to priorities for interventions and were not appropriate to measure the impact of the Project. Although 70% of the Loan was the responsibility of the Ministry of Health, the parent Ministry with responsibility for the Project was not the Ministry of Health. Poor documentation of expenditures resulted at the outset in an inability to seek reimbursement from the World Bank. This was rectified in the latter stages of the Project when reimbursement for prior year expenditures was sought. 9. Conclusion (i) Lessons Learnt and Recommendations • The planning period with regard to loans of this nature should be extended to ensure that any hindrances to completion of conditions prior to first disbursement are addressed before loan signature. • Careful evaluation should be undertaken to ensure that the Project is manageable given the resources of Barbados. • Extensive review of proposed indicators should be undertaken including giving stakeholders an opportunity to comment and taking the cultural context into consideration before they are approved. • There should be careful evaluation of the financial requirements for inclusion in the Estimates of Expenditure over the life of the loan to satisfy the requirements in the Loan Agreement. (ii) Sustainability • The Government of Barbados established a National AIDS Programme in advance of its relationship with the World Bank, which subsequently resulted in the establishment of the NHAC. • Having successfully implemented two loans funded by the World Bank, Barbados intends to build on the successful work which has been done by continuing the implementation of all activities falling under the National AIDS Programme guided by the National Strategic Plan 2014-2018. 46 Annex 7b. Comments on Draft ICR Comments by the Government of Barbados DRAFT IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-75850) 1. Page v ICR Data Sheet B. Key Dates: The Appraisal date given in the document as 02/26/2008 is not the same that in the PAD 02/28/2008. 2. Page v: Under the heading Implementing Agencies, the official name of the Social Care Ministry is incorrect. Replace Constituent with Constituency and insert Development after Community. 3. Page vi, Section C.3 – The innovative nature of this Project design should have flagged that there could have been potential challenges. 4. Section 1.1, paragraph 2. The Bank was the only anticipated source of external funding for the NAP at the time of Project preparation, however, within the first year of Project implementation PEPFAR funding became available within the Region and in Barbados. PEPFAR supported key population behavioural surveys, provided laboratory equipment, provided additional staff support, improved HIV case based surveillance and supported the initiation of rapid testing. 5. Page 4: Under Component 1 Sub-Component 2: Scaling up Prevention Efforts insert HIV & STI Prevention and treatment after provision of condoms 6. Section 2.1, paragraph 13: The environment within which the Project was designed is not adequately considered here. We believe that the background analysis was flawed as it was based on information which was not context specific. Furthermore, assumptions were made with respect to the key populations at higher risk based on international evidence. These assumptions were not necessarily translatable to the Barbados situation. 7. Section 2.1, paragraph 15: The general tone of this paragraph is misleading as the PAD made no reference to the need for a standalone or discrete Project Implementation Unit. 8. Section 2.1, paragraph 15: The quote “there needed to be a piece on transition management from the First to the Second Project” should be removed. A more accurate statement might be “greater consideration could have been given to the changes in the size and scope of the Second Project”. A general statement is preferable to a direct quote. Any direct quotations should be excluded from the document. 9. Section 2.1, paragraph 15: Inconsistencies in the Project documents concerning expenditures eligible for reimbursement contributed to confusion throughout the life of the Project. To acquire certainty the GOB sought a formal Legal Opinion on the interpretation of eligible expenditure. That Opinion is attached for ease of reference. 47 10. Section 2.1, paragraph 16: The challenges referred to this paragraph were not solely due to the implied inability of the local project team to understanding and executing reimbursable expenditures but also to the 1) provision of conflicting (i.e. ambiguous) information and poor explanations from the World Bank team; 2) flaws in the project design relating to project scope and unrealistic activities; and 3) ambiguous information within the principal project documents. 11. Section 2.1, paragraph 18: The rating process used seems inconsistent given that a rating of “low” was assigned in relation to the assessment of risks. Although the transparent fiduciary environment in Barbados is referred to in the PAD as a benefit, the PAD also acknowledged that the economy was vulnerable to external shocks particularly given high debt levels. Despite this the Project was given a low risk rating which in retrospect was a mistake. Economic impact should have been given more consideration in light of the economic climate at the time. 12. Section 2.1, paragraph 18: This section is misleading – “the mitigation measures highlighted that the Bank would provide funding, technical assistance, and training for planning and M&E purposes. However, in at least one instance, training was provided only at the later stages of the Project when it should have been provided much earlier”. The mitigation measures should be stated clearly. It is not clear which training is referred to here. Is it training on M&E? It should be noted that little training was provided for M&E purposes. Also the last sentence – “However, in at least one instance, … earlier” - should be deleted as it adds no value to the paragraph. 13. Section 2.2, paragraph 19. Although the multisectoral framework necessary to implement the Project existed at Project start, the same capacity utilized for the first project was erroneously expected to be sufficient for the Second Project which was of much larger size and scope and with a more complex design. This paragraph therefore ignores the fact that the institutional assessment conducted by the Bank failed to factor in the complexity of the Second Project relative to the First Project and failed to highlight the need for additional support and resources. Please delete the sentence – 14. “Finally, as noted by the PAD, despite the existence of a multisectoral framework to implement the National HIV/AIDS Project, the reality was quite different on the ground.” 15. Section 2.2, paragraph 19: Reference, without explanation, is made in this paragraph to a “lack of capacity” and “lack of technical expertise among key staff”. This needs to be addressed. 16. Section 2.2, paragraph 20: The statement – “The Second Restructuring resulted in a turning point for the Project which had previously been facing challenges and which was rated Moderately Satisfactory for PDO and Moderately Unsatisfactory for IP after the MTR” requires evidence to support this view. The move to address the 48 underutilization of funds started before the mid-term review and was initiated by the NHAC with assistance solicited from the Public Investment Unit and the Director of Finance and Economic Affairs. 17. Section 2.2, paragraph 20 a: This information is misleading given the role of a NHAC chair – “The possibility of the chair of the National HIV/AIDS Commission, who led the team in the implementation of the First Barbados HIV/AIDS Project, stepping down prior to the Second Project commencing was raised as a concern. This concern was heightened considering a successor would be new to the role, not carrying the mandate from the Prime Minister’s office, be tasked with implementing cross- Ministry coordination and reporting without the previous mandate, and implementing a new, larger-scale project with a more elaborate design.” The ICR should speak to the fact that the shift of the NHAC from the Prime Minister’s office resulted in a noticeable lack of visibility and authority which Project implementation. Past and present chairmen of the NHAC have never run the HIV Projects. Furthermore, the Chairman referred to in the aforementioned text was Special Envoy for HIV, at no time did this individual assume Project managerial responsibility. The section starting from “The possibility … new to the role”, should be deleted. 18. Section 2.2, paragraph 20 a: With respect to the sentence, “In addition, the NHAC had insufficient capacity in the beginning. Staff with certain areas of technical expertise was needed at the NHAC, and the MOH was theoretically able to provide this expertise, but ended up not doing so due to a lack of coordination between the two agencies as a result of poorly-defined roles at the beginning of the Project.”, it should be noted that the Procurement Officer MOH was not assigned at the Project start. Her post was later added at the Bank’s recommendation. Unfortunately the need for a dedicated Procurement Officer had not been recognized during Project preparation. 19. Section 2.2, paragraph 20 b: The explanation provided in this paragraph is fully supported by the Project team since it accurately captures and explains the basis of the implementation challenges. 20. Section 2.2, paragraph 21: The involvement of the Permanent Secretary (Investment) was at the behest of the NHAC through the Director of Finance and Economic Affairs and subsequently formalized by the Prime Minister. 21. Section 2.2, paragraph 21, Area 1: The sentence which reads “The technical expertise of the Project Director and those under her also meant that technical oversight could be provided to relevant MOH activities” is misleading and should be removed or reworded. The sentence currently implies that there was a lack of technical expertise at the MOH prior to the Project Director post. 22. Section 2.2, paragraph 21, Area 1: It is inaccurate to say that the Project Director was responsible for improving multisectoral coordination and clarifying the roles and responsibilities of different ministries. 49 23. Section 2.2, paragraph 21, Area 2: In the following sentence – “By the time of the MTR, training in M&E had begun to take place, and the momentum generated by the Permanent Secretary further highlighted the importance of these activities.”, please change “place” to “effect” which would reflect the provision of M&E training since Project start. 24. Section 2.3, paragraph 23: Since this paragraph refers to the catalytic impact on behaviour change, consideration should be given to raising the ICR rating to satisfactory. 25. Section 2.3, paragraphs 25 & 26: There is a disconnect between the information presented in para 25 and 26. Para 25 speaks to the challenges with the BSS and para 26 loosely addresses the Ministry of Health’s “M&E skills which would have overcome these difficulties”. It is not logical for the NHAC to tap into MH’s skills for a MH activity (BSS MSM). 26. Section 2.3, paragraph 25 requires clarification. The problem with tracking of key population indicators rested with the fact that comprehensive key population programming did not exist at Project outset. It was therefore very difficult to measure activities/ track indicators for programmes now being developed. In addition to this, as mentioned previously in the ICR there is a reluctance for persons to identify as MSM or FSW, which also hampers monitoring of access to services. 27. Section 2.3, paragraph 26: It is inaccurate to say that M&E capacity was increased. An alternative statement could be “there was strengthened capacity and improved M&E performance during the Project”. It should be noted that coordination of indicator data collection was being done from the start of the project. In the latter stages of the Project the M&E process was further streamlined. 28. Section 2.3, paragraph 27: The Project did not strengthen the NHAC Management Information System (MIS), which never materialized. The Project was supposed to develop a MIS for the NHAC but this was vetoed by the Bank on the premise that the NHAC could use the MH’s HIS which in one of its earliest iterations included a prevention component. As a result, these funds were diverted with World Bank approval to the development of the Virtual HIV Research Unit. 29. Section 2.3, paragraph 28: This sentence on the NHAC strengthening capacity to take over KABP administration is incorrect and misleading. First, the Ministry of Family assumed responsibility for the KABP because they were conducted among youth and the youth portfolio was the responsibility of MFYS. The NHAC assumed responsibility for the 2013 survey because 1) the focus was different, adults as well as youth, 2) a new methodology was being used and 3) as a quality control measure. In addition the core group working on the survey protocol and submissions included personnel from the NHAC and the Ministry of Health. Also the NHAC adopted the same approach to the Adult KABP as the MH to MSM BSS where there was outsourcing re manpower for the surveys while the financing was provided by the NHAC. 50 30. Section 2.3, paragraph 28: The final sentence relating to MSM and FSW epidemiological data needs to be in a separate paragraph as the KABP surveys were never intended to be the primary sources for these data. 31. Section 2.4, paragraph 32: Contrary to the wording of this paragraph there was a well- defined reporting system for HIV related expenses, with a designated “head” used for all ministries. One major challenge was related to the definition of eligible expenditures. In addition accounting staff shifted throughout the Project period, there was restructuring of ministries, and a lack of continuity of financial management staff with institutional memory. Despite these challenges the Project was able to fully document all of the eligible expenditures identified within the closing period of the Loan. For these reasons, we believe that the fiduciary compliance of the Project deserves a satisfactory rating. 32. Section 2.4, paragraph 33: There was always coordination between the NHAC Senior Accountant and the MH Procurement Officer irrespective of who was leading on the process. The comment regarding ownership of the procurement plan is immaterial since the procurement plan was the procurement plan of the Project. 33. Section 2.5, paragraph 35: This paragraph implies a direct benefit to the NHAC as a result of implementation of the Health Information System. This is misleading given the intended function of the HIS which is expected to measure clinical performance and monitor health sector data primarily. The VHRU and its contribution to building M&E capacity should be highlighted here. 34. Section 3.1, paragraph 36: Please remove the direct quotation “discrimination and stigma has improved but there is still a lot of work to be done”. Reference can be made to the evidence provided such as the data on the National Stigma and Discrimination surveys. 35. Section 3.1, paragraph 37: - “The design and implementation of the Project continue to be highly relevant” – contradicts previous paragraphs on project complexity and its negative impact on implementation. It would be more accurate to state that the focus areas of the Project (which are in fact HIV Programme components) remain highly relevant i.e. scaling up prevention, treatment and institutional strengthening. The HIV epidemic in Barbados is described as a “mixed” epidemic with concentrated pockets. 36. Section 3.2, paragraph 38: Although the Bank was the primary agency providing funds to the NAP during the course of the Project, it was not the only agency providing funds to the NAP during the course of the Project. The Ministry of Health received grant funds from two (2) cooperative agreements with the Centers for Disease Control and Prevention within the Project time frame. It should be noted that credit may be given to other agencies as well. 37. Section 3.2, paragraphs 40 – 52: This section could be expanded to include the efforts involved in the achievement of these results, the context in which these results were 51 achieved or the efforts in train to sustain these results. 38. Section 3.2, paragraph 45: This paragraph refers to a “PMTCT rate” but is describing “PMTCT coverage”. The word “rate” needs to be replaced with “coverage”. 39. Section 3.2, paragraph 48: See comment relating to Section 2.3, paragraph 28 with respect to the NHAC administration of the Adult KABP survey. 40. Section 3.3, paragraphs 54 and 55: The Efficiency rating given to the Project seems to be based mainly on health sector performance as described in this section of the ICR. Paragraph 54 states that “on balance, investments in prevention and treatment, as well as in the health sector were efficient”. On the other hand paragraph 55 refers to “untapped potential” specifically with respect to PMTCT and surveys on key populations, resulting in a reduced rating of “modest”. 41. This seems unfair given the favorable description put forward in paragraph 54. It also gives unbalanced weighting towards PMTCT and key population surveys without significant credit or acknowledgement being given other health related achievements such as the advances in HIV testing efforts and interventions, or improvements in ARV coverage during the Project time frame. 42. Although the specifics of the strengthening of primary health care initiative are listed in Annex 2: subcomponent 1, it would be worthwhile to re-emphasize them in this section of the document as a positive contribution to overall Project efficiency, especially since the primary health care system serves the entire population of 277,000 as alluded to in paragraph 55. 43. Benefits of the Project therefore went beyond the NAP and extended to the wider health system. Given the initial scope of the Project this was an efficient use of resources which allowed strengthening of the health system and potential long term impact beyond the original anticipated Project scope. 44. The final Project rating should be consistent with the final ISR rating of “satisfactory” rather than “moderately satisfactory”. 45. Section 3.5, paragraph (a): It is perplexing that a discussion on poverty impacts should be included in the ICR given WB pronouncements that the Project is not a welfare Project. In addition expenditures deemed to be related to poverty alleviation were explicitly rejected by the Bank as noted in the following statement – “Subsequent review by the Bank found that US$13.641 million of these expenditures did not qualify as they were purely poverty alleviation-related expenditures”(reference page 56 paragraph 3 lines 8 to 10). 46. Section 5.1, paragraph 57: The inability of the Project team to understand reimbursement and other financial aspects despite the provision of fiduciary and financial management training begs the question of the training’s relevance to the Project’s needs. Training was conducted on a large scale only at the start of the Project in October 2008 with over 30 52 people being trained. The fact that such large numbers were trained discounts the fact that government restructuring meant changes in staff. 47. Section 5.1, paragraph 62: This sentence is inaccurate – “The appointment of the Project Director assisted greatly with multisectoral coordination efforts, and with building capacity within each of the agencies.” Despite the shift in project management to the MOH, the multisectoral coordination remained the responsibility of the NHAC and was executed by that agency throughout the life of Project. Apart from not having a multisectoral coordination role, the Project Director did not have responsibility for building capacity within agencies. If there is evidence to support such it should be included in the document. 48. Section 5.1, paragraph 62 and 64: The Project Team’s ability to implement the Project was compromised by challenges with the Project design and conflicting information in the guidance documents. As the Project progressed, the Team proactively adopted and implemented measures to address the attendant performance challenges. 49. Pages 51 to 53 Annex 12 Detailed Project Components: If this section is supposed to correspond to Annex 4 Detailed Project Description of the PAD pages 47 to 50, there are some discrepancies noted. First, it is not clear whether this section was intended to provide an overview of what the Project was supposed to entail or whether it was designed as another forum for presenting Project results. Given there is not a detailed Project description anywhere else in the document this section should fulfil that function. Second, at different intervals in this section, there are sporadic reports on Project results primarily for the MOH namely: a. Page 52 para (i) lines 3 to 4 research…factors. b. Page 52 paras (i) and (ii) under Sub-component 3: Improving Diagnosis, Treatment &Care – results are reported on HIV testing services and treatment respectively. Additionally there are some inaccuracies stated with respect to HIV testing services, and decentralization of treatment services. c. Page 53 para (iii) – results are reported on referral systems to social care d. Page 53 Component 2. Institutional Strengthening (US$4.47 million, US$3.6 million loan) In this section results are being reported – “While routine surveillance, seroprevalence and behavioral surveys, and quality audits were carried out under Component 1, the second component included non-routine training and technical assistance to review the surveillance system, putting sero- and behavior surveillance and quality audits in place, and assisting with standardization of data collection methodologies, particularly in the case of behavior surveillance. This component financed training and technical assistance on M&E, management, surveillance, prevention, diagnosis, treatment and care of HIV/AIDS and other Sexually Transmitted Infections (STIs), to support the implementation of the Strategic Plan.” 53 e. The purpose of this section needs to be clear as such there should be a level of consistency in the information provided. 50. Pages 54 to 56 Annex 13: An updated financial plan similar to that contained in the POM has been provided along with these comments and should be appended to the ICR. Explanatory notes will be included in the plan. Within Annex 13, page 56, Component 2: Institutional Strengthening and M&E, line 10, please replace “Virtual AIDS Info Centre” with “Virtual HIV Research Unit”. 51. Page 56 Annex 13, Special Audit line 10: Please delete “such as groceries and rent” Conclusion 52. We recommend that the overall Project rating should be Satisfactory rather than Moderately Satisfactory for the following reasons: 53. Despite the poor Project design, erroneous background analysis, inaccurate risk assessment, unclear foundation documents, conflicting task team leader interpretations of the foundation documents, mismatched resources and capacity - the Government was able to overcome the challenges which plagued Project implementation while in the midst of a financial crisis. 54. The Project achieved all of the Development Objectives, surpassed many of the indicator targets, was able to demonstrate impact on at least two key population groups, MSM and youth. 55. The rate of Project implementation significantly increased post Mid Term Review and surpassed the expectations of the Bank and the Government of Barbados. 56. Although financial management capacity issues were identified during the first half of the Project funds were fully disbursed by Project end. (END) 2015-05-14 54 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable. 55 Annex 9. List of Supporting Documents 1. Project Appraisal Document 2. Loan Agreement 3. First Restructuring Paper 4. Second Restructuring Paper 5. Third Restructuring Paper 6. Implementation Status and Results Report1-12 7. Financial Management Implementation Status and Results Report (December 2014) 8. Global AIDS Response Progress Report 2012 (Barbados) 9. Economic Impact Analysis of the HIV Epidemic in Barbados for 1984-2008 10. National Strategic Plan for the Prevention and Control of HIV 2008-2013 11. National Strategic Plan of Barbados: 2005-2025 12. GOB/IBRD HIV/AIDS Prevention and Control Project Indicator Framework 13. CSO Grant System Consultant Report 56 Annex 10. Map 57 Annex 11. Summary of KPI Modifications at Restructuring End-of-Project End-of-Project Target Value: Target Value: Baseline Value Key Performance Indicators First Second Restructuring Restructuring Number/Text Number/Text Number/Text (Date) (Date) (Date) Maintain a Maintain a 1. Percentage of young women and 94.3% knowledge rate of at knowledge rate of at men aged 15-24 indicating sexual (2005-2006) least 90% least 90% relations as a way of transmitting HIV (2008-2013) (2008-2013) 2. Percentage of female sex workers 80% 85% 85% reporting the use of a condom with (2006) (2012) (2013) their most recent client 3. Percentage of men reporting the use 65% 15 70% 70% of a condom the last time they had sex (2007) (2011) (2013) with a male partner. 4. Percentage of young men and women aged 15-24 years reporting the 21% 65% 60% 16 use of a condom the last time they had (2005-2006) (2013) (2013) sex with non-marital non cohabitating sexual partner. 5. Percent of most at-risk populations MSM: 75% MSM: 75% (female sex workers and men who NA FSW: 70% FSW: 70% have sex with men) reached with HIV (2008) (2012) (2013) prevention services. 6. Percentage of identified HIV positive pregnant women who gave Maintain a PMTCT Maintain a PMTCT birth in the last 12 months receiving a 96% rate > 95% rate > 95% complete course of ARV to reduce (2006) (2008-2013) (2008-2013) Mother to Child Transmission (MTCT) 7. Percentage of people living with Maintain a virologic Maintain a virologic HIV on first line antiretroviral regimen 60% success rate above success rate above achieving virologic success within the (2008) 70% 70% first 6 months of treatment. (2012) (2013) 8. Amount of funds spent by CSOs US$0 US$150,000 US$150,000 under results-based agreements. (2007) (2013) (2013) Strategic Plan Strategic Plan 2014- Strategic Plan 2014- 9. Evidence-based Strategic Plan 2014- 2008-2013 2018 2018 2018. (2008) (June 2013) (June 2013) 15 This figure is 64% in the PAD. 16 This figure is 65% in ISR 12. 58 Annex 12. Detailed Project Components Sub-component 1: National Program Coordination and Institutional Strengthening. This program aims at strengthening the ability of the public sector, private sector and civil society partners to coordinate, monitor and evaluate their activities and use data to continually increase the quality of their programs. Specific activities would include: (i) Building capacity that would help the GOB and civil society increase their ability to formulate a vision, policies, strategies, and plans of action; mobilize financial resources; and conduct operations relevant to HIVIAIDS. These activities would address issues in intra- and inter-agency communications and coordination, leadership, division of labor and adequate work practices and management practices. An existing Public Sector-CSO Grant System would be further developed under the project. This is described in Annex 6 and in more detail in the Project Operational Manual. (ii) Strengthening surveillance, including the following: (a) continue to routinely report on identified HIV and AIDS cases and AIDS deaths, with close attention to confidentiality matters; (b) routinely report other STIs cases (Syphilis, Gonorrhea and Chlamydia); (c) Continue to carry out sentinel surveillance on pregnant women attending antenatal care and blood donors, to monitor general population prevalence; (d) carry out periodic seroprevalence surveys among key populations at higher risk; (e) carry out periodic KAPB studies of high risk behaviors among key populations at higher risk, looking for changes in behavior which may lead to spread of HIV infection, concentrating on behavior links between members of these groups and the general population; and (f) carry out audits of quality of treatment and care for PLHIV and ST1 patients. (iii) Addressing the critical issue of M&E within the HIVIAIDS Program. Additional M&E staff would be hired within the NHAC and MHNI and a system would be set up to ensure a pool of trained M&E personnel in the country. A consultant would be hired to provide technical assistance and coaching to the new staff. The current M&E system would be redesigned to more effectively gather high quality data on a smaller number of critical indicators. The new system would facilitate the sharing of data with NHAC, the MHNI, and all relevant partners. Analysis and action on data would also be strengthened via new institutional arrangements and training. In addition, the safe storage of data to prevent loss or breaches of confidentiality would be improved. M&E capacity would be increased in the MHNI and other agencies. Capacity building includes training in implementation of the M&E plan, training in specific technical areas such as questionnaire design and data analysis, and translating results into action. Capacity would also be built through the procurement of needed computer systems and software. Implementation of the M&E strategy would include collection of baseline data for planned activities, monitoring of ongoing work, and final evaluation of the HIVIAIDS programming at the end of the project. Sub-component 2: Scaling up Prevention Efforts. This program aims at increasing access to preventive services, especially among key populations at higher risk, and including BCC and provision of condoms. Prevention activities would be 59 implemented in close cooperation between public agencies and CSOs. Among other activities, the BCC Strategy would be implemented as follows: The BCC Strategy included: (i) Evidence-Based BCC Design. High quality research would be conducted to determine the KAP of key populations at higher risk regarding HIV/AIDS practices. In addition, research would be conducted to determine proximal determinants of behavior as well as barriers and enabling factors. The results of the assessment of the most vulnerable groups (sex workers, men who sex with men, prisoners, and youth) would be used to create and test multi-channel high-impact BCC programs tailored to each of the key populations at higher risk, as well as to the general population. (ii) Implementation of BCC programs. BCC programs would be implemented over the course of the five years to reach key populations at higher risk. Promising existing BCC programs in Barbados and the Caribbean would be identified, tested and fine-tuned to ensure that they are appropriate for the Barbados context. The interventions would be multi-channel including peer communications, counseling, mass media, and social marketing. (iii) Management of the BCC strategy. The NHAC manages the BCC strategy and in addition to program design and implementation, it is responsible for needed policy changes and reporting. The BCC unit is currently understaffed and does not have the capacity to meet the challenge of effective BCC. This subcomponent would be responsible for the recruitment and hiring of a senior behavior change officer, a junior behavior change officer responsible for edutainment, and a junior behavior change officer responsible for advocacy. Given the multi-sector nature of Barbados HIV/AIDS response, the success of the new BCC program requires the training and coordination of the partners including civil society, faith-based organizations (FBOs), PLHIV groups, the Ministry of Education, and the Ministry of Tourism, among others. Sub-component 3: Improving Diagnosis, Treatment & Care. The goal of this program is to increase the length and quality of life of PLHIV. The program aims at increasing PLHIV access to diagnostic services, treatment services (ART and treatment for 01s) and social care and support (counseling, support groups, drug addiction therapy, and home care), as follows: (i) HIV testing services would be expanded into community organizations, including those working with vulnerable groups. The HIV testing training program would be adjusted to the new rapid testing protocol, and training would be condensed and made available at more convenient times for private providers. The LRU's laboratory would be strengthened so that it can offer viral loads and CD4 counts not only for Barbados but for other Caribbean countries at cost. (ii) Treatment would be decentralized on a phased basis to the polyclinics that provide free government health services to the entire island. This would help eliminating the stigma currently associated with the LRU in the MHNI, and would seek to integrate HIV 60 services with other services provided at the polyclinics. The subcomponent would provide HIVIAIDS training for health care workers and social workers in the polyclinics. Training would also be made available for private providers. In addition, a quality control system would be introduced to ensure that the decentralization of care does not compromise the quality of outcomes. (iii) Referral systems to social care would be strengthened, including assigning each PLHIV to a social worker. A protocol for systematic evaluations of social care needs would be instituted. Counseling for drug use would be made available to PLHIV and other highly vulnerable individuals. Component 2. Institutional Strengthening (US$4.47 million, US$3.6 million loan) The objective of Component 2 is to provide institutional strengthening via training and technical assistance that would not be funded under the SWAP component. While routine surveillance, seroprevalence and behavioral surveys, and quality audits would be carried out under Component 1, the second component would include non-routine training and technical assistance to review the surveillance system, put in place sero- and behavior surveillance and quality audits and assist with standardization of data collection methodologies, particularly in the case of behavior surveillance. This component would finance training and technical assistance on M&E, management, surveillance, prevention, diagnosis, treatment and care of HIVIAIDS and other STIs, to support the implementation of the Strategic Plan. 61 Annex 13. Project Expenditures 1. As of the final Financial Management Implementation Support and Supervision Report (FMISSR), the Project had US$5.75 million of undocumented expenditures remaining with documentation required to be completed soon after closing, but this has not yet been submitted. 2. Project expenditures as listed in the final FMISSR (covering the duration of the Project only), supplemented by a detailed breakdown provided by the GOB using preliminary figures where available from FY2008-2015 (which includes the Project duration) are provided in the Table below. Component 1 contains expenditures in four mutually exclusive categories: (i) prevention and care (activities related to the prevention and testing of HIV/AIDS), (ii) care and support, (iii) treatment, and (iv) management. Costs associated with PLHIV are covered under care and support and treatment. While there is some overlap in subcategory names (e.g. clinical staff cost under prevention and care and care and support) these reflect different expenditures. For example, clinical staff costs listed under prevention and care refers to staff solely working on prevention and care activities, while clinical staff costs listed under care and support reflect costs of clinical staff solely related to PLHIV. Funds Received Cumulative Detailed Breakdown (US$) Project Receipts – Preliminary Figures from & Expenditures 2008-2015 17 (US$) Loan from IBRD 34,912,480 Contribution from GOB 41,256,819 Total Project Funds 18 76,169,299 Component 1: Prevention and Care (Consists of EPEs which had three components: (i) National Program Coordination and Institutional Strengthening; (ii) Scaling up Prevention Efforts; and (iii) Improving Diagnosis, Treatment & Care) Prevention and Care 20,651,455 11,437,429 19 • Supplies/Materials (E.g. condoms, posters, syringes for testing events) 3,832,152 • Professional Services (I.e. consultant fees for outreach) 9,948 • School Outreach (E.g. Guidance counselor salaries across schools) 2,962,427 • Operating Expenses (E.g. HIV testing events and health fairs, outreach to youth, tent rental) 4,400,121 • Capital Expenditure 232,781 17 Totals may not match the previous column given differences in period covered (Jan 2009-Nov 2014 v. April 2008-Mar 2015). 18 Excludes Front End Fee of $87,500. 19 Figures for 2010-2012 from the Ministry of Social Care and the Prime Minister’s Office have not been included. 62 • Clinical Staff Cost (Salaries for counselors and social workers at polyclinics) 194,940 Care and Support (Focused on PLHIV) 10,085,091 8,945,41119 • Supplies/Materials (E.g. Food Bank, Elroy Philips Center (hostel for PLHIV)) 1,943,258 • Operating Expenses (E.g. Elroy Philips Center and Vashti Inniss Empowerment Center 20; stipend for hotline and Food Bank volunteers) 3,858,566 • Capital Expenditure 14,136 • Clinical Staff Cost (E.g. Staff related to providing care and support specifically to PLHIV) 3,129,451 Treatment (Specific to the LRU Clinic 25,318,103 26,988,029 unless specified) • ARV Drugs 9,565,115 • Reagents 4,090,415 • Other (Non-ARV) Drugs 968,994 • Medical Supplies (E.g. Blood 615,402 Tubes, Needles) • Consumables (E.g. Bandages, 432,896 Cotton) • Operating Expenses 1,475,054 • Capital Expenditure (E.g. 183,002 Equipment, Computers) • Patient Treatment (Treatment at 5,368,436 Queen Elizabeth Hospital) • Clinical Staff Cost (Salaries and 4,288,715 Wages) Management 16,170,324 18,692,735 • Operating Expenses (NHAC and 2,645,233 Programming within MoH) • Management Staff Cost (Salaries 5,294,691 and Wages of NHAC and Program Staff within MoH) • Equipment (E.g. Computers, 3,437,923 Software) • Professional Services (Consultant 3,481,383 fees for financial, research, medical, and technical management consultants) 20 This center houses the Food Bank, HIV-related NGOs, staff providing psychosocial support and the HIV hotline. 63 •Professional Services-MSM Study 456,980 (Consultant fees for MSM Project) • Conferences/Training (In-house 1,026,400 and overseas for Project staff) • Monitoring & Evaluation 510,832 • Behavioral Change 716,425 • Subventions & Grants (Funds 271,098 allocated to civil society) • Supplies/Materials (E.g. Cleaning 851,770 materials, stationery) Component 1 Total 72,224,973 Component 2: Institutional Strengthening and M&E (Consists of goods and consultants’ services) Trend Study 172,788 Technical Audit 11,500 Training 46,525 Financial Audit 171,327 Study Tours 62,233 Legislative Review 37,500 Development of Strategic Plan 129,090 Procurement & Financial Management 152,875 Health Information System 2,799,087 Virtual HIV Research Unit 110,760 Training for Clinical Health Providers 183,141 Component 2 Total 3,856,826 Total Project Expenditures 21 76,169,299 3. Special Audit: Soon after the October 2012 mission (which was ~1 year away from the original closing date of the Project and at which point only $11 million of the Project’s $35 million had been disbursed), a working group led by the Permanent Secretary of Investment of the Ministry of Finance and Economic Affairs with representatives from 17 line ministries was convened to review HIV/AIDS-related expenditures from April 1, 2010-March 31, 2012 that had not been captured under previous IFRs but that qualified for reimbursement. The working group identified US$26.3 million of expenditures; a Special Audit requested by the Bank found that US$23.349 million of expenditures could be certified. Subsequent review by the Bank found that US$13.641 million of these expenditures did not qualify as they were purely poverty alleviation-related expenditures, such as groceries and rent. This left a balance of US$9.707 million in eligible expenditures, of which 65% could be reimbursed as EPEs for a total of $6.309 million. In summary, of the $26.3 million of expenditures identified by the working group as eligible, only $9.707 million was found to be eligible and reimbursement was in the amount of $6.309 million. 21 Excludes Front End Fee of $87,500 64