Document of The World Bank Report No: ICR00004140 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-45960) ON A CREDIT IN THE AMOUNT OF SDR 150.4 MILLION (US$ 225 MILLION EQUIVALENT) TO THE FEDERAL REPUBLIC OF NIGERIA FOR A SECOND HIV/AIDS PROGRAM DEVELOPMENT PROJECT November 30, 2017 Health, Nutrition and Population Global Practice Africa Region i CURRENCY EQUIVALENTS (Exchange Rate Effective November 30, 2017) Currency Unit = Nigerian Naira (NGN) NGN 1.00 = US$ 0.003 US$ 1.00 = NGN 360.00 XDR 1.00 = US$ 1.41 FISCAL YEAR July 01 – June 30 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immune Deficiency Syndrome IBBSS Integrated Bio-Behavioral Surveillance Survey ART Antiretroviral Therapy HIV Human Immunodeficiency Syndrome BBFSW Brothel-Based Female Sex Worker ICR Implementation Completion Report BCC Behavior Change Communication IDA International Development Association CBO Community-based Organizations IDU Intravenous Drug User CSO Civil Society Organization IFR Interim Financial Reports CPS Country Partnership Strategy IMF International Monetary Fund CDS Community Development Services IP Implementing Partners DFID UK Department for International Development IRI Intermediate Results Indicators DHIS District Health Information System IRR Internal Rate of Return ENR Enhancing Nigeria’s Response to HIV/AIDS ISR Implementation Status Report Program, DFID JFA Joint Financing Arrangement ESMF Environmental and Social Management KPA Key Population Assessment Framework LACA Local Action Committee on HIV/AIDS FCT Federal Capital Territory LGA Local Government Areas FEC Federal Executive Council LHPMIP Logistics and Health Program Management FGON Federal Government of Nigeria Information Platform FM Financial Management LM Line Ministry FMOF Federal Ministry of Finance LSHTM London School of Hygiene and Tropical FMOE Federal Ministry of Education Medicine SMoE State Ministry of Education M&E Monitoring and Evaluation FMOH Federal Ministry of Heatlh MAP Multi-Country HIV/AIDS Program FMOWSD Federal Ministry of Women Affairs and Social MARP Most-at-Risk Population Development MDA Ministries, Departments and Agencies FMOY Federal Ministry of Youths Development MDG Millennium Development Goal FMOYS Federal Ministry of Youth and Sports MICS Multi-Indicator Cluster Survey FSW Female Sex Worker MPPI Minimum Package of Interventions GDP Gross Domestic Product MSM Men Who Have Sex with Men GFATM The Global Fund to Fight AIDS, Tuberculosis MTCT Mother to Child Transmission and Malaria MTR Midterm Review GHAP Global HIV/AIDS Program, World Bank MWMP Medical Waste Management Plan GNI Gross National Income NACA National Agency for the Control of HAF HIV/AIDS Fund AIDS/National Action Committee on AIDS HEAP HIV/AIDS Emergency Action Plan NARHS National AIDS and Reproductive Health Survey HCT HIV/AIDS Counseling and Testing NASCP National AIDS and STI Control Program of the HMIS Health Management Information System Federal Ministry of Health HPDP1 First HIV/AIDS Program Development Project NBBFSW Non Brothel-Based Female Sex Worker HPDP2 Second HIV/AIDS Program Development Project NEEDS National Economic Empowerment and HSS Health System Strengthening Development Strategy ii NGO Non-Governmental Organization PRS Poverty Reduction Strategy NNRIMS Nigeria National Response Information QER Quality Enhancement Review Management System QIPR Quarterly Implementation Progress Report NPV Net Present Value RF Results Framework NSF1 National Strategic Framework for HIV/AIDS, SACA State Agency for the Control of HIV/AIDS/State 2005-2009 Action Committees on AIDS NSF2 National Strategic Framework for HIV/AIDS, SHARHS State-specific HIV/ AIDS, Reproductive Health 2010-2016 and Child Health Survey NSF3 National Strategic Framework for HIV/AIDS, SPHCDA State Primary Health Development Agencies 2017-2022 STC Short Term Consultant NSHDP National Strategic Health Development Plan, STI Sexually-Transmitted Infection 2010-2015 TB Tuberculosis NYSC National Youth Service Corps TOR Terms of Reference OVC Orphans and Vulnerable Children TSA Single Treasury Account OI Opportunistic Infection TTL Task Team Leader PAD Project Appraisal Document TWG Technical Working Groups PBF Performance Based Financing UNAIDS The Joint United Nations Programme on PAC Presidential AIDS Commission HIV/AIDS PCRP President’s Comprehensive Response Plan UNDP United Nations Development Program PDO Project Development Objective UNFPA Unites Nations Population Fund PEPFAR U.S. President's Emergency Plan for AIDS Relief UNICEF United Nations Children's Fund PFMU Project Financial Management Units USAID United States Agency for International PHF Private Health Facility Development PIU Project Implementation Unit UNGASS United Nations General Assembly Special PMTCT Prevention of Mother-to-Child Transmission Session on AIDS PLHIV People Living with HIV USG United States Government PLWHA People Living with HIV/AIDS WHO World Health Organization Vice President: Makhtar Diop Country Director: Rachid Benmessaoud Practice Manager: Trina Haque Project Team Leader: Enias Baganizi ICR Team Leader: Enias Baganizi ICR Author: F. Brian Pascual ii NIGERIA HIV/AIDS Program Development Project II Table of Contents Data Sheet ...................................................................................................................... iv A. Basic Information .................................................................................................. iv B. Key Dates .............................................................................................................. iv C. Ratings Summary .................................................................................................. iv D. Sector and Theme Codes........................................................................................ v E. Bank Staff .............................................................................................................. vi F. Results Framework Analysis ................................................................................. vi G. Ratings of Project Performance in ISRs ........................................................... xviii H. Restructuring (if any) .......................................................................................... xix I. Disbursement Profile ............................................................................................ xx 1. Project Context, Development Objectives and Design ............................................... 1 1.1 Context at Appraisal ............................................................................................. 1 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved in the Project Appraisal Document, PAD) .................................................. 3 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification .............................................................................................. 3 1.4 Main Beneficiaries ................................................................................................ 4 1.5 Original Components (as approved) ..................................................................... 4 1.6 Revised Components ........................................................................................ 5 1.7 Other significant changes ...................................................................................... 5 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 2.1 Project Preparation, Design and Quality at Entry ................................................. 7 2.2 Implementation ..................................................................................................... 8 2.3 M&E Design, Implementation and Utilization ................................................... 11 2.4 Safeguard and Fiduciary Compliance ................................................................. 13 2.5 Post-completion Operation/Next Phase .............................................................. 16 3. Assessment of Outcomes .......................................................................................... 16 3.1 Relevance of Objectives, Design and Implementation ....................................... 16 3.2 Achievement of Project Development Objectives .............................................. 18 3.3 Efficiency ............................................................................................................ 22 3.4 Justification of Overall Outcome Rating ............................................................ 24 3.5 Overarching Themes, Other Outcomes and Impacts .......................................... 24 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ... 25 4. Assessment of Risk to Development Outcome......................................................... 26 5. Assessment of Bank and Borrower Performance ..................................................... 26 5.1 Bank Performance ............................................................................................... 26 5.2 Borrower Performance ........................................................................................ 28 6. Lessons Learned ....................................................................................................... 29 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 30 Annex 1. Project Costs and Financing .......................................................................... 31 (a) Project Cost by Component (in USD Million) .................................................... 31 ii (b) Financing ............................................................................................................. 31 Annex 2. Outputs by Component ................................................................................. 32 Annex 3. Economic and Financial Analysis ................................................................. 35 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 42 (a) Task Team members............................................................................................ 42 (b) Staff Time and Cost ............................................................................................. 42 Annex 5. Beneficiary Survey Results ........................................................................... 44 Annex 6. Stakeholder Workshop Report and Results................................................... 45 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 46 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders ...................... 58 Annex 9. List of Supporting Documents ...................................................................... 59 Annex 10. Changes to Indicators in Results Framework.............................................. 61 Annex 11. MTR details ................................................................................................. 69 Annex 12. Revised Disbursement Estimates ................................................................ 72 MAP .............................................................................................................................. 74 iii Data Sheet A. Basic Information Nigeria HIV/AIDS Country: Nigeria Project Name: Program Development Project II Project ID: P102119 L/C/TF Number(s): IDA-45960 ICR Date: 11/30/2017 ICR Type: Core ICR FEDERAL Lending Instrument: SIL Borrower: GOVERNMENT OF NIGERIA Original Total XDR 150.40M Disbursed Amount: XDR 126.50M Commitment: Revised Amount: XDR 137.2M Environmental Category: B Implementing Agencies: National Agency for the Control of AIDS (NACA) and State Agencies for the Control of AIDS (SACAs) Co-financiers and Other External Partners: External partners included The Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM), U.S. President's Emergency Plan for AIDS Relief (PEPFAR), U.K. Department for International Development (DFID), and The Joint United Nations Programme on HIV/AIDS (UNAIDS). B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 01/28/2008 Effectiveness: 02/25/2011 12/16/2011 Appraisal: 12/02/2008 Restructuring(s): 05/23/2014 11/19/2015 Mid-term Approval: 06/16/2009 11/15/2013 10/30/2013 Review: 11/30/2015 Closing: 12/31/2013 02/28/2017 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Satisfactory iv Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Satisfactory Quality of Moderately Implementing Moderately Satisfactory Supervision: Satisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Moderately Satisfactory Performance: Satisfactory Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Quality at Entry Project at any time Yes None (QEA): (Yes/No): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Major Sector/Sector Health Health 27 27 Public Administration – Health 20 20 Social Protection Social Protection 53 53 Major Theme/Theme/Sub Theme Human Development and Gender Disease Control 61 61 HIV/AIDS 61 61 Tuberculosis 10 10 Social Development and Protection v Social Inclusion 12 12 Participation and Civic Engagement 12 12 Social Protection 17 17 Social Safety Nets 17 17 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Rachid Benmessaoud Onno Ruhl Practice Trina S. Haque Lynne D. Sherburne-Benz Manager/Manager: Project Team Leader: Enias Baganizi John Elder ICR Team Leader: Enias Baganizi ICR Primary Author: F. Brian Pascual F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is i) to reduce the risk of HIV infections by scaling up prevention interventions and ii) to increase access to and utilization of HIV counseling, testing, care and support services. For the ease of measuring progress, the indicators were categorized under one of three PDO categories: reduce risk, increase access, and increase utilization. Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original End- Formally Formally Actual Value of-Project Revised Revised Achieved at Baseline Targets Targets at Targets at Project Indicator Value (2009-2011) First Third Completion Restructuring Restructuring (2011-2015) (2015-2017) PDO I: Reduce the risk of HIV infection Indicator 1: Percent of young women aged 15-24 reporting the use 34.3 45.4 40.0 No change 46.6 of a condom during the last sexual intercourse with a non- regular partner Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 vi Active Y Y Y Y Original – Target revised at First Restructuring – Target Surpassed: The actual value at the end of the Project surpassed Comments revised target by 116% and the original target by 11%. (incl. % achievement) Data source: 2007 NARHS for baseline and MICS 2017 for final data. Indicator 2: Percent of young men aged 15-24 reporting the use of a 52.2 69.2 60.0 No change 61.4 condom during the last sexual intercourse with a non-regular partner Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original – Target revised at First Restructuring – Target Surpassed: The actual value at the end of the Project surpassed the Comments revised target by 18%. (incl. % achievement) Data source: 2007 NARHS for baseline and MICS 2017 for final data. Indicator 3: Percentage of women aged 15-49 who have had sexual 1.0 0.4 0.6 Dropped 1.1 intercourse with more than one partner in the last 12 months Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 30-Sept-2015 Y Y N N Original – Original Target Revised at First Restructuring and Dropped at Third Restructuring: At the time of the First Restructuring, the End-of-Project Target was revised to account for the delayed Project effectiveness. During the third restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based Comments surveys on which data for these indicators relied were infrequently (incl. % achievement) conducted, the Project had difficulties in tracking their progress. In addition, the follow-up surveys would not be completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. Data source: 2007 NARHS for baseline and FMOH program data for final data. Indicator 4: Percentage of men aged 15- 49 who have had sexual 7.2 5.0 No change Dropped 10.8 intercourse with more than one partner in the last 12 months Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 30-Sept2015 Active Y Y N N vii Original – Dropped at Third Restructuring: At the time of the restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based surveys on which data for these indicators relied were infrequently conducted, the Project had difficulties in tracking Comments their progress. In addition, the follow-up surveys would not be (incl. % achievement) completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. Data source: 2007 NARHS for baseline and FMOH program data for final data. Indicator 5: 68.0 Percentage of Female Sex (revised Workers (Brothel-Based) baseline at First Restructuring) reporting consistent condom 72.0 No change No change 92.5 use with casual partner in the (Original last 12 months baseline in PAD 64.8%) Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original – – Baseline revised at First Restructuring – Target Surpassed: The actual value at the end of the Project surpassed the target by 29%. The last IBBSS was conducted in 2014. The KPA 2017 has provided an update for the indicator, the KPA was conducted Comments specifically for the project with the same methodology as the IBBSS. (incl. % achievement) The baseline data provided in the PAD (64.8%) was incorrect and was corrected as part of the first restructuring. Data source: IBBSS 2007 for baseline and KPA 2017 for final data. Indicator 6: 61.0 Percentage of Female Sex (revised Workers (Non Brothel- baseline at First Restructuring) Based) reporting consistent 75.0 65.0 No change 87.0 condom use with casual (Original partner in the last 12 months baseline in PAD 67.1%) Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original – Baseline and Target revised at First Restructuring – Target Surpassed: The actual value at the end of the Project surpassed revised target by 550% and the original target target by 86%. The last IBBSS was conducted in 2014. The KPA 2017 has provided Comments an update for the indicator, the KPA was conducted specifically for the (incl. % achievement) project with the same methodology as the IBBSS. The baseline data provided in the PAD (67.1%) was incorrect and was corrected as part of the first restructuring. viii Data source: IBBSS 2007 for baseline and KPA 2017 for final data. Indicator 7: Percentage of most-at-risk populations (FSWs) who both correctly identify ways of 32.9% 50% No Change Dropped 35% preventing the sexual transmission of HIV and who reject major misconception about HIV transmission Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 30-Sept-2015 Active Y Y N N Original – Dropped at Third Restructuring: At the time of the restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based surveys on which data for these indicators relied Comments were infrequently conducted, the Project had difficulties in tracking (incl. % achievement) their progress. In addition, the follow-up surveys would not be completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. Data source: IBBSS 2007 for baseline. Indicator 8: Percentage of most-at-risk populations (MSM) who both correctly identify ways of 44.0% 57% No change Dropped 33% preventing the sexual transmission of HIV and who reject major misconception about HIV transmission Date achieved 31-Dec-2010 31-Dec-2013 30-Nov-2015 19-Nov-2015 10-May2015 Active Y Y N N Original – Dropped at Third Restructuring: At the time of the restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based surveys on which data for these indicators relied Comments were infrequently conducted, the Project had difficulties in tracking (incl. % achievement) their progress. In addition, the follow-up surveys would not be completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. Data source: IBBSS 2007 for baseline. ix Indicator 9: Percentage of most-at-risk populations (IDUs) who both correctly identify ways of 34.0% 47% No change Dropped 31% preventing the sexual transmission of HIV and who reject major misconception about HIV transmission Date achieved 31-Dec-2010 30-Dec-2013 30-Nov-2015 19-Nov-2015 30-Sept-2015 Active Y Y N N Original – Dropped at Third Restructuring: At the time of the restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based surveys on which data for these indicators relied Comments were infrequently conducted, the Project had difficulties in tracking (incl. % achievement) their progress. In addition, the follow-up surveys would not be completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. Data source: IBBSS 2007 for baseline. PDO II: Increase access to HIV counseling, testing, care and support services Indicator 10: 2,287,805 Number of women and men (updated aged 15-49 (Project numeric baseline at Beneficiaries) who received Third an HIV test in the last 12 Restructuring) 10,728,872 months and who know their No change in results (numeric target, but (No progress 96% 10,000,000 baseline baseline data collected introduced at changed before First First Restructuring) Restructuring, 4,400,000) (Original baseline 72.7%) Date achieved 31-Dec-2010 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original – Revised at First Restructuring –Target Surpassed: Originally provided as a percentage. The actual annual cumulative value at the end of the Project surpassed the target by 7%. Over 25 million persons have been counselled tested and received result since the inception of the project. This CORE indictor introduced as part of Comments first restructuring in 2011; previously given as a %. The 2008 NDHS (incl. % achievement) baseline data initially provided in the first restructuring (4,400,000) was updated with 2010 FMOH data (2,287,805). Data source: FMOH Program Data 2010 for baseline and 2016 for final data. Indicator 11: 38.8 NA NA 43.0 90.1 x Percentage of female sex workers who were tested for HIV in the last calendar year Date achieved 27-Sept-2011 28-Feb-2017 28-Feb-2017 Active N N Y Y New – Added at Third Restructuring – Target Surpassed: The actual value at the end of the Project represented an average of data points from brothel (92.1%) and non-brothel (88%) based sex workers, and which surpassed the target by 110%. The target has been surpassed Comments at the end of the project, with the key population assessment which (incl. % achievement) was conducted specifically for the project, with the same methodology as the IBBSS. Data source: IBBSS 2010 for baseline and KPA 2017 final actual data. PDO III: Increase utilization of HIV counseling, testing, care and support services. Indicator 12: 26,133 53,677 Number of pregnant women (new numeric living with HIV who receive baseline at First (Progress data a complete course of Restructuring) 30% 40,000 No Change collected at antiretroviral prophylaxis to First reduce the risk of mother-to- (Original Restructuring: child transmission baseline 5.25%) 18.7%) Date achieved 31-Dec-2010 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original Core Indicator – Revised at First Restructuring– Target Surpassed: The actual value at the end of the Project surpassed the target by 34%. The target for this indicator has been surpassed. This Comments CORE indicator introduced as part of first restructuring in 2011; (incl. % achievement) previously given as a %. Data source: FMOH Program Data 2010 and 2016. Indicator 13: NA Ratio of current school attendance between orphans 75% 87% Dropped N/A (No progress and non-orphans. data collected before First Restructuring) Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Original Core Indicator – Dropped during First Restructuring: Measurement and collection complexities by the Government and Comments UNICEF. (incl. % achievement) Data source: CRS 2006 and FMOWA OVC Situation Analysis for baseline. xi (b) Intermediate Outcome Indicator(s) Original End- Formally Formally Actual Value of-Project Revised Revised Achieved at Baseline Targets Targets at Targets at Project Indicator Value (2009-2011) First Third Completion Restructuring Restructuring (2011-2015) (2015-2017) Component 1: Expanding the Public Sector Response Indicator 1: Percentage of schools that provided life skills based HIV 22.8% 30% No change No change 30% education in the last academic year Date achieved 31-Dec-2009 31-Dec-2013 30-Nov-2015 28-Feb-217 28-Feb-2017 Active Y Y Y Y Original – Target Achieved: The Project reached the target by 100%. Comments (incl. % achievement) Data source: FME 2009 Program Data for baseline and for end of Project target Indicator 2: Number of Federal Line 28 13 Dropped NA NA Ministries implementing HIV/AIDS workplans Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 3: Number of State Line 281 259 Dropped NA NA Ministries implementing HIV/AIDS workplans Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 4: Number of health facilities 1,064 1,600 No change 8,308 that provide HIV counseling and testing services Date achieved 16-Dec-2011 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active N Y Y Y New – Added at First Restructuring – Target Surpassed. The Project increased the number of health facilities that provided counseling and testing by 681% over the baseline, surpassing the target Comments by 419%. (incl. % achievement) Data source: FMOH Program Data 2010 and 2017 for end of Project data xii Indicator 5: Percentage of health facilities 4% 6% 5% NA that provide HIV counseling and testing services Date achieved 16-Dec-2011 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active N Y Y Y Comments New – Added at First Restructuring: Progress data never collected. (incl. % achievement) Indicator 6: Percentage of HIV-positive patients who were screened 87% NA 90% Dropped NA for TB in HIV care or treatment centers. Date achieved 16-Dec-2011 30-Nov-2015 19-Nov-2015 Active N Y N N Comments New – Added at First Restructuring but dropped at Third (incl. % achievement) Restructuring: Component 2: Expanding the Civil Society and Private Sector Response to the HIV/AIDS Epidemic Indicator 7: Number of PLWHA groups and associations that receive a basic package of support 0 TBD Dropped NA NA from CSOs that implement the HIV/AIDS Fund (HAF) program Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 N N Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 8: Number of orphans that receive a basic package of 0 TBD Dropped NA NA support from CSOs that implement the HIV/AIDS Fund (HAF) program Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 9: Percentage of Private Sector Revised and CSOs that report indicator quarterly and on time on 0 50% Dropped NA description project indicators: Federal for specificity level Private Sector Organizations (PSOs) Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 xiii Active Y Y N N Original – Revised at First Restructuring and dropped at Third Comments Restructuring. The original indicator only asked for reporting on a (incl. % achievement) regular basis, but a change was made to specific quarterly timeframe reporting during First Restructuring. Indicator 10: Percentage of Private Sector Revised and CSOs that report indicator 71% 95% Dropped NA quarterly and on time on description project indicators: Federal for specificity level CSOs Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 Active Y Y N N Original – Revised at First Restructuring and dropped at Third Comments Restructuring. The original indicator only asked for reporting on a (incl. % achievement) regular basis, but a change was made to specific quarterly timeframe reporting during First Restructuring. Indicator 11: Percentage of Private Sector and CSOs that report 74% 90% 80% Dropped NA quarterly and on time on project indicators: State level CSOs Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 19-Nov-2015 Active Y Y N N Original – Revised at First Restructuring and dropped at Third Restructuring. The original indicator only asked for reporting on a Comments regular basis, but a change was made to specific quarterly timeframe (incl. % achievement) reporting during First Restructuring. End of Project Target was also reduced due to delay in HAF implementation that financed CSO activities. Indicator 12: Percentage of organizations that submit quarterly Financial Monitoring and Procurement reports 0 NA 80% No Change 92% according to specific guidelines (disaggregated by Private Sector Organizations and CSOs) Date achieved 16-Dec-2011 31-Dec-2015 28-Feb-2017 30-Nov-2016 Active N Y Y Y New – Added at First Restructuring – Target Surpassed: The actual value at the end of the Project surpassed the target value by Comments 15%. Data not disaggregated by PSO and CSO (incl. % achievement) Data source: NACA QIPR 2016 for final data. xiv Indicator 13: Number of new most-at-risk 0 NA NA 10,000 110,196 populations (MARPs) reached with MPPI Date achieved 19-Nov-2015 28-Feb-2015 28-Feb-2017 Active N N Y Y New – Added at Third Restructuring – Target Surpassed: The Project surpassed the target by 1,336%, reaching 14 times as many Comments number of MARPs as planned. (incl. % achievement) Data source: DHIS 2016 for final data. Indicator 14: Number of project supported private health care facilities 0 NA NA 25 360 providing prevention of mother-to-child transmission services Date achieved 19-Nov-2015 28-Feb-2017 28-Feb-2017 Active N N Y Y New – Added at Third Restructuring – Target Surpassed: The Comments Project surpassed the target by 1,340%, increasing the number of (incl. % achievement) private health facilities providing prevention of mother-to-child transmission services by 13 times more than planned. Indicator 15: Number of project supported private health care facilities 0 NA NA 25 360 providing HIV testing and counseling services Date achieved 19-Nov-2015 28-Feb-017 28-Feb-2017 Active N N Y Y New – Added at Third Restructuring – Target Surpassed: The Comments Project surpassed the target by 1,340%, increasing the number of (incl. % achievement) private health facilities providing testing and counseling services by 13 times more than planned. Indicator 16: Number of female sex 25,596 NA NA 40,000 45,725 workers who were tested for HIV in the last calendar year Date achieved 19-Nov-2015 28-Feb-2017 28-Feb-2017 Active N N Y Y New – Added at Third Restructuring – Target Surpassed: The Project increased the number of female sex workers tested within a Comments calendar year by 79% over the baseline, exceeding the target by 14%. (incl. % achievement) Data source: DHIS 2015 for final data. Component 3: Strengthen Mechanisms for Project Coordination and Management Indicator 17: 0 100% Dropped NA NA xv Percentage of Public Sector, Private Sector, and CSOs with improved financial accounting practices Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 18: Percentage of staff NACA and SACA receiving TA/training who report that 0 100% Dropped NA NA they have used their new knowledge and skills and can give examples Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 19: Percentage of actual funds NA 100% Dropped NA NA utilized relative to planned targets Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 20: Number of States that link the National program priorities 35 37 Dropped NA NA with their annual operational plans Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 21: Number of States that link the 18 37 Dropped NA NA State program priorities with their annual operational plans Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 22: All Participating States and 13 35 Dropped NA NA the Federal Capital Territory have transformed their xvi SACAs into financially autonomous statutory agencies Date achieved 16-Jun-2009 31-Dec-2013 16-Dec-2011 Active Y N N N Comments Original – Dropped at First Restructuring. (incl. % achievement) Indicator 23: Number of SACAs, State and Federal level public sector organizations that have a 2 13 No Change 4 4 written strategic plan clearly linked to the National Strategic objectives and priorities: Federal LMs Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Y Y Y Y Original – Revised at Third Restructuring – Target Achieved: End of Project Target was revised downward as a result of transition to a Comments more focused approach and support to limited number of LMs. The (incl. % achievement) Project reached the revised target by 100%. Data source: NACA 2010 and 2017 Indicator 24: Number of SACAs, State and Federal level public sector organizations that have a 18 37 No Change No Change 37 written strategic plan clearly linked to the National Strategic objectives and priorities: SACAs Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 30-Sept-2015 Active Y Y Y Y Original – Target Achieved: The Project reached the target by 100%. Comments (incl. % achievement) Data source: NACA 2010 and 2017 Indicator 25: Percentage of SACAs and Public Sector Organizations 30% 100% 80% No Change 86% that hold quarterly partner forums to report on planning, decisions and progress Date achieved 16-Jun-2009 31-Dec-2013 30-Nov-2015 28-Feb-2017 Active Y Y Y Y Original – Revised at First Restructuring and Third Restructuring Comments –Target Surpassed: Changes under the First Restructuring included (incl. % achievement) moving indicator from Component I; revising meeting schedule from biannually to quarterly; adding PSOs as part of indicator; and reducing xvii end of Project Target to 80%. Changes under Third Restructuring involved increasing baseline from 0 to 30%. The Project increased the number of SACAs and Public Sector Organizations that held quarterly partner forums by 187% over the baseline, surpassing the target by 12%. Data not disaggregated by SACAs and PSO; Baseline revised at Third Restructuring from 0% to 30%. Data source: NACA QIPR 2011 and 2017. Indicator 26: Number of LACAs 0 TBD No change 272 600 implementing HIV/AIDS workplans Date achieved 31-Dec-2010 31-Dec-2013 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active Y Y Y Y Original – Revised at First Restructuring and Third Restructuring – Target Surpassed: Changes under the First Restructuring involved moving indicator from Component I. Changes under Third Comments Restructuring included assigning baseline and end of Project Target (incl. % achievement) values that were previously not determined. The Project surpassed the target by 121%. Data source: SACA reports 2010 and 2017. Indicator 27: Availability of Annual HAF N NA Y No Change Y performance report prepared by NACA Date achieved 16-Dec-2011 30-Nov-2015 28-Feb-2017 28-Feb-2017 Active N Y Y Y Added at First Restructuring – Target ACHIEVED: The Project contributed to ensuring that NACA completed HAF performance Comments reports on an annual basis. (incl. % achievement) Data source: NACA Program Data 2010 and 2017 G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/22/2009 Satisfactory Satisfactory 0.00 2 05/10/2010 Satisfactory Satisfactory 0.00 3 12/22/2010 Moderately Satisfactory Moderately Satisfactory 0.00 4 06/28/2011 Moderately Satisfactory Moderately Satisfactory 0.85 5 12/25/2011 Moderately Satisfactory Satisfactory 8.03 6 06/23/2012 Satisfactory Satisfactory 25.57 7 12/25/2012 Satisfactory Satisfactory 45.86 Moderately 8 06/22/2013 Moderately Satisfactory 61.22 Unsatisfactory xviii Moderately 9 12/28/2013 Moderately Satisfactory 74.34 Unsatisfactory 10 06/14/2014 Moderately Satisfactory Moderately Satisfactory 89.25 11 12/22/2014 Moderately Satisfactory Moderately Satisfactory 111.26 12 06/09/2015 Moderately Satisfactory Moderately Satisfactory 133.23 13 11/03/2015 Moderately Satisfactory Moderately Satisfactory 154.27 14 11/13/2015 Moderately Satisfactory Moderately Satisfactory 155.21 15 05/23/2016 Satisfactory Moderately Satisfactory 166.25 16 01/13/2017 Satisfactory Satisfactory 182.05 17 02/28/2017 Moderately Satisfactory Satisfactory 184.90 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Extension of closing date and revisions to results framework, especially to update baselines to reflect the delayed start of the project implementation and align 12/16/2011 N MS S 5.30 the target values accordingly. Amount disbursed represented 2.9% of the US$186.00 million total Project disbursement. Change in Component 1 reallocation of unallocated resources, establish framework contracting for procurement of goods, and adjustment of 05/23/2014 N MS MU 82.21 disbursement estimate. Amount disbursed represented 44.2% of the US$186.00 million total Project disbursement. Extension of Closing Date by 15 months to Feb 28, 2017, revision to Results Framework, change 11/19/2015 N MS MS 155.85 financial management arrangements, and incorporated Kano State into the Project. xix ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Amount disbursed represented 83.8% of the US$186.00 million total Project disbursement. I. Disbursement Profile xx 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. The Federal Republic of Nigeria, Africa’s most populous country, had a population of 144 million at the time of project appraisal on December 2008. The country was divided administratively into 36 States – which were further subdivided into 774 local government areas (LGAs) – and one Federal Capital Territory (FCT) located at Abuja. During the first half of the 2000s, Nigeria benefitted from oil revenues and had an average Gross Domestic Product (GDP) growth rate of 11.5%. With the overall decline in the economy in 2005 outside of the energy sector, however, inflation rate rose to 15.6% while the Gross National Income (GNI) per capita hovered at US$660, making Nigeria one of the poorest countries in Africa. Poverty became pervasive with more than 70% of the population living on less than $1 per day. At the time, Nigeria ranked at the lowest scale of the United Nations Development Program (UNDP) Human Development Index, where it placed 158 out of 177 countries. Moreover, the country suffered significantly from the burden of communicable diseases, including HIV. Nigeria’s HIV and AIDS epidemic was a combination of a ‘generalized’ epidemic that affected the general population, including rural areas, and also of pervasive sub-epidemics with high prevalence rates among particular geographic and social groupings involved in high risk practices. 2. In 1999, political support through the newly established Presidential AIDS Commission (PAC) changed the landscape on HIV/AIDS in Nigeria by shifting the HIV/AIDS response from a health sector led approach to a multisectoral response. By 2000, the Office of the President had established the National Action Committee on AIDS (NACA, subsequently converted to Federal agency status as the National Agency for the Control of AIDS) tasked with coordinating the various HIV/AIDS response activities in the country. The 2000-2004 National HIV/AIDS Emergency Action Plan (HEAP) and the 2006-2010 National Health Sector Strategic Plan for HIV/AIDS provided the framework that ensured everyone had access to quality health care and adequate treatment or management of their conditions, including the provision of HIV/AIDS counseling and testing (HCT), Prevention of Mother to Child Transmission (PMTCT) services and Antiretroviral Therapy (ART). 3. In the aftermath of Nigeria’s overall response, the country witnessed a decline in its HIV prevalence rate from 5.8 in 2001 to 4.4% in 2005. However, prevalence in six states and the FCT remained above 6%, with Benue State (at 10% prevalence) as the epicenter of the infection. Also, certain sub-groups, like the most-at-risk population (MARP), experienced rates well above the national average, including female sex workers (FSW) at greater than 30%, men who have sex with men (MSM) at 13.5%, and intravenous drug users (IDUs) at 5.5%. By 2007, approximately 2,600,000 were people living with HIV (PLHIV), out of which 370,000 were new infections and 170,000 AIDS deaths recorded for the year. Cumulative deaths from AIDS at the time stood at 1.48 million. Although the situation remained stable, Nigeria accounted for the second highest burden of the disease in the world according to the Joint United Nations Programme on HIV/AIDS (UNAIDS). The Federal Government of Nigeria’s (FGONs) ensuing 2005-2009 HIV/AIDS National Strategic Framework (NSF1), a follow up to the HEAP, aimed to ‘reduce HIV/AIDS incidence by at least 25%, and provide equitable prevention, care, treatment and support while mitigating its impact amongst women, children and other vulnerable groups and the general population in Nigeria by 2009.’ 4. Between 2001-2010, the Bank’s support for the initial HIV/AIDS Program Development Project (HPDP1) was credited with creating the necessary institutional framework within the public sector and civil society for a subsequent large scale multisectoral response to HIV/AIDS, including establishing State Action Committees on AIDS (SACAs, subsequently converted from committees 1 to State agencies similar to NACA at the Federal level) at the State level. When HPDP1 closed, the Bank’s continued involvement was considered critical given the Bank’s contributions to i) bringing in significant amount of financing, which was still needed despite substantial Government and other donor financing; ii) playing a key role as a key investor in comprehensive prevention services and demand-driven community based interventions; iii) supporting a multisectoral response; iv) learning simultaneously from other countries pursuing similar approaches as a result of the Bank’s Multi- Country HIV/AIDS Program (MAP) initiative for the Africa Region; v) designing and implementing social funds and targeted cash transfers for civil society and community-based HIV interventions; vi) helping the FGON through its convening power to implement the ‘three ones’ principle supported by all partners – one HIV/AIDS action framework, one national HIV/AIDS coordinating authority, and one monitoring and evaluation (M&E) system; vii) leading Nigeria’s first Joint Financing Arrangement (JFA) with other donors that ensures greater harmonization and alignment of partner support; and viii) financing system development and capacity building to scale-up activities. 5. Given the burden of HIV/AIDS, the FGON and the Bank agreed to engage on a second HIV/AIDS Program Development Project (HPDP2). The US$225 million International Development Association (IDA) Credit financing for HPDP2 was intended to support Nigeria for four and a half years (June 2009 to December 2013) and serve all states1 along with the FCT (referred to as 36+1 States). The Bank-financed Project focused on prevention and care and support services that complemented other programs being implemented around the same time, namely the UK Department for International Development’s (DFID’s) Enhancing Nigeria’s Response to HIV/AIDS (ENR) program (US$155 million), the Global Fund to Fight AIDS, Tuberculosis and Malaria (GFATM, US$520 million), and UNAIDS (US$192 million). These programs aimed at providing the Federal and State Governments substantial technical support to strengthen their stewardship and coordination roles in effectively implementing a multi-sectoral and evidence-based HIV/AIDS response, mainly focused on boosting prevention, treatment, care and support, and health system strengthening (HSS) activities. The largest funders towards the national response were the FGON (US$1,176 million) and the US Government (USG, US$2,235 million), with the USG focused on providing services on treatment, prevention, and care. 6. The proposed HPDP2 supported the goals of the NSF1 and minimized the country’s funding gap. Moreover, HPDP2 complemented the FGON’s 2005 National Economic Empowerment and Development Strategy (NEEDS) – Nigeria’s Poverty Reduction Strategy (PRS) – that emphasized the development of strategic plans for HIV/AIDS as a specific action under the health component of the PRS. Specifically, the FGON recognized that the epidemic, if not adequately contained, would prove to be the greatest single obstacle to reaching national poverty reduction and other targets for social and economic development. The Project also aligned with the FGON’s existing 2009 National Policy on HIV/AIDS since the HPDP2 involved providing care and support to people infected and affected by HIV/AIDS. Lastly, the HPDP2 followed the lead of the 2005-2009 Country Partnership Strategy (CPS) that expected to achieve the following results: i) improved service delivery for human development; ii) improved environment and services for non-oil growth; and iii) improved transparency and accountability for better governance. 1 Kano State opted not to participate in the Project from the outset, making 35 the total number of States involved in HPDP2, plus the FCT. According to the Governor of Kano, the State had enough resources to fund its own program, and as such declined World Bank support. At the time, the Kano State Government made a decision to withdraw from a portfolio wide engagement with the Bank, affecting other ongoing Projects such as the Malaria Control Booster Project and the National Urban Water Sector Reform Project. 2 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved in the Project Appraisal Document, PAD) The PDOs were i) to reduce the risk of HIV infections by scaling up prevention interventions, and ii) to increase access to and utilization of HIV counseling, testing, care and support services. For ease of measuring progress, the indicators were categorized under one of the three PDO categories. PDO1: Reduction of risk through prevention: 1) Percentage of young women aged 15-24 reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a non-regular partner in the last 12 months); 2) Percentage of young men aged 15-24 reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a non-regular partner in the last 12 months); 3) Percentage of women aged 15-49 who have had sexual intercourse with more than one partner in the last 12 months; 4) Percentage of men aged 15-49 who have had sexual intercourse with more than one partner in the last 12 months; 5) Percentage of most-at-risk groups (Brothel-Based Female Sex Workers) reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a nonregular partner in the last 12 months); 6) Percentage of most-at-risk groups (Non Brothel-Based Female Sex Workers) reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a nonregular partner in the last 12 months); 7) Percentage of most-at-risk populations (FSWs) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission; 8) Percentage of most-at-risk populations (MSM) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission; 9) Percentage of most-at-risk populations (IDUs) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission. PDO2: Access to HIV counseling, testing, care and support services: 10) Percentage of persons aged 15 and older who received counseling and testing for HIV and received their test results; 11) Percentage of female sex workers who were tested for HIV in the last calendar year (new indicator added in 2015 at the third Project restructuring) PDO3: Utilization of HIV counseling, testing, care and support services: 12) Percentage of pregnant women who are HIV positive and require antiretroviral treatment who receive a complete course of antiretroviral prophylaxis to reduce the risk of MTCT. Although prophylaxis helps reduce risk of HIV transmission, the focus of the indicator is on documenting access and utilization of services among women with HIV, one of the main beneficiaries targeted by the Project. 13) Ratio of current school attendance between orphans and non-orphans. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 7. PDOs were not revised, but changes to PDO and intermediate indicators were undertaken twice. 3  First Project Restructuring, 2011. The number of PDO indicators changed from 12 to 11: 1 was dropped2; 3 were revised with lower end-of-Project targets to make indicators more realistic with the implementation progress; 1 was updated with a corrected baseline; 1 was revised with a lower end-of-Project target as well as with a corrected baseline; 1 was rephrased from a percentage to a number measurement for more specificity and alignment with UN reporting; 1 was redefined from a percentage to a number measurement to match Bank-mandated core indicator requirement; and 4 continued without changes. The number for the intermediate indicators changed from 18 to 13: 9 were dropped; 3 were revised; 4 new ones were added; and 3 continued without changes. The list of changes is summarized in Table 2 of Annex 10.  Third Project Restructuring, 2015. Changes were made to make the Results Framework (RF) more measurable and to better reflect progress of key Project indicators. The number of PDO indicators changed from 11 to 7, and the number for the intermediate indicators remained at 13. Of the 11 outcome indicators, 5 were dropped 3; 1 was updated with new baseline data; 1 new outcome indicator was added that better aligned with the PDO since it could monitor HIV testing scale-up among FSW target population; and 5 continued without changes. The list of changes is summarized in Table 2 of Annex 10. 1.4 Main Beneficiaries 8. The intended primary beneficiaries were to be multifold. First, women above 15 years of age who made up 61.5% of all adults living with HIV. Second, vulnerable people – people living with HIV/AIDS (PLWHA) who needed access to services that managed opportunistic infections and provided nutritional supplements, and Orphans and Vulnerable Children (OVC) who needed increased schooling, counseling and support services – were anticipated to gain from the Project. Third, MARPs and youths representing major drivers of the epidemic were to benefit from increased focus on HCT, Behavior Change Communication (BCC), and condom social marketing. Beyond these particular beneficiaries, the majority of Nigerians were also expected to directly and indirectly benefit from the Project given the impact of HIV/AIDS within the generalized population. 1.5 Original Components (as approved) 9. Component 1: Expanding the Public-Sector Response (estimated cost at appraisal: US$68 million in IDA financing). This component was to support HIV/AIDS activities as part of the NSF and was to be carried out by selected key Federal and State Line Ministries (LMs) that can scale-up ongoing high impact, evidence-based interventions that targeted their clients, and to support the implementation of their strategic HIV/AIDS workplans. 10. Component 2: Expanding the civil and private sector response to the HIV/AIDS epidemic (estimated cost at appraisal: US$79 million in IDA financing). This component was intended to provide financing to civil society and private sector through the HIV/AIDS Fund (HAF) for prevention, care and support activities at the community level. It was to provide for technical 2 Ratio of current school attendance between orphans and non-orphans – dropped because of measurement and collection complexities by the Government and UNICEF. 3 The percentage of (i) women and (ii) men aged 15-49 who had sexual intercourse with more than one partner in last 12 months, and percentage of most-at-risk populations – (iii) FSWs, (iv) MSM, and (v) IDUs – who both identify ways of preventing sexual transmission of HIV and who reject misconception about HIV transmission. At the time of the restructuring, progress towards meeting the targets for these indicators were slow with the most recent data being from 2010. Because the population based surveys on which data for these indicators relied were infrequently conducted, the Project had difficulties in tracking their progress. In addition, the follow-up surveys would not be completed until after the Project closed. Given these challenges, the Project decided to drop these indicators. 4 assistance, training and implementation to national and local non-governmental organizations (NGOs) and community-based organizations (CBOs), associations of PLHIV, religious organizations, and private sector to prepare and implement programs that they propose. Funding was to be allocated in line with HAF guidelines which detail the eligibility criteria, application process, responsibilities and accountabilities and M&E requirements of proposals. 11. Component 3: Strengthening mechanisms for project coordination and management (estimated cost at appraisal: US$78 million in IDA financing). This component was to support the development and implementation of a comprehensive and systematic capacity building and supervision plan that addressed the needs of implementers (LMs, CSOs, and private sector) to deliver the public sector and civil society response; and NACA and SACAs to fulfill their coordination mandate, including the provision of technical support to LMs and CSOs. NACA held the overall coordination responsibility for the Project, tasked with overseeing the implementation of the public sector component, the HAF requests, and overall M&E. At the State level, the SACAs were designated to develop priorities for State-level action; review and approve proposals prepared by State LMs, as well as CSOs requesting state-level HAF funding; monitor epidemics in the State; and evaluate the performance of State implementing agencies. With regards to capacity building, the Project aimed to deliver strengthened evidence-based planning, increased coordination, harmonization and alignment by all stakeholders, and stronger, sustainable and more responsive HIV/AIDS financing and programming. 1.6 Revised Components 12. While no components were deleted or added, changes to recipients of funds within Component 1 were undertaken as part of the second restructuring that entailed reducing funding from 13 to 4 high impact LMs. The move was intended to shift additional resources to LMs already involved in scaling up and strengthening ongoing high impact service delivery activities, especially where service delivery gaps existed. These included the Federal Ministry of Health (FMOH), Federal Ministry of Education (FMOE), Federal Ministry of Youth and Sports (FMOYS), and Federal Ministry of Women Affairs and Social Development (FMOWSD). Also, unallocated funds were assigned accordingly, including towards priority interventions. Since the unallocated amount had already been built into each component at the design stage, the reallocation did not change overall component allocations. Along this line, 70% of newly allocated resources in Component 1 was assigned to the FMOH, thereby allowing the Project to focus on high impact services such as the scale-up HCT and PMTCT services. Under Component II, support focused more specifically on CSOs that implemented high impact services. Thus, the newly allocated resources created better targeted and intensified activities for high risk groups, while continuing prevention activities for the general population (see May 23, 2014 Restructuring in the next section for details). 1.7 Other significant changes 13. The first Level II restructuring (approved by the Country Director, CD, on December 16, 2011). The changes involved the following:  Extended Closing Date. The project was due to close on December 31, 2013. Because of the 20-month delay between Project approval (June 16, 2009) and project effectiveness (February 25, 2011), the Bank team and FGON had only three years to implement the Project and to achieve the PDOs. FGON therefore requested to extend the project Closing Date by 23 months to November 30, 2015.  Updated RF. See Section 1.3. 5  Revised disbursement estimates. Formally revised disbursement estimates in line with the revised implementation period. The revised schedule is in Annex 12. 14. The second Level II restructuring (approved by the CD on May 23, 2014). Based on findings from the June 2013 Government-World Bank quality enhancement review (QER) and the November 2013 Project mid-term review (MTR), the restructuring was intended to immediately accelerate implementation with focused effort on scaling up HIV prevention and service delivery in a more targeted manner. The changes undertaken were as follows:  Reduced the number of implementing LMs from 13 to 4 under Component 1.  Designated unallocated resources under each component (US$15 million) to priority interventions by adjusting disbursement categories accordingly4.  Conducted a safeguard audit. To avert a safeguards red flag, the FGON agreed to (i) strengthen institutional capacity of project staff; (ii) recruit environment and social safeguards officers at the national, state and local levels; and (iii) design an easy to use template for safeguards reporting (see additional discussion under Safeguards in Section 2.4).  Established framework contracting for the procurement of certain goods. The framework contracts with manufacturers and suppliers of HIV test kits, consumables and condoms was intended to (i) facilitate procurement of goods; (ii) ensure the quality of products; (iii) control costs; and (iv) improve efficiency of the process. It was a response to NACA/SACAs findings that local bidders manipulated the bidding process and colluded to share the contracts for the supply of test kits and reagents at sky-rocketing prices. Ultimately, the United Nations Children's Fund (UNICEF) and United Nations Population Fund (UNFPA) signed an agreement developed through a contract framework that ensured quality and cost efficiencies in the purchase of supplies. Framework contracts were added to the list of procurement methods in the FA (see additional discussion under FM in Section 2.4). 15. The third Level II restructuring (approved by Regional Vice President on Nov 19, 2015). Because the restructuring resulted in extending the closing date beyond two years from the original closing date of December 31, 2013, Regional Vice President (RVP) approval was required.  Extended Closing Date by 15 months to February 28, 2017, bringing the cumulative extension to 38 months.  Strengthened RF. See Section 1.3.  Addition to safeguards. In response to a September 2015 country-wide safeguard audit, that found that many states had an unstructured and ad-hoc beneficiary feedback and grievance redress mechanism, an enhanced system was established to better clarify the Project’s benefits and communicate with stakeholders. The addition did not require amending existing safeguards instruments (see additional discussion under Safeguards in Section 2.4).  Revised disbursement estimates. The disbursement estimates were adjusted based on the remaining funding envelope during the 15-month extension period (see changes in Annex 12). Also, it budgeted an amount for Kano State which had previously opted out but then became interested in participating in the Project’s activities towards the last year and a half.  Changed Financial Management (FM) arrangement. An FY15 FM review conducted by the Bank to assess the continuing adequacy of the FM arrangement and compliance with internal 4 The US$15 million allocation is as follows: (i) assessment of FSW prevention programs – US$1 million; (ii) central procurement of test kits, consumables, condoms and lubricants – US$8.5 million; (iii) demand creation for PMTCT services including an evaluation of the impact of these services – US$4 million; (iv) operational research – US$1 million; and (v) end of project evaluation and documentation – US$0.5 million. The funds were reallocated between disbursement categories. 6 controls, outlined the need to strengthen the Project’s internal control, financial reporting, and accountability arrangements. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 16. Soundness of background analysis: The Bank participated in a joint scoping mission with DFID in 2007 to assess current and projected AIDS financing and to delineate key thematic service areas covered in varying degrees by partners. Compared to other partners, the Bank Credit served as the only source of external financing that covered a country-wide program that provided prevention, care and support services. The team considered the lessons learned from the previous project, including the importance of i) political commitment at all tiers of government; ii) strong M&E systems to scale-up effective interventions that target key sectors and high transmission populations; iii) the vibrant involvement of civil society, despite needing better targeted support; iv) greater focus in the health sector to strengthen effectiveness in delivering HIV/AIDS services; and v) building capacity and strengthening systems through partners and networks within various implementing sectors. 17. Assessment of the Project Design: The Project was tailored to support the action framework of NSF1’s final years and the more recently updated 2010-2016 HIV/AIDS National Strategic Framework (NSF2), both of which framed the national response in the context of the key sectors and thematic areas covered by several partners. This created a concerted effort in which each partner focused on their comparative advantage in contributing to the overall response. HPDP2 supplemented the work of partners at the Federal level by providing flexible funds to support prevention and care and support services as part of the national response. At the State level, HPDP2 augmented the capacity building work of the HIV/AIDS Program supported by the UNDP. 18. The original four-and-a-half-year timeframe for the Credit (June 2009 to December 2013) was guided by the framework of the country’s own multi-year NSF2. The component allocations appeared to have been based on findings from the previous Project that the Government institutions it helped establish (namely, NACA and SACAs) needed to be strengthened at the time to manage the HIV response at both the federal (NACA) and state level (SACAs) for all 36+1 States. Therefore, resource allocation for coordination and management appeared justified. Hence, a third of the funds went to Component 3. Alternative designs were considered but rejected. First, a project design solely based in the health sector was rejected on the grounds that it would not have succeeded in coordinating and stimulating a multi-sectoral multi-partner response, where the support of the World Bank was considered a comparative advantage. Second, a design focused on interventions targeting exclusively populations at increased risk of HIV infection was rejected since the Nigerian epidemic in many areas had already spread to the general population. Beyond this, political sensitivities in the country resulted in the client requesting a country-wide Project that ensured equal access for all States to the HIV/AIDS Project. Also, the Government at the time found it challenging to make MARPs a priority based on their understanding of the epidemic, even though the Bank felt that a focused approach on high risk areas and sub-groups was likely to be much more effective. There was the Government’s concern of a political backlash from supporting a project aimed at FSWs and other MARPs. To the Bank’s credit, it decided to continue engaging with the country to help promote policy changes. But as the Project went on and significant lessons were learned from the HIV/AIDS MAP Program, as well as from in-country assessments that showed the MARPs as the driving force of the epidemic, the Project transitioned to place greater emphasis on high risk groups. No Quality Enhancement Review (QER) was conducted during the preparation stage. 7 19. Risks and mitigation measures. The PAD highlighted two risks rated as substantial, both of which were related to financial management. The first entailed the Federal level’s capacity to handle the increased pressure resulting from challenges to financial oversight due to the increase in external funding and activities from numerous partners. As a mitigation measure, the Bank team proposed to strengthen NACA, LMs, CSOs, and FM systems through direct financial and technical support and on-going training. In addition, the team proposed establishing a JFA focused on supporting one FM, M&E, and reporting system for all development partners. The second substantial risk concerned the States’ overall weak FM capacity and limited expertise on Bank FM requirements. The Bank team proposed that the Project Financial Management Units (PFMUs) at the states receive ongoing support from the Bank’s FM unit, while the project accountants received regular financial management training. 2.2 Implementation Phase I (from approval on June 16, 2009 to the first restructuring on December 16, 2011, disbursed amount US$5.3 million) 20. Delayed Project Signing & Effectiveness. There was a 20-month delay period between Bank approval (June 16, 2009) and the project effectiveness (February 25, 2011). When the project was initially presented to the Federal Executive Council (FEC) for approval, the FEC raised concerns about the lack of funds allocated for treatment and questioned the high allocation designated for CSO activities. Although the Government through ongoing discussions agreed to approve the Project in November 2009, the President fell ill causing FEC meetings to be postponed or delayed, hampering opportunities to get approval from the FEC. The eventual death of President Yar'Adua on May 2010 further delayed the process. During this time, however, the task team carried out work to establish a conducive environment to successfully launch project implementation as assessed by Bank management in the Implementation Status Report (ISR). After lengthy delays arising from internal Government processes, the FEC approved the Project, which became effective on February 25, 2011. 21. Achievements and Challenges: 31 SACAs (31 of 37 at the time) had put in place the minimum Project staff required to implement the Project per Bank guidelines. Moreover, the 31 States had successfully transformed SACAs into agencies (up from 13 when the Project was prepared), thus entitling them to a line item budget in their States’ annual funding allocation. With regards to planning purposes and mapping effectiveness and efficiency of prevention programs, SACAs were beginning to utilize evidence-based epidemiologic summaries that had been prepared for States through technical assistance jointly provided by NACA and the Bank’s Project team, as well as experts from the Bank’s Global HIV/AIDS Program (GHAP) and DFID’s ENR Program. NACA worked with the Bank team to assist SACAs prepare for Project implementation. HAF contracting guidelines were also revised to strengthen coordination with CSOs and Service Support Organizations (SSOs). Beyond these, the Project showed little progress. In addition to the delayed Project Effectiveness, there were lags in implementation start off at the States due to delays with approval from the FMOF’s internal reviews that determined whether or not States met conditions for disbursement of Credit. Also, the Project showed no progress in meeting the PDOs, further hampered by a delay in conducting the Integrated Bio-Behavioral Surveillance Survey (IBBSS) and National AIDS and Reproductive Health Survey (NARHS), population-based surveys which many of the outcome indicators relied on to track progress. As a result of the delayed effectiveness and slow progress, the Project underwent its first restructuring. 8 Phase II (from the first restructuring on December 16, 2011 to the third restructuring on November 19, 2015, disbursed amount US$82.4 million) 22. A Joint Government-Bank mini-QER on June 27, 2013 and a subsequent MTR between October 30, 2013 and November 15, 2013 were conducted to assess the implementation phase (refer to Annex 11 for detailed findings). Achievements noted during the MTR mission included i) progress in four out of six PDO indicators towards reaching the end Project targets; ii) increased use of State level epidemiology to drive policy at all levels and prioritize programming; iii) FGON adoption of methodology and tools used in epidemic appraisals in States as national standard to ensure consistency in approach irrespective of the funder; and iv) increased political commitment to the HIV/AIDS response through the 2013-2015 President's Comprehensive Response Plan for HIV/AIDS (PCRP). 23. Despite these achievements, the QER and MTR highlighted the continued slow pace of implementation and disbursement that required the following resolutions: (i) revise the PDOs so that it focused only on the core deliverables aligned with priority interventions – To increase access to and utilization of HIV prevention interventions, care and support services in Nigeria – as well as to revise the RF indicators and targets; (ii) limit the number of LMs supported under Component 1 (public sector response) to four key LMs, including the FMOH which was to receive up to 70% of the component's resources to rapidly scale up HCT and PMTCT services; (iii) conduct a safeguards audit of the project and implement the actions towards ensuring compliance with the environment and social safeguards; and (iv) allocate resources from the unallocated amount of US$15 million to support an assessment of the national sex worker prevention program, central procurement of test kits, consumables, condoms and, lubricants, demand creation for PMTCT services and operational research. Through the MTR, the Project used the additional funds to increase its focus on MARPS based on findings from the epidemic appraisals that showed that MARPs drove the epidemic. During this time, the Project rating for progress towards achieving the PDO was MS, while the rating for implementation progress was MU, along with risk flags for the following categories: Country Record, Project Management, Financial Management, Slow Disbursement, and Safeguards. 24. Second Restructuring: Through the restructuring, the Project took action on all resolutions made during the MTR, except for changing the original PDOs based on management decision due to the passage of the 2013 Same-Sex Marriage (Prohibition) Act in January 2014. The Act not only coincided with the second restructuring, but more importantly made those who engaged in same sex relations more vulnerable to being arrested. The Bank instead undertook a Level II restructuring to address immediate actions deemed to quickly improve Project implementation. Due to the political sensitivity in the country at the time, it took longer than anticipated to approve the second restructuring after the MTR. With regards to implementation, the MTR and the ensuing second restructuring refocused activities towards greater emphasis on high impact activities and in reaching MARPs. The overall disbursement rate moved from 31.5% at the MTR to 46.6% one year after the MTR. This compares to a 15.1% rate in one year after the MTR compared to a 7% rate per year for the period between Project approval and the MTR. The jump in disbursement points to the impact of the changes undertaken in the MTR and ensuing restructuring. 25. Achievements and Challenges: The second restructuring spurred the following activities under Component 1: SMOHs scaled-up HCT and PMTCT services in health facilities across the country, which contributed significantly to achieving the HC and PMTCT targets; the Ministry of Women’s Affairs provided care and support services to PLHIV and OVCs; the MOE in addition to coordinating FLHE in schools, were able to train students in tertiary institution to provide HCT and other HIV prevention services on campus, also contributing to reaching HCT targets; and the Federal 9 and State Ministries of Youth and Sports added to its ongoing work on HCT with out-of-school populations, using the new support to work with the National Youth Service Corps (NYSC) on establishing anti-HIV/AIDS community development services (CDS). Under Component 2, a rapid assessment conducted by the Bank on HAF implementation in 6 States indicated that CSOs were focused on the appropriate target population using recommended national guidelines to deliver package of interventions to clients. More than half of the CSOs were providing preventive services to MARPs and pregnant women compared to 14% in the previous Project. However, private sector engagement had been very limited, particularly in the area of clinical HIV services. The Project had not leveraged the large number of private providers in Nigeria to expand service delivery of HIV/AIDS especially PMTCT services. The private sector also provided 40% of treatment and care to the general population, as well as accounted for as much as 43% of facility-based deliveries. Moreover, safeguards and FM remained problematic based on findings from a 2015 country-wide safeguard audit and a 2015 FM review (refer to Section 2.4). The Bank and Nigeria agreed to a third restructuring to address the remaining shortcomings. Phase III (from the third restructuring on November 19, 2015 to Project Closing on February 28, 2017, disbursed amount US$98.3 million) 26. Performance-based financing (PBF). During the extension period, the Project engaged with the private sector through a PBF scheme to pay private health providers and implementation partners (IPs) for results on pregnant women. The decision to implement PBF, which had not been in the original design, worked well in bringing in PHFs, likely due to the clear and straightforward transaction involved in financing results. Specifically, PHFs received funds upon verification on how many pregnant women received PMTCT services. HPDP2 successfully leveraged PHFs ability to offer services directly to those infected and affected by HIV/AIDS, going beyond the engagement with multinational corporations in HPDP1, thereby effectively expanding the cadre of service providers and increasing access to and utilization of services. During a span of 5 months, close to 100,000 pregnant women in 13 States were counselled, tested, and received results, of whom 1% were referred to for PMTCT. 27. Donor Coordination and Harmonization. The Bank through GHAP, University of Manitoba, and LSHTM provided technical support over a 6- year period that helped the Government generate and apply evidence base for planning HIV prevention programs at the local level. As a result of the long-term engagement and mentoring, NACA and SACAs were able to design and implement local epidemic analyses for use in guiding the development of national HIV prevention workplans, strengthening the county’s capacity to coordinate and manage the Project (Component 3). Moreover, collaboration with GFATM and USG resulted in these donors adopting the tools developed by the Bank and NACA for the local epidemic appraisals as part of their own assessments. 28. Country-wide Project implementation. From the outset, implementation support to 36+1 States remained a challenge. Delays in contracting staff specialists, which further hampered the already weak country capacity, only amplified difficulties involved, including managing HPDP2 with thinly spread resources. While the Project did take action towards increased emphasis on reaching MARPs, as well as focused its support to LMs with the greatest impact, the overall implementation remained country-wide. Although the Bank team stepped up to provide necessary support, it faced hurdles itself. Up until the MTR, one in-county Task Team Leader (TTL) was assigned to the Project with support from an in-country operational officer. A request from the TTL for additional budget to recruit technical consultants resulted in hiring two Short Term Consultants (STCs) in 2013. Each STC supported 16 states and the FCT, a demanding task given the coordination and capacity problems at SACAs. While the team stretched itself to meet demands as accounted by the country team and 10 Government representatives, it was consumed with responding to a multitude of No Objections from 36+1 states, NACA, and four LMs on a regular basis, leaving little time to provide quality technical assistance. Prior to the MTR, a Washington-based TTL took over the Project and frequently traveled to Nigeria for support. While not unusual for the Bank to assign Project TTLs not stationed in- country, the massive scale of HPDP2, as well as the limited country capacity, necessitated placing an in-country presence. Despite challenges, the daily support provided by the STCs helped address bottlenecks such as improving coordination of LM activities and reducing duplication efforts across various State-level implementation partners of the United States Agency for International Development (USAID) and GFATM programs. The STC support also dramatically advanced awarding of HAF grants. 2.3 M&E Design, Implementation and Utilization 29. M&E design. The Project aimed to support M&E strengthening efforts at the national, state, and local levels, including civil society and the private sector, initiated under the previous HIV/AIDS development project. HPDP2 also intended to collaborate further with development partners, such as the DFID, UNAIDS, and USG, in order to harmonize and integrate a cohesive, coherent national M&E system that can assist with progress measurement, accountability, learning and planning. Overall, NACA had the responsibility of monitoring the epidemiologic situation in the country and of evaluating the national response. Its role extended to providing M&E technical assistance to the States. When HPDP2 started, all 35 Project States through the SACAs and most Ministries, Departments and Agencies (MDAs) already had existing M&E Units and/or designated M&E focal persons through World Bank support. In addition, HPDP2 was designed to support stronger collaboration between SACAs, development partners, and research and academic institutions through the establishment of M&E Technical Working Groups (TWGs) where there were none present, and strengthen coordination where TWGs already existed. Program monitoring (dashboard and other output indicators) and financial reporting were to be accomplished through a harmonized reporting template already agreed to by NACA, SACAs and the Bank. The system design was intended to generate clearly defined set of products on a quarterly, annual or periodic basis to allow information to be used strategically. To ensure that implementation at the state level succeeded, joint bi-annual supervision missions were to be organized with key partners, including NACA, DFID and Federal Ministry of Finance (FMOF). The Bank project team also conducted quarterly joint supportive TA/supervision missions with NACA. 30. The Project’s M&E activities aligned with Nigeria’s 2007 National M&E Operational Plan that served as the country’s HIV Response M&E Roadmap. In addition, the design took advantage of the existing M&E framework under the Nigeria National Response Information Management System (NNRIMS) to collect routine outcome and impact level indicators consistent with NSF1, United Nations General Assembly Special Session on AIDS (UNGASS), and Millennium Development Goal (MDG), as well as of the Logistics and Health Program Management Information Platform (LHPMIP) for a real-time data reporting at the local facility level. Routine monitoring and reporting data on health and non-health indicators from the NNRIMS and the LHPMIP systems were compiled into the national District Health Information System (DHIS), Nigeria’s national HMIS. The decentralized architecture aimed to strengthen capacity of sectors at national, state and local levels, while at the same time create one national M&E framework at all levels. Particular data points on HIV prevention, care and support services incorporated into the RF were to be tracked using periodic population surveys, including the IBBSS to monitor progress in intervention coverage, particularly of HCT and ART, and the NARHS. 11 31. Exploratory studies were conducted to help the development of the Project RF. The outcome indicators focused on measures that assessed i) behavioral practices of population groups known to likely participate in risky behaviors such as those aged 15-24 and MARPs (sex workers, MSMs, and IDUs); and ii) access to HCT among those infected and affected by HIV/AIDS, including HIV positive pregnant women (a mandatory World Bank core sector indicator monitored across projects and countries for corporate reporting), orphans, and MARPs. In total, the RF included 12 original outcome indicators that fell into one of three categories that helped monitor progress in reducing the risk of HIV infections (PDO I) and increasing access (PDO II) and utilization to HCT (PDO III). The number and selection of indicators were reflective of the complexity and impact of HIV/AIDS in the population. The design relied on already existing M&E systems and agreements (as in the case of future population surveys to be conducted by third party entities) to both monitor the performance of NACA and SACAs and measure the Project’s progress. Survey data were anticipated to take place every two years. In the end however, data verification for 5 of the original 12 (42%) outcome indicators became problematic due to infrequent/delayed data collection. For example, the 2007 IBBSS was used for baseline data on 5 indicators; but since only one other IBBSS was eventually conducted in 2010 to track progress and given the Project’s life of 2009 to 2013, it was limiting in terms of quickly adjusting programmatically based on emerging conditions. 32. M&E implementation. The RF underwent revisions as part of the first restructuring in 2011 and third restructuring in 2015, as described in Section 1.3 above. The change from 12 to 7 PDO indicators by the end of the Project also ensured consistency with the government’s own monitoring frameworks. Importantly, the changes to the RF did not reflect a change in scope of the project. Instead, it represented the need to adjust and correct data to make them consistent and relevant to measuring the PDOs. Given the ongoing problem with Safeguards (see Section 2.4 for details), there was a missed opportunity after the first and second restructurings to incorporate safeguard-specific indicators to help track progress and address challenges. 33. NACA and FMOH progressed throughout the first two years after project effectiveness in conducting regular data quality assurance reviews. At the same time, routine reporting and monitoring at the State level improved. One key aspect of HPDP2 was its use of mobile reporting at the facility level, including non-health sector and CSOs; this helped with more timely and accurate reporting compared to sending in paper reports. The need to improve their submissions toward more result- based reporting however was identified by the Bank, who agreed to support hiring two M&E consultants to review the Project's monitoring framework with a view of improving state targets for indicators and reporting systems. Nigeria also conducted an assessment of HAF Implementation in 18+1 States to find out whether (i) CSOs were implementing activities as agreed in their contracts, targets were being met, and activities documented; (ii) Performance Monitoring SSOs were delivering as per their signed contracts; and (iii) PIUs at SACA and NACA were performing their roles as per their mandate to deliver a successful HAF. 34. During the implementation stage, Nigeria collaborated with the Bank and several partners to conduct various assessments. For example, the Project financed State epidemic appraisals that mapped the locations of MARPs and estimated their sizes, as well as provided profiles of venues for high-risk activities used by higher risk segments of the general population (beyond MARPs). The appraisals aimed to strengthen prevention programs put into place for the MARPs and to better direct the grants provided to CSOs intended to support MARP prevention programs in States. Results confirmed that MARPs drove Nigeria’s epidemic, mainly in high burden States, accounting for a much as 70% of the HIV burden in Nigeria. Moreover, NACA and the National AIDS and STI Control Program of the Federal Ministry of Health (NASCP) received joint technical support from the Bank, London School of Hygiene and Tropical Medicine (LSHTM), and the University of 12 Manitoba to design an impact evaluation of Nigeria’s female sex worker HIV prevention program that could be implemented. The design mainly focused on 12 + 1 key States identified by the Government as priority States for HIV prevention efforts because they accounted for about 70% of the HIV burden in the country. The ability to increase the coverage of HIV prevention programs among sex workers and assess the impact is indispensable given that Nigeria is among the countries with less than 25% service coverage for sex workers despite the 24% HIV prevalence rate among the group, compared to the 4.1% prevalence among the general population. 35. The IBBSS and NARHS conducted in 2007 provided data for the baselines. It was anticipated that during implementation, follow-up IBBSS and NARHS would provide updated data. However, delays in the availability of up-to-date survey data, mainly due to the time it took for NASCP to release survey results, posed a challenge throughout the life of the Project. Ultimately, updated NARHS results became available in 2012 and IBBSS in 2015. At the end of the Project, however, the slow progress on completing the final update to NARHS meant that results would not be available until 2018, well after the Project closing. In order to not miss out on the opportunity presented by conducting a multi-indicator cluster survey (MICS) that included the same NARHS indicators used in the RF, the Bank and NACA supported UNICEF in conducting the 2017 MICS with a sum of US$1 million; results from the survey provided final Project data. A Key Population Assessment (KPA) was also conducted at the end to provide updates to indicators that previously relied on the IBBSS. While the Project should be commended for proactively seeking alternative options for collecting end of Project data, comparing data findings from different surveys raises issues of sampling inconsistencies and modeling assumptions. 36. Utilization. Routine surveillance data (NNRIMS) and populations surveys (IBBSS, NARHS, MICS, and KPA) were utilized to assess progress in meeting the RF targets. Other reports like the Quarterly Implementation Progress Report (QIPR) and State-specific HIV/AIDS, Reproductive Health and Child Health Survey (SHARHS), which was carried out in 18 States November 2011, also provided valuable input. Nigeria also benefitted from results of the Bank-supported epidemic appraisals. Findings from the appraisals provided the FGON a greater understanding of the HIV/AIDS epidemic in Nigeria and the importance of using epidemiologic data to drive evidence- based policy. The FGON responded by dramatically shifting the focus of its national response, including (i) the prioritization and scale-up of HIV prevention programs for FSWs, MSM, and other MARPs; (ii) more emphasis on the PMTCT; (iii) prioritization of States with a high HIV burden that included 12 States plus the FCT (referred to as the 12+1 States); and (iv) less effort on ineffective general HIV awareness raising. These were incorporated into the 2010-2015 National HIV/AIDS Strategic Plan, affecting how the national response was managed by implementers and global partners. The findings also helped generate State-specific epidemiological summaries that the States used to strengthen prevention programs put in place for the MARPs, as well as assisted with identifying thematic priorities to be used by CSOs in developing proposals to be funded by the HAF. In general, the technical, process and thematic specific reports were used to guide the development of the next set of NACA and State-level operational plans and to develop policy briefs for wide dissemination. 37. Overall, Monitoring and Evaluation was Modest under the Project. 2.4 Safeguard and Fiduciary Compliance 38. Safeguard. One safeguard policy was triggered by the Project: Environmental Assessment under the Bank’s Operational Policy/Bank Procedure 4.01. The Project received a Category B environmental rating at the appraisal stage. The potential environmental and social impacts of sub- 13 projects under components 1 and 2 were considered small-scale, site-specific, and easily remediable. The national Medical Waste Management Plan (MWMP) and project specific medical waste management plan were to provide guidance on the management of medical waste generated by the Project. While the Project was anticipated to generate some hazardous medical waste in HIV/AIDS testing and treatment centers and with building renovations and constructions, an Environmental and Social Management Framework (ESMF) was prepared to address these concerns. Furthermore, HPDP2 was not intended to fund activities that caused adverse environmental effect, land acquisition, or restriction of access to sources of livelihoods. The MWMP and ESMF were prepared with extensive consultation with stakeholders, including the private sector and CSOs. 39. The absence of safeguards implementation did not come up at all both at the NACA and Bank until the preparatory activities for the MTR in 2013. The MTR highlighted safeguard weaknesses, including i) delay in recruiting technical consultant and focal persons to coordinate between the Federal and State-levels and ensure that disposal of HIV/AIDS related medical wastes at service delivery sites took place; ii) missing baseline data and indicators for reporting on Safeguard Compliance; iii) delays by NACA in reporting and submitting collated state reports, as there was no basis for reporting due mainly to the absence of baselines; and iv) absence of a hot line where beneficiaries and citizens of States can report any anomalies found and or concerns during implementation as highlighted in the PAD. These gaps resulted in non-compliance of the safeguards legal covenant. The Borrower’s ICR highlighted that there was no expert at the Project launch and through the first half of the Project’s life to guide Project implementers. Consequently, the MTR recommended an expanded Safeguard Audit that covered all the Project States and FCT. Moreover, safeguard focal persons were accordingly appointed at various levels to ensure adequate attention to safeguard issues with the preparation of safeguards audit. The safeguard action plan included in the audit report contained the following specific mitigation measures for implementation tied to responsible parties and their budgets: i) designation of environmental and social safeguards officers at NACA and SACAs; ii) capacity building for project staff, especially safeguard officers and medical waste management at designated health facilities; iii) procurement of personal protective equipment for staff; iv) establishment and integration of an environmental and social baseline data and indicators into the M&E framework; v) development of a grievance redress and beneficiary feedback mechanism; and vi) designing of an easy to use template for safeguards reporting. Implementing the plan required enhanced implementation supervision support from safeguards specialists at the Bank (mission & virtual). In addition, Government carried out a compliance audit of the safeguards action plan to ascertain the level of conformance to the agreed terms, six months into the last project extension. 40. Financial Management. Generally, there were no delays experienced in funds flow from the Bank to the States, and disbursement to beneficiaries was equally smooth and effective. However, a new Government in 2015 mandated that all funds from external donors be combined into a Single Treasury Account (TSA), which dissolved individual donor balance accounts. The policy made it difficult to disburse allocations specifically designated for HPDP2, affecting funding needs for NACA, which in turn slowed disbursement and implementation of overall activities. During this time, the Project could not disburse for specific activities as agreed on workplans for LMs, CSOs, and PHFs. It took the Government almost 5 months to resolve the problem by redirecting funds back to relevant programs. 41. IFRs from most States were submitted in a timely manner; however, desk reviews across implementing agencies revealed several deficiencies in the reports, including inadequate documentation and adherence to Bank/NACA guidelines, inefficient budgeting process, and poor management of funds. As a result of IFR review, in-depth financial reviews in 8 states were 14 recommended. These in-depth reviews identified fiduciary issues, including ineligible expenditures, insufficient documentation, and suspected fraudulent transactions. Overall, 82.3% of all the questionable or ineligible expenses (US$1,738,956) originated from Borno, Sokoto, and Niger States. 42. The Bank recommended remedial actions for improving FM arrangements, internal control, and accountability in States. The changes entailed (i) establishing a Financial Management Accountability Framework that was operational for the remaining life of the project; (ii) building capacity of internal auditors to conduct risk-based internal audits, (iii) NACA and States preparing IFRs and submitting to the Bank quarterly, and (iv) replacing audited consolidated financial statements with individual annual audited financial statements to be submitted by each State PIU within 6 months after the end of the year cycle. The extension of the Project allowed the Government to address these issues and implement more resilient actions to strengthen the FM process at NACA and States level, as well as provided sufficient time to recover unjustified or ineligible expenditures. NACA and SACAs responded quickly and effectively, with NACA working with all the States involved to ensure refund of the ineligible sums. By the end of the Project ineligible expenditures from NACA and the States were refunded back to the project. 43. Procurement. The weak procurement capacity at NACA and at some states caused delays in implementing key procurement packages such as the HAF contracts and purchase of commodities. The No Objection process experienced long delays, estimated at around two months due to protracted response times from both NACA and the Bank. This in turn hindered designated implementers from receiving funds for workplans that had already been approved. The delays, which lasted as long as six months according one LM, in acquiring funds meant that as much as 50% of agreed activities were not undertaken or completed based on estimates provided by interviews with the Bank team and implementers. Since workplans were not tied to disbursements, there was a disconnect between what might have been approved in the workplan versus how much funding was received. And due to the rapid pace of inflation in Nigeria, any delayed funds became insufficient to cover predetermined cost of activities, particularly towards the last year of the Project. 44. A World Bank Post Procurement Review of approved procurement plans from States revealed that the weak capacity led to inadequate compliance of the Bank's Procurement Guidelines. In response to the slow progress of awarding HAF grants, the Bank continued to provide additional TA to the states for guidance and to help with ensuring that HAF grants were processed and awarded as quickly as possible to qualified CSOs and for evidence-based interventions in line with the epidemic of each state. In addition, the Bank approved terms of reference (TOR) for hiring two Procurement Specialists to fast track these packages. Improved capacity was quickly evident even after only hiring the first Specialist, as the number of states with signed HAF contracts increased from two to 25 within six months. 45. Apart from the key challenges pertaining to patent, algorithm, poor quality of the bidding documents and bid evaluation reports, a review by NACA and SACAs identified problems with bidders manipulating the bidding process and colluding to share the contracts for the supply of test kits and reagents at sky-rocketing prices. In response, NACA and the Bank put in place an ongoing procurement training and hands-on mentoring by Bank procurement specialists. Furthermore, SACAs, with support from NACA and the Bank, developed a framework contract for the procurement, delivery, and distribution of test kits, reagents and lubricants that ensured quality and cost effectiveness, as well as enhanced fiduciary compliance. To this end, NACA signed agreements with UNICEF to provide test kits and drugs to treat sexually-transmitted infections and opportunistic infections (STI/OI) and with UNFPA to provide condoms and lubricants to each State based on State- specific needs. The budget envelope for Federal level procurement of commodities including delivery 15 to the States totaled US$8.5 Million (US$4.25 million for test kits; US$2.95 million for condoms and lubricants; and US$1.3 million for STI/OI drugs). The direct purchase and supply of commodities through UNICEF and UNFPA created a more accountable and streamlined process for NACA and SACAs that alleviated pressures on staff resources and transaction cost. As the Project progressed, compliance with the Bank’s guidelines and procurement procedures improved. The procurement units developed capacity to ably process World Bank No Objections for prior review contracts, including procurement plans, and clear all post review contracts during World Bank post review missions. The quality of the implementation of procurement activities by the Borrower during implementation was reviewed during the last support mission before project closure and was rated ‘satisfactory’. 46. Disbursement. As illustrated in Annex 1, the Project disbursed US$186.04 million (or 82.68% of original IDA financing) and cancelled the unutilized amount of US$23.74 million. With regards to FGON’s planned contribution of US$5 million, the Government disbursed US$4.35 million for various support, including the cost of staffing NACA, as well as individual contributions of US$100,000 from each state as a precondition for participating in the Project. 2.5 Post-completion Operation/Next Phase 47. The Project contributed to strengthening further the established infrastructure under the HIV/AIDS national response. Specifically, the Project supported the full transition of all 36 SACAs to agency status and advanced the Government’s commitment to a multisector response through stronger links with the CSOs and private sector. Despite the achievements of HPDP2, Nigeria needs to continue its battle against the HIV epidemic given the high proportion of disease burden that remains globally attributable to Nigeria. While the infrastructure has been put in place, capacity of the SACAs remained ineffectual and required extensive Bank support through to the end. To compound the challenges in responding to the epidemic, the financing landscape in Nigeria is experiencing decreased contribution from donors it had relied on in the past. 5 Along the decreasing revenues due to falling oil prices, the Government’s financing constraint led to a new request on December 2016 for the third HIV/AIDS Program Development Project in the amount of US$300 million. The request is being considered by the Bank within the framework of its new multiphased approach to supporting the health sector. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 48. Relevance of objectives: Substantial for all Phases. The PDOs reflected the priorities of the Bank’s 2010-2013 CPS for Nigeria that underscored the daunting task of combating HIV and the need for Nigeria to reduce HIV/AIDS prevalence and provide adequate care to PLWHA. The two PDOs were sufficient to cover specific indicators focused on reducing transmission of HIV, including among several MARPs, as well as indicators that track provision of HIV counseling, testing, and care services. At the time of the Project closing, the PDOs remained consistent with the 2014-2017 CPS that asserted the importance of increased access and utilization of quality services for infectious diseases that represented a significant burden of disease. In particular, the CPS stated the need to increase the proportion of pregnant women with HIV who receive ART to reduce risk of MTCT, which the Project further scaled-up through additional funding and support after the MTR. The Project also contributed to increasing the proportion of adults who received an HIV test. With regards 5 PEPFAR commitment down from US$503 million in 2015 to US$469 million in 2016 and focused only on three States; GFATM funding now solely focused on providing services to Lagos State; and DFID financing for HIV/AIDS effectively terminated in December 2016. 16 to the country’s own development goals, the PDOs reinforced the principles of the Nigeria’s 2010- 2015 National Strategic Health Development Plan (NSHDP) that highlighted working jointly towards complementarity between health and related sectors, expanding utilization and delivery options and coordinating technical assistance. The shift in the Government’s understanding of the HIV/AIDS epidemic, which came about as a result of the Bank-financed epidemic appraisal, was reflected in the 2010-2015 National HIV/AIDS Strategic Plan and the 2013-2015 PCRP. Key lessons from the Project’s success provided valuable input into the new 2017-2022 NSF (NSF3). To the Banks’ credit, it responded to the emerging situation, as evident in the MTR and restructurings, by refocusing the Project towards priority, high impact services. 49. Relevance of design: Substantial for all three phases. The overall institutional architecture of HPDP2 remained similar to the previous project so as to take advantage of the progress made in establishing the coordinating and implementing institutions under HPDP1. The Project provided ongoing support to SACAs, which the Bank helped establish under HPDP1. Given the sub-optimal capacities at SACAs, the support signaled the Bank’s commitment to strengthening the national response since SACAs remained responsible for implementing many of the activities that reached the beneficiaries, through their coordination with local CSOs, PHFs, and State LMs. With regards to who should be targeted by the Project, the Bank team was aware that there was mounting evidence from Projects elsewhere that a focused approach on high risk areas and sub-groups was likely to be much more effective. However, the geopolitical sensitivities in the country at the time of appraisal made it improbable to design such a project. As mentioned above, there was also a conscious decision to stay engaged country-wide in order to affect change in the Government’s whole approach through constructive policy engagements. It was clearly felt that not engaging with the Government in that way at a time when Nigeria was experiencing the second largest HIV epidemic in the world was not possible. Also, the Bank’s continued support in the low capacity setting helped ensure that hard gained achievements at the time were not lost. This strategy adopted by the Bank team was clearly successful since it was a large part of the reason that the FGON completely altered its entire approach to HIV control. 50. Relevance of implementation: Substantial for all three phases. The Project experienced a rebalancing of its focus after the MTR due to findings from the Project-financed epidemic appraisals. The analyses revealed that MARPs and their sexual partners accounted for about 50% of new infections despite only representing less than 5% of the population, much more than previously thought. Moreover, a high percentage of married and unmarried men and women participated in transactional sex with more than one sexual partner. The appraisals also highlighted further how prevalence vary across States. This led the FGON to better understand the epidemic and to characterize it as a mixed and complex situation that required varying interventions that addressed key sources of new infections. In response, the FGON revised its National Prevention Plan to reflect this new knowledge and defined a minimum package of combination prevention packages specific to the key drivers of the epidemic, and provided an updated 2010-2015 National HIV/AIDS Strategic Plan that established a new commitment to an evidence-based approach to planning and implementing interventions. 51. HPDP2 increased efforts to engage the private sector and CSOs to provide HIV services to key populations, which contributed significantly to achieve the PDOs. It should be noted that the Bank was the only partner to work with CSOs and non-health sector as part of the national response. Moreover, incorporating PBF into the payment scheme fully engaged the private sector and represented the first of its kind under the national response. It was not only successfully implemented, but it also contributed to improving service delivery dramatically. Despite PEPFAR’s and GFATM’s work on prevention activities, only the Bank-financed Project implemented prevention measures that 17 targeted MARPs. Other significant impacts that came out of the HPDP2 related to the expansion of the use of mobile reporting to other disease projects, including TB, malaria, and others, as well as its introduction into programs managed by GFATM. The Project further leveraged Government support to the national response by supporting the full transition of all SACA’s to agency status, thereby ensuring capacity building, as well as providing the infrastructure to commit State finances for HIV/AIDS related activities. The approach undertaken by the Project was considered by the international community as effective and efficient, including the focus on interventions targeting MARPs and the inclusion of PMTCT services, both for which the Bank Credit served as the main source of funding. The focus on increasing PMTCT coverage, in particular, remained globally imperative since Nigeria represented approximately 30% of the global PMTCT coverage gap. 52. The overall ratings for Relevance is Substantial for all three phases. 3.2 Achievement of Project Development Objectives PDO I: Reduce the risk of HIV infection by scaling up prevention interventions. 53. Based on outcome indicators that contributed towards achieving PDO I, the rating for Phase I is Negligible, for Phase II is Modest, and for Phase III is High. Table 1. Summary of PDO I Achievement Level of Achievement Phase I Phase II Phase III (2009 –2011) (2011 –2015) (2015 –2017) PDO Indicators Surpassed (100%+) 3 4 4 Achieved (85-100%) 0 0 0 Not Achieved 6 5 0 Total 9 9 4 % Surpassed & Achieved 33% 44% 100% Rating N M H 54. The focus of the objective is on reducing risk. Most of the progress occurred after the MTR. Condom use among young men, young women, and FSWs all surpassed the end-of-Project target. The observed change in behavior practice was temporarily correlated with the downward trajectory in prevalence from 3.3% to 2.9% among adults aged 15-49 years over the same time period between 2011 and 2016. Comparing the counterfactual based on the past trend with the number of new HIV infection in adults (15-49 years of age) actually observed during the life of the project, the project could have prevented as many as 192,700 new infections over five years (see Annex 3 for calculations). 55. Among FSWs, the IBBSS survey in 2014 found that 9 in 10 used condoms with casual partners. It was reasonable to expect that the high rate of condom use reduced infection within this group, as had been observed in other prevention programs such as in India where increased condom use reduced HIV infections among FSWs by between 48% and 67%. 6 In Nigeria, 2014 IBBSS results found that Brothel Based Female Sex Workers (BBFSWs) exhibited a downward trajectory from a prevalence of 27.4% in 2010 to 19.4% in 2014, while Non Brothel Based Female Sex Workers (NBBFSWs) saw its prevalence decline from 21.1% in 2010 to 8.6% in 2014. 6MC Boilya, M Picklesa, CM Lowndes, et al. Positive impact of a large-scale HIV prevention programme among female sex workers and clients in South India. AIDS 2013, 27:1449–1460. 18 56. Bank-financed activities led by the Ministries of Youth and of Women Affair contributed to reaching youths and women in the general population. With regards to higher risk groups, the Bank played a critical role in financing focused activities that targeted MARPs, especially after the MTR and second restructuring. The expansion of the programs targeting the key populations (FSWs, MSM, and IDUs) could have avert about 17,800 new HIV infections over the life of the project (Annex 3). The Project succeeded in engaging CSOs and the private sector to reach out to MARPs shown to be a major source in infection (Figure 2). Along with the decline in prevalence among FSWs, prevalence among IDUs dropped from 4.2% in 2010 to 3.4% in 2014 (IBBSS 2014). Figure 2. Number of FSWs tested and Number of MARPs reached with MPPI. Baseline Target End of Project Number of FSWs tested for HIV in the last year Number of new MARPs reached with MPPI 0 25,000 50,000 75,000 100,000 125,000 150,000 175,000 57. The prevalence rate among MSMs however not only remained high, but actually increased during the last ten years. According to IBBSS conducted in 2007, 2010, and 2014, the annual rates for this sub group started at 13.5% in 2007, and subsequently increased to 17.2% in 2010 and 22.9% in 2014. The challenge faced by implementers in changing the course of MSM infections may have to do with current stigma associated with MSMs, making the population harder to reach and less likely to benefit from prevention strategies, made difficult due to conflicting messages. For example, the Government passed the HIV/AIDS Anti-Discrimination Act of 2014 aimed at eliminating discrimination towards PLHIV. During the same year however, a law maintained homosexuality as a punishable offense that carried a mandatory 14-year prison term, while those who assisted couples of same sex relationships faced up to 10 years in prison. Not surprisingly, a government survey the following year in 2015 found that only 25% reported testing for HIV in the past 12 months. 7 Given that MARP prevalence rates remained well above the rate for the general population, BBFSWs, NBBFSWs, and MSMs continue to drive the epidemic. Ongoing interventions need to further address how best to reach out to these groups, including likely efforts to resolve legal barriers that underpin social stigma and discrimination. 58. The work performed by public sector agencies at all levels – which included NACA, all SACAs, the four main LMs at the Federal and State levels – through the support of Bank financing contributed to a focused approach to the national response (see Annex 2 for list of various initiatives and accomplishments). In the end, the coordination between Federal and State MDAs helped foster strategic plans that reinforced the overall goal of reducing HIV infection. The active participation of the private sector and civil society, both of which were fully funded through the Bank project, also played a significant part in expanding the reach of prevention and care to MARPs, youths, and pregnant women at the local level. The following IRIs, whose targets were successfully met, provided evidence of this coordination effort: i) SACAs and State level public sector organizations that have a written strategic plan clearly linked to the National Strategic objectives and priorities; ii) Federal LM that have a written strategic plan clearly linked to the National Strategic objectives and priorities; and iii) Number of LACAs implementing HIV/AIDS workplans. 7 NACA (2015) ‘Nigeria GARPR 2015’ 19 PDO II: Increase access to HIV counseling, testing, care and support services. 59. Based on outcome indicators that contributed towards achieving PDO II, the rating for Phase I is Negligible, for Phase II is High, and for Phase III is High. Table 2. Summary of PDO II Achievement Level of Achievement Phase I Phase II Phase III (2009 –2011) (2011 –2015) (2015 –2017) PDO Indicators Surpassed (100%+) 0 1 2 Achieved (85-100%) 0 0 0 Not Achieved 1 0 0 Total 1 1 2 % Surpassed & Achieved 0% 100% 100% Rating N H H 60. The objective focused on increasing access to HIV counseling, testing, care and support services. As a result of setbacks in Phase I, the Project initially failed to collect progress data on the adult population getting tested for HIV as a measure of access to services. By the end of the Project however, 10,728,872 adults aged 15-49 years received HIV tests and results within the last 12 months, with 90.1% of FSWs tested within the last calendar year. 61. Several Bank-supported activities contributed to achieving the PDO indicator targets. With regards to the increased counseling and testing among the general population, the Project contributed to increasing HCT sites. Figure 3 illustrate the correlation between the increase in HCT sites and the increase in access to HCT services in the population, with both following the same progression pattern. The FGON also increased coverage among at-risk populations by rolling out national intervention programs targeting MARPs, based on the epidemic appraisals financed through the Project Credit. Figure 3. Number HCT Sites and Number of People Tested per Year. 10,000,000 8,114 8,308 10,000 No. Tested per Year No. of HCT Sites 8,000,000 8,000 7,075 8,523,118 6,000,000 7,101,636 6,000 4,000,000 3,517,441 4,000 2,056,578 2,628,103 2,000,000 2,626 2,000 1,357 - 0 2011 2012 2013 2014 2015 No tested annually Year No of HCT Sites 62. According to the FMOH’s 2014 HHS data, the country witnessed a decline in national HIV prevalence among pregnant women attending antenatal clinics from 4.1 in 2010 to 3.0 in 2014, which represented 26% decline.8 The trend among pregnant youths aged 15-24 years also showed a decrease from 4.1 in 2010 to 2.3 in 2014, representing a 44% decline. 63. Beyond the services offered through the public space within the government, the Project also supported the private sector’s engagement in offering similar services, the only international donor partner to do so. This entailed involving five implementing partners that introduced HTC services in 8FMOH. National HIV and Syphilis Sero-Prevalence Sentinel 19.4 Survey among Pregnant Women Attending Antenatal Clinics in Nigeria. December 2015. 20 PHFs across 12+1 high burden states. By the end of the implementation period, the Project succeeded in recruiting 96% of private facilities (376 out of 390 total sites) to offer HTC services, greatly enhancing access and entry points for beneficiaries. Given that Nigeria’s national response when the Project began did not include the private sector, the subsequent engagement of 376 private health facilities represented a remarkable achievement. In doing so, the Project helped add a significant facet to a response focused on a multisector approach. 64. Along with the private sector, 557 HAF-supported CSOs from 33+1 states provided prevention, care, and support interventions to targeted populations, including delivering HIV services to MARPs. In particular, CSOs succeeded in reaching 161,124 MARPs for MPPI. Moreover, CSOs ensured that 2,433,798 people were counselled, tested, and received results. With USAID and GFATM not providing support to CSOs on the national response, HPDP2 represented the sole partner to engage CSOs. PDO III: Increase utilization of HIV counseling, testing, care and support services. 65. The ratings under this objective for Phase I is Negligible, for Phase II is High, and for Phase III is High. Table 3. Summary of PDO III Achievement Level of Achievement Phase I Phase II Phase III (2009 –2011) (2011 –2015) (2015 –2017) PDO Indicators Surpassed (100%+) 0 1 1 Achieved (85-100%) 0 0 0 Not Achieved 2 0 0 Total 2 1 1 % Surpassed & Achieved 0% 100% 100% Rating N H H 66. As an indicator to track utilization, the Project monitored the number of pregnant women throughout the life of the Project who used services to complete ART in order to reduce the risk of mother-to-child transmission (MTCT) of HIV. Despite faltering to meet the target in Phase I, 53,677 pregnant women by the end of the Project were documented to have received ART to reduce MTCT, a 105% increase over the baseline figure. Based on the most recent epidemiological and program data, it was assumed that the ART coverage was 41.8% (Annex 3). 67. The increase in the number of sites the Bank supported that offered PMTCT services and the increase in the ensuing number of pregnant women who came in to get counseling, testing, and results followed the same run up as observed in the increase in the number of pregnant women who received ART (Figure 4). Thus, increasing access helped increase utilization. The project funding allowed the screening of about 5.2 million pregnant women through the public, private, and the civil society programs. As WHO has indicated, PMTCT interventions can have significant impact by reducing the likelihood of vertical transmission of HIV to below 5%, compared to 15%-45% without treatment. 9 68. The Project Credit served as the major source of financing the FGON’s goal of increasing PMTCT coverage. In general, the Project provided the funds to train 3,710 health workers on HCT and 3,939 on PMTCT, as well as the procurement of various commodities (test kits, reagents, 9World Health Organization (2014). Elimination of mother-to-child transmission (EMTCT) of HIV and syphilis Global guidance on criteria and processes for validation 21 lubricants, and condoms) and services (transport and distribution costs of delivery to the states, facilities, and community outreach campaigns). About 1.6% of the women screened in the public sector were enrolled in and completed the course of PMTCT (NACA, 2017). Figure 4. Number HCT Sites and Number of People Tested per Year. 63,350 3,500,000.00 No. of pregnant women counselled, tested, and 53,677 No. of pregnant women 60,000 3,000,000.00 40,465 received results 3,067,514 50,000 2,500,000.00 2,844,737 33,891 40,000 2,000,000.00 on ART 1,500,000.00 30,000 1,000,000.00 1,181,296 20,000 500,000.00 907,387 10,000 0.00 0 2010 2012 2014 2016 Year No of pregnants who recieved counseling, testing, and results No pregnant women tested 69. The private sector played a significant role in facilitating PMTCT services, particularly in the 12+1 high burden states. With HAF financing, CSOs helped create the demand for PMTCT services among pregnant women. The CSOs contributed to reaching 646,513 pregnant women for prevention services and ANC referrals, distributing 16,710,570 condoms to targeted beneficiaries, delivering care to 323,510 PLHIV, and providing necessary support to 274,815 OVCs. 70. The overall Efficacy ratings for Phase I is Negligible, for Phase II is Substantial, and for Phase III is High. Table 4. Summary of Efficacy Rating PDO Phase I Phase II Phase III (2009-2011) (2011-2015) (2015-2017) PDO I: Reduce risk to infection N M H PDO II: Increase access to HIV counseling, testing, care and N H H support services PDO III: Increase utilization of HIV counseling, testing, care N H H and support services Overall PDO Rating N S H 3.3 Efficiency 71. Overall Cost-effectiveness and IRR Rating: Modelling of the project impact on infant mortality and HIV incidence showed that, over the life of the project, the expansion of PMTCT coverage could have prevented about 4,500 infant deaths and about 13,400 new HIV infections (Figure 5). Based on this projected impact, preventing one death cost about US$32,160 and preventing one vertical transmission of HIV – about US$10,800. Furthermore, counting only the infant death prevented, PMTCT expansion has likely averted over 111,400 of disability-adjusted life years (DALYs; discounted at 3%) with a cost per DALY averted of US$800. 22 Figure 5: Projected number of AIDS-related infant deaths prevented observed and projected under a counterfactual scenario of no expansion in PMTCT coverage. Number of AIDS deaths in infants (<1yo) 12,000 11,000 10,000 9,000 8,000 7,000 6,000 2008 2009 2010 2011 2012 2013 2014 2015 2016 Counterfactual Observed 72. It was also estimated that the expansion of the programs targeting MARPs could have averted between 17,800 and 192,000 new HIV infections, depending on the assumption of the total number of infections averted between 2009 and 2016 in Nigeria that could be directly attributed to the project (see Annex 3 for details), with a cost per infection averted between US$191 and US$2,100. Adopting a conservative assumption that the ART coverage will not expand beyond current level (42%) over the lifetimes of the person infected with HIV in Nigeria, and using disability weights from the most recent Global Burden of Disease study (Solomon et al, 2015), it was estimated that the expansion of prevention services targeting MARPs averted between 14,900 and 161,000 DALYs with a cost per DALY averted between US$141 and US$1,530 (both DALYs and costs are discounted at 3%). 73. Jointly both component could have averted between 154,000 and 272,500 DALYs with the cost per 1 DALY averted between US$411 and US$1,420. This is significantly below the cost- effectiveness threshold set by the World Health Organization (WHO) Commission on Macroeconomics and Health (WHO, 2003) of a cost per discounted DALY equal to three times per capita GDP. Therefore, the investment of the project can be considered as highly cost-effective. 74. Overall Internal rate of return (IRR): The economic impact of the project, including internal rate of return (IRR) and the benefit cost ratio was calculated based on infant AIDS deaths (it was not possible to calculate the economic benefits for the MARPS component due to insufficient data). The child infections and deaths prevented alone would translate into gains to the Nigerian economy with a net present value (NPV) of about US$576 million. The investment in the expansion of the PMTCT had an annual IRR of 5% and a benefit-cost ratio of 4.0. Put differently, each dollar invested has the potential to generate four times as much in economic benefits. Therefore, it can be concluded that the costs involved in achieving the key project objectives were reasonable and justified in comparison with the estimated benefits and with recognized international norms. Furthermore, because IRR reported here captures the costs but not the benefits of the interventions targeting MARPs, it substantially underestimates the true economic benefits resulting from the project. 75. Implementation efficiency: During the initial stages of the project, the efficiency of implementation was hindered by low procurement capacity of NACA and SACAs, personnel constraints at World Bank side, and challenges in the procurement of the key inputs (e.g. test kits and reagents). In addition, exogenous factors (e.g. changes in the government following president’s Yar’adua’s death; high inflation, which exacerbated the impact of procurement delays) affected the efficiency of implementation. However, as noted in the preceding sections, implementation efficiency improved substantially following the MTR and project restructuring. The disbursement rate increased 23 and the initial implementation bottlenecks, both at the client and the World Bank sides were addressed. Furthermore, new framework contracting for procurement established under the second restructuring allowed for reducing inefficiency by lowering the cost of the key inputs. The Project compensated for the initial inefficiency and recovered during Phase II of the Project by being proactive and responsive. The Project benefitted from the MTR and restructurings that took place, as well as the intensive Bank staff support. Nonetheless, given the 38 months extension and the inability to fully disburse the Credit is notable. 76. Given the cost-effectiveness of project interventions, with its high allocative/technical efficiency and high IRR, along with the 20-month delay in effectiveness, implementation challenges, and 38-month extension to complete implementation, the overall efficiency rating on balance is Modest. 3.4 Justification of Overall Outcome Rating 77. Rating: Moderately Satisfactory. The overall rating is based on a weighted rating of Phases I, II, and III as calculated in the table below. Table 5: Project Overall Outcome Ratings Phase I Phase II Phase III (2009-2011) (2011-2015) (2015-2017) Relevance Substantial Substantial Substantial Objective Substantial Substantial Substantial Design & Implementation Substantial Substantial Substantial Efficacy Negligible Substantial High PDO I N M H PDO II N H H PDO III N H H Efficiency Modest Overall (Outcome) Unsatisfactory Moderately Moderately Satisfactory Satisfactory Rating Value 2 4 4 Total Disbursed (US$ million) $5.3 $82.4 $98.3 Weight in % (total disbursed amount of US$) 0.03 0.46 0.54 Weighted value (Rating Value X % Weight) 0.1 1.8 2.2 Final Outcome rating 4.1→→→→→→→→→→ MS Note: HU (1); U (2); MU (3); MS (4); S (5); HS (6) 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development. 78. The Project greatly benefited women and adolescent girls given that they were more likely to experience sexual abuse, difficulties in negotiating condom use with partner, and discrimination if they became infected with HIV. Because women above 15 years of age made up around 61.5% of all adults living with HIV, the Project impacted a significant portion of PLHIV. In addition, about one- third of infected women will transmit HIV to their unborn children, making PMTCT services vital for the next generation. The Project was also intended to make an impact on PLWHA who were likely to lose income during their most productive age, as well as face increasing medical expenses. By providing counseling, care, and support services, the Project offered PLWHA an opportunity to lead longer and more productive lives. This helped control care and treatment costs at the household and 24 societal levels, and stem any ensuing reduction in size and growth of the nation’s labor force. The marginalized populations, many of whom were represented by MARPs – FSWs, MSMs, IDUs, and OPVs – benefited from the Project. By ensuring that they had access to services, the Project contributed to driving down the overall epidemic. Although the Project was carried out in accordance with the United Nations HIV/AIDS and Human Rights International Guidelines (United Nations, New York and Geneva, 1998), much more needs to be done, including easing social and legislative barriers that might stigmatize and discriminate against MSMs. (b) Institutional Change/Strengthening 79. The Project, through the Bank’s policy dialogue, helped increase the Government’s understanding and use of epidemiological data to drive evidence-based policy. This change in mindset was reflected in the national response developed for the 2013-2015 President’s Comprehensive Response Plan (PCRP) for HIV/AIDS, namely i) the prioritization and scale-up of HIV prevention programs for FSWs, MSM, and other MARPs; ii) more focus on PMTCT; iii) prioritization of states with a high burden of HIV; and iv) less focus on ineffective general HIV awareness raising. The shift in strategic focus represented a significant departure from previous policies. With regards to institutional strengthening, both NACA and SACAs improved their capacities to manage a nationwide response to HIV/AIDS, along with coordinated activities that involve multiple donors. Through intensive Bank technical support, NACA and the SACAs gained the exposure necessary to implement a Project that included following appropriate financial management, procurement, and safeguard guidelines. Also, their involvement in various Project activities aided in increasing the level of expertise and knowledge in the country, including in conducting epidemiologic analyses and in using the results to design evidence-based strategies for the national response. The project also supported efforts to engage newly found partners who coordinated in scaling-up HIV services, namely the private sector and CSOs. The public health response was bolstered through this partnership. With the increased capacity required of NACA and the SACA’s generated by the rising demand for services, the private sector and CSOs provided the model to fill the gaps, including i) promoting HHCT; ii) engaging in BCC; iii) participating in condom social marketing and universal precautions; (iv) increasing support to PLWHA and OVC; and (v) targeting high risk behaviors and high-risk groups. (c) Other Unintended Outcomes and Impacts (positive or negative) 80. The Bank with technical support from the experts at the University of Manitoba conducted an analytic work that examined the status of HIV epidemiology at the sub-national level. The findings proved to be eye-opening, such that it shifted the emphasis of the national response towards (i) greater prioritization and scale-up of HIV prevention programs for FSW, MSM, and other MARPs; (ii) more focus on PMTCT; (iii) additional attention to states with a high burden of HIV; and (iv) less focus on ineffective general HIV awareness raising. The shift in mindset was evident in the approach included in the President’s Comprehensive Response Plan (PCRP) for HIV/AIDS for 2013-2015 and represented a significant departure from previous policies. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 81. NA 25 4. Assessment of Risk to Development Outcome 82. Rating: Substantial. Nigeria’s commitment to the progress made under this Project continues as evident from the Government’s request for the third HIV/AIDS Project, emphasizing ongoing efforts to provide HIV/AIDS services to those most at risk through partnerships with community organizations and the private sector. Moreover, the country remains engaged with the World Bank through the SOML initiative priority interventions that HIV counselling and testing among women attending antenatal care. NACA has set itself to achieve the 90-90-90 percent objective by 2030 (90% will have HIV; 90% on treatment; and 90% are biologically suppressed). While institutional arrangements, such as NACA and SACAs, are in place to administer the national response, running the program effectively will require strong political and management support, especially given the capacity challenges that still persisted when the Project closed. States have already committed to allocating between 0.5 and one percent of their budget towards HIV/AIDS programs. While Nigeria and the USG just agreed to a US$469 million 2017 Country Operation Plan (COP) financing to support HIV prevention, treatment, and care programs, overall resources from external donors have been on a downward trend. For example, GFATM now only supports services in Lagos, while DFID has terminated its funding for HIV in Nigeria. With contributions from international donors covering 70% of the HIV/AIDS Program, Nigeria’s initiatives are heavily dependent and influenced by external sources. This makes federal and state commitments to funding NACA and SACAs, respectively, even more critical, especially in the face of fiscal shocks due to decreasing oil prices. 83. Any new financing from the Bank will need to be designed efficiently based on epidemic evidence and the Bank’s comparative advantage, including focusing on selected priority States and population with high prevalence rates, utilizing a performance-based approach, and partnering with existing donors and stakeholders to ensure service coverage precludes potential beneficiaries outside of where the Bank may be present. One option might be to coordinate with partners so that the Bank focuses on the 12+1 high prevalence States, with GFATM continuing work in Lagos as the only State to which it remains committed, PEPFAR covering the three States it currently maintains, and other donors engaging in local provision of services to the rest of the population. Given the importance of the SACAs that the Bank helped established throughout the country, future support may include minimal financing for their operational tasks. Also an RBF approach, akin to the scheme used successfully with PHFs where payments were made against deliverables, might be used even for Government agencies and unlike the input-based arrangement applied to LMs under Component I where funds received were not tied to results. In this respect, the SOML initiative in Nigeria could be a relevant model. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 84. Rating: Moderately Satisfactory. The Bank financed project was aligned with Nigeria’s National Strategic Framework by focusing on a comprehensive, multisector approach. This was clearly expressed in the design given the emphasis on partnership with various LMs, civil society, and the private sector across the federal and state governments. The HPDP2 team incorporated the existing institutional framework previously envisioned, though not operationally fully realized given the weak capacity at the time, in HPDP1, while at the same time noting lessons learned from HPDP1. The design also considered the environmental impact of the Project by incorporating the Nigeria’s national MWMP as well as addressing project specific wastes as part of a medical waste management plan and ESMF. Furthermore, the Bank took advantage of existing structures (NACA and SACAs) 26 from which to build the Project, following on with the achievements from the previous project. The Bank also coordinated with the government and other development partners regarding the scope of the work. Although the Project was country-wide, which the Bank team viewed as risky, it allowed for continued engagement with Nigeria and helped pave the way to assist the Government understand the drivers of epidemic in the country and how HIV/AIDS should be managed. Despite these, the Bank team could have brought in consultant and technical experts as agents to address upfront the country’s weak capacity as identified in the risk assessment. A more proactive approach could have alleviated the financial management and procurement challenges that the Project ultimately experienced. The extra support might have also helped manage such a big Project designed to cover 36+1 States. (b) Quality of Supervision (including of fiduciary and safeguards policies) 85. Rating: Moderately Satisfactory. From the outset, the Project suffered a 20-month delay that was beyond the Bank team’s control. In the interim, however, the team undertook work that would facilitate progress once the Project became effective. For example, the team initiated a detailed review of the effectiveness of the HAF design put in place and an analysis of the process in order to provide insights needed to strengthen the contracting with NGOs. During this time, the task team also leveraged on an ongoing basis the expertise of GHAP and the University of Manitoba to undertake an in-depth state-based analysis of Nigeria’s epidemic, and assist in evaluating targeted prevention interventions for sex workers that allowed states to scale-up effectively proven prevention interventions to high levels of coverage. Results from these analyses caused transformative shifts in the knowledge and strategic outlook by the Government, both of which influenced changed in the Project towards a more focused approach. In addition, the country benefited from the experience due to the mentoring and capacity building involved in the process. 86. Once the Project became effective, the Bank team remained proactive in working with the country to move the Project forward. For example, the Bank coordinated a joint Government-Bank QER prior to the MTR to stay ahead of ongoing issues, as well as be adequately prepared for an MTR process understood to result in a restructuring to set right a Project hampered by a prolonged delay. In addition, task team engaged with implementing partners through trainings, technical support, and ongoing supervision throughout the life of the Project. Despite these, the number of Bank staff designated to manage the Project did not match the scope, size, and needs of the Project. This was evident from the additional supervision request from the TTL to hire STCs to assist in the daily management of the Project. Even then, the team was stretched thin in responding promptly to 36+1 states. The complexity and scope of the Project also necessitated a TTL based in the country. In NACA’s own account, Bank responses on No Objections took an average of two months, assuredly delaying implementation. The M&E remained inconsistent throughout the Project, making it difficult to measure progress on some critical indicators because they were dropped, added late, or had revised baselines/targets. A stronger and more realistic M& E plan at the beginning might have helped. With regards to the safeguards, problems that persisted for much of the Project could have been eased with early presence and more intensive technical support. (c) Justification of Rating for Overall Bank Performance 87. Rating: Moderately Satisfactory. The overall rating is based on the following sub-ratings: (i) Bank Performance at Ensuring Quality at Entry – Moderately Satisfactory and (ii) Quality of Supervision – Moderately Satisfactory. 27 5.2 Borrower Performance (a) Government Performance 88. Rating: Moderately Satisfactory. The FGON, along with participating States who entered into subsidiary agreements, as the Borrowers, remained committed to the aims of the Project as evident from various strategic plans advanced throughout the duration of the Project. Despite delays in disbursing spending budgets towards operational costs of SACAs, the States were fully involved and contributed their own portions of financing to help fund the national response, as agreed in the FA. Throughout the Project, the Government and States collaborated with the Bank team in addressing challenges identified during the MTR and various reviews. They were also open to findings from the epidemic appraisals that resulted in changes in the approach and management of the Project and National Response. The FGON’s two requests for Closing Date extensions reflected the Governments desire to ensure that the Project fulfilled its commitments. A new request for a third HIV/AIDS Program Development Project show the Government’s desire to continue the trajectory achieved in HPDP2, in the face of dwindling resources from revenues domestically and donor funding internationally. The Project however suffered a prolonged delay in effectiveness, which was compounded by political transitions, associated with the passing of President Yar'Adua. Nonetheless, the first meeting with the FEC that intended to present the Project for approval caused an initial delay that might have been avoided had representatives from the FMOH and/or NACA attended the meeting as envoys to explain the aim, design, and importance of the Project. Moreover, the change in Government in 2015 also affected implementation for almost a year, causing delays due to new policies that combined Project accounts from all donors, hampering disbursements of funds to relevant activities. (b) Implementing Agency 89. Rating: Moderately Satisfactory. The NACA and SACAs maintained flexibility in responding to emerging implementation needs, such as scaling up PMTCT services and devising MARP-specific interventions. In order to effectively implement these targets programs, agencies supported many of the trainings to Federal and State level LMs, civil organizations, and private sector participants. Prior to the MTR, NACA also conducted an independent assessment of HPDP2 and undertook a safeguards audit, both of which provided input into the review. Afterwards, NACA was receptive to making changes based on the MTR recommendations and findings from the epidemic appraisals. The Government also ensured that critical surveys were completed, despite the short delays experienced, as well as took action in revising the RF so that progress towards the PDOs could be assessed as accurately as possible. NACA and SACAs successful partnership with the private sector was not only significant in reaching target populations, but was also the first public-private health partnership of its kind in Nigeria to control the burden of HIV/AIDS. In spite of these, implementation delays related to the weak procurement capacity at the Federal level hindered contracting consultant experts such as Procurement Specialists, as well as disbursements of public sector funds to LMs and of HAF to the CSOs and private sector. To fill in stop gaps quickly and temporarily, specially upon Project launch, staff with relevant expertise to provide additional support could have been seconded from other departments or agencies within the Government. With regards to the legal covenants, Safeguards remained a problem for much of the Project due to non-compliance, requiring a Safeguard audit and extensive remedial actions to resolve. As for FM and Procurement, the Project faced many challenges throughout, including poor management of funds and lack of adherence to FM guidelines. The LMs and CSOs also experienced long delays in getting funds for approved workplans, delaying and at times unable to complete work under the Project. From the LMs and CSOs end, there were also delays in their submission of complete documentation of prior disbursements that were required before new funds were released. Component 2 suffered the most 28 with only 67% of the original allocation being disbursed, despite its effectiveness in reaching target populations. (c) Justification of Rating for Overall Borrower Performance 90. Rating: Moderately Satisfactory. The overall rating is based on the following sub-ratings: (i) Government Performance – Moderately Satisfactory and (ii) Implementing Agency – Moderately Satisfactory. 6. Lessons Learned 91. Need for a more focused approach. Designing a project that covered all the 36+1 States in Nigeria created inefficiencies in project implementation, given the time required to respond to all No Objection requests from the States while at the same time providing quality technical assistance to the implementing entities. Bank-financed Projects could leverage its expertise by focusing on specific target populations and areas, while working with other donors and organizations to provide comprehensive support in other segments of the population based on their comparative advantage or organizational interest. 92. Engage with CSOs and the Private Sector for service delivery. Local CSOs and private sector provides an opportunity to extend the reach of a national response to targeted populations, especially in situations where these organizations are already providing services to populations not typically reached by government agencies. In countries with strong CSOs and private sector, future Projects can extend its impact by leveraging their capacity. HPDP2 showed that engaging CSOs and the private sector for targeted services can be successfully implemented in Nigeria. 93. MARPs. Designing future HIV/AIDS Projects in Nigeria must focus on MARPs. Bank financed analyses undertaken as part of this Project provided evidence that MARPs drive the epidemics. It will also be important to address legal barriers that might serve as the basis of social stigma and discrimination, hindering opportunities to reach out to MARPs. 94. Performance-based financing allows for better accountability and tracking of results against disbursed funds. As this Project showed, the incentives from this type of financing helped detail their contribution and spur their participation in the National Response, its first ever. The payment for results approach could be utilized for disbursements to other designated implementers, including Government agencies and CSOs. 95. Delegate implementation, such as purchase of commodities, to other vetted third party. Weak coordinating agencies may not be equipped to take on the full responsibility of managing every tasks. Since coordinating complex World Bank-financed Projects are in themselves challenging, there is always the risk of inefficiency when managing too much. For complex Projects, the design needs to be open to delegating implementation to other stakeholders when possible, or through direct agreement with vetted third party entities that bypasses the coordinating agency. In this Project, establishing the direct purchase of commodities from UNICEF and UNFPA not only ensured the quality of commodities but also helped alleviate pressures from an overburdened NACA. 96. Assigning country-based TTL. When faced with supervising a project of this magnitude, the Bank would be wise to put additional resources at the local level and also to have the TTL based in country. 29 97. Sustainability. Transitioning SACAs to agency status provided States with the jurisdiction to manage decision based on their understanding of the epidemic, which varied across States. This contributed to strengthening the national response to HIV/AIDS. Moreover, SACAs played a critical role in establishing stronger links with the CSOs and private sector, which helped expand services and access as already mentioned above. Given the driving force behind the epidemic, the next Project will need to continue its focused engagement with CSOs and the private sector to provide services to MARPs. HPDP2 showed that this approach can be effectively implemented. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies. See Annex 7. 98. See Annex 7. (b) Cofinanciers. 99. NA (c) Other partners and stakeholders. 100. NA 30 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million) Appraisal Actual Percentage of Components Estimate Estimate Appraisal (USD millions) (USD millions) Component 1: Expanding the Public Sector Response 68.00 61.77 90.84 Component 2: Expanding the civil and private sector response 79.00 53.24 67.39 to the HIV/AIDS Epidemic Component 3: Strengthening mechanisms for project 78.00 71.03 91.06 coordination and management Total Baseline Cost 225.00 186.04 82.68 Front-end fee PPF 0.00 0.00 0.00 Front-end fee IBRD 0.00 0.00 0.00 0.00 Total Financing Required 225.00 186.04 82.68 (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (USD millions) (USD millions) Borrower 5.00 4.35 87.00 International Development Association 225.00 186.04 82.68 (IDA) 31 Annex 2. Outputs by Component Component 1: Expand the public sector response 1. Ministries Departments and Agencies (MDAs) and LACAs were supported through component One. Thirteen (13) MDAs benefited at the initial stage of the project but they were reduced to 4 after mid-term review when the project was restructured. The Four Ministries received funds to directly implement activities in the following areas. 2. The Ministry of Health provided HIV counseling and testing (HCT) and prevention of mother-to-child transmission (PMTCT) services through public and private health facilities. 3. The Ministry of Education provided curriculum and non-curriculum family and life health education (FLHE) in secondary schools as well as HCT and condom promotion services targeting students in tertiary institution (most of which falls within the age bracket of 15-24years) through the formation of anti-HIV/AIDS Club composed of trained students and health workers. 4. The Ministry of Youth provided HCT services and condom promotion to the youth and general population in most of the local government administrations (LGAs) in the country through the National Youth Service Corps (NYSC) HIV/AIDS community development services (CDS) group made up of trained NYSC members. 5. The interventions provided by the Ministry of Women Affairs included demand creation for PMTCT, provision of care and support services for people living with HIV (PLHIV) and orphans and vulnerable children (OVC). The Local Action Committees on HIV/AIDS (LACAs) conducted massive community outreaches with HCT and condom promotion using trained community volunteers. 6. Some of the key achievements under this component include: 7. Ministry of Health:  1,639 health facilities were supported by the project to provide HCT services.  1,320 health facilities were supported to provide PMTCT services.  649,005 persons were counselled, tested and received results.  1,317 health workers were trained on HCT and PMTCT.  54 Extension workers were trained on coordination of HIV/ AIDS prevention activities, from (3- Geo-political zones) North Central, South- East, South-South.  30 selected Nigeria prisons’ health facilities were assessed, 25 of which implemented HCT in the prisons.  25 health care workers trained on STI management. 8. Ministry of Education:  Family Life and HIV/AIDS Education (FLHE) curriculum developed and integrated into primary and secondary schools.  1,969 schools implementing the FLHE through HPDP2 support  208 MIS 002, 104 Fact sheets and 104 peer education manuals printed for the Federal Unity colleges and State Schools on FLHE Delivery Services  37 SMoEd officers had their capacities strengthened on FLHE coordination and delivery.  66 Unity College staff school head teachers sensitized on FLHE delivery  2080 Unity College FLHE subject teachers sensitized on curriculum delivery.  In collaboration with the SACAs & SMYDs, the SMoEd reached 18,000 out-of- school youths from 18 states with HTS services.  1,244 schools provided life-skills based HIV education  Supportive supervision was conducted to HIV Desk Officers in the SMoE in 36+1 States. Data is currently being collated from the States. 32 9. Ministry of Youth and Sport Development:  HCT outreaches were conducted for out- of- school youths in 18 states.  20,160 persons were counselled, tested and received their results.  215 positive cases were referred to healthcare facilities for treatment and care.  Mechanisms were strengthened between FMYD & NYSC for effective coordination of out of school youth and other youth related activities.  22 NYSC & State Ministry of Youth HIV schedule officers were trained on coordination of out of school youth HIV prevention.  84 state HIV desk officers of MOY trained on the reviewed HTS tools  27,562 persons were counselled, tested and received result from the SMoY outreaches.  15,848 persons were counselled, tested and received result from Anti-HIV/AIDS CDS group outreaches.  262,536 condoms were distributed. 10. Ministry of Women Affairs:  7,482 OVC were reached with at least one support service.  2,148 OVC care givers were enrolled in long-term income-generating training.  47,565 PLHIV were provided with a minimum of one care.  Capacity building was conducted for 85 OVC desk officers and HIV / AIDS focal persons on National Standards for Improving the Quality of Life of OVC in Nigeria and National Priority Agenda (NPA) in coordinating the national OVC response.  32 officers from the SMWASD were trained on coordination and service provision using the National Standard for programming for OVC, OVC caregivers and PLHIV.  Strong synergy and collaboration were established between the Federal and state structures in the country for OVC programming. 11. Component 2: Expand the civil society and private sector response 12. This component supported contracting of civil society organizations (CSOs) using the HIV/AIDS Fund (HAF) to provide targeted interventions to prioritized populations in line with individual state’s epidemiological profile. 13. The process of engaging the CSOs in each state include mapping of the CSOs, formation of HAF advisory committee, development of brief epidemiological summary, prioritization based on epidemiological profile, advertisement of expression of interest (EOI), issuance of RFP, review of proposals, capacity assessment, contract negotiation and contract award. The program science approach was adopted for the HAF implementation and a minimum package of prevention interventions (MPPI) were provided to the prioritized target population. 14. In addition to the contracting of CSOs using HAF, NACA also awards engage national-level implementing partners (IPs) to provide PMTCT services in private health facilities through a performance based financing (PBF) scheme. 15. The key achievements of component two include:  564 CSOs implemented the HAF from 34 states and FCT.  94 CSOs were re- engaged by 21 states to provide HIV services to key populations and demand creation for PMTCT services among pregnant women.  2 States (Ebonyi and Kano) could not engage CSO at all on the HAF for HPDP II. 33 16. Private sector engagement:  Supported engagement of 4 implementing partners (IPs) – APIN, SFH, IHVN, CCCRN - to provide PMTCT services in private health facilities across 13 States through a performance-based financing (PBF) scheme.  A total of 376 sites have been activated out of 390 sites, bringing achievement to 96%.  206,243 pregnant women have been counselled, tested and received results over seven months.  1888 (0.92 % prevalence) pregnant women tested positive and referred for PMTCT while 1123 (59.5%) of them completed referral for PMTCT. The defaulting patients refused referral on account of not believing the test results.  Eight (8) firms were engaged as Independent Verification Agents (IVAs) to verify services across the thirteen states. 17. Sexual prevention learning sites (LS)  Proceeded to the identification of the Learning Site (LS) model for HIV prevention programs in Nigeria.  Conducted a gap analysis and development of LS operational manual in Bwari.  Conducted capacity building of SACAs and LACAs.  Coordinated inclusion of partners’ programs in the LS implementation plan e.g. SFH and GFATM supported CS0s.  Engaged 2 consultants for consultancy services to develop learning sites curriculum and methodology for the four States.  Production and adoption of the national implementation guidelines for FSW.  Local epidemic appraisals conducted (Geographical mapping of MARPs and size estimation) in 22 states.  Confirmed outlines for MSM and PWID implementation guidelines.  SBCC: A draft PMTCT TV spot and HIV/AIDS knowledge ‘5 over 5’ TV spot is in existence and awaiting production.  National HIV Strategy for AYP launched in March 2016.  Capacity building and dissemination workshops for the implementation of the AYP Strategy held in 5 out of 12+1 priority states (Taraba, Benue, FCT, Rivers, and Lagos). 18. Sexual ethnography study: characterization of sexual activities and typology:  Protocol for the assessment of sexual behaviour patterns and influences among women and men in venues that facilitate sexual networking in urban areas in Nigeria developed.  A community sensitization meeting conducted which brought together the FSW community to inform them of the project and its impact.  Sexual ethnography studied conducted. Study led by FSWs community members.  Data collated, analyzed and report finalized. 19. Component 3: Strengthen mechanisms for project coordination and management 20. This component supported a cohesive approach to capacity building with the objectives to deliver strengthened evidence-based planning, increased coordination, harmonization and alignment by all stakeholders, and stronger, sustainable and more responsive HIV/AIDS financing and programming. 21. Key achievements under this component include:  35 out of 36+1 States now have a coordinating agency with direct budgetary allocation for HIV/AIDS activities.  Strengthening of the M&E system for the HIV/AIDS response through deployment of both web-based and mobile DHIS which significantly improved the reporting rate of service delivery data.  Over 1500 health facilities now report their service delivery data directly to the DHIS platform through mobile phones.  Reporting rate of HIV prevention programs have improved from 0.3% in 2013 to 72.8% in 2015. 34 Annex 3. Economic and Financial Analysis Methodology: The project included two principal modalities of interventions aimed at reducing the incidence of HIV: 1) prevention of vertical transmission of HIV from mother to child through the expansion of the PMTCT program and 2) prevention of HIV transmission in key populations at risk of HIV infection: female sex workers (FSWs), people who inject drugs (PIDs), and men who have sex with men (MSM). Cost estimation: The estimation of the cost was done from the project perspective and only included the incremental costs incurred by the project. The annual costs were calculated as the annual project disbursements (see Figure 1) and include all three project components, including capacity building at the federal, state, and local levels. Figure 1: Annual Project Disbursement. $60.0 $50.0 $49.2 $40.0 $37.3 $37.1 $30.0 $28.8 $20.0 $20.2 $10.0 $8.8 $0.0 $0.0 $0.0 2009 2010 2011 2012 2013 2014 2015 2016 Data on disbursement by project component or by intervention were not available. It was assumed that the annual disbursements for PMTCT and the key population prevention services followed the overall disbursement pattern. Furthermore, it was assumed that the proportion of disbursement allocated for each of the components was equivalent to the difference in the unit cost of PMTCT services and services targeting the key populations. Based on available data, the average annual cost of PMTCT services in Sub-Saharan Africa is about US$368 (average of the unit costs reported in Desmond et al., 2004; Orlando et al., 2010; and Roberstad et al., 2010). The annual cost of providing prevention services to key populations was estimated to be about US$94. This total included an estimated average cost of HIV testing and counseling (HTC) of about US$7.3 (based on Aliyu at al. 2012; Kombe, 2004) and an estimated average cost of outreach services of about US$86.8 (based on Futures Institute, 2010; Lafrort et al. 2010, Thomsen et al. 2006). Based on those assumptions, it was estimated that, out of the US$181.3 disbursed between 2009 and 2016, US$144.4 million (79.6%) was allocated to the PMTCT program and US$36.9 million (20.4%) – to programs targeting the key populations at risk of HIV infection. Impact estimation: The potential impact of the project on three primary outcomes: new HIV infections in adults, new HIV infections in infants, and infant AIDS deaths was assessed by comparing the observed numbers of infections and deaths with a counterfactual (i.e. a hypothetical scenario of what would have happened in the absence of the project) based on long- term trends. More specifically, an average annual reduction in the number of HIV adult and infant infections and in infant deaths was estimates from 2001 through 2011. To construct the counterfactual, that trend was projected from 2012 through 2016. Those estimates were then compared to the number of HIV adult and infant infections and in infant deaths actually observed. The difference between the counterfactual and the observed numbers of new infections and deaths was interpreted as the net impact of the project. Sensitivity analysis PMTCT: To assess the plausibility of this approach and the assumption that all observed reductions in infant HIV infections and AIDS deaths could be attributed to the expansion of the PMTCT coverage financed by the project, 35 additional impact estimation for the PMTCT component was conducted using the AIM module of the SPECTRUM software suite (Avenir Health, Glastonbury, CT). The module allows users to estimate the impact of expanding PMTCT services on a number of outcomes, including the number of new HIV infections in infants and a number of AIDS-related infant deaths. Two scenarios were modelled using SPECTRUM. In the first scenario, the coverage of PMTCT was kept at the 2009 baseline of 5.25% reported in the PAD. In the second scenario, the expansion of the PMTCT coverage was modelled based on project coverage data. The project funding allowed to screen about 5.2 million pregnant women through the public, private, and the civil society programs. About 1.6% of the women screened in the public sector were enrolled in and completed the course of PMTCT (NACA, 2017). Data on enrollment and completion rates for the private and the civil society sectors were not available and it was assumed that about the same percentage for women (1.6%) who were screened in those sectors completed PMTCT. It was estimated that, over the life of the project, about 83,000 pregnant women completed PMTCT thanks to the project investment (cumulative number of beneficiaries). Annual numbers of beneficiaries were not available and were estimated based on the project disbursement rates. Figure 2 presents the estimated increases in PMTCT coverage in Nigeria financed through the project. Figure 2: Estimated Annual PMTCT Coverage. 25.0% 20.0% 15.0% 10.0% 5.0% 0.0% 2009 2010 2011 2012 2013 2014 2015 2016 Counterfactual coverage Modelled project coverage MARPS: AIM does not allow to model the annual scale up of prevention interventions targeting the key population at risk of HIV infection. Therefore, the plausibility of the assumption that observed declines in the HIV adult prevalence could be attributed directly to the project was assessed as comparative statics using a simple Excel model. The model projected the number of infections averted among the members of the key populations reached with outreach programs. It did not estimate the reduced risk of onward transmission (reduced transmission to partners of the members of the key populations whose infections were averted thanks to the programs). Equation 1 blew summarizes the model: (1) KPr= ∑1-gKPg*Br*Ag*Pg*MSM*STIg*STIm*CE*CPg*PI Where g – Key population group (FSWs, PIDs, MSM) Br – baseline risk of HIV transmission per sex act A - average number of sex acts per partner in each KP group P – average number of sex partners per year in each KP group MSM – relative risk increase for sex acts between MSM STI – STI prevalence in each KP group STIm – relative risk increase for sex acts in the presence of STIs CE – effectiveness of condoms in preventing HIV infection CP – prevalence of condom use in each KP group PI – impact of the program on the likelihood of condom use in each KP group Table 1 presents the model parameters used. The prevalence of risk behaviors and relative risk ratios were obtained from SPECTRUM (default values for Nigeria). 36 Table 1: Key Population HIV Infection Model Parameters. Target group Model parameter FSWs PIDs MSM Baseline risk of HIV infection per sex act 0.0008 0.0008 0.0008 Acts per partner 4 36 15 Partner per year 134 5.3 6 MSM multiplier 1 1 3.4 STI multiplier 8 8 8 STI prevalence 20.1% 14.9% 15.1% Age at first sex 21 21 21 Condom effectiveness 80.0% 80.0% 80.0% Condom use 64.6% 47.7% 49.9% Increase in likelihood of condom use due to outreach 1.365 1.56 1.25 DALYs DALYs averted through the programs were calculated based on the number of deaths and infections prevented. One death prevented was counted as 1 DALY averted per year (over the standard life expectancy). One HIV infection prevented was valued as 0.221 DALYs per year for persons with CD4 count>200 not on ART, 0.547 for persons for CD4 count <200 not on ART and 0.053 for persons on ART. Based on the most recent epidemiological and program data, it was assumed that the ART coverage was 41.8% and that 91.9% of people infected with HIV had the CD4 count greater than 200. Economic impact Economic impact was estimated based on the infant mortality and morbidity averted through the PMTCT program. Each infant death averted was valuated at one times GDP per capita per year, over 35 years, from the year the child would have turned 18 through 53 - the current life expectancy in Nigeria. In addition, the averted cost of the treatment of opportunistic infections was added to the economic benefits. It was assumed that HIV-positive children would require treatment for opportunistic infections for 5 years. No cost of pediatric treatment of opportunistic infection was available. Therefore, the cost of treatment for adults in Nigeria (US$268.04 per person per year) was used, based on Menzies et al. (2011). Equation 2 summarizes the way economic benefits were calculated. (2) EB=[(DP*GDPpc*35*IM/1000)+(IP*OIc*5*IM/1000)]*1/(1+r)n Where: EB – total economic benefits (in US$) DP – number of infant deaths prevented GDP pc – GDP per capita IM – infant mortality rate OIc – annual cost or treatment of opportunistic infections IP – number of HIV infections in infants prevented R – discount rate The economic benefit estimated do not include the benefits from averted infections in MARPs, because data on the economic impact of adult HIV infection in Nigeria are not available. The impact of HIV infection on productivity of adults infected with HIV depends on ART regimens, treatment protocols and ART eligibility threshold, and coverage rates. Those factors change, sometimes dramatically, over time and it would be difficult to develop a credible scenario of the future trajectory of ART treatment programs in Nigeria. However, it needs to be noted that the economic benefits based solely on reductions in HIV-related infant mortality and morbidity are substantially under-estimating the total economic benefits generated by the project. Results: Economic Impact and Efficiency 37 Figures 3, 4, and 5 show the number of infants AIDS deaths, the number of new infections in infants, and the number of new infections in adults, respectively, under the counterfactual and those actually observed during the life of the project. Figure 3: The Number of AIDS Deaths in Infants Observed and the Number of AIDS Deaths in Infants Projected Under the Counterfactual, 2006-2016. 12,000 11,000 10,000 9,000 8,000 7,000 6,000 2008 2009 2010 2011 2012 2013 2014 2015 2016 Counterfactual Observed Figure 4: The Number of New HIV Infections in Infants Observed and the Number of New HIV Infections in Infants Projected Under the Counterfactual, 2006-2016. 50,000 45,000 40,000 35,000 30,000 25,000 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Counterfactual Observed Figure 5: The Number of New HIV Infections in Adults Observed and the Number of New HIV Infections in Adults Projected Under the Counterfactual, 2006-2016. 200,000 150,000 100,000 50,000 0 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 Counterfactual Observed Overall, the expansion of the PMTCT component prevented about 13,300 new pediatric HIV infections and about 4,500 infant deaths. Figure 6 below shows the annual numbers of infections and deaths averted. Spectrum modelling showed very similar results. Based on the model, the expansion of the PMTCT could have prevented about 4,900 infant deaths and about 15,300 vertical HIV infections. 38 Figure 6: Annual Estimated Number of New HIV infections and AIDS Deaths in infants (<1yo) Prevented through the PMTCT Component of the Project, 2011-2016. 5,000 4,000 3,000 2,000 1,000 0 2011 2012 2013 2014 2015 New HIV infections (infants;<1year old) AIDS deaths (infants;<1 year old) Preventing one death cost about US$32,160 and preventing one vertical transmission – about US$10,800. Furthermore, counting only the infant death prevented, the PMTCT component has likely averted over 111,400 of disability-adjusted life years (DALYs; discounted at 3%), with a cost per DALY averted of about US$800. Comparing the counterfactual based on the past trend with the number of new HIV infection in adults (15-49 years of age) actually observed during the life of the project shows that, over five years, the project could have prevented as many as 192,700 new infections (see Figure 7 below). Preventing one new infection would cost about US$191. Adopting a conservative assumption that the ART coverage will not expand beyond the current level (42%) over the lifetimes of the person infect with HIV in Nigeria and further assuming that about 8% of new HIV diagnoses are made in people with the CD4 count lower than 200, and using disability weights from the most recent Global Burden of Disease study (Solomon et al, 2015), it was estimated that the MARPs component of the project could have averted 161,900 DALYs with a cost per DALY of about US$141 (both DALYs and costs are discounted at 3%). Figure 7: Annual Estimated Number of New HIV Infections in Adults (15-49 years old) Prevented during the Life of the Project, 2011-2016. 70,000 60,000 50,000 40,000 30,000 20,000 10,000 0 2011 2012 2013 2014 2015 2016 However, it is unlikely that all of the observed reductions could be directly attributed to the project, given the project’s focus on MARPs, and the fact that other development partners and the FGON have made concurrent and substantial investments in HIV prevention. Therefore, while the 192,700 infections averted can be considered a possible upper bound estimate, it is likely that at least a portion of the infections averted should be attributed to expansion of HIV prevention efforts carried out by the FGON and its partners using financial resource coming from sources other than the project. The excel model used as an alternative modeling strategy (see methods above), showed that the expansion of the programs targeting the key populations: female sex workers (FSWs), men who have sex with men (MSM), and people who inject drugs (PIDs) could have avert about 17,800 new HIV infections over the life of the project. Preventing one infection would cost about US$1,600 and averting one DALY - about US$1,530 (both DALYs and costs are discounted at 3%). This can be considered as the reasonable lower-bound estimate of the impact of the MARPs component of the project. 39 In sum, the HIV prevention component targeting the MARPs could have prevented between 17,800 and 192,000 new infections and between 14,900 and 161,000 DALYs, depending on assumptions made, with the cost per DALY averted between US$141 and US$1530. Jointly both component of the project could have averted between 154,000 and 272,500 DALYs (discounted at 3% rate). The total cost per 1 DALY averted is between US$411 and US$1,420. Economic benefits and internal rate of return: The deaths and infections averted would translate into economic gains with a net present value of about US$576 million. The gains would result from mortality prevented and from averted costs of treatment of opportunistic infections in children who would live past infancy. The investment in the expansion of the PMTCT component had an annual internal rate of return (IRR) of 5% and a benefit-cost ratio of 4.0. Put differently, each dollar invested in PMTCT had the potential to generate over 4 time as much in economic benefits. The WHO Commission on Macroeconomics and Health has proposed a standard approach for evaluating cost- effectiveness and technical efficiency, of health interventions (WHO, 2003). According to the Commission’s guidance, an intervention can be considered cost-effective if the cost per DALY, discounted at 3%, is lower than 3 times GDP per capita; an intervention can be considered very cost effective if the cost per DALY discounted at 3% is lower that one time GDP per capita. The cost per DALY averted for both programs (the PMTCT program and the key population program) is lower than Nigeria’s GDP per capita and so is the cost per DALY for the entire project. Therefore, the economic analysis indicates that, based on the international benchmarks, the investment of the project can be considered to be very cost-effective. The attractive internal rate of return of 4% and a benefit cost ratio substantially higher than 1 provide further economic justification of the project investment. In sum, it can be concluded that the costs involved in achieving the key project objectives were reasonable and justified in comparison with the estimated benefits and with recognized international norms. References: Aliyu HB, et al. 2012. Title What is the cost of providing outpatient HIV counseling and testing and antiretroviral therapy services in selected public health facilities in Nigeria? Publication Information Journal of Acquired Immune Deficiency Syndrome. 2012; 61(2): 221-225. Desmond C, Franklin L, Steinberg M Title The prevention of mother-to-child transmission. Costing the service at four sites in South Africa. Publication Information 2004. Health Systems Trust. Durban. Futures Institute 2010 (Unpublished). Avenir Health HIV Unit Cost Repository. http://www.avenirhealth.org/policytools/UC/app.php Kombe G, Galtay D, Nwagbara C. 2004. Scaling up antiretroviral treatment in the public sector in Nigeria: A comprehensive analysis of resource requirements. Bethesda, MD: The Partners for Health Reformplus Project, Abt Associates Inc. Lafort Y, Geelhoed D, Cumba L, Lazaro CDDM, Delva W, Luchters S, Temmerman M. 2010. Reproductive health services for populations at high risk of HIV: Performance of a night clinic in Tete Province, Mozambique. BMC Health Services Research. 2010: 10/144: 1-9. http://www.biomedcentral.com/1472-6963/10/144 Menzies N., Berruti, A, Berzon r., Filler S., Ferris R, Ellbrock T. et al. 2011. The Cost of Providing Comprehensive HIV Treatment in PEPFAR-Supported Programs. AIDS 10(14):1743-1760. Orlando S, Marazzi MC, Mancinelle S, Liotta G, Ceffa S, Giglio P, Alumando E, Ziegler I, Shawa M, Palombi L. 2010. Cost-effectiveness of using HAART in the prevention of Mother to Child Transmission in the DREAM- Project Malawi. Journal of Acquired Immune Deficiency Syndromes. 2010; 55(5):631-634. Study link http://www.ncbi.nlm.nih.gov/pubmed/21934555 Robberstad B, Evjen-Olsen B. 2010. Preventing mother-child transmission of HIV with highly active antiretroviral treatment in Tanzania- a prospective cost-effectiveness study. Journal of Acquired Immune Deficiency Syndromes. 2010; 55(3): 397-403 http://www.ncbi.nlm.nih.gov/pubmed/20739897 40 Thomsen SC, Ombidi W, Toroitich-Ruto C, Wong EL, Tucker HO, Homan R, Kingola N, Luchters S. 2006. A prospective study assessing the effects of interoducing the female condom in a sex worker population in Mombasa, Kenya. Sexually Transmitted Infections. 2006; 82: 397-402. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2563858/pdf/397.pdf 41 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Sunday Achile Acheneje Procurement Specialist AFCW2 Procurement Adewunmi Cosmas Adekoya Financial Management Specialist AFTFM Financial Management Nevena Alexieva Senior Operations Officer Operations Mary Asanato Procurement Specialist AFTPC Procurement Bayo Awosemusi Lead Procurement Specialist GGO01 Procurement Aissatou Diallo Finance Officer LOAFC Financial Management John Elder AFTH3 Task Team Leader Abiodun Elufioye Program Assistant AFCW2 Program Assistant Ella Omomene Iklaga Team Assistant AFCW2 Program Assistant Rita Obioma Itoro-Godfrey Program Assistant GGO19 Program Assistant T. Mpoy-Kamulayi Lead Counsel LEGAF Counsel Oluwole Odutolu AFTH3 Consultant HIV/AIDS Ogo-Oluwa Oluwatoyin Jagha Senior Operations Officer OPSRR M&E Specialist Akinrinmola Oyenuga Akinyele Financial Management Specialist AFTFM Financial Management Masahiro Matsumoto E T Consultant EASHD - HIS Operations Susan Joyce Elizabeth A. Mshana Consultant AFCW2 Operations Joanna Lee Nicholls Senior HIV/AIDS Specialist AFTH3 - HIS HIV/AIDS Specialist Chita Oje Team Assistant Program Assistant Foluso Okunmadewa Lead Social Protection Specialist GSP07 Social Protection Adenike Sherifat Oyeyiola Sr Financial Management Specialist GGO24 Financial Management Rajiv Sondhi Senior Finance Officer LOAFC Financial Management Therese Tshamala Program Assistant AFTH3 Program Assistant Juliana Victor Senior M&E Specialist AFTRL M&E Supervision/ICR Adewunmi Cosmas Ameer Sr Financial Management Specia GGO25 Financial Management Adekoya Francisca Ayodeji Akala Senior Health Specialist GHN19 TTL Thomas Kwasi Siaw Anang Senior Procurement Specialist GGO01 Procurement Enias Baganizi Senior Health Specialist GHN TTL Ella Omomene Iklaga Team Assistant AFCW2 Program Assistant Daniel Rikichi Kajang Sr. Procurement Specialist GGO25 Procurement Specialist Jakub Jan Kakietek Economist GHN Economist Public Health Benjamin Loevnisohn Sr. Public Health Specialist GHN Specialist NDella Njie Operations Officer GHN Operations Ogo-Oluwa Oluwatoyin Jagha Senior Operations Officer OPSRR Operations AFTHW - Michael O'Dwyer Lead Health Specialist Health Specialist HIS Adenike Sherifat Oyeyiola Sr Financial Management Specia GGO24 Financial Management F. Brian Pascual Consultant GHN ICR Author Juliana Chinyeaka Victor Senior Monitoring & Evaluation GEESO M&E (b) Staff Time and Cost Stage of Project Cycle Staff Time and Cost (Bank Budget Only) 42 USD Thousands (including travel No. of staff weeks and consultant costs) Lending FY07 0.0 10.75 FY08 25.74 91.87 FY09 13.72 150.08 Total: 39.46 252.70 Supervision/ICR FY10 21.45 112.19 FY11 24.57 149.52 FY12 30.08 243.57 FY13 39.83 297.94 FY14 28.24 213.79 FY15 45.35 283.94 FY16 38.65 260.45 FY17 19.63 175.93 FY18 5.23 28.21 Total: 253.03 1,765.54 43 Annex 5. Beneficiary Survey Results (if any) 44 Annex 6. Stakeholder Workshop Report and Results (if any) 45 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR 1.1 Achievements of the Project Development Objectives (PDO) The Project Development Objectives (PDO) of HPDP II is to reduce the risk of HIV infections by scaling up prevention interventions and to increase access to and utilisation of HIV counselling, testing, care and support services. Progress towards achieving the PDO had been rated satisfactory throughout the course of the project in the recent ISR (ISR P102119- 01- 13- 2017). The outcome indicators were achieved by scaling up high impact interventions that addressed the PDO through public sector agencies, CSOs and the private sector. Capacity building, which is the third component of HPDP II, contributed indirectly to the PDO through dedicated strengthening of the institutional arrangements of NACA / SACAs and implementing partners for more responsive planning, monitoring and evaluation, policy development and ultimately, results-based implementation. The percentage of Female Sex Workers (FSWs) (brothel-based and non-brothel-based) reporting consistent condom use with casual partner in the last 12 months (IBBSS 2014) was 83% for brothel-based FSW and 74% for non-brothel based FSW as at 2016 compared to the baseline target of 68% for brothel and 61% for Non-Brothel Based FSWs in 2010. The target achieved has surpassed the end of project targets of 72% and 65% respectively. Percentage of Female Sex Workers who were tested for HIV in the last calendar year (IBBSS 2014) was 88% for Brothel Based FSW and 75% for Non-Brothel Based FSW in 2016 compared to the baseline target of 38.8% in 2010. The end of project target of 43% was surpassed with significant achievement. The percentage of young women and men aged 15-24 reporting the use of a condom during the last sexual intercourse with a non-regular partner in the last 12 months (NARHS 2012) has increased from 34.3% (women) and 52.2% (men) in 2010 to 43% (women) and 61.2% (men) in 2012. More recent 2017 MICS showed additional progress made with women increasing to 46.6% and men increasing to 61.4%. The number of women and men aged 15-49 who received an HIV test in the last year and who knew their results rose from 2,287,805 at baseline in 2010 to 10,163,249 year in 2016.This is an annual cumulative figure. The number of pregnant women living with HIV who received a complete course of antiretroviral prophylaxis to reduce the risk of Mother-To-Child-Transmission (MTCT) was 51,797 as at 2016 compared to the base line target of 26,133 in 2010. The achieved target has surpassed the end of project target of 40,000. 1.2 Original Project Development Objective (PDO) and Initial Key Performance Indicators The achievements of key indicators that were initially identified for measuring performance were: The percentage of young women and men aged 15-24 reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a non-regular partner in the last 12 months) has increased from 34.3% to 45.4% for females and from 52.2% to 69.2 for males. The percentage of women and men aged 15-49 who have had sexual intercourse with more than one partner in the last 12 months has decreased from 1.1% to 0.4% for females and from 7.2% to 5% for males. The percentage of most-at-risk groups reporting the use of a condom during (last) sexual intercourse with a non-regular partner (of those reporting sexual intercourse with a non-regular partner in the last 12 months) has increased from 64.8% to 72% for female sex workers’ brothel based and from 67.1% to 75% for female sex workers’ non-brothel based. The percentage of persons aged 15 and older who have received counselling and testing for HIV and have collected their HIV test results has increased from 72.7% to 96%. The percentage of most-at-risk populations who both correctly identified ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission has increased from 33% to 50% for sex workers; from 44% to 57% for men who have sex with men; and from 34% to 47% for injecting drug users. 46 The percentage of pregnant women who are HIV positive and require antiretroviral prophylaxis and who have received a complete course of antiretroviral prophylaxis to reduce the risk of mother to child HIV transmission has increased from 5.25% to 30%. The ratio of current primary school attendance between orphans of HIV/AIDS and other pupils has increased from 75% to 87%. 2.3.2 Monitoring and Evaluation In the spirit of the third arm of the universally accepted ‘three ones’ principle, the monitoring & evaluation framework of the HPDP2 is aligned with the country’s national M&E system for HIV/AIDS also known as the NNRIMS Operational Plan 2 (NOP2). HPDP II funds contributed significantly to strengthening the national M&E system in Nigeria. M&E units have been established in all 36 SACAs. HPDP II also supported the harmonisation and development of standard data collection and reporting tools in 2012. SACA M&E staff, LACA staff and facility staff were trained on the use of the standard national tools. Nigeria adopted the use of the electronic reporting platform known as the District Health Information System (DHIS) in 2009 for HIV routine data reporting. HPDP II also supported the expansion and rollout of DHIS to 33 states and FCT as at March 2016. These states, including their LGAs, some secondary and tertiary facilities are currently using the platform for reporting on a monthly basis. Under the HPDP II NACA developed and rolled out a DHIS application for monthly reporting of data by PHCs using mobile phones. As at the end of 2016, 1701 mobile phones with the DHIS mobile application technology had been rolled out in 19 states. HPDP II funds were used to support the development of data collection tools for tracking the implementation of the Minimum Package of Prevention Interventions (MPPI). Civil Society Organisations implementing the HAF deployed the data tools at the community level for use. A total of 557 CSOs implemented the HAF in 33 states + FCT. They were trained on the use of the HIV prevention, Home Based Care (HBC) and Orphans and Vulnerable Children (OVC) data collection tools in 2014. Similarly, all the 557 CSOs were also trained on how to use DHIS for reporting. Reporting rates for the various programmes under HAF remained high at 81% for Prevention, 84% for OVC and 82% for HBC respectively. HPDP II provided technical assistance for the use of a programme science approach to programming for HIV. This ensured that state and LGA level epidemiology is better understood and is used to inform a more targeted and efficient HIV prevention programme for key populations and the general population within geographic coverage areas. Special studies and reviews were also conducted by NACA using HPDP2 funds. Epidemic appraisals aimed at better understanding of the HIV/AIDS epidemic among key populations and targeting them with interventions were conducted in eight states in 2012. A HAF assessment to review implementation of the HAF by CSOs was also conducted in 18 states. Annual reports on the project were submitted in a timely manner from 2012 to 2016. The project also provided support to surveys during the life span of the project. HPDP2 has supported the conduct of a Nigeria Multi Indicator Cluster Survey (MICS) in 2017 and conducted a Key Populations Assessment in 2017. 3.0 EVALUATION OF OTHER PROJECT ASPECTS (RISKS, FIDUCIARY & SAFEGUARDS) 3.1 Financial Management (FM): Evaluation of Risks The Financial Management risk of the Project was assessed as substantial as at appraisal. This initial assessment was not based on the control risks associated with basic elements of the project financial management, but rather on the inherent risks associated with high community involvement in the project. However, these inherent risks were well mitigated by the effective use of the well-established Project Financial Management Unit (PFMU) in the states and the Finance Department of National Agency for the Control of AIDS (FD/NACA). The FD/NACA at the federal level and the PFMUs in the SACAs deployed sufficient financial management staff and resources to the project which ensured: (i) that funds were used only for intended purposes; (ii) the production of timely information for project management and government oversight; and (iii) compliance with IDA fiduciary requirements. However, there were still a few issues around internal control and documentation, such as questionable and ineligible expenditures, unretired advances and insufficient documentation. The reports of a Financial Management monitoring 47 carried out by the World Bank in 2013 raised concerns on these three components of the internal control systems at both NACA and SACAs. It is note-worthy that both NACA and the SACAs had developed strategies to implement the recommendations contained in the report. A follow up FM mission is currently with the project at the time of preparing this completion report and the final report of that mission would demonstrate how much the project had done to ensure a more robust internal control system. 3.2 Compliance with Legal Covenants Finance Comments S/N Agreement Description of Covenant Status Reference Each participating SACA shall replenish its account by the Varying Schedule 2, 1 first day of each calendar semester by at least $25,000 | degree of Section I, A.4. Frequency: Yearly compliance Schedule 2, NACA and each SACA shall prepare and submit annual Complied 2 Section 1,A, 6 work plans to the Association not later than September of with (a) the preceding year. Where applicable, NACA and each SACA shall compile Issue on Environmental Safeguards Complied Schedule 2, and submit to the Association consolidated reports on the were not referenced until shortly 3 with but Section I, E, 3 status of compliance with the ESMF, EMPs and MWMP after the Mid-Term Review Mission after delay quarterly. of Oct/Nov. 2013 Only 13 States had promulgated the Schedule 2, All participating States and the FCT shall transform their Complied necessary Legal Instrument. By 4 Section II, A SACAs into Financially autonomous statutory Agencies not with project closure all SACAs had Subsection 2h later than June 30, 2011. partially become statutory Agencies. Fully Schedule 2, NACA and each SACA shall submit to the Association 5 complied Section II B 2 Quarterly Interim Financial Reports with NACA and each SACA shall submit Annual Audit and Fully Schedule 2, 6 Financial statement and Reports to the Association not later complied Section II B 3 than six (6) months after the end of each financial year. with The ICR rating of the Financial Management is moderately satisfactory. 3.3 Internal Audit Under the HPDP II the Internal Audit ensured the orderly and efficient conduct of operations including, among others, the adherence to project management policies, rules and regulations, safeguarding of project assets, prevention and detection of errors and irregularities as well as promoting the accuracy and reliability of the accounting records. It was also to ensure that all expenses in the project were wholly, exclusively, reasonably and necessarily incurred. 3.4 Procurement Procurement performance was generally satisfactory. The procurement performance in the states was enhanced by the support provided by the national office. Two Procurement Consultants that have good knowledge and wide experience in implementing procurement activities under the World Bank also supported the national office. In addition, the Procurement Units of NACA and the SACAs received tremendous assistance and support from the Procurement Team of the Bank. The Procurement Methods that were employed for the procurement of goods, and consultancy services were identified at appraisal and included in the Project Appraisal Document (PAD) and the Financing Agreement. This provision was amended during the Mid Term Review with the inclusion of the use of Framework Agreement and the UN Agencies for the procurement of Goods. 48 Overall, the procurement performance was rated satisfactory 3.5 Environmental and Social Safeguards (ESS) The HPDP II triggered the World Bank’s safeguards Policy OP 4.01 (Environmental Assessment). The project is also classified “category B” under the World Bank’s environmental guidelines. This means that anticipated significant adverse environmental and social impacts that would result from the implementation of the project are expected to be minor, site specific, none cumulative and relatively easy to avoid, prevent, and manage to acceptable levels. At the design stage, the following activities were identified to have environmental and social impacts: Minor renovations and repairs within existing facilities to help expand counselling and testing. Provision of rapid testing kits to counselling and testing centres. Provision of condoms. Implementation of waste management plans and sustainable systems related to above at facility level. 3.5.1 Safeguards Instruments for the Project In accordance with World Bank Policies to mitigate risks and to satisfy the Bank's Environmental Assessment requirement, two instruments were prepared for the project at inception: Environmental and Social Management Framework (ESMF): for the renovation of old buildings, construction of new ones and other minor civil works. Medical Waste Management plan (MWMP): provided guidance on how the medical waste generated in World Bank supported health care facilities would be managed. 3.5.2 Safeguards Issues Although there was a whole chapter in the Project Appraisal Document (PAD) on Environmental Safeguards, there was no expert at the project launch to guide project implementers. The absence of a Bank staff to provide the necessary support for adherence to environment and safeguards continued for up to half of the project life. At the Mid Term Review (MTR), it was identified that the project had not been fully compliant with the safeguards requirements, as spelt out in the PAD, ESMF and MWMP. As a result, an Environmental and Social safeguards audit was carried out nationwide in March 2015 to obtain the baseline on ESS implementation, identify gaps and proffer solution. The audit exercise identified series of issues requiring adequate mitigation and concluded that the existing and potential issues could be mitigated. Sequel to this, an action plan and budget detailing mitigation activities/corrective actions to be carried out, with responsible parties proposed and means of monitoring the performance of the activities clearly stated was developed. As a fall out of the audit, a joint Bank/NACA supervision mission (9 - 22 September and 7 – 9 October, 2015) carried out to states clarified the need to introduce Grievance Redress and Feedback Mechanism (GRM) into the project; though the project did not trigger OP 4.10 or 4.12 (Indigenous Peoples, Involuntary Resettlement) for which GRM were mandated. However, the capacity building on ESS carried out to SACA plus HCFs in 2016, and the HCW monitoring logbook field test that was conducted in January 2017 revealed that GRM may not be relevant to healthcare waste management. Nonetheless, a section on GRM has been included in the healthcare waste-monitoring logbook for facilities. Since inception, the project had inadequate ESS supervision (both at the World Bank level and NACA level): This manifested when safeguards were rated unsatisfactory from inception of project up until MTR. In terms of assessing the Bank’s and borrowers’ responsibilities prior to the MTR, there was no clear guidance on implementation and framework to assess the performance and impacts of implementation. Performance indicators, though specified in the PIM, were not integrated into the project’s results framework. Also, data for monitoring and evaluation were not routinely collected or used. Post MTR, there was a substantial improvement in effort in terms of following up on the mitigation measures and action plan prepared to address the safeguards policy triggered. Monitoring logbook (daily register and a monthly summary form), quarterly tracking tool and an assessment checklist were developed for monitoring of safeguards activities and shared with the SACA. A safeguards M&E Plan was also developed at the National level and adapted by the States. Safeguards supervision at national level was integrated into Integrated Supportive Supervision of the States, CSOs and Healthcare facilities. 49 These Borrower’s safeguards supervision responsibilities were fulfilled post MTR: Implementation of agreed technical and institutional measures specified in safeguards action plan Report on status of implementation of agreed actions, as part of regular project monitoring. The third responsibility, which is monitoring and evaluation of environmental and social impacts and risks associated with project was only partially fulfilled due to several reasons (stated in the table). World Bank’s safeguards supervision roles also became more prominent post MTR. Ensuring proper coverage of safeguards compliance aspects in Bank’s regular supervision of project progress: WB’s safeguards specialists were part of WB-NACA missions for project supervision until 2016. Review the status of implementation of ESMF: this was also done mostly after MTR. Assisting and guiding the borrower in addressing the issues identified by the supervision. This was done satisfactorily when the WB safeguards specialist started to regularly attend the WB-NACA missions. Review of documentation: proposals and requests, consultancy ToRs etc. This was erratic due to non-availability of the specialists at a point in the project. 3.5.3 Environmental Safeguards Achievements Eight out of thirteen activities approved in the Environmental Safeguards mitigation action plan to be conducted by NACA were implemented as follows: Safeguards Audit Report finalisation and disclosure in-country and World Bank info-shop. Appointment of Safeguards Focal Officers: 2 safeguards officers at national level, 37 from the SACAs. The SACAs appointed focal officers for World Bank supported facilities. Capacity building: On environmental and social safeguards with emphasis on medical waste management and grievance redress/feedback mechanism for NACA, SACA and relevant stakeholders. 35 NACA staff, 10 officers from Federal Line Ministries, 120 SACAs (36+1 states). Step down training for Healthcare Workers, CSOs working on MARPS, LACA Staff and stakeholders on Safeguards issues and effective Waste Management at the State level in 23 States: 1560 state healthcare facility workers, CSOs and LACAs and relevant ministries were trained. Integration of environmental and social indicators into the M&E framework: Development, printing and distribution of quarterly tracking tool and assessment checklist to States. This however came late (January, 2017) and therefore the states were not able to use them for data collection. Development, Printing and Distribution of logbook for recording safeguards events at the healthcare facility level: Procurement: PPE and colour coded bags to support states: Weighing machine, Trolleys, sharps boxes, biohazard bags, bin liners, overalls, leather hand gloves, boots, Gas masks, eye goggles, and hand sanitisers. HCW storage and treatment equipment for project facilities: 72 Septic tanks (2 per state) procured, casted and installed in 2 World Bank supported facilities per state (36 states and the FCT). 4.0 BENEFICIARIES OF HPDP II 4.1 Primary Beneficiaries The primary beneficiaries targeted were Female Sex Workers (FSW); Injecting Drug Users (IDUs); Men who Have Sex with Men (MSM); Orphans and Vulnerable Children (OVCs); Health Care Workers (HCWs); People Living with HIV (PLHIV); People with Disabilities (PWDs); Transport Workers and other men who travel often (inclusive of all mobile itinerant, Uniformed service personnel); Women and Men of Childbearing Age and Young People. The strategy for determining interventions for implementation of activities for these beneficiaries of the project, based on priorities of the National Prevention Plan was Behaviour Change Communication. States were to choose interventions for one or a combination depending on the nature of the epidemic in their area. The project also had some unintended beneficiaries identified during the needs assessment and during the course of implementation. Such target groups included clients of FSWs, OVC caregivers and PLHIV caregivers. Interventions were provided through a variety of programmes such as HCT outreaches, MPPI, FLHE, HBC and OVC intervention packages. 4.2 Secondary Beneficiaries The first sets of secondary beneficiaries were the 36+1 states of the Federation that signed SCAs with the FMOF and NACA that signed the Financial Credit Agreement (FCA). They gained from the project in terms of capacity building. 50 They primarily played a coordination role toward other beneficiaries of the project, provided oversight functions, technical assistance, capacity building and strengthened institutional arrangements. These beneficiaries signed contracts with the private sector to provide services to the primary beneficiaries, while those in the public sector (LMs and LACAs) were supported based on agreements guided by approved work plans. 4.3 Civil Society and Private Sector CSOs and private sector organisations (IPs and PHFs) were engaged in order to expand civil society and private health sector response through the HIV/AIDS Fund (HAF). A total of 557 Civil Society Organisations (CSOs) from 33+1 states provided prevention, care and support interventions to different target groups. During the extension 93 high performing CSOs in 20+1 states that had completed and closed their initial round of HAF contracts were reengaged. The reengaged CSOs focused on providing HIV services to key populations and demand creation for PMTCT services among pregnant women. 3 States (Imo, Ebonyi and Kano) could not work/ engage with CSOs at all on the HAF for HPDP II. For the private healthcare sector engagement, five (5) implementing partners were engaged to coordinate activities that increased the uptake of PMTCT through 376 PHFs using PBF 5.0 ASSESSMENT OF BORROWER AND BANK PERFORMANCE 5.1 Borrower Performance: 5.1.1 Government Performance Rating: Satisfactory As agreed at design, most states pursued with vigour and were able to establish the institutional structure for the project. States that did not have an agency for the control of HIVAIDS were able to set up backed them with necessary legislation. Other states were able to back up their agencies with law. This was a good step towards the sustainability of the national response even after the expiration of the World Bank credit. Project effectiveness was extended for about eighteen months due to delay in meeting certain conditions for effectiveness. This extension accounted for the first restructuring of both the effective date and the project objectives and targets. But despite this initial delay in project effectiveness, the borrower met all legal covenants but delayed in meeting the legal requirements on environmental and social safeguards. Some states did not adhere strictly to the legal covenant in the agreement to retain project staff, as much as possible, for the period of project implementation. There were changes in project teams in many states during the project period. This caused delay in project implementation as the new staff took time to understand Bank’s procedures and guidelines, particularly in procurement and accounting. The states affected included Edo, Anambra, Imo, Yobe, Enugu, Abia, Oyo and Lagos. Added to this was glaring evidences of political interferences in the administration and day to day running of the project in some states. The most glaring case was in Imo state, which led to suspension of disbursement for this state during much of the implementation period. In one or two other cases, the interference had led to some financial improprieties, which were identified during financial management review missions carried out by the Bank. The FM reports had flagged this ineligible expenditures and states affected had to make commensurate refunds. One area of concern is the low level of ownership of the project by some states. Some states did not meet their obligations of paying the agreed counterpart contributions to the project. After the initial deposit for disbursement effectiveness, most states could not pay the required counterpart funds. This raises concern over the sustainability of the project after the credit. Again, there is the likelihood that some state governments may not be able to integrate the project into their budgeting systems to ensure that the current achievements on the project are sustained and improved upon. At the grassroots’ level, The Local Agencies for the Control of AIDS (LACAs) were very active in implementing outreaches. However, they depended on their being included in the SACAs work plans and budgets Above all, the leadership and support from the Federal Government through the Federal Ministry of Finance and state governments were ultimately the determinants of the success of the HPDP II. The FMoF implemented 2 borrowers mission to states that had specific problems during the project. 5.1.2 Implementing Agency or Agencies Performance Rating: Satisfactory The responsibilities of NACA as identified during the project launch in June 2011 were: Overall responsibility for the coordination of the project; 51 To provide overall national strategic directions within which States tailored their activities based on particular characteristics; Responsibility for setting priorities for activities and financing; Provision of technical assistance to States across a full range of project activities and promoting the sharing of experiences of best practices; Approval of Action Plans prepared by Federal Ministries and national civil society organisations proposing multi-state programmes for HAF financing; Planning and implementation of capacity-building activities at the national Level; Preparation of quarterly reports on implementation, compliance with financial, environmental and other legal covenants; Monitoring of the epidemiological situation in the country; and Evaluation of the National response. During the MTR, after almost three years of implementation, HPDP 2 disbursement stood at 31.5% (US$ 72.4 Million) and the project was in the Implementation Status Report (ISR) for Project Development Objectives (PDO) and Implementation Progress (IP) as Moderately Satisfactory (MS) and Moderately Unsatisfactory (MU) respectively. However, timelines were set and aggressive technical assistance as well as advocacy visits were done to improve the situation. NACA staff were deployed to assist SACAs with low capacity and the National level staff also endeavoured to adhere to action plans drawn between NACA and the World Bank. By 30th November 2015, the initial proposed closing date; the overall disbursement rate had increased to about 67 percent thereby justifying the project for an extension of 15 months. The day-to-day implementation coordination work was done through the Partnerships Coordination Department of NACA and a project team made up of relevant skill mix from different units and departments of the Agency. The 37 SACAs comprising of the 36 states and the Federal Capital Territory also replicated this at the states level. The teams were guided by project documents such as the PIM, PAD, HAF Manual, Procurement Guidelines and Financing Agreement. Most of the project staff learned on the job and even though this was cost effective and improved performance on the long run, certain delays could not be avoided. 5.1.3 Justification of Rating for Overall Borrower Performance Rating: Satisfactory Based on the combination of the two ratings above, overall Borrower performance is rated Satisfactory. The experiences gained and lessons learnt on the project are already being applied on the implementation of other development partners and will be useful for the follow up project being processed by the borrower. 5.2 Bank Performance 5.2.1 Bank Performance in Ensuring Quality at Entry Rating: Satisfactory It was evident from the Project Appraisal Document that the Bank provided the necessary support to the project team to ensure a sound project design. The analysis of the economic and human capacity situations of the country, the in- depth review of the achievements, lessons learnt and challenges of the HPDP 1 provided good background for the design and implementation arrangement for the project. The Bank provided an in-depth analysis of the success story of the multi-sectoral approach to the fight against the HIVAIDS scourge and the goal of the National Strategic Framework (2005-2009) to reduce the HIVAIDS incidence by at least 25 percent through equitable prevention, care, treatment and support. The Bank involved State Governments closely in project preparation. The Bank initiated a joint review on HPDP I in November 2006, conducted a joint scooping mission with DFID in September, 2007, the HAF and Public Sector Impact evaluation in October, 2008 and conducted epidemiology and behavioural surveys and supported government to develop a strong and sustainable structure and post credit impact. The end product was the establishment and institutionalisation of the control agencies at the state level and backed by law enabling law. Right from the project launch, the Bank provided support for the government team through cross-sectoral contributions, The Bank Team to the launch included experts in HIVAIDS and social protection, monitoring and evaluation, procurement and financial management, These experts guided the various project implementers and other stakeholders on the goal of the project, the development objectives, implementation arrangements, fiduciary arrangements and 52 framework for monitoring and evaluation. In addition, the Bank facilitated the development of a Medical and Waste Management Plan prior to project effectiveness. However, two issues were of concern and noticeable from entry. The Bank’s support and guidance for the effective and efficient implementation of the social and environmental safeguard components of the project was weak at the entry of the project. Though the Bank facilitated the development of a Medical Waste Management Plan in 2008 and prior to project effectiveness, there was no expert from the Bank on environment and social safeguard at the project launch and no reference was made to the plan until shortly before the Mid-Term Review in October 2013. As will be seen in the subsequent section, this lack of Bank staff on environment and safeguard continued even to halfway in the implementation of the project. Secondly, the Project Launch was designed to provide the necessary orientation on the World Bank guidelines and procedures for project implementation. The orientation should provide stakeholders the opportunity to learn and appreciate the implementation arrangements, fiduciary aspects of project implementation, compliance with legal and social requirements and the first training on project management. The HPDP II project launch, which was held in June 2011. However, in spite of breakout sessions, 5.2.2 Quality of Supervision Rating: Moderately Satisfactory The Bank’s roles and responsibilities were defined in the PAD and these roles were re-echoed and clearly defined during project launch in June 2011. Immediately after the project launch and at the commencement of implementation, the Bank provided support and consistently stressed the need to adhere to guidelines and procedures and work towards the achievements of targeted outcomes. The Bank started early during project implementation with supervision missions which were problem- solving in approach, and provided the necessary support to overcome the problems that were encountered during the project launch and the early implementation period. The Bank provided access to cross support from the multi-sectoral teams that participated in supervisions. The Aide Memoires and other supervision reports were well prepared and shared with the Government and other development partners. In order to facilitate timely review and clearance of project requests, both the Bank and the borrower introduced the monthly review meetings with the national project team. This innovation worked and promoted good communication between the Bank and the implementation team at the early stage. However, later in the project life, the meeting became non-regular and created situations for delays. The outcomes of such meetings were hardly implemented and same issues were being re-visited several times. Since the meeting became the basic method for the revision of requests and proposals, in most instances, the written comments of the Bank were not obtained and the required follow up actions were not clear to the project. Added to this was the observed delay in processing requests for no-objection. The key Bank staffs that worked on the project were deeply engaged in the supervision of many other projects and in some cases in other countries where they visit at intervals. The average turnaround time for processing requests was two months. The Task Team Leader (TTL) was changed twice. The first TTL was based in Nigeria and project implementation started to gather momentum after the initial teething problems. The second was not based in country and supervision was done by the rest of the in-country team, aided by teleconferences and implementation support missions. Generally, there was moderate to high commitment by the task team 5.2.3 Justification of Rating for Overall Bank Performance Rating: Satisfactory The Bank teams were active and played a pivotal role in ensuring quality at entry and in resolving problems and exploiting identified opportunities during supervision missions. However, supervision of safeguards implementation was low for the greater part of project implementation. There were also avoidable delays in processing requests for No Objection, uncoordinated responses to requests that led to back and forth mails between Bank and borrower and unavailability of key staff at critical periods during the project. The Bank made significant contributions to the success of the project. 53 6.0 OVERARCHING THEMES AND OTHER ISSUES 6.1 Gender Aspects and Social Development HPDP II recognised that Gender inequality had been identified as one of the key drivers of the HIV and AIDS epidemics in Nigeria. This was evidenced in 59% of infection among women (NDHS Plus 2012). Women are more vulnerable to HIV infection due to biological, social, economic, cultural issues etc. Women carry the greater burden of care and support, which has effects on their productive lives. Therefore, the project paid particular attention to access of women to HIV/AIDS prevention, treatment, and care and support services. 6.1.1 Achievements in Terms of Gender Mainstreaming Gender was a cross cutting issue mentioned as part of guiding principle in the expired NSP with no clear strategic guidance on its operationalisation. Human rights issues were not clearly mentioned in the Framework and Plan. Following the Joint Annual Review and Mid Term Review of the HIV response in 2013, which indicated weakness of the response in gender mainstreaming and recent reports on human rights issues, this aspect of the national response was strengthened. The introduction of combination prevention domesticated as MPPI, thus creating a platform that supported the delivery of key interventions and services to women, marginalised groups and to PMTCT. CSOs and line ministries were encouraged to work directly with women groups such as market women association, women religious groups, women professionals and artisans. NACA and SACAs ensured that proposals under the HAF targeted women and young girls and that the work plan of line ministries incorporated activities directed at women. For example, the Ministry of Education paid special attention to girls in the roll out of Family Life HIV/AIDS Education (FLHE). SACA Community Mobilisation Officers were trained in gender mainstreaming and promoting access to services by UN-Women through NACA. 6.2 Partnerships In line with the declaration on Aid Effectiveness, the Three Ones and Global Task Team (GTT) recommendation for harmonised procedure by development partners in 2005, the HPDP II supported a range of collaboration with other funding organisations including the United States Government (USG), the Global Fund, DFID, UNAIDS, UNICEF and Government of Nigeria. NACA enjoyed the close working relationship with DFID in financing the development of National Organisational Harmonised Capacity Assessment Tool (NOHCAT), which guided the SACAs to carry out capacity assessment of CSOs. HPDP II also continued to complement Global Fund, UNAIDS and USG support to the national response by providing flexible funds to support their programmes focusing on the scale up of treatment programmes through building capacity of health workers and upgrading of health facilities. University of Manitoba collaborated with NACA to provide technical support and capacity building to SACAs and CSOs that benefitted from HPDP II Fund. Further collaboration existed with the Society for Family Health (SFH), University of Manitoba and other IPs to provide technical assistance on epidemic appraisal (rural appraisal), mapping of MARPs, venue profiling of MARPs, and Learning sites for FSWs and prevention Sexual Ethnography. HPDP II collaborated with UNICEF and Nigeria Bureau of Statistics (NBS) to carry out Multi indicator cluster survey (MICS) on HIV/AIDS. This population-based survey helped to provide information on one of the outcome indicators that measures the PDO before the end of HPDP II in 2017. 6.3 Sustainability Measures The project was expected to be institutionally sustainable in the medium term because it was built upon strong government ownership and broad partnerships. It was expected to: Scale up existing interventions and mainstream HIV/AIDS activities into major government sectors and civil society. Emphasises institutional capacity building at national, state and local levels further enhanced the sustainability of the programme. The transition of NACA and some of the SACAs to agency status will help ensure continued government funding by assuring access to government budget lines. Federal and states government pay staff salaries and also provide operational cost for running of agencies. 54 Current arrangements for sustainability of results include the following. Proposed Inclusion of HIV services into national and state health insurance scheme. Availability of National Anti-Stigma Law to protect the right of people living with HIV since 2014. This law seeks to prevent the stigmatisation of and discrimination against those living with or affected by HIV and AIDS in Nigeria, and seeks to protect their human rights and dignity, while encouraging those that are infected to declare their HIV status in a friendlier community and country. This law has also been passed in 16 states and is at various stages of approval for passing into law in other states. Combination prevention approach was adopted and domesticated as Minimum Prevention Package of Interventions (MPPI) for MARPs and general population programming in Nigeria. Programme science approach was developed to drive evidence-based programming. This was used to assign targets, prioritise geographic locations and populations for engagement of CSOs on HAF implementation. Development of strategic documents and guidelines (e.g. National Strategic Framework, National Strategic Plans, National HIV Workplace Policy, NACA-SACA-LACA Standard Operational Manual, Community Strategy Document and Community Strategy training manual) Availability of National and State programme and technical working platforms for effective programme coordination ( National Council on AIDS, NACA-SACA Forum, NACA-Line Ministries Forum, State Management Team and NACA- CSOs Forum). Availability of monitoring and evaluation infrastructure for capturing and reporting of data. Electronic reporting of data using the DHIS has improved reporting rates, timeliness and completeness of reported data by the states. Improved National and States M&E system for HIV response. Developed environmental and social safeguards logbook for monitoring medical waste generated at health facilities. Step-Up of Budgetary Releases for HIV/AIDS Response due to advocacies. Set up of a HIV/AIDS Trust fund outside the normal budgetary provision. Capacity-building in form of Training and Technical Assistance for funds management, governance, leadership; management systems development and technical skills 7.0 LESSONS LEARNED, CHALLENGES & RECOMMENDATIONS 7.1 Lessons Learned Agency status improved SACA operations: Agency status has provided access to on-going financial support from the state government and gives the SACAs a better institutional platform from which to coordinate other partners. This is verified by state allocation of funds to the SACAs in their yearly budget, although there is need for high-level advocacy to ensure release in some states. Also, SACA staffs are already members of the state’s civil service and as a result, the state governments pay salaries. Strengthening LACA proved effective for successful implementation of activities at community level: States that were able to ensure well-trained staff worked for the LACAs recorded successes in implementation at the community level. For example, in some states LACA officers had linkages with all the facilities in their LGAs, made and tracked referrals, served as community liaison persons, supervised CBOs and TBAs and conducted community mobilisation activities and awareness campaigns. Development of intervention packages contributed significantly to standardisation of prevention effort as implementation of these packages increased within the national response. Focus on targeted interventions (e.g. HCT for MARPs) has been introduced and this facilitated resource mobilisation for prevention activities (Global Fund, World Bank HAF-Fund, PEPFAR and GoN/PCRP). A shortcoming was that SACAs were trained along with the CSOs they engaged in order to save time. This was because the process of standardisation was not completed by the project launch in 2011. Strengthened programme intelligence was key for identifying National and State HIV programme priorities: Other lessons learned include the production of more evidence based programming (MARPs programming EPI-Appraisal), development of guidelines and frameworks (e.g. PMTCT demand strategy, Female Sex Workers (FSWs) implementation guidelines and tools, SBCC users guide) and facilitated the development of SBCC short courses in research and academic institutions. Service Support Organisations were useful for filling CSO capacity gaps: Service Support Organisations (SSOs) that were engaged to provide M&E technical support to CSOs implementing the HAF helped to bridge technical capacity gaps at the CSOs and this boosted M&E supervision and mentoring for the CSOs. 55 Improved technical capacity at the state level led to better coordination of stakeholders by SACAs: HPDP2 has contributed significantly to strengthening the national M&E system for HIV at all levels, particularly at the state level. Improved technical capacity at the state level has led to better coordination of stakeholders by SACAs leading to better organisation and output from state monthly M&E meetings, quarterly data validation exercises as well as mentoring & data verification at facilities by states. Mobile technology and electronic reporting of data using the DHIS platform has improved data quality and reporting rates (timeliness and completeness) by the states (both at health facility and CSO levels): The use of mobile phones for reporting has been shown to improve the data management in terms of improved reporting rates and timeliness of report from hard-to-reach PHCs. The reporting rates and timeliness of report from these PHCs increased considerably, averaging 90% and 85% respectively. By design, the project did not provide for the procurement of ARVs. The project relied on the supply of drugs by other donors. However, this became an issue during project implementation when other donors put a cap on the supply of the product. The PAD states that the potential impacts of medical waste generated by project activities are minor and site specific, however reports of safeguards audit and Integrated Supportive Supervisory (ISS) visits carried out showed that medical waste generated have grave implications for health care workers, waste handlers and the community at large and pose potential negative impacts. Therefore, issue of safeguards should be given more emphasis in any follow-on project. In all the facilities visited for the safeguard logbook field- test, respondents revealed that there were no tools for recording data on waste management and none had records of waste generated and disposed. The logbook for monitoring of HCW developed by NACA in collaboration with FMoH, FMOEv, NESREA and other stakeholders can be used as a National tool for recording data on waste management as a sustainability measure. The provision and installation of incinerators within 70 km radius of the HCF was emphasised in the MWMP and Safeguards audit report. The ISS carried out also confirmed that most facilities did not have standard incinerators. A follow-on project should make provision of incinerators a priority. 7.2 Challenges Late implementation of clustering exercise caused delays in implementing the cluster model approach for MARPs, which would have made their interventions more effective, increased targets reached and reduced barriers to access Passing the law against same sex relationships made it difficult to programme for and reach MSM in most parts of the country Project effectiveness was delayed by 22 months due to a long ratification process at the National Assembly based on a misinterpretation of project objectives. The processes of engagement of CSOs were delayed till 2012 after capacity of SACAs were built to adequately implement HAF. However, the extension granted gave the opportunity to cover the lost time. The modification of key indicators in 2015 during project extension redefined the focus of PDO to prevention interventions only. This to a large extent limited the achievement made on care and support services for OVC and PLHIV. Delay in getting No Objections from the Bank slowed implementation of planned activities. Reminders helped to elicit responses The knowledge gap on safeguards implementation both at NACA and SACA levels led to slower implementation pace. A lot of time was spent on technicalities of developing the Waste Management logbook, printing and distribution of safeguard materials due to non-existence of national tools (for guidance) that capture data on medical waste in the healthcare facilities or even with the stakeholders (FMoH, FMoE, JSI and NESREA). The existence of parallel procurement system for the World Bank and NACA resulted in bureaucratic delays due to waiting for approval from several lines before commencement of activities. The Treasury Single Account (TSA) policy of the government in compliance with the requirement significantly curtailed the ability of the project to access and disburse funds for close to 5 months (October, 2015-February, 2016). 56 The process invariably stagnated the disbursement rate of the project. Also, some State PIUs overdrew their allocated amount of $5million, and some overdrew their category schedule. The FM of NACA was only able to carry out one supervisory mission to SACAs, which was largely inadequate for a project with a magnitude of $225million credit. Due to these inadequacies, NACA was not able to make more effective performance update from SACAs. The Project experienced late submission by SACAs of IFR to NACA, audited financial statement for consolidation, and non-withdrawal applications for further funding of the project. Removal of SACA project staff by some State Governments delayed implementation of activities as some were the key signatory to the project accounts. Incomplete documentations, late retirement and ineligible expenditures by PIUs affected speedy implementation 7.3 Recommendations The design of the project credit not to finance procurement of ART affected the project effectiveness in the area of impact mitigation. This became obvious when the supply of ART drugs from other projects in the country were not sufficient for the HPDP II activated health facilities. For sustaining the gains of the project; Government (or subsequent projects) should consider including the procurement of ART in order to reduce reliance on donor-supported projects. The proposed follow on project should consider treatment and PMTCT as one of its major area of financial support. Streamlining of procurement by both the Bank and the implementing agency processes to hasten turnaround time for commencement of activities. World Bank should improve on fast tracking process of issuing No Objections for implementation of activity. Engagement of World Bank safeguard specialist for training at the National and State level should be done at the start of follow on project with simplified standard training modules. SACAs should ensure submission of IFR and audited financial statement on time, and sanctions imposed for non- compliance. When there are “rules of engagement” for a process the likelihood of non-compliance is reduced or eliminated. Bank and NACA to re-programme the project to flag exception to mitigate PIUs from overdrawing their allocated amount and their category schedule. Frequent supervisory mission to SACA PIUs to reduce/eliminate incomplete documentations/late retirement and ineligible expenditures by stakeholders. The World Bank should provide adequate training for the Internal Audit and recognition as an independent Unit. To overcome most of the implementation challenges such as inadequate supervision, delayed rendition of reports and scale up efficiency and effectiveness of implementation, subsequent projects should operate as a separate PIUs similar to many best-practice in World Bank-assisted projects in Nigeria and elsewhere. 57 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders 58 Annex 9. List of Supporting Documents Federal Government of Nigeria. 2005 National Economic Empowerment and Development Strategy (NEEDS) Federal Government of Nigeria. 2000-2004 National HIV/AIDS Emergency Action Plan (HEAP). Federal Government of Nigeria. 2006-2010 National Health Sector Strategic Plan for HIV/AIDS. Federal Government of Nigeria. National Strategic Framework for HIV/AIDS, 2005-2009. Federal Government of Nigeria. National Strategic Framework for HIV/AIDS, 2010-2014. Federal Government of Nigeria. National Policy on HIV/AIDS, 2009. Federal Government of Nigeria. National Strategic Health Development Plan, 2010-2015. Federal Government of Nigeria. National HIV/AIDS Strategic Plan, 2010-2015. Federal Government of Nigeria. Global AIDS Response Country Progress Report (GARPR) 2015. Federal Government of Nigeria. President’s Comprehensive Response Plan (PCRP) for HIV/AIDS, 2013-2015 Federal Ministry of Health (FMOH). Nigeria Integrated Biological and Behavioural Surveillance Survey 2007. Federal Ministry of Health (FMOH). Nigeria Integrated Biological and Behavioural Surveillance Survey 2010. Federal Ministry of Health (FMOH). Nigeria Integrated Biological and Behavioural Surveillance Survey 2014. Federal Ministry of Health [Nigeria] (2008). National HIV/AIDS and Reproductive Health Survey, 2007 (NARHS Plus). Federal Ministry of Health Abuja, Nigeria Federal Ministry of Health [Nigeria] (2013). National HIV/AIDS and Reproductive Health Survey, 2012 (NARHS Plus). Federal Ministry of Health Abuja, Nigeria. MC Boilya, M Picklesa, CM Lowndes, et al. Positive impact of a large-scale HIV prevention programme among female sex workers and clients in South India. AIDS 2013, 27:1449–1460. LG Bekker, L Johnson, F Cowan, et al. Combination HIV prevention for female sex workers: what is the evidence? Lancet 2015; 385: 72–87. National Agency for the Control of AIDS. 2016 Stigma and Discrimination Reduction in the National HIV/AIDS Response. Abuja, Nigeria. International Development Association. Financing Agreement for a Second HIV/AIDS Program Development Project between Federal Republic of Nigeria and the International Development Association. Credit Number 45967-NG. November 25, 2010. The World Bank and DFID. Country Partnership Strategy for the Federal Republic of Nigeria (2005-2009). Report No. 32412-NG. June 2, 2005. The World Bank Group. Implementation Completion and Results Report on a Credit to the Federal Republic of Nigeria for a HIV/AIDS Program Development Project. Report No: ICR00001674-NG. May 28, 2011. 59 The World Bank Group. Project Appraisal Document on Proposed Credit to the Federal Republic of Nigeria for a Second HIV/AIDS Program Development Project. Report No: 46343-NG. May 20, 2009. The World Bank Group and UK DFID. Country Partnership Strategy for the Federal Republic of Nigeria (2005- 2009). June 2, 2005. The World Bank, UK DFID, USAID, and the African Development Bank. International Development Association Country Partnership Strategy for the Federal Republic of Nigeria (2010-2013). July 2, 2009. The World Bank Group. International Bank for Reconstruction and Development, International Development Association, International Finance Corporation, and Multilateral Investment Guarantee Agency – Country Partnership Strategy for the Federal Republic of Nigeria for the Period FY2014-2017. March 13, 2014. 60 Annex 10. Changes to Indicators in Results Framework Table 1: Original Project indicators approved in PAD, May 20, 2009. s/n Indicators Baseline Project Target PDO Indicators 1 Percentage of young women aged 15-24 reporting the use of a 34.3% 45.4% condom during the (last) sexual intercourse with a non-regular partner of those reporting sexual intercourse with a non- Data Source: regular partner in the last 12 month). NARHS 2007 2 Percentage of young men aged 15-24 reporting the use of a 52.2% 69.2% condom during the (last) sexual intercourse with a non-regular partner of those reporting sexual intercourse with a non- Data Source: regular partner in the last 12 month). NARHS 2007 3 Percentage of women aged 15-49 who have had sexual 1.0 0.4% intercourse with more than one partner in the last 12 months Data source: NARHS 2007 4 Percentage of men aged 15-49 who have had sexual 7.2 5.0% intercourse with more than one partner in the last 12 months Data source: NARHS 2007 5 Percentage of most-at-risk groups (Brothel Based Female 64.8% 72% Sex Workers) reporting the use of a condom during (last) sexual intercourse with a non-regular partner of in the last 12 Data source: months. IBBSS 2007 6 Percentage of most-at-risk groups (Non-Brothel Based 67.1% 75% Female Sex Workers) reporting the use of a condom during (last) sexual intercourse with a non-regular partner of in the Data source: IBBSS last 12 months. 2007 7 Percentage of persons aged 15 and older who received 72.7% 96% counseling and testing for HIV and received their test results Data source: NARHS 2007 8 Percentage of pregnant women living with HIV who receive a 5.25% 30% complete course of antiretroviral prophylaxis to reduce the risk of MTCT Data source: FMOH Program Data 2007 9 Ratio of current school attendance between orphans and non- 75% 87% orphans Data source: CRS 2006 and FMOWA OVC Situation Analysis. 10 Percentage of most-at‐ risk populations (Female Sex 32.93% 50% Workers) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major Data source: misconception about HIV transmission. IBBSS 2007 61 11 Percentage of most-at‐ risk populations (Men who have Sex 44.0% 57% with Men) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major Data source: misconception about HIV transmission. IBBSS 2007 12 Percentage of most-at‐ risk populations (Injecting Drug 34.0% 47% Users) who both correctly identify ways of preventing the sexual transmission of HIV and who reject major Data source: misconception about HIV transmission. IBBSS 2007 Intermediate Results Indicators Component I 1 Percentage of SACAs that hold partner forum at least twice a 0 100% year to report on planning, decisions, and progress 2 Number of Federal Line Ministries implementing HIV/AIDS 28 13 workplans 3 Number of State Line Ministries implementing HIV/AIDS 281 259 workplans 4 Number of LACAs implementing HIV/AIDS workplans TBD TBD 5 Percentage of schools that provided life skills-based HIV 33.61% 100% education in the last academic year Component II 6 Number of PLWHA groups and associations that receive a 0 TBD basic package of support from CSOs that implement the HIV/AIDS Fund (HAF) program 7 Number of orphans that receive a basic package of support 0 TBD from CSOs that implement the HIV/AIDS Fund (HAF) program 8 Percentage of Private Sector and CSOs that report regularly 0 50% and on time on project indicators: Federal level Private Sector Organizations (PSOs) 9 Percentage of Private Sector and CSOs that report regularly 71% 95% and on time on project indicators: Federal level CSOs 10 Percentage of Private Sector and CSOs that report regularly 74% 90% and on time on project indicators: State level CSOs Component III 11 Percentage of Public Sector, Private Sector, and CSOs with 0 100% improved financial accounting practices 12 Number of SACAs, State, and federal level public sector 2 13 organizations that have a written strategic plan clearly linked to the National Strategic objectives and priorities: Federal LMs 13 Number of SACAs, State, and federal level public sector 18 37 organizations that have a written strategic plan clearly linked to the National Strategic objectives and priorities: SACAs 14 Percentage of staff receiving TA/training who report that they 0 100% have used their new knowledge and skills and can give examples 15 Percentage of actual funds utilized relative to planned targets NA 100% 62 16 Number of States that link the National program priorities 35 37 with their annual operational plans 17 Number of States that link the State program priorities with 18 37 their annual operational plans 18 All Participating States and the Federal Capital Territory have 13 35 transformed their SACAs into financially autonomous statutory agencies Note: Outcome Indicator to track the number of adults and children with HIV receiving antiretroviral combination therapy was included in the results framework in ISR #1 (dated 10/22/2009) and ISR #2 (dated 5/10/2010) before Project Effectiveness, but was never commenced. This indicator was not mentioned either previously in the PAD or afterwards in subsequent ISRs or in the First Restructuring Paper. Table 2: Project indicators at the First Restructuring, December 16, 2011. s/n Indicators Baseline Project Target Comment PDO Indicators 1 Percentage of young women aged 15-24 34.3% 40% End of Project target reporting the use of a condom during the revised downward 5.4% (last) sexual intercourse with a non- Data Source: (from 45.4%) regular partner of those reporting sexual NARHS 2007 intercourse with a non-regular partner in the last 12 month). 2 Percentage of young men aged 15-24 52.2% 60% End of Project target reporting the use of a condom during the revised downward 9.2% (last) sexual intercourse with a non- Data Source: (from 69.2%) regular partner of those reporting sexual NARHS 2007 intercourse with a non-regular partner in the last 12 month). 3 Percentage of women aged 15-49 who 1.0 0.6% End of Project target have had sexual intercourse with more revised upward 0.2% than one partner in the last 12 months Data source: (from 0.4%) NARHS 2007 4 Percentage of men aged 15-49 who have 7.2 5.0% No change had sexual intercourse with more than one partner in the last 12 months Data source: NARHS 2007 5 Percentage of most-at-risk groups 68.0% 72% Incorrect 2007 baseline (Brothel Based Female Sex Workers) data in PAD was updated reporting the use of a condom during Data source: (last) sexual intercourse with a non- IBBSS 2007 regular partner of in the last 12 months. 6 Percentage of most-at-risk groups (Non- 61.0% 65% Incorrect 2007 baseline Brothel Based Female Sex Workers) data in PAD was updated reporting the use of a condom during Data source: (last) sexual intercourse with a non- IBBSS 2007 End of Project target for regular partner of in the last 12 months. Non-Brothel Based Sex Worker revised downward 10% (from 75%) 63 7 Number of women and men aged 15-49 4,400,000 10,000,000 Previous ambiguous who received an HIV test in the last 12 indicator name (percentage months and who know their results Data source: of persons aged 15 and NDHS 2008 older who received counseling and testing for HIV and received their test results) revised in line with UNGASS indicator that measures progress by number. Baseline and end of project target revised to reflect number as a unit of measurement 8 Percentage of most-at‐ risk populations 32.9% 50% No Change (Female Sex Workers) who both correctly identify ways of preventing the Data source: sexual transmission of HIV and who IBBSS 2007 reject major misconception about HIV transmission. 9 Percentage of most-at‐ risk populations 44.0% 57% No Change (Men who have Sex with Men) who both correctly identify ways of preventing the Data source: sexual transmission of HIV and who IBBSS 2007 reject major misconception about HIV transmission. 10 Percentage of most-at‐ risk populations 34.0% 47% No Change (Injecting Drug Users) who both correctly identify ways of preventing the Data source: sexual transmission of HIV and who IBBSS 2007 reject major misconception about HIV transmission. 11 Number of pregnant women living with 26,133 40,000 Indicator previously HIV who received a complete course of provided as a percentage. antiretroviral prophylaxis to reduce the Data source: risk of MTCT [Core Indicator] FMOH 2010 Intermediate Results Indicators Component I 1 Number of health facilities that provide 1046 1600 New HIV counseling and testing services 2 Percentage of health facilities that provide 4% 5% New HIV counseling and testing services 3 Percentage of HIV-positive patients who 87% 90% New were screened for TB in HIV care or treatment settings 4 Percentage of schools that provided life 22.8% 30% No change skills-based HIV education in the last academic year Component II 64 5 Percentage of organizations that report 0% 50% Timing changed to quarterly and on time on agreed project “quarterly” indicators: Federal level Private Sector Organizations 6 Percentage of organizations that report 71% 95% Timing changed to quarterly and on time on agreed project “quarterly” indicators: Federal level CSOs 7 Percentage of organizations that report 74% 80%? Timing changed to quarterly and on time on agreed project “quarterly”; target revised indicators: State level CSOs from 90% 8 Percentage of organizations that submit 0% 80% New quarterly Financial Monitoring and Procurement report to specific guidelines: (CSOs and Private Sector Organizations) Component III 9 Availability of Annual HAF performance 0 1 New report prepared by NACA 10 Percentage of SACAs and Public Sector 0 80% Moved from component I; Organizations that hold quarterly partner changed meeting schedule forums to report on planning, decisions, to quarterly; Added Public and progress. Sector Organization aspect (MDAs); target revised from 100% to 80%. 11 Number of SACAs that have a written 18 37 No change strategic plan clearly linked to the National Strategic objectives and priorities 12 Number of federal level public sector 2 13 No change organizations that have a written strategic plan clearly linked to the National Strategic objectives and priorities 13 Number of LACAs implementing TBD TBD Moved from Component I HIV/AIDS work plans. Dropped PDO indicator: i) Ratio of current school attendance between orphans and non-orphans – dropped because of measurement and collection complexities by the Government and UNICEF. Dropped Intermediate Indicators: i) Number of Federal implementing HIV/AIDS workplans (Compo I); ii) Number of State Line Ministries implementing HIV/AIDS workplans (Compo I); iii) Number of PLWHA groups and associations that receive a basic package of support from CSOs that implement the HIV/AIDS Fund (HAF) program (Compo II); iv) Number of orphans that receive a basic package of support from CSOs that implement the HIV/AIDS Fund (HAF) program (Compo II); v) Percentage of Public Sector, Private Sector, and CSO organizations with improved financial accounting practices (Compo III); vi) Percentage of NACA and SACA staff that were trained in past year and are demonstrating new skills in job responsibilities (Compo III); vii) Percentage of actual funds utilized relative to planned targets (Compo III); viii) Number of States that link the National programme priorities with their annual operational plans (Compo III); ix) Number of States that link the State programme priorities with their annual operational plans (Compo III); 65 x) All Participating States and the Federal Capital Territory have transformed SACAs into financially autonomous statutory agencies (Compo III). Table 3: Project Indicators at the Third Restructuring, November 19, 2015 S/N Indicators Baseline End of Project Comment Target (Feb 2017) PDO Indicators 1 Percentage of young women aged 15-24 34.3% 40.0% No change reporting the use of a condom during the last sexual intercourse with a non-regular Data source: partner in the last 12 month). NARHS 2007 2 Percentage of young men aged 15-24 52.2% 60% No change reporting the use of a condom during the last sexual intercourse with a non-regular Data source: partner in the last 12 month). NARHS 2007 3 Percentage of Female Sex Workers 68% 72% No Change (Brothel-Based) reporting consistent condom use with casual partner in the last Data source: 12 months. IBBSS 2007 4 Percentage of Female Sex Workers (Non 61% 65% No Change Brothel-Based) reporting consistent condom use with casual partner in the last 12 Data source: months. IBBSS 2007 5 Number of women and men aged 15-49 2,287,805 10,000,000 Baseline data of 4,400,000 who received an HIV test in the last 12 from 2008 NDHS was months and who know their results Data source: updated. FMOH 2010 6 Number of pregnant women living with 26,133 40,000 No change HIV who receive a complete course of antiretroviral prophylaxis to reduce the risk Data source: of MTCT [Core indicator] FMOH 2010 7 Percentage of female sex workers who were 38.8% 43% New outcome indicator tested for HIV in the last calendar year added as part of the Data source: restructuring. IBBSS 2010 Intermediate Results Indicators Component I 1 Number of health facilities that provide 1,064 (4%) 1,600 No change HIV counseling and testing services (UNGAS 2010) 2 Percentage of health facilities that provide 4% 5% No change HIV counseling and testing services 3 Schools that provided life skills- based 22.80 30 No Change HIV education in the last academic year Component II 66 4 Percentage of organizations that submit 0% 80% No Change quarterly Financial Monitoring and Procurement report to specific guidelines: CSOs and Private Sector Organizations 5 Number of new most-at-risk populations 0 10,000 New (MARPs) reached with MPPI 6 Number of project supported private 0 25 New health care facilities providing prevention of mother to child transmission services 7 Number of project supported private 0 25 New health care facilities providing HIV testing and counseling services 8 Number of female sex workers who were 25,596 40,000 New tested for HIV in the last calendar year Component III 9 Availability of Annual HAF performance No Yes No change report prepared by NACA 10 Percentage of SACAs and Public Sector 30% 80% Baseline changed from 0 Organizations that hold quarterly partner forums to report on planning, decisions and progress 11 Number of SACAs, State and federal level 37 37 No change public sector organizations that have a written strategic plan clearly linked to the National Strategic objectives and priorities: SACAs 12 Number of SACAs, State and federal level 2 4 Target revised downward public sector organizations that have a as a result of transition to written strategic plan clearly linked to the a more focused approach. National Strategic objectives and priorities: Federal LMs/MDAs 13 Number of LACAs implementing 0 272 Designated baseline and HIV/AIDS work plans. End-of-Project Target. Dropped PDO indicators: i) Percentage of women and men aged 15-49 who have had sexual intercourse with more than one partner in the last 12 months. ii) Percentages of most-risk populations (FSWs, MSMs, and IDUs) who both identify ways of preventing the sexual transmission of HIV and who reject misconception about HIV transmission. Dropped Intermediate Indicators: i) Percentage of HIV-positive patients who were screened for TB in HIV care and treatment settings (Compo I); ii) Percentage of Private Sector and CSOs that report quarterly and on time on agreed project indicators (Federal level Private Sector Organizations, Federal level CSOs, State level CSOs) (Compo III); iii) Percentage of Private Sector and CSOs that report quarterly and on time on agreed project indicators (Federal level CSOs) (Compo III); iv) Percentage of Private Sector and CSOs that report quarterly and on time on agreed project indicators (State level CSOs) (Compo III). 67 Annex 11. MTR details 1. Joint Government-Bank mini-QER of the HPDP2’s implementation stage was held on June 27, 2013 to address ongoing challenges. The discussion focused on three broad categories as described in Table 1. Table 1. QER Discussion Points and Agreed Next Steps Discussion Categories Action plan moving forward 1. M&E Data: i) difficulty tracking PDO achievements in absence of i) Requested that FGON release 2012 NARHS data routine data from comprehensive Health Management Information results to track progress; ii) Reviewed RF indicators System (HMIS) and infrequent national surveys. and targets to ensure they remained realistic and achievable by Project closing date. 2. Implementation of Component 1: Public sector response had been less i) Increased focus on health sector response by than optimal due to i) service delivery gaps in PMTCT and HCT given making funds more available to meet service that Nigeria accounted for about 30% of the global burden of MTCT of delivery gaps in PMTCT and HCT; ii) Reduced HIV and the FGON’s own efforts to scale-up PMTCT activities; and ii) involvement of Federal and State LMs to health, the non-health sector that hardly implemented service delivery services youth, women affairs, and education for greater but focused instead on workshops, trainings, and meetings that had no service delivery impact that reach target direct impact on contributing to PDO achievement or project results10. populations. 3. Implementation of Component II through HAF grants to CSOs and i) Prioritized evidence-based interventions to 12+1 Private Sector: No disbursement to date since no contracts had been high prevalence States by providing extensive TA, finalized and signed. Challenges revolved around a revised HAF process developed agreed template for contract negotiation that was i) not well understood by NACA and SACAs, and ii) required and focused much of HAF resources at providing use of epidemic appraisal findings from each state in order to develop intervention for MARPs who were major drivers of contacts focused on qualified entities; clearly defined services; and HIV epidemics and demand creation for PMTCT geographic priority areas specific to each state. and HCT services. 2. At the MTR, which was conducted on 10/30/2013-11/15/2013, the Project rating for its progress towards achieving the PDO was MS, while the rating for implementation progress was MU, along with risk flags for the following categories: Country Record, Project Management, Financial Management, Slow Disbursement, and Safeguards. Achievements noted during the MTR mission included i) progress in four out of six PDO indicators towards reaching the end Project targets; ii) increased use state level epidemiology to drive policy at all levels; iii) FGON adoption of methodology and tools used in epidemic appraisals in states as national standard to ensure consistency in approach irrespective of the funder; iv) increased use of epidemiological data to prioritize programming; and v) increased political commitment to the HIV/AIDS response through the 2013-2015 President's Comprehensive Response Plan for HIV/AIDS (PCRP). Despite these achievements, the MTR highlighted the slow pace of implementation and disbursement as described in the table below: Table 2. MTR findings – challenges and mitigation measures Challenges Mitigation measures 1. The original PDO stated 3 objectives subsumed into The new proposed PDO, To increase access to and utilization of one: Reduce the risk of HIV infections (PDO 1) by HIV prevention interventions, care and support services in scaling up prevention intervention (PDO 2) and to Nigeria, was intended to focus on the core deliverable of the increase access to and utilization of HIV counseling, Project as designed, improve accountability, and better align testing, care, and support services (PDO 3). deliverables with priority interventions supported by the Project. The revised PDO was to be reflected in a revised legal agreement The ICR author notes that the approved PAD states 2 as part of an upcoming Project Restructuring. objectives as: To reduce the risk of HIV infections 10 An analysis of 111 randomly selected requests for No Objection between January and June 2013 revealed that less than only 1% of requests were for service delivery activities while over 50 were for trainings and workshops. 69 by scaling up prevention interventions (PDO 1) and to increase access to and utilization of HIV counseling, testing, care and support services (PDO 2). 2. Weak M&E data collection systems. Data sources to i) SACAs to collect data through regular supervisions (every 2 track progress of performance indicators were either months) using quantitative supervisory checklist; ii) NACA to problematic (DHIS 2) or infrequently conducted support completion of 2013 IBBSS for baseline data on FSW national surveys (NARHS and IBBSS conducted every assessment and conduct two other IBBSS-like surveys before end 3-5 years). of Project to track progress among MARPs; iii) conduct NARHS every two years as was done in the past. 3. Ineffective non-health sector response in scaling-up i) Allocate at least 70% of the public sector response resources to service delivery. Activities of LMs focused on the health sector response to rapidly scale-up HCT and PMTCT trainings, meetings and workshops rather than services; ii) Limit Project financing to the following four LMs with delivering services to clients at large enough scale to specified mandates for service delivery: have an impact. While the training is an integral part of 1) Min. of Health – scale-up HCT and PMTCT services, scaling up quality services, they are not sufficient to prevention interventions for MARPS and general population; 2) achieving the desired impact on their own. Also, HIV Min. of Youth and Sport – peer education & social mobilization of plans for most LMs fully dependent on Bank Credit, in and out of school youths; 3) Min. of Education – scale-up FLHE limiting their effectiveness and sustainability beyond and other in-school interventions; 4) Min. of Women Affairs – the project life. work with OVC and their care givers; income generation activities (IGA) for PLWHA; and create demand for PMTCT services. 4. Slow disbursement rate due to delay in implementing i) Conducted workshops to share experience of states with HAF component designed to provide grants to CSOs effective HAF processes; ii) establish framework contracts with and private sector. A revised HAF process to improve manufacturers and suppliers if HIV test kits, consumable, and effectiveness required state specific data to coordinate condoms to facilitate efficient and quality procurement. responses with defined geographic coverage area, target population, and priority interventions. Adjusting to the new concept took longer than expected as it required intensive technical support, strong coordination, and buy-in at the national and state levels. 5. Institutional arrangements for HIV/AIDS remained i) Strengthen Government’s oversight of HPDP 2 and Institutional weak, specifically i) NACA and SACA financing Arrangements; ii) follow up by NACA with the Office of the through national and state budgets did not improve Secretary to the Government of the Federation (SGF) to expedite despite their conversion from committees to official constitution of NACA Board and with states without Boards to do agencies; ii) non-existent Board oversight at NACA and the same; iii) form State Management Teams (SMT) under the some SACAs; iii) lack of project oversight function by leadership of the State Commissioners for Health to improve FMOH; v) most LM did not allocate resources to fulfill resourcing, coordination, reporting and oversight responsibilities of their mandates on the multi-sectoral HIV/AIDS the state governments HIV/AIDS response. response thereby making mainstreaming efforts under the project ineffective. 6. Limited human resource capacity hindered i) The Bank provided assistance in hiring second procurement coordination and procurement services, including, i) consultant to assist initially hired procurement consultant to inadequate procurement capacity to undertake technical accelerate contract negotiations under the HAF and review bidding nature of HAF process; ii) poor quality of bidding documents by the States; ii) SACAs to procure HIV test kits, documents and evaluation reports that delayed reagents and lubricants through framework contracting to be procurement of goods, such as test kits and reagents; established with technical support from both NACA and the Bank iii) delayed recruitment of national Service Support to ensure quality and cost efficiencies. Organizations (SSOs) due to frequent changes in scope and TOR of SSOs. 7. Unallocated funds (US $15 million) needed to be Allocated funds as follows: (i) assessment of FSW program - allocated to emerging priorities. $1,000,000; (ii) procurement of test kits, condoms, lubricants, and STI drugs - $8,500,000; (iii) demand creation for PMTCT - $4,000,000; (iv) operation research - $1,000,000; and (v) end of project evaluation and documentation - $500,000. The proposed allocation was to be reflected in the legal agreement. 70 8. Preliminary safeguards audit highlighted challenges (i) complete safeguards in-depth audit in 8 states; (ii) strengthen in the use of safeguard instruments by states, such as (i) institutional capacity for ESMP preparation, execution and little emphasis made on social safeguards, (ii) weak monitoring; (iii) recruit/designate environment and social implementation and reporting; and (iii) absence of focal safeguards officers at the NACA, SACAs, LACA, and health points at the national, state and health facility levels. facility levels to coordinate safeguards related issues and ensure Rating for the safeguards performance was “U”. compliance; and (iv) design an easy to use template for safeguards reporting. 3. The proposed changes were formally incorporated in the Project through a restructuring that was requested by the FGON on March 3, 2011 and approved by the Bank on May 23, 2014. Also, the PDO as stated in the PAD was not changed and maintained throughout the Project. At the time of the MTR, disbursement rate was at 23%. After the MTR the project team was more involved in States preparation of annual work plans which were reviewed on their technical and procurement merits. Once approved the States had the latitude to implement the work plans and only come to the Bank team to get no objections on fewer prior review activities. This increased drastically the overall disbursement rate of the project to 83%. 71 Annex 12. Revised Disbursement Estimates Revised Disbursement from First Restructuring, December 16, 2011. Quarter Original (USD) Formally Revised (USD) Q4-FY-2009 0 0 Q1-FY-2010 0 0 Q2-FY-2010 0 0 Q3-FY-2010 3,000,000.00 0 Q4-FY-2010 5,000,000.00 0 Q1-FY-2011 10,000,000.00 0 Q2-FY-2011 15,000,000.00 0 Q3-FY-2011 15,000,000.00 0 Q4-FY-2011 15,000,000.00 1,500,000 Q1-FY-2012 15,000,000.00 3,000,000 Q2-FY-2012 15,000,000.00 3,000,000 Q3-FY-2012 15,000,000.00 3,000,000 Q4-FY-2012 15,000,000.00 3,000,000 Q1-FY-2013 15,000,000.00 4,000,000 Q2-FY-2013 15,000,000.00 4,500,000 Q3-FY-2013 15,000,000.00 6,500,000 Q4-FY-2013 15,000,000.00 7,000,000 Q1-FY-2014 22,000,000.00 10,000,000 Q2-FY-2014 20,000,000.00 10,000,000 Q3-FY-2014 0 12,000,000 Q4-FY-2014 0 12,000,000 Q1-FY-2015 0 15,000,000 Q2-FY-2015 0 18,000,000 Q3-FY-2015 0 18,000,000 Q4-FY-2015 0 20,000,000 Q1-FY-2016 0 25,000,000 Q2-FY-2016 0 25,000,000 Q3-FY-2016 0 25,000,000 TOTAL 225,000,000.00 225,000,000.00 Revised Disbursement from Second Restructuring, May, 23, 2014. Quarter Original (USD) Formally Revised (USD) Q4-FY-2009 0 0 Q1-FY-2010 0 0 Q2-FY-2010 0 0 Q3-FY-2010 3,000,000.00 0 Q4-FY-2010 5,000,000.00 0 Q1-FY-2011 10,000,000.00 0 Q2-FY-2011 15,000,000.00 0 Q3-FY-2011 15,000,000.00 0 72 Q4-FY-2011 15,000,000.00 1,550,000 Q1-FY-2012 15,000,000.00 4,200,000 Q2-FY-2012 15,000,000.00 3,078,518.29 Q3-FY-2012 15,000,000.00 6,822,472.97 Q4-FY-2012 15,000,000.00 15,250,650.49 Q1-FY-2013 15,000,000.00 6,727,768.33 Q2-FY-2013 15,000,000.00 8,475,675.71 Q3-FY-2013 15,000,000.00 6,447,905.15 Q4-FY-2013 15,000,000.00 9,529,128.73 Q1-FY-2014 22,000,000.00 5,458,003.64 Q2-FY-2014 20,000,000.00 7,325,313.68 Q3-FY-2014 0 10,134,562.77 Q4-FY-2014 0 15,000,000 Q1-FY-2015 0 20,000,000 Q2-FY-2015 0 25,000,000 Q3-FY-2015 0 25,000,000 Q4-FY-2015 0 25,000,000 Q1-FY-2016 0 10,000,000 Q2-FY-2016 0 10,000,000 Q3-FY-2016 0 7,500,000 TOTAL 225,000,000.00 225,000,000.00 Revised Disbursement from Third Restructuring, November 19, 2015. With a 15- month project extension from November 30, 2015 to February 28, 2017, the disbursement estimates will change to ensure that activities under the 3 components of the project are implemented in 2016 and within the current funding envelope. Fiscal Year Current (USD) Proposal (USD) 2009 0.00 0.00 2010 8,000,000.00 0.00 2011 55,000,000.00 750,000.00 2012 60,000,000.00 23,750,000.00 2013 60,000,000.00 50,000,000.00 2014 42,000,000.00 38,000,000.00 2015 0.00 42,000,000.00 2016 0.00 70,500,000.00 TOTAL 225,000,000.00 225,000,000.00 73 MAP 74