Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) Report Number: ICRR0022503 1. Project Data Project ID Project Name P130299 National AIDS Control Support Project Country Practice Area(Lead) India Health, Nutrition & Population L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-52360 31-Dec-2017 237,122,284.72 Bank Approval Date Closing Date (Actual) 01-May-2013 30-Jun-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 255,000,000.00 0.00 Revised Commitment 255,000,000.00 0.00 Actual 237,122,284.72 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Salim J. Habayeb Joy Maria Behrens IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (p. 4), the project's objective was "to increase safe behaviors among high risk groups in order to contribute to the Recipient's national goal of reversal of the HIV epidemic by 2017." At a May 2018 restructuring, the objective was revised to reflect the fact that the project's closing date had been extended by two years at a 2017 restructuring. The revision removed the date reference, to read "to Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) increase safe behaviors among high risk groups to contribute to the national goal of reversal of the HIV epidemic." When the closing date was extended in 2017, outcome targets were revised upward to reflect the additional implementation time. As outcome targets were all adjusted upward in 2017, and the project's scope was not affected by the change in wording of the objective in 2018, a split rating is not warranted. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 03-May-2018 c. Will a split evaluation be undertaken? No d. Components The project had three components, all of which included implementation at the national, state, and district levels. The ICR did not provide information on actual project costs, total or by component, to reflect the government's contribution. Only actual Bank costs are reported here: Component 1: Scaling Up Targeted Prevention Interventions (appraisal: US$440 million, of which US$220 million from the Bank; actual Bank costs US$183.01 million). This component was to support the scaling up of targeted interventions (TIs) with the aim of reaching out to the hard-to-reach population groups who did not yet access and use the prevention services of the National AIDS Control Program (NACP), and saturate coverage of high-risk groups (HRGs). The Project Appraisal Document (PAD) defined the HRGs to be targeted as female sex workers (FSWs), men having sex with men (MSM), the transgender (TG)/Hijra population, and people who inject drugs (PWIDs). In addition, the component was to support partners of PWIDs and bridge populations, especially migrants and truckers. Prior to implementation, the Department of AIDS Control (DAC)/National AIDS Control Organization (NACO) had identified a standard methodology to validate HRG estimates and site assessment, identifying and confirming locations within states and districts where TIs were required to teach populations most at risk. To understand mobility patterns among HRGs, community-led social network analysis was ongoing to identify hard-to-reach HRGs, such as new or young sex workers. The component included two sub-components:  Scaling up coverage of TIs among HRGs: Implementation of proven TIs with a focus on FSWs, MSM, the TG/Hijra population, and IDUs, through the contracting of non-governmental organizations (NGOs) and community-based organizations (CBOs). Support was to cover ongoing TIs as well as the launching of new interventions, encompassing: the provision of behavior change interventions to increase safe practices, testing and counseling, and adherence to treatment, and demand for other services; the promotion and provision of condoms; provision or referral for services for sexually Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) transmitted infections; and needle and syringe exchange for PWIDs as well as scaling up opioid substitution therapy. This sub-component also was to finance operating costs for State Training Resource Centers and participant training costs for five years.  Scaling up interventions among other vulnerable populations: Mapping of peer networks to improve access to prevention services among partners of sex workers, partners of PWIDs, and bridge populations including migrants and long-distance truckers moving between high- and low-prevalence areas and engaging in unsafe practices. Support was to include: risk assessment and size estimation of migrant population groups and truckers at transit points and workplaces; behavior change communication (BCC) for creating awareness about risk and vulnerability, prevention methods, and availability and location of services; promotion and provision of condoms through various channels including social marketing; development of linkages with local institutions, both public and NGO-owned, for testing, counseling, and sexually transmitted infection treatment services; creation of peer support groups and safe spaces for migrants at their destinations; establishment of need-based and gender-sensitive services for partners of PWIDs; and strengthening of networks of vulnerable populations with enhanced linkages to services centers and risk reduction interventions, especially condom use. In 2017, after several years of evidence had accumulated proving that anti-retroviral therapy (ART) reduced viral load to the point that persons on treatment were highly unlikely to transmit HIV to others, the project added a "treatment as prevention" approach and added provision of ART under this component. Component 2: Behavior Change Communications (appraisal: US$40 million, of which US$20 million from the Bank; actual Bank costs US$6.11 million). This component was to coordinate and implement BCC activities, including: (i) communication programs (media campaigns, creative development campaigns, and short films) for risk reduction and safe behavior, including advocacy, social mobilization, and BCC to integrate people living with HIV/AIDS and HRG into society and to encourage normative changes aimed at reducing stigma and discrimination in both society at large and in health facilities; (ii) financing of a research and evaluation agency to assess the cost-effectiveness and program impact of BCC activities; and (iii) establishing and evaluating a helpline at the national and state level to further increase access to information and services. Component 3: Institutional Strengthening (appraisal: US$30 million, of which US$15 million from the Bank; actual Bank costs US$48 million). This component was to support DAC/NACO's steering coordination, and managerial roles in managing the prevention component of the NACP. It was to support innovations to enhance performance management, including fiduciary management, at the national and state levels. Support for institutional capacity was also to help strengthen procurement and supply chain management. Staff and operating costs of Technical Support Units (TSUs) were to be covered for a period of 3-4 years to ensure oversight of the quality of TIs through monitoring and supportive supervision, building capacity of states and assisting them in effective use of available information for evidence-based planning, program roll-out, and performance monitoring. This component was also to support the services of a procurement agent for buying medications for opioid substitution therapy. Finally, this component was to support an annual knowledge dissemination forum and audits. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Financing, costs, and Borrower contribution: The project was to be financed by a US$255 million specific investment loan and a US$255 million contribution from the Borrower, for total project financing of Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) US$510 million. The Bank loan fully disbursed at US$237.21 million, with the difference between planned and actual financing explained by US$/XDR exchange rate fluctuations. Most planned funds under Component 1 (89%) were disbursed. Only one-third of planned funds were spent under the second component, as the behavior change activities required the development of fewer centralized materials than anticipated, producing savings that were informally reallocated to the third component. Component 3 spent over three times what was originally planned for two reasons: in 2017, it was decided to support all 17 state-level TSUs rather than only 9 as initially envisioned; and the human resource costs of contracting staff for oversight at the state level were financed by the component, which was not initially planned. Dates: The project was approved on May 1, 2013 and became effective on July 22, 2013. It underwent a mid-term review in May 2016. Its original closing date was December 31, 2017. The project underwent four restructurings:  March 17, 2017: Given new evidence on the effectiveness of treatment-as-prevention, rapid rollout of HIV treatment became a priority. HIV treatment was therefore added to Component 1. To accommodate the implementation of the added activity, the project end date was extended from December 31, 2017 to December 30, 2019. Intermediate indicators focusing on HIV treatment were added, and targets for most other indicators were revised upward to reflect two additional years of project implementation.  May 3, 2018: The phrase "by 2017" was removed from the objective.  June 23, 2019: The Central Medical Services Society was added as an implementing agency to facilitate procurement of goods, specifically those related to HIV treatment.  November 25, 2019: The closing date was extended for six months, to June 30, 2020, to allow for remaining disbursements and implementation of activities. 3. Relevance of Objectives Rationale At the time of appraisal, India's national burden of HIV and AIDS ranked third globally in terms of number of infections, after South Africa and Nigeria. Although it was on track to meet the Millennium Development Goal to halt and reverse the epidemic, it was still experiencing a highly diverse and heterogeneous epidemic scenario concentrated among HRG, driven by sex work, unprotected sex among men having sex with men, and injection drug use. Infection rates were also a concern among "bridge populations," people who interact with HRG and are "bridges" to the general population, primarily male migrants and long- distance truck drivers. National HIV prevalence had started to level off in the late 1990s, with decline accelerating over the following decade due to high coverage of prevention interventions, but progress among states was highly variable. Some states still faced emerging epidemics among HRG, and at the national level, migration, the low status of women, and widespread stigma remained challenging. The project's objectives were highly responsive to this country context. At the end of 2012, India was wrapping up its third National AIDS Control Program (NACP III, 2007-2012), which had scaled up targeted HIV prevention interventions for most at-risk population groups and expanded Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) the surveillance system. During NACP III, targeted prevention interventions had reached 81% of FSW, 66% of MSM, and 81% of PWIDs. Anti-retroviral treatment for adults had increased by 30% between 2009/2010 and 2010/2011, and estimated annual deaths from HIV had declined from almost 200,000 in 2006 to about 172,000 in 2009. Although this progress was impressive, some states lagged behind, and some population groups -- young sex workers, PWIDs and their partners, MSM, and transgender persons -- remained disproportionately at risk. In the fourth phase of the NACP (2012-2017), India aimed to accelerate the reversal of the epidemic by reaching out to HRG with targeted prevention interventions through innovative approaches; increasing access to comprehensive care, support, and treatment; expanding information, education, and communication with a focus on behavior change, demand generation, and stigma reduction; further strengthening institutional capacity and processes of integration and convergence of program components with those common to other diseases, conditions, or programs; and continuing to innovate to generate knowledge and lessons. Programmatically, the fourth phase aimed to shift more responsibility for financing, managing, and implementing critical components and activities (such as facility-based testing and treatment, blood safety, and other health services) from the DAC/NACO and State AIDS Control Society (SACS) to the National Rural Health Mission and government health services. HIV prevention programs that rely on effectively reaching HRG and vulnerable populations through peer outreach were to continue to be separately managed through the contracting of NGOs and CBOs under DAC/NACO. The project was highly relevant to this government strategy in its support for outreach to the most vulnerable and marginalized population groups, increasing their access to and utilization of services, and reducing stigma and discrimination against them through TIs and BCC. Although domestic financing for India's HIV response had increased significantly, the saturation of HRG with TI programs, including the expansion of those programs to new geographies and new emerging HRG and vulnerable populations, required new, external resources. The project responded to this need. It specifically supported three of NACP IV's five elements: prevention, behavior change, and institutional strengthening. The other two elements of the strategy, care and treatment for people living with HIV and AIDS and strategic information systems, were being supported by the national budget and other donors at appraisal. The project was relevant to the Bank's Country Assistance Strategy for India at appraisal (CAS, 2012-2017) through its support for increasing the effectiveness of service delivery (a CAS pillar), and its focus on achieving the 2015 Millennium Development Goals. It remained relevant to the Country Partnership Framework at closing (2018-2022), which included "Investing in Human Capital" as a Focus Area, with specific reference to supporting NACP IV; a specific objective on improving the quality of health service delivery and financing; and an outcome indicator for percentage of HRG receiving ART. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) OBJECTIVE 1 Objective Increase safe behaviors among high risk groups to contribute to the national goal of reversal of the HIV epidemic Rationale The project's intervention logic held that funding for civil society and communities to implement targeted interventions, complemented by behavior change communication and supported by institutional strengthening, would enable implementation of an array of TIs focused on key populations: peer outreach activities, condom and lubricant distribution, HIV and STI counseling and testing, harm reduction services, and crisis management/legal/social services. Underpinning these activities were behavior change efforts through mass media, development of information, education, and communication (IEC) materials, and stigma reduction campaigns. Strengthened institutions were assumed to be better able to support the capacity, targeting, and quality of the TIs through improved HRG population size estimates, grading of NGOs to ensure quality of services, on-time contracting of NGOs, and reduction in stock-outs of condoms and other supplies. It was assumed that increased availability of TIs, coupled with increased knowledge among HRG and reduced stigma, would lead to increased demand among HRG and bridge populations for the TIs being offered under the project. As the project was revised in 2017 to include treatment as prevention, the project's logic expanded to acknowledge the impact of increased HIV testing, ART coverage, and ART retention on preventing the spread of infection. Ultimately, increased TI coverage and uptake would lead to safer sex practices among HRG and bridge populations and safer injection practices among PWIDs, and expanded coverage of ART would lead to increased viral suppression, all contributing to reversal of the HIV epidemic. Outputs The project supported NGOs and CBOs to conduct 1,426 peer-led TIs that provided prevention services to, on average, 85% of members of core group HRGs:  662,000 of the estimated total 868,000 FSWs  238,000 of the estimated total 427,000 MSMs  40,550 transgender persons  140,000 of the estimated 177,000 PWIDs NACO led the development of innovative implementation strategies to accelerate TI coverage: community mapping to guide TI activities, use of social media and community platforms for TIs, opioid substitution therapy services for PWIDs, community-based HIV and STI screening and testing, indexed testing, and community-based HIV treatment through linked ART centers and differentiated service delivery models. These innovations culminated in the development of a TI implementation revamping strategy, published in 2019 and based in part on the experiences of this project. The project implemented 90% of its planned prevention interventions for bridge populations, reaching 4.8 million migrants and 1.3 million truckers with BCC, provision of condoms, and referral services for sexually transmitted infections, integrated counseling and testing, and ART. As of March 2020, SACS and TSU had Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) partnered with 743 private-sector partners in 22 states serving 2.7 million workers/migrants for various HIV/AIDS-related services. The program established three million condom outlets across the country. Condom distribution met estimated need at a level of 87% among FSWs, 88% among MSM, 94% among PWIDs, and 90% among transgender persons. Needle and syringe distribution met 77% of estimated demand. About 80% of high-burden states and districts implemented an IEC/BCC strategy with a focus on demand generation and stigma reduction. IEC materials were developed for reduction of stigma and discrimination, reduction of unsafe behaviors among PWIDs, information-seeking behaviors, regular testing, correct and consistent condom use, and facilitation of ART adherence. As of May 2020, the National AIDS Helpline had recorded over 3.1 million calls, of which almost 70% were inquiries about service provision. At project closing, the Helpline was addressing about 200 distress calls per day. Intermediate Outcomes The percentage of FSWs reached by TIs over the previous year increased from 80% in 2012 to 90% in 2020, exceeding the target of 85%.The percentage of FSWs who had been counseled and tested over the previous year increased from 35% in 2012 to 85% in 2020, exceeding the target of 80%. The percentage of high-risk MSM reached by TIs over the previous year increased from 67% in 2012 to 79% in 2020, exceeding the original target of 75% and the revised target of 72%. The percentage of high-risk MSM who had been counseled and tested over the previous year increased from 35% in 2012 to 86% in 2020, exceeding the target of 80%. The percentage of PWIDs reached by TIs over the previous year increased from 81% in 2012 to 94% in 2020, exceeding the original target of 83% and the revised target of 85%. The percentage of PWIDs who had been counseled and tested over the previous year increased from 28% in 2012 to 83% in 2020, exceeding the target of 70%. The percentage of TIs that validated high-risk group data size in the previous year increased from 60% in 2012 to 85% in 2020, exceeding the original target of 70% and the revised target of 75%. The percentage of TIs graded according to the performance indicators of the program's Strategic Information Management System (SIMS) increased from 65% in 2012 to 94% in 2020, exceeding the original target of 85% and essentially meeting the revised target of 95%. The percentage of TIs reporting condom stock-outs in the previous quarter decreased from 10% in 2012 to 2% in 2020, meeting the original target of 2% and exceeding the revised target of 7%. The percentage of prevention interventions for migrants that were implemented as planned increased from 70% in 2012 to 98% in 2020, exceeding the original target of 85% and the revised target of 90%. The Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) percentage of prevention interventions for truckers that were implemented as planned increased from 70% in 2012 to 95% in 2020, exceeding the original target of 85% and the revised target of 90%. The percentage of NGOs contracted as per the SACS annual plan increased from 60% in 2012 to 96% in 2020, exceeding the original target of 85% and the revised target of 95%. The percentage of states updating, reporting, and responding to dashboard indicators increased from 75% in 2012 to 97% in 2020, exceeding the original target of 85% and the revised target of 90%. Outcomes Baseline values were taken from 2009, prior to the start of the project, because of a delay in the planned Integrated Bio-Behavioral Surveillance (IBBS) survey that was initially scheduled for 2012 and was to provide project baseline data.  The percentage of FSWs who reported using a condom with their last client increased from 80% in 2009 to 96% in 2020, exceeding both the original target of 85% and the revised target of 95%.  The percentage of MSM who reported using a condom during sex with their last male partner increased from 45% in 2009 to 85% in 2020, exceeding both the original target of 65% and the revised target of 90%.  The percentage of PWIDs who did not share injecting equipment during their last injecting act increased from 45% in 2009 to 88% in 2020, exceeding both the original target of 65% and the revised target of 85%. While the outcome indicators did not measure progress toward the national goal of reversing the HIV epidemic, the ICR (pp. 28-29) provided relevant data indicating that TIs leading to safer behaviors among HRG may have contributed to this goal. HIV prevalence among FSWs decreased from about 3% in 2012 to below 1.6% in 2017. Declines among MSM were steady, from about 4% in 2013 to 2.7% in 2017. Figure 7 of the ICR (p. 29) indicated that prevalence also declined, though less sharply, among PWID; the project team later provided source data for that figure indicating that prevalence among PWID declined from 13.19% in 2003 to 6.26% in 2017. There was also a steady reduction in prevalence among bridge populations, from about 1.6% in 2006 to 0.5% in 2017 among migrants, and from about 2.4% in 2006 to 0.8% in 2016 among truckers. The ICR stated that no incidence data were available for HRG, and therefore it is not possible to determine definitively whether declines in prevalence were due to decreased incidence or increased mortality. The ICR stated that "the ICR team believes that HIV prevalence reduction among HRG is likely due to lowered HIV incidence because HIV-related deaths also declined in this same period," but no specific mortality data were provided for HRG. Rating Substantial Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Attribution: There was no impact evaluation or systematic counterfactual analysis reported in the ICR. The contribution of domestic financing for the national response had increased significantly since NACP I (1992-1999), when it was 15%. According to the ICR (p. 7), the government provided 63% of HIV financing in India from 2012-2017; the Global Fund provided 14%; the United States and other partners provided 13%; and the Bank provided 10% through this project. Other donors focused primarily on treatment, while the Bank's focus was primarily on targeted prevention interventions. The ICR does not state whether there were any other interventions during the project's lifetime that focused specifically on targeted prevention interventions among HRG; the project team later confirmed that the Bank was the only donor that specifically focused on targeted interventions among HRGs during NACP IV, although other donors did finance innovations and focused prevention, care, and support interventions in select high-HIV burden districts during this time period. Although observed outcomes cannot therefore be attributed exclusively to the project, the integrity of the project's internal logic indicates that its interventions contributed significantly to the adoption of safer behaviors by FSWs, MSM, PWIDs, and bridge populations. Efficacy is rated Substantial, due to convincing evidence that project-financed interventions contributed significantly to safer behaviors among high risk groups and their partners, evidence that the project adapted its interventions appropriately as new evidence about HIV prevention became available, and indications that the observed behavior changes contributed to the national goal of reversing the HIV epidemic. Overall Efficacy Rating Substantial 5. Efficiency The PAD's economic analysis (pp. 71-74) focused on the cost-effectiveness of saturation coverage of HRGs with targeted interventions. It presented analysis demonstrating that government and development partner spending had amounted to US$104 per HIV infection averted, and US$10.7 per disability-adjusted life year (DALY) averted. Discounting at 3%, TIs specifically for FSWs were found to cost US$105.5 per HIV case averted and US$10.9 per DALY averted. Given India's per-capita gross domestic product, the PAD noted that TIs were therefore a cost-effective strategy for HIV prevention. The ICR did not conduct a formal economic analysis because the project did not measure HIV infections averted. Allocative efficiency of the original project design was high, as the targeted interventions supported by the project had been previously demonstrated to be cost-effective. A 2015 analysis using the Optima model was conducted by the Burnet Institute and the Public Health Foundation of India in the states of Karnataka (declining epidemic) and Punjab (increasing epidemic). The analysis confirmed that NACP IV's approach of targeting HRG and bridge populations was the most appropriate and efficient for minimizing new HIV infections and deaths. This study also found that incorporating ART, increasing HIV counseling and testing programs, and Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) increasing opioid substitution therapy coverage would be the most cost-effective ways to reduce new infections and deaths. The restructuring of the project to add treatment-as-prevention therefore increased its allocative efficiency. Technical efficiency was reported to be strong, with unit costs of five key HIV/AIDS commodities (condoms, HIV test kits, anti-retroviral (ARV) drugs, first-line ART, and second-line ART) all markedly lower under NACP IV than global reference prices (drawn from data from the Global Health Cost Consortium, the Global Fund, and UNAIDS). Similarly, unit costs to reach the project's targeted populations were lower than global reference prices, although no comparative data were provided for transgender persons/Hijra, migrants, or truckers. Administrative efficiency was enhanced through the use of implementation structures and institutional arrangements that were consistent with the overall NACP. The project was managed by DAC/NACO at the central level and SACS at the state level. Few contractual issues were reported, and procurement processes were efficient (ICR, p. 32). Joint mission reviews that combined inputs from development partners "saved time for NACO and local implementing partners" (ICR, p. 38). The project responded flexibly to changes of implementation arrangements and some implementation challenges:  Under NACP III, a National Technical Support Unit (NTSU) was created to improve the technical quality of activities and provide guidance to 17 state-level TSUs. During the implementation of NACP IV, the government abolished the NTSU, and responsibility was shifted to the 17 state TSUs. The project responded by reallocating funds to support them.  The project's initial approach relied on reaching members of HRG in physical locations where they congregated. At the 2017 restructuring, intervention strategies were revamped to acknowledge widespread use of various social media platforms in addition to in-person gatherings, identifying and reaching hidden and hard-to-reach populations through social media channels.  ART provision was decentralized and employed differentiated service delivery models, adapting to meet the varied needs of HRG and eliminating the challenges of attending frequent ART visits at crowded ART centers (ICR, p. 27). These models included fast-tracking of ART refills, decentralized ART dispensation at community support centers, multi-month dispensation, and community ART refill groups. However, implementation efficiency was negatively impacted by inconsistent provision of government financing for the HIV and AIDS response over the project's lifetime. Just after project approval, the central government enacted a policy change that required state governments to provide counterpart financing for the NACP in a 60:40 central:state arrangement. States were not prepared for this added budget responsibility, producing a three- to six-month "disruption of TI activities, delays in implementers receiving funds (and some TI activities had to close down as a result of it), and shortage of HIV test kits and ARV drugs" (ICR, p. 19). The counterpart financing policy was reversed in FY2017-18, but TI implementation was nonetheless slowed. These delays were compounded by a 32% cut in the central AIDS budget in 2015-2016, which partially recovered through a reversal of course from 2017 onward. Financing from other development partners was also inconsistent over the project's lifetime. According to the ICR (p. 19), from 2018 forward, "it took a concerted effort to get the national HIV program back on track to achieve the results set out in this project." Efficiency Rating Substantial Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project's relevance is rated High, as its objective was fully derived from the country's national HIV response strategy and was consistent with the Bank's Country Partnership Framework at project closing. Efficacy is rated Substantial due to evidence that project-financed interventions contributed significantly to safer behaviors among high risk groups and their partners. Efficiency is also rated Substantial, even in the absence of a formal economic analysis, as allocative, technical, and administrative efficiency were strong, and implementation inefficiencies were only moderate. These ratings indicate that the project experienced essentially minor shortcomings in preparation and implementation, leading to an Outcome rating of Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The Indian government took significant ownership of the HIV response and now finances over 60% of the HIV program budget. It has adopted important legislative changes during NACP IV that will contribute to sustainability of achieved outcomes: the Indian Supreme Court passed a landmark judgment in 2018 to decriminalize homosexuality; a Transgender Persons Protection of Rights bill was passed in December 2019; an amendment to the Narcotics Drugs and Psychotropic Substances Act was passed allowing for management of drug dependence through opioid substitution therapy and other harm reduction services; and the Human Immunodeficiency Virus and Acquired Immunodeficiency Syndrome Act was passed in 2017 criminalizing any discrimination against persons living with HIV and ensuring the right of persons living with HIV to ART treatment. The project contributed to significant institutional strengthening that is likely to be sustained. The ICR (pp. 33- 34) reported that TSUs enhanced networks of civil society and grassroot-level CBOs and NGOs to ensure improved quality of interventions and data collection. M&E and surveillance systems, researchers, and research institutions were strengthened through participation in sentinel surveillance activities, technical reviews and meetings, and participation in publications and scientific conferences. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) The ICR (p. 45) noted that adequate steps have not been taken to ensure the continued integration of HIV activities with other health services provided at the community level. 8. Assessment of Bank Performance a. Quality-at-Entry Project design drew lessons from implementation of previous NACPs, including the impact and cost- effectiveness of TIs targeted at the right beneficiary groups, the need to allocate resources to highest- priority drivers of the epidemic, the importance of structural amplifiers like stigma and discrimination, the role that functional technical support for SACS/NACO can play in ensuring the quality of TIs, and the importance of generating and disseminating information on successful innovations and good practices. Numerous best practices were carried over from NACP III, including the computerized financial management system, annual performance reviews of participating NGOs, biannual financial audits, IBBS and size estimation methods, and continuing support for TSUs. Overall implementation risk was assessed as moderate, with substantial risk specifically related to state-level governance. Mitigation measures included institutional capacity building and strengthening of fiduciary arrangements for SACS, enhanced monitoring control observed by DAC/NACO over SACS, and capacity development for NGOs and CBOs. Project preparation and design were highly consultative. During project preparation, DAC/NACO and SACS worked closely with various CSOs, associations, and advocacy groups representing various stakeholders and affected populations. The results framework and M&E arrangements were realistic, reflecting the project's tight intervention logic and allowing for tracking of progress through routine as well as surveillance data. Safeguards were well prepared (see Section 10a). The ICR (p. 44) noted two minor shortcomings at entry: insufficient differentiation of implementation strategies for hard-to-reach HRG, and failure to anticipate the possibility of government funding cuts. Quality-at-Entry Rating Satisfactory b. Quality of supervision The supervision team had been part of the previous two phases of India's HIV response and included well- known experts in HIV prevention and treatment (ICR, p. 35). The task team leaders and key staff had experience working in the country and good understanding of its context. Monitoring of safeguards and fiduciary compliance was adequate (see Section 10), and the project team was proactive with project restructuring to raise outcome targets and add treatment-as-prevention activities in response to emerging evidence of its effectiveness. The Bank's relationships with government counterparts and key development partners were strong, and coordination was smooth. The ICR (p. 45) described coordination with other donors as "best practice." The ICR noted that the Bank team could have more effectively encouraged the government to scale up pre-exposure prophylaxis and HIV self-testing and revised and updated methodologies for estimating the size and characteristics of HRG. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design DAC/NACO was responsible for project monitoring and evaluation. Its computerized management information system had been incorporated into a web-based SIMS that, as the project was being prepared, was in the process of roll-out across the country. These systems were to generate data on key indicators to track performance. This core performance data was also to be included in a national dashboard, to be updated for the project and to serve as a management tool for SACS, DAC/NACO, and development partners. The project's results framework was aligned with the broader DAC/NACO framework for 2012-2017 and included a subset of those indicators and targets. Indicators, baselines, targets, data sources, and data collection arrangements were well specified. Targets were set based on triangulation of multiple data sources. For the evaluation of key behavior indicators, DAC/NACO was to implement a second-generation surveillance system and conduct the first round of national IBBS in 2012-2013, building on the experience of an integrated bio-behavioral assessment carried out in selected high-prevalence states in 2006 and 2009. The first-round national IBBS was to provide project baseline data to complement the existing annual HIV sentinel surveillance (HSS) system that generated data on HIV prevalence among HRG and the general population; that survey was delayed, however, and project baseline levels were based only on earlier HSS data and other sources At restructuring in 2017, outcome targets were revised to accommodate two additional years of project implementation, and four intermediate targets were added to reflect the addition of treatment services to the first component. In 2019, some intermediate indicator targets were revised downward to accommodate the slowdown and closure of some TI activities (described in Section 5). b. M&E Implementation India has had a long-standing M&E system for its HIV response, and the three pillars of that system -- routine data, HIV sentinel surveillance, and research -- continued and were strengthened under the project. The project institutionalized a routine reporting system to monitor performance. In 2014-2015 (two years after originally planned), the world's largest and most comprehensive IBBS survey was conducted. In 2020, based on subsequent new thinking about the types of epidemiological data needed to inform an effective HIV response, a "BSS Lite" was implemented. The BSS Lite was used to estimate the prevalence of HIV-related risk and safe behaviors, knowledge, attitudes, practices, and service update among key population groups. A National HIV/AIDS Research Plan was created to address gaps Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) in the national response and generate systematic evidence; 33 studies and evaluations, chosen from 91 identified priority areas for evidence generation, were commissioned during the project's lifetime. c. M&E Utilization According to the ICR (p. 37), monthly review of local data collected from TIs and quarterly reviews with technical support from the TSU "created a tight feedback loop that enabled rapid course correction and informed the many adaptations and refinements during implementation." Project-supported capacity building for M&E officers, statisticians, and program managers stationed at the state and TSU levels helped them to analyze data and disseminate findings at the site level to inform program adjustments and adaptive management. The national surveillance system was used to provide data on levels and trends in the HIV epidemic, make epidemiological projections, and guide resource allocation. IBBS and BSS Lite data were disseminated through workshops at regional and national levels, and over a dozen peer-reviewed articles were published based on the data and analysis. M&E Quality Rating High 10. Other Issues a. Safeguards The project was Environmental Assessment category "B" and triggered OP/BP 4.01, Environmental Assessment, and OP/BP 4.12, Involuntary Resettlement. Environmental and social safeguard risks were considered low, as the project was to finance prevention rather than treatment or basic services like blood banks or ART. The primary environmental risks associated with the project related to the handling and disposal of infectious waste resulting from HIV preventive activities. The safeguards approach for the project mirrored that for NACP III, which had been rated satisfactory for safeguards performance. A well- functioning infection control and waste management system had been institutionalized at the national and state levels. In preparation for NACP IV, DAC/NACO undertook a situation and gap analysis with recommendations for improved implementation. These recommendations were detailed into an Action Plan that was posted on the DAC/NACO website. The ICR (p. 36) reported that no major environmental safeguard violations were documented during implementation. The project team later confirmed that the environment health and safety risks associated with the provision of ART (added at restructuring) were manageable within the existing safeguards management framework, and no critical issues were reported. DAC/NACO and SACS had been strengthening focus on gender, social inclusion, greater involvement of persons living with HIV, and reaching out to vulnerable and tribal populations. A Social Assessment was conducted, contributing to a Gender Equity and Social Inclusion Strategy prepared specifically for the project. The ICR (pp. 32-33) noted that gender was a cross-cutting theme of the project and was included in all operational guidelines, strategies, and activities under NACP IV, and that gender equity issues among HRG were addressed by focusing TI activities among sexual minority and other most vulnerable groups who faced social exclusion. HRG were involved at all levels of project strategy development, coordination, Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) and implementation, and many implementing NGOs and CBOs were led by members of HRG. No physical structures or facilities were constructed, and therefore there were no involuntary resettlement risks. b. Fiduciary Compliance Financial management: NACO had a robust financial management system. Despite strong pre-existing guidelines, however, project financial management was rated Moderately Unsatisfactory in 2015 and 2016 due to the new state government counterpart financing provisions described in Section 5, uneven performance across states, and delayed submission of some financial reports. New staff were recruited and supervision was intensified, after which the rating improved to Moderately Satisfactory. Of the 28 interim unaudited financial management reports submitted during the project's lifetime, 45% were received on time. Most delays were between 2014 and 2016. There were minimal deviations between reported unaudited expenditures and audited expenditures. Procurement: Project procurement followed established procedures. When HIV treatment activities was added after the mid-term review, a third-party procurement agent was engaged to ensure that drugs were rapidly procured and distributed as needed. This arrangement worked well, and no significant stockouts were experienced. During a post review in June 2020, it was discovered that three consultancy contracts had been awarded to a firm that is a dependent agency of a public sector enterprise owned by the Ministry of Health and Family Welfare (and therefore not eligible for project contracts). NACO reallocated these expenditures to its own budget codes. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Highly effective M&E arrangements and institutionally Quality of M&E Substantial High embedded systems, as well as high performance, triangulation, and adaptation. Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) Quality of ICR --- Substantial 12. Lessons The ICR (pp. 46-48) provided lessons and recommendations that were grounded in the project's experience but, in many cases, extended that experience to other disease areas and contexts. They included: Peer networks can be effective agents of communication and change, as this project's experience illustrates. This lesson led the ICR to pertinently recommend the use of peer networks to advance COVID-19 vaccination programs. Differentiated care models that provided client-specific services can facilitate the delivery of precise and targeted interventions. During this project, the fast-tracking of ART refills, decentralized ART dispensation at community support centers, multi-month dispensation, and community-led dispensation produced reduction in wait times and improved treatment adherence. Similar differentiated care strategies might be effectively extended to treatment and management of chronic diseases. Successful interventions are underpinned by good data. This project's success was predicated on the findings of well-crafted and implemented epidemiological and behavioral surveys and surveillance. The ICR observed that continued progress will depend on continuing the project's efforts to integrate HIV data with other routine data in the health sector, so that individuals can be followed through the entire continuum of care. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well-written and thorough. It remained focused throughout on factors that contributed to achievement of outcomes, with particular attention to challenges related to reaching the high-risk groups that were targeted in the development objective. It explained thoroughly the sources of data used in the analysis and the evolution of data collection methods across the project's lifetime. It was candid about implementation shortcomings, specifically about delays in key behavioral surveys and the funding interruption that impacted project activities in 2014-2015. Its lessons were particularly well crafted with an eye toward applicability as the fight against HIV evolves and in other health contexts. The ICR did not update the economic analysis conducted at appraisal, and no explanation was offered for this decision. a. Quality of ICR Rating Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National AIDS Control Support Project (P130299) Substantial Page 17 of 17