Document of The World Bank Report No: ICR00001651 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37760 IDA-H0440 TF-58225 TF-93986) ON A CREDIT IN THE AMOUNT OF SDR20.2 MILLION (US$27.83 MILLION EQUIVALENT) AND GRANT IN THE AMOUNT OF SDR6.7 MILLION (US$9.28 MILLION EQUIVALENT) TO THE GOVERNMENT OF PAKISTAN FOR THE HIV/AIDS PREVENTION PROJECT December 27, 2010 Human Development South Asia Region Pakistan Country Office CURRENCY EQUIVALENTS (Exchange Rate Effective November 30, 2010) Currency Unit = Pakistani Rupee Pakistani Rupee 1.00 = US$0.01 US$ 1.00 = Pakistani Rupee 85.73 FISCAL YEAR July 1 - June 30 ABBREVIATIONS AND ACRONYMS AIDS Acquired immunodeficiency syndrome AJK Azad Jammu and Kashmir ANC Antenatal care ARV Antiretroviral pharmaceuticals BCC Behavior change communication CAS Country Assistance Strategy CIDA Canadian International Development Agency DFID UK Department for International Development DDO Drawing and Disbursing Officers DP Development partners ELISA Enzyme-linked immunosorbent assay FM Financial management FSW Female sex worker GOP Government of Pakistan HASP HIV/AIDS Surveillance Program HIV Human Immunodeficiency Virus HMIS Health management information system HSW Hijra sex worker IBBS Integrated biological and behavioral surveys IDU Injecting drug user INT Integrity Vice Presidency at the World Bank MARP Most at risk population MSM Men who have sex with men MSW Male sex worker MTR Mid-term review NACP National AIDS Control Program NGO Non-governmental organization NPV Net present value NWFP North Western Frontier Province (now Khyber Pakhtunkhwa Province) NZ Nai Zindagi PACP Provincial AIDS Control Program PPP Public private partnership SDP Service Delivery Package QA Quality assurance STI Sexually transmitted infection VCT Voluntary Counseling and Testing Vice President: Isabel M. Guerrero Country Director: Rachid Benmessaoud Sector Manager: Julie McLaughlin Project Team Leader: Shahnaz Kazi ICR Team Leader: Ghulam Dastagir Sayed Pakistan HIV/AIDS Prevention Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome......................................................... 20 5. Assessment of Bank and Borrower Performance ..................................................... 21 6. Lessons Learned ....................................................................................................... 24 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 27 Annex 1. Project Costs and Financing .......................................................................... 28 Annex 2. Outputs by Component ................................................................................. 31 Annex 3. Economic and Financial Analysis ................................................................. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 41 Annex 5. Beneficiary Survey Results ........................................................................... 43 Annex 6. Stakeholder Workshop Report and Results................................................... 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 62 Annex 9. List of Supporting Documents ...................................................................... 63 MAP A. Basic Information HIV/AIDS Prevention Country: Pakistan Project Name: Project IDA-37760,IDA- Project ID: P074856 L/C/TF Number(s): H0440,TF-58225,TF- 93986 ICR Date: 12/29/2010 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: PAKISTAN Original Total XDR 26.9M Disbursed Amount: XDR 23.9M Commitment: Revised Amount: XDR 26.9M Environmental Category: B Implementing Agencies: National AIDS Control Programme (NACP) Cofinanciers and Other External Partners: UK Department for International Development (DFID) UNAIDS Canadian International Development Agency (CIDA) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 09/06/2001 Effectiveness: 12/23/2003 12/23/2003 Appraisal: 06/29/2002 Restructuring(s): Approval: 06/05/2003 Mid-term Review: 06/30/2006 05/27/2006 Closing: 12/31/2008 12/31/2009 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Moderately Satisfactory Government: Unsatisfactory Implementing Quality of Supervision: Unsatisfactory Moderately Satisfactory Agency/Agencies: i Overall Bank Moderately Overall Borrower Moderately Performance: Unsatisfactory Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 100 100 Theme Code (as % of total Bank financing) HIV/AIDS 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Isabel M. Guerrero Mieko Nishimizu Country Director: Rachid Benmessaoud John W. Wall Sector Manager: Julie McLaughlin Anabela Abreu Project Team Leader: Ghulam Dastagir Sayed Benjamin P. Loevinsohn ICR Team Leader: Ghulam Dastagir Sayed ICR Primary Author: Ghulam Dastagir Sayed Tayyeb Masud Henri A. Aka Naoko Ohno F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To prevent HIV from becoming established in vulnerable populations and spreading to the general adult population, while avoiding stigmatization of the vulnerable populations. ii Revised Project Development Objectives (as approved by original approving authority) N/A (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : HIV prevalence among vulnerable populations. All intervention cities: IDUs (18%); FSW IDUs (20%); FSWs (0%); Value (0%); MSW/HSW and MSW/HSW (1% / quantitative or Remain <5% (1% / 4%) 3%). Qualitative) Source: IBBS Round 3 Source: IBBS Round 1 Date achieved 09/30/2005 12/23/2003 12/31/2008 As for IDU population, the IBBS revealed the original target was not relevant as Comments actual prevalence significantly exceeded the target at baseline in all but 2 cities. (incl. % Where baseline was below 5% at baseline, the epidemic exceeded this by end- achievement) line. Indicator 2 : STI (Syphilis) prevalence among vulnerable populations. IDUs (23.5%); FSWs (6.7%) MSWs (38.2%) Value Hijras (60.2%) Truckers <50% of baseline quantitative or (6.9%) Not available estimate Qualitative) Source: DFID STI Study (2004) Date achieved 07/30/2004 12/23/2003 12/31/2009 Comments DFID study covered Lahore and Karachi. End line data were to come from CIDA (incl. % financed surveillance system, which was not implemented. achievement) Indicator 3 : HIV prevalence among ANC attendees Value quantitative or Not available <1% Not available Qualitative) Date achieved 12/23/2003 12/23/2003 12/31/2009 Comments (incl. % achievement) Condom use during last sex act among vulnerable population (as measured by Indicator 4 : behavioral surveillance). All intervention cities: FSW: 51%; FSW (35%); MSW/HSW MSW/HSW: 35% / Value (22% / 17%) 33% quantitative or 60% - 80% Qualitative) Source: IBBS Round 1 Source: IBBS (2005) Round 2/3 (2007/8) iii Date achieved 09/30/2005 12/23/2003 12/31/2008 No intervention city achieved the target. Condom use for FSW and MSW/HSW Comments increased substantially by end line. However, condom use rates in all cities (incl. % surveyed (including non intervention cities) showed similar levels and achievement) improvements as project areas. Use of clean needles during last week by IDUs as measured by behavioral Indicator 5 : surveillance. All intervention cities: 60% Value 26% quantitative or 80% IBBS Round 3 Qualitative) Source: IBBS Round 1 (2008) (2005) Date achieved 09/30/2005 12/23/2003 12/31/2008 Comments Significant increases in rates of sharing were observed, but most cities did not (incl. % reach end line target. achievement) Percentage of blood transfusions in public sector screened for HIV in labs Indicator 6 : meeting QA standard. 46% 59% Value quantitative or 100% Source: Third party Source: Third party Qualitative) assessment (Round assessment (Round 1) 3) Date achieved 12/31/2006 12/23/2003 12/31/2008 Comments This is percentage of blood banks in public sector which had screened for HIV in (incl. % labs where staff had adequate knowledge of HIV testing procedure. achievement) Percentage of general population expressing #positive# attitude towards AIDS Indicator 7 : patients. Value quantitative or Not available 85% Not available Qualitative) Date achieved 12/23/2003 12/23/2003 12/31/2009 Comments The only reference to this indicator is in Pakistan DHS 2007 #Accepting attitudes (incl. % towards those living with HIV/AIDS# that is 47.7%, which cannot be used for achievement) comparison purpose. Percentage using condoms (at last sex) among general population adults who Indicator 8 : have had sex with a non-regular partner (past 12 months). Value quantitative or Not available 40% Not available Qualitative) Date achieved 12/23/2003 12/23/2003 12/31/2009 Comments Given the concentrated nature of the HIV epidemic in Pakistan, this indicator (incl. % was not relevant. It should have been revisited by the Bank team. achievement) iv (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : N/A Value (quantitative N/A N/A N/A or Qualitative) Date achieved 12/23/2003 12/23/2003 12/23/2003 Comments Intermediate outcome indicators were not systematically measured during the (incl. % project implementation. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 11/22/2003 Satisfactory Satisfactory 0.00 2 06/07/2004 Satisfactory Satisfactory 1.15 3 10/26/2004 Satisfactory Satisfactory 3.49 4 12/30/2004 Satisfactory Satisfactory 3.98 5 06/08/2005 Satisfactory Satisfactory 7.07 6 12/15/2005 Moderately Satisfactory Moderately Satisfactory 9.32 7 06/09/2006 Moderately Satisfactory Moderately Satisfactory 12.39 8 12/08/2006 Moderately Satisfactory Moderately Satisfactory 16.55 9 06/07/2007 Moderately Satisfactory Moderately Satisfactory 20.35 10 12/07/2007 Moderately Satisfactory Moderately Satisfactory 25.12 11 06/15/2008 Moderately Satisfactory Moderately Satisfactory 29.68 12 12/18/2008 Moderately Satisfactory Moderately Satisfactory 33.08 Moderately Moderately 13 06/26/2009 35.47 Unsatisfactory Unsatisfactory Moderately Moderately 14 01/29/2010 35.70 Unsatisfactory Unsatisfactory H. Restructuring (if any) Not Applicable v I. Disbursement Profile vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal HIV prevalence and means of transmission: At the design stage of the project in 2002 and 2003, the human immunodeficiency virus (HIV) prevalence in Pakistan was low. In September 2002, Pakistan ­ with a population of 154 million -- had reported only 1,741 HIV cases and 231 AIDS cases with the most common forms of transmission being heterosexual sex (41%) contaminated blood or blood products (17%), homosexual or bisexual sex (4%), injecting drug use (IDU) (3%), and mother to child transmission (1%). Despite this low reported prevalence rate, it was believed that Pakistan needed to enhance its response because: (i) the number of actual cases was envisaged to be much more than reported (based on limited surveillance data and computer modeling, UNAIDS had estimated the number of HIV­infected persons to be between 70,000 and 80,000 -- about 0.1 percent of the adult population; and (ii) the presence of high risk behaviors among the vulnerable population sub-groups such as high-volume, unprotected commercial sex among both female and male sex workers, and sharing of needles among injecting drug users (IDU) posed a significant threat of increasing prevalence. Vulnerable populations: At appraisal, four population groups/behaviors were identified as at high-risk of HIV infection. Although illegal, there were a total of about 100,000 female sex workers (FSW) in Lahore, Multan, and Karachi, only 5% of whom were reached by any HIV prevention services. According to a United Nations Office on Drugs and Crime (UNODC) report in 2001, there were about 60,000 IDU (12% of the total drug users) in Pakistan. A study in Lahore in 1999 found that 89% of IDU tested positive for Hepatitis C and 64% shared needles in the previous week alone. In Lahore in 2002 there were approximately 38,000 men who have sex with men (MSM) with multiple sexual partners and low condom use (17 percent) and a lot of them reported symptoms suggestive of sexually transmitted infections (STIs). Lastly, while the incarceration rate in Pakistan was not particularly high, prisoners appear to be involved in high risk behaviors more than the average population both during and prior to their confinement. Knowledge, attitudes, and behaviors in the general population: Despite improvements in the general population's knowledge of HIV prevention, it was still inadequate and risky behaviors remained prevalent. Attitudes towards people living with AIDS were mixed, discrimination against people with HIV/AIDS evidently needed to be addressed. Safety of blood transfusion services: About 1.2 million units of blood were transfused annually in Pakistan at the time of appraisal, and a review by WHO and the Swiss Red Cross found that at least 20% of blood used was inadequately tested for HIV and Hepatitis B. The actual figure was probably higher because in many instances good records were not being kept, despite the existence of national guidelines for blood transfusion services. The Government's AIDS Control Program: The GOP established a National AIDS Control Program (NACP) in the late 1980s. The program made slow progress due to several reasons including the lack of recognition of the HIV threat, a vague strategy with lack of focus on high risk groups, weak surveillance, and limited resources. Towards the mid-1990s, there was a gradual improvement in the level of GOP commitment to the program which resulted in its inclusion in the second Social Action Program Project (SAPP-II; 1998-2003) financed by the Bank and other development partners (DP). This enhanced commitment was reflected in slowly increasing expenditures, increase in staffing levels, and establishment of provincial AIDS control programs (PACP), with most programs being one-man structures. However, the overall strategy remained unchanged with the exception of an increased focus on health promotion and HIV education aimed at the general public. In 2000, the GOP, through a broad consultative process, developed a National Strategic Framework for HIV/AIDS that set out the broad strategies and priorities for effective control to prevent the HIV epidemic. The comprehensive framework had an increased focus on provision of services to the vulnerable populations. At the time of appraisal, the program was spending about US$2.5 million per year. The project was in line with the Bank's Country Assistance Strategy (Document number: 24114-PAK) (CAS) objectives and the health sector goals in the Interim Poverty Reduction Strategy of empowering people by creating opportunities for improving access to health services, including HIV/AIDS and TB, as well as other pro-poor services such as education and safety nets. With the Bank's extensive experience in HIV prevention programs globally, the project was expected to bring regional and global expertise to assist Pakistan in its national response to HIV/AIDS. The project envisaged containing the epidemic and building capacity by contracting NGO to deliver services to vulnerable populations with high risk behaviors. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The PDO was to prevent HIV from becoming established in vulnerable populations and spreading to the general adult population, while avoiding stigmatization of the vulnerable populations. The principal indicators for judging achievement of development objectives were : (i) HIV prevalence among vulnerable populations; (ii) STI prevalence among vulnerable populations; (iii) HIV prevalence among women coming for antenatal care, iv) condom use during last sex act among vulnerable; (v) use of clean needles during last week by IDU; (vi) proportion of blood transfusions in public sector screened for HIV in laboratories meeting quality assurance (QA) standards; (vii) proportion of the general population who admitted having sex with non-regular partner using a condom; and (viii) percent of general population expressing positive attitude towards AIDS patients. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification Not applicable 1.4 Main Beneficiaries Although the project assumed that the largest group of beneficiaries of the project was the general adult population, its main targeted beneficiaries were the vulnerable populations such as FSW, IDU, MSM, and prisoners. It was envisaged that the services provided by NGOs to these vulnerable groups could have benefits beyond simply preventing HIV, if the project was successful, including; (i) the experience acquired by, and mechanism developed for, contracting with NGOs could have broader application in the future for service delivery in the health and other sectors; (ii) the project would strengthen the capacity of the NACP and the PACPs to manage an effective and large communicable disease control program; and (iii) the attention paid 2 to monitoring and evaluation in the project could demonstrate to the Government the value of doing this well and dedicating adequate resources for this public function. 1.5 Original Components (as approved) Component 1: Expansion of Interventions among Vulnerable Populations (total base cost US$10.53 million) Subcomponent 1.1 Service Delivery Contracts with NGOs to provide and expand HIV prevention services to FSW, IDU, and MSM living in large cities. The service package included: (i) behavior change communication; (ii) promotion of condom use; (iii) voluntary counseling and HIV testing; (iv) proper STI management; (v) promotion of safe injection practices among IDU; and (vi) empowering the vulnerable populations. Subcomponent 1.2 Small Grants to: (i) help develop the capacity of NGOs and other institutions to carry out work on HIV/AIDS; (ii) allow participation of small NGOs, and NGOs which have not previously worked in HIV and (iii) create opportunities for testing and rigorously evaluating innovative approaches and carrying out needed operational research. Component 2: Improved HIV Prevention among the General Population (total base cost US$11.81million) including: (i) Behavior change communication (BCC) aimed at the general adult population to promote condom use, use of STI treatment services, use of sterile syringes, reduction in the numbers of injections, voluntary blood donation, HIV-screened blood for transfusion, and reducing discrimination and stigma towards people living with HIV/AIDS; (ii) Advocacy to support activities aimed at having decision makers and opinion leaders better understand HIV preventive methods, and help avoid stigmatization or harassment of vulnerable populations; (iii) Targeted interventions for youth, the police, and formal sector workers, covering a) HIV BCC messages for Pakistani external migrant workers in Karachi, Lahore and Rawalpindi, b) training on STIs using the syndromic approach for health workers of the Employees Social Institutions, c) targeting youth particularly high risk groups through mass-media HIV BCC, c) advocacy sessions for new police recruits and the Pakistan Armed Forces Medical Crops to address HIV prevention issues; and (iv) Improved and expanded management of STI cases, based on a protocol developed by WHO and the Government that used a "syndromic" approach. Component 3: Prevention of HIV/STI Transmission through Blood Transfusion (total base cost US$9.21million) including; (i) Establishing and building the capacity of provincial blood transfusion authorities, and building the latter's capacity to regulate private and public sector blood banks; (ii) Implementation of a quality assurance system; (iii) Screening of blood for HIV and other STIs through provision of the necessary materials and reagents for testing all blood in the public sector for HIV and hepatitis B; (iv) Improving waste management, the Ministry of Health would develop guidelines for proper handling of wastes in blood banks and needle exchange programs. Staff of health facilities with blood banks and NGOs involved in needle exchange would be provided with training in these guidelines and the materials for properly handling bio-hazardous wastes. Component 4: Capacity Building and Program Management (total base costsUS$10.55 million) including: (i) Strengthening of federal and provincial AIDS Control Programs covering: (a) recruitment of a firm to help build the capacity of the federal and provincial staff to manage contracts and procurement; (b) recruitment of more full time staff to work in the federal and 3 provincial AIDS control programs; (c) office support including furniture, equipment, vehicles, and access to the world wide web; (e) an annual conference that would bring together staff from government, NGOs, and research institutions to discuss lessons learned and latest findings; and (f) short term attachments of technical staff to other AIDS control programs in the region to learn first-hand about the successes and difficulties encountered; (ii) NGOs capacity development by the management and procurement firm hired under the project on applicable procurement procedures and project implementation techniques; (iii) Second generation HIV surveillance and evaluation, systematic behavioral surveillance and surveillance of HIV prevalence among the vulnerable populations on a regular basis using consistent methodologies. Technical and financial support for surveillance activities was provided by the Canadian International Development Agency (CIDA); and (iv) Care for people living with AIDS covering: (a) counseling for patients and their families; (b) treatment of opportunistic infections; (c) palliative care; (d) supportive care for the patient and their families; (e) linkages with other programs and services such as the TB Control Program; and (f) prevention of mother to child transmission of HIV in cases of HIV positive mothers. 1.6 Revised Components Not applicable 1.7 Other significant changes The Closing Date of the project was extended from December 31, 2008 to December 31, 2009, with the intention to complete project activities and to provide sufficient time for the planned follow up operation to be in place and avoid disruption of service provision to the vulnerable populations. The project benefitted from co-financing from DFID and CIDA through Trust Fund arrangements as follows: In a co-financing agreement, the DFID Grant became effective in April 2009 to supplement the HIV/AIDS Prevention Project during the project extension period. The trust fund of US$4.12 million was to finance services to high risk groups and procurement of screening kits. When the Bank-financed project closed on December 31, 2009, it was decided to extend this Trust Fund for a period of six months through June 30, 2010 to ensure continuity of critical services, specifically (i) a successful harm reduction intervention for injecting drug users in Punjab; and (ii) procurement of anti retroviral pharmaceuticals (ARV) for treatment and care centers nationwide. Unfortunately, due to management issues in Punjab and the National Program, there was little progress in TF activities during the first extension period. A second extension was granted to the TF from June 2010 to March 31, 2011 for one category, the procurement of ARV to cover the gap during the interim period of the closing of the Global Fund Round 2 support for ARV and future Global Fund support under Round 9. Through parallel financing, the CIDA contributed US$1,041,906 with the objective of providing technical assistance to female sex workers; this became effective on August 21, 2007. While the project design included support for building organizational capacity for NGOs, there was no provision for technical support. Given the lack of in-country 4 expertise, the CIDA Grant provided support from an international organization to strengthen implementation of female sex worker interventions as well as support to the ACPs in managing supervising and monitoring these interventions. The procurement of the consultancy was delayed and the contract with the selected organization was not signed until March 30 2009. Subsequently the TF was extended until April 1, 2011. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The project was prepared over a period of 33 months and included analytical studies, which looked into behaviors of vulnerable populations and lessons learned from NGO contracting. The available information, however, did not include information on the size of MARP and the HIV prevalence among MAPR, therefore efforts for collection of this critical information was built into the project design which included mapping and behavioral assessment of targeted populations in large cities along with the CIDA- supported integrated biological and behavioral surveys (IBBS). As one of the strategic choices to be taken under the project, the possibility of public private partnership (PPP) was extensively explored during preparation of the project. Given the importance of NGOs involvement especially in service delivery to high risk groups, the preparation team had a series of consultations with NGOs to assess their capacity to implement HIV/AIDS prevention activities. The team also learned key factors for a successful PPP from the Bank-financed study in 2002 on the analysis of the experience of government partnerships with NGOs, including: the need to ensure well-defined deliverables by NGOs, systematic monitoring and reporting of activities, strengthening fiduciary capacity, smooth transfer of funds to the NGOs, and transparency and good governance in the NGO selection process. The results of these consultations were reflected in the project design and implementation arrangements. The project design went through a consultative process focusing on prioritized areas, looked at the best possible approach to provide services, undertook consultation and analytical studies to understand possible options and finally settled on final design. The design was strategic in the context at that time and focused on most at risk populations (MARP). It was innovative in the use of a systematic approach to NGOs delivering services. The project was a new experience in Pakistan, both from the viewpoint of focusing on MARP, who are normally marginalized in society, and in contracting with NGOs. Hence despite having an appropriate approach, it was clear from the start that this project was ambitious (but also courageous on the part of GOP), and would pose a formidable implementation challenge. The design, however, had shortcomings such as the BCC campaign for the general population and blood transfusion, which were not perhaps as appropriate to a concentrated epidemic but were included due to limited information at that time and likely due to political compulsions. In addition, it was overly complex with too many implementing agencies in a context of low capacity. Experience in Pakistan had highlighted difficulties of implementing umbrella projects covering multiple provinces. With the benefit of hindsight it would have been more strategic and effective, although politically difficult, to focus on the two largest provinces of Punjab and Sindh likely to have the highest concentration of MARPs at the first phase and consider expansion of the coverage to other cities in a second phase, based on the experience of the first phase. 5 Inadequate risk analysis also posed implementation challenges, particularly risks related to NGO contracting including payments delays (the NGO consultation during preparation had identified this as the key constraint of working with government), insufficient monitoring and limitations of technical capacity particularly lack of experience of working with highly stigmatized groups such as sex workers. Another area of inadequate risk assessment was related to high staff turnover at GOP. Mitigation measures for payment mechanisms and the high staff turnover did not receive sufficient attention. Technical support for NGOs should have been central to the capacity building component and was required at an early stage. Neglect of these risks contributed to the main implementation bottlenecks. The PAD did not elaborate on these risks and did not provide any explicit risk mitigation measures. In addition, the PAD played down the critical role of HMIS and routine M&E, which later on undermined the ability of the project to monitor the outputs. Project interventions in the government project document (PC-1) and the Bank document were the same, with exception of M&E arrangements/ mechanisms; where there appears to be a contradiction. Page 18 para 1 of PAD states: " many of the activities under the project will be difficult to monitor so two mechanisms are proposed for M&E: a) Integrated biological and behavioral surveys (IBBS) and b) Third party assessments; while page 31 of Original PC1 on M&E arrangements, after a reference to the indicators, and annex on IBBS and Third party surveys states: "the information will flow from the district level to the PACPs who in turn will send the consolidated reports to the NACP on quarterly basis. A two way information flow will be established between the NACP, PACP and other stakeholders". In other words, it appears that while the PC1 document included generating routine information from the district level up to the NACP, this was not explicit in the PAD. Participatory Process: the project was prepared in close consultation with different groups of stakeholders, including FSW, MSM and Hijras 1 truckers and NGO service providers through social assessment and mapping exercises along with group meetings and stakeholder workshops. In addition, key development partners (DP), including UNICEF, UNAIDS, CIDA, DFID, and USAID were directly involved in the project preparation. The common understanding of HIV/AIDS problem and the commitment to assisting in a fight against HIV/AIDS among donor partners formed the nucleus of project design and enabled mobilization of financial commitment for HIV/AIDS in Pakistan from the development partners.. The overall Quality at Entry is rated as Moderately Satisfactory. 2.2 Implementation The project was never formally restructured with regard to its objectives and indicators. The project was generally rated as moderately satisfactory during its implementation but was downgraded during its last year of implementation. Implementation challenges have included slow procurement of services, recurrent problems in contract management including delayed payments, capacity issues of the ACPs and NGOs and 1 Biological males who are usually fully castrated 6 limitation of coverage of targeted interventions. Overall performance varied significantly across implementing agencies. For example, the operation of preventive services for IDUs were well executed in Punjab which benefited from strong ownership of the provincial government, effective project management with considerable administrative powers of the program manager, barely any staff turnover and a good partnership with an experienced NGO. Unfortunately, this positive experience in Punjab was not replicated project-wide for the following reasons: Delays in effectiveness and start-up: The project faced serious start-up problems in the first three years. After a long preparation of 33 months (including the time around September 11, 2001 which exacerbated delays in preparation due to limited possibility of travel), prolonged administrative procedures in government required another six months (ratification by ministry of law) before the approved project became effective. Poor contract management: Despite advance procurement actions during preparation, the contracting of NGO took on average 19 months partly due to the required clearances at various levels, excessive time taken in evaluation of proposals due to large unwieldy committees and lack of capacity of staff to deal with service contracts. The procurement method was Quality and Cost Based Selection (QCBS). To address these issues, Expressions of Interest (EOIs) were advertised widely, selection committees were broadened to include independent representatives, a Procurement Management Firm was recruited and program staff trained. However, the measures were only partially successful. The situation was compounded by frequent turnover of the procurement officers of the Programs, leading to a loss of capacity that was built. In the case of services for IDU, a more rapid Fixed Budget Selection (FBS) procurement method was used, whereby the terms of reference and the prices for the expected package of services were advertised and a single stage evaluation done by the procurement committee of the respective AIDS program. The initiative was successful in Punjab where it resulted in contracting of services within four months. In Sindh it took nine months, but was still faster than the average time taken for similar NGO service procurements. The delay was the worst in Baluchistan where only one contract (that of IDU) out of a planned six contracts materialized throughout the life of the project. There was also an issue of the sequencing of the contracts processing, as the contract of the management and procurement firm, which was supposed to facilitate the procurement process, was only the fifth contract --which should have been first-- as the program lacked capacity to properly manage the contracting process. Delayed procurement of screening kits: Provision was made to procure screening kits through WHO. After experiencing long delivery delays, the project Financing Agreement (FA) was amended to allow the project to also procure screening kits through UNICEF. However, the possibility of procuring screening kits through these two different channels did not help speed up the procurement and delivery of screening kits. Delays in signing the agreement with UNICEF were essentially due to the extended discussions between the Bank and UN Agencies at headquarters' level to reach an agreement on the templates of Agreement to be used when UNICEF supports governments in procuring health related goods under Bank financed projects. Disbursement delays: the project suffered from long delays in payments to NGO in Sindh and NWFP, and over the last year of the project at the Federal level, which adversely impacted on quality and coverage of services. For example one NGO in NWFP was not paid for 18 months after submission of invoices. The payment process involved a system of monitoring by a 7 committee, certification by the procurement management firm and final clearance by the Health Ministry/Department. The main reason for delays was the excessive time consumed by the clearance from the Health Ministry/Department. In Punjab, where financial powers were delegated to the Program Manager, project execution has been relatively prompt; the manager could approve payment transaction without each payment approval going up through the hierarchical steps. Some of the delays can be attributed to a lot of time taken in organizing monitoring visits and the delayed report submission. Another important cause for these delays was lack of a clearly defined system/mechanism for monitoring of outputs and process level indicators that could serve as an objective basis for assessing short-term performance of the NGO. In the absence of a system to monitor overall outputs, the payment verification process was primarily an audit of the progress report submitted by the NGOs and it involved some serious breaches of individual privacy of the beneficiaries. As a result, conflicts arose between the Government and NGOs over the financial resources allocated to the delivery of services. At time of project closing, several NGOs had not received full payment from Government counterpart funds to the project. Lack of staff and the high turnover: Delays in staffing in the project implementing agencies became problematic prior to project effectiveness. High managerial and staff turnover rendered project implementation more problematic. For example, in Sindh the project experienced seven secretaries of health and ten AIDS Control Program managers. Similarly four PACP managers were transferred to other positions in NWFP in three years. The project also suffered from lack of procurement and FM staff throughout its life, especially in Sindh and NWFP. Due to a new management philosophy brought by each new manager, the direction of the project was not always compatible with the project development objectives. Service delivery by NGO suffered from the high project management turnover, since each new manager would further scrutinize NGO's financial management and focus on input to service delivery rather than numbers and quality of the outputs, not appreciating the performance based lump-sum contracts. Staff from the procurement management firm responsible for building capacity was stretched due to competing demands. Quality and coverage of targeted interventions: Quality, particularly of sex worker interventions, suffered due to limitations of technical capacity. Considerable experience had been built up in working with drug users including IDU as part of demand reduction efforts by NGO since 1990s, and from 2001 to early 2005, prior to the project; DFID had supported provision of harm reduction services for IDU. However, there was little experience of HIV prevention with highly stigmatized groups of male and female sex workers and transgender people and most contractor NGOs were new to this work. The design of the project included support to NGOs in contract management and organizational capacity through a management firm but largely neglected the important area of technical capacity. Subsequently, short term assistance was arranged for male sex worker interventions. Technical assistance to female sex worker interventions intended to be implemented over a two-year period with support from CIDA, materialized towards the end of the project; by this time, most of the interventions were in the process of closing down. Project resources were also not adequate to expand coverage of targeted interventions to the extent required to contain the epidemic. Punjab was able to increase allocation for this activity due to the support of the provincial planning department, but in Sindh only a limited increase in 8 resources was made possible through re-allocation within the PC-1 2 budget for high risk interventions. Mid-Term Review (MTR): The MTR, conducted in May 2006, recognized that considerably greater effort was required both in terms of capacity building and additional resources to scale up services for containing the spread of the HIV epidemic. For rapid expansion of coverage and quality enhancement of HIV prevention services to high risk groups, the MTR recommended provision of supplementary financing for the subsequent five years. The financing was conditioned to prior actions to be taken by the GOP to ensure timely procurement of services, and to improve the shortcomings of the modalities for recruitment of NGOs. It was agreed that service contracts would be province-wide for each of the priority vulnerable groups including IDU, FSW, and MSW and the contracts would be flexible to the requirements of the epidemic and be signed by mid-November 2006. To facilitate quick processing of the contracts, the Bank agreed to allow fixed price procurements and the Government was supposed to streamline the evaluation committees and reduce the layers of bureaucratic approval. Additional Financing and the extension of the project Closing Date: The provision of Additional financing of three years was considered during implementation. Its preparation required the revision of the PC-1s (costed implementation plans) of the National and Provincial Programs. The process was complex, particularly for a multi-province umbrella project and required several layers of approval at the provincial and federal levels. The revised plans were approved by the Central Development Working Party at the federal level in November 2008 and final approval by the Executive Council of the National Economic Committee (ECNEC) was granted in September 2009. Following the CDWP approval Punjab was the only province that was able to scale up services to IDUs through a province-wide contract for services to IDUs signed in April 2009. As a result of these delays, Additional Financing was not processed by the Bank. Instead, a follow-up operation of 5 years was considered as more appropriate for future program support. In the interim, to allow sufficient time for the follow-up operation and to ensure continuity of services, it was decided to extend the Closing Date of the project for one year until December 31, 2009. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Selection of Key Performance Indicators The M&E framework had a focus on vulnerable populations for many of the indicators, particularly those at the outcome level. PAD and legal documents originally defined the eight principal indicators for judging achievement of development objectives. The PAD also defined 15 intermediate output indicators for the project. In ISRs however, out of eight PDO indicators, the team selected two (HIV and STIs prevalence rates) as PDO indicators, and three (clean needle condom use, and blood transfusion) as intermediate outcome indicators, to monitor the progress. Only data for four PDO indicators have been collected systematically, suggesting feasibility 2 PC1 stands for Planning Commission 1 ­ it is a GOP document for project evaluation and approval; and includes project description along with the budget needed for implementation. 9 issues in measuring the selected indicators. It also appears that there was limited collection of credible monitoring data to measure progress of the defined output indicators throughout the project implementation. Difficulty in measuring the progress and achievements by PDO indicators may indicate that the selection of PDO indicators was inappropriate. HIV prevalence data may not be the best indicator to measure project progress, as prevalence depends on many external factors, including effective prevention, survival time of HIV positive people and migration. In addition, more emphasis should have been placed on indicators of actual program coverage of services for vulnerable populations to assess the link between the program's contributions to changes in epidemiologic or behavioral outcomes. The only measure of program coverage included in the original results framework was the proportion of estimated vulnerable populations covered by NGO contracts. This indicator is a useful measure of appropriate planning, but does not reflect the actual proportion of vulnerable populations benefiting from the prevention interventions. Output indicators for Component 2 (Knowledge and Prevention among the general population) described a mix of preventive behaviors and non discriminatory attitudes toward people living with HIV/AIDS. While these indicators were consistent with the proposed objectives of Component 2, as discussed earlier, the BCC campaign for the general population itself was focused on issues that should not have been program priorities for a country facing a concentrated HIV epidemic. For Component 3, output indicators related to blood safety focused only on public sector system strengthening, even though weaknesses in the regulation of private blood transfusion centers appeared to be a key gap in services and a primary area of work of the project, as documented in the project aide-memoires. Monitoring and Evaluation Data Systems The primary mechanisms instituted by the project for collecting monitoring and evaluation data were the integrated biological and behavioral surveys (IBBS) and third party assessments to assess quality of various components of services. The IBBS surveys were conceived and housed primarily as tools for HIV surveillance, i.e., under the auspices of the HIV AIDS Surveillance Program (HASP). As such, the IBBS was a critical contribution by the project providing invaluable size estimates of vulnerable population and epidemiologic data about vulnerable populations in a large number of cities. The implementation of three rounds of IBBS during the project period is an important achievement for the country and a model for second generation surveillance in the region. However, the utility of the IBBS data as a primary source of monitoring and evaluation was limited in part because of decisions made about the sampling design and the lack of analysis to calculate project defined performance indicators. It is also important to recognize the effective application of the IBBS data in the area of advocacy and program planning. For example, high HIV prevalence measured in the IBBS of IDU was used to justify additional resources for prevention interventions. Similarly, size estimates of the vulnerable populations resulting from the mapping component of the IBBS directly fed into budgets based on planned population coverage and the placement of specific intervention sites. The third party assessment mechanism was a promising effort to measure service quality in a standardized, objective manner. However, there appeared to be some lack of ownership and 10 support on the part of provincial AIDS control programs to endorse the metrics and tools adopted for the assessment. This resulted in findings which were not perceived to focus on the priority areas of the service and disputed as an accurate assessment of the project achievements. The reliance on IBBS survey data and annual third party assessments to measure program outputs de-emphasized the value of using standardized interim output level indicators from routine monitoring systems for tracking program progress. A sample of quarterly reports submitted from NGO show that such data are collected; however, these data were not systematically collated in a way that provided an overview at provincial or national level. There was a lack of clarity on who had responsibility for collecting and analyzing these data. The government project document identified the management firm being responsible for monitoring of the NGO outputs. However, the management firm did not appear to have the technical expertise to identify a standardized set of indicators that would focus on the key aspects of these types of interventions. The responsibility for collating and analyzing the data to provide periodic assessments of program performance did not appear to be included in any agency's terms of reference. Toward the project completion in 2008, an effort to assess NGO program performance synthesizing behavioral data from the IBBS and the limited commodity distribution data available from a sub-set of sites was conducted by an external consultant. The management firm did make efforts to lay out guidance for routine monitoring systems for the vulnerable population interventions in 2007 in a manual. It provided sample formats for data collection, but did not specify a core set of indicators with which to measure program performance and the effort did not result in a standardized dataset available from all project sites that could be used to describe overall program outputs. Some of the larger NGO contracted to provide services for IDU (e.g. Nai Zendagi) had independently developed sophisticated systems for tracking services to their beneficiaries; however, this local expertise was not leveraged to extend good monitoring practices to other NGOs. There was a general failure of the system to produce consistent project wide data on program coverage. This also reflected a broader lack of program guidelines defining minimum quality standards for prevention services for vulnerable populations. The detailed project description included in the PAD provided clear, concrete standards for the prevention program; however, these do not appear to have been adopted by implementation partners or reinforced by NACP/PACP, the management firm (nor by Bank supervision teams). In addition to the assessment of MARP interventions, an evaluation of the behavior change communication strategy for the general population was conducted. The report from this evaluation acknowledged the lack of baseline data because for this a planned demographic health survey could have provided baseline levels of knowledge and attitudes. However, these data were not collected and therefore were unavailable for the evaluation. The results of the surveys and focus group discussions conducted at the end of the project period focused on exposure to the media campaign and assessed levels of knowledge, but did not appear to correspond to the key behavior change messages identified as PDO, i.e. those related to attitudes about HIV/AIDS and MARP or condom use among adults who had non-regular sex partners. With respect to knowledge of HIV prevention and modes of transmission, the report from the evaluation team of Socio-Economic and Business Consultants (SEBCON) found that "in general, the results of the quantitative and qualitative survey point to a significant increase in the knowledge about HIV and AIDS." For example, the most commonly identified mode of transmission was unsafe sex (34% 11 of the respondents). However, because there was no baseline, it is difficult to substantiate the conclusion that the levels of knowledge measured reflect an increase from prior to the implementation of the BCC campaign. In conclusion, while distinguishing between outputs/process indicators and outcomes, the project failed to develop an adequate system for tracking upstream indicators. However, at the same time, there was a strong focus on monitoring outcomes and data on MARP behavioral, biological outcomes and utilization of services was the basis for programmatic changes, particularly the increased allocation for IDU services in Punjab and to a lesser extent in Sindh. 2.4 Safeguard and Fiduciary Compliance The environmental category for the project was B. The environmental management plan, and the guidelines related to infection control and proper handling of waste in blood banks were developed and published prior to project appraisal. A third party contractor (Hagler Bailly Pakistan), carried out a baseline study which indicated that waste management procedures applied by health centers were inappropriate and not in keeping with environmental management plan developed by NACP. There is no evidence that remedial actions were taken to address the environmental management weaknesses identified in the baseline study. Deficiencies in financial management were noted during project implementation period despite the intensive NGO consultation and capacity assessment during the project preparation period. The Bank temporarily suspended the use of Statement of Expenditures (SOEs) in Baluchistan for lack of receipt of acceptable audited financial statement for FY2004-05. The use of SOEs was reinstated after the Baluchistan project unit complied with agreed Financial Covenants. The deficiencies in financial management can be linked to the untimely and limited availability of qualified financial management staff, and complicated bureaucratic procedures in the government. Federal and Provincial authorities' efforts to control input, rather than focusing on quality and delivery of services, further weakened NGOs' performance in delivering services to population targeted. At time of project closing several NGOs had not received full payments coming from counterpart funds. There were two instances of complaints specifically on possible corruption in payments to NGO. Communications related to corruption were forwarded to the World Bank Integrity Vice Presidency (INT) by the project task team leader. The INT followed up on some occasions with requests for more information. In one case INT inquired about the findings of the Bank's financial management and procurement review. In another instance INT requested contact details of all NGOs that had participated in the bidding process. It is not known if INT initiated a further investigation. Procurement management was a constraint as well despite the support provided by the procurement management firm. This was due to several factors: (i) the firm came very late on board, while 4 contracts had been already in place; (ii) the firm had limited knowledge about the specifics of Bank procurement guidelines (iii) the TOR for the firm were very ambitious, where the firm was responsible for the training on procurement and capacity building for both government and NGOs, the monitoring of the NGO performance, and approval of payments to NGOs; (iv) the responsibilities of the firm increased overtime, without any change to their contract cost (A member of the management firm said to ICR team: "we lost our shirt with this contract!") and; (v) the firm also faced staff shortage. For example, in Sindh they did not have any 12 staff for several months. Notwithstanding the advance procurement undertaken prior to project effectiveness, the project suffered from delays in finalizing the procurement of service providers, including the selection of the procurement and management firm. Particularly since national and local procedural requirements included as many as 6 to 7 approvals at various levels. 2.5 Post-completion Operation/Next Phase The project was extended once and given its performance issues in the Sindh Program, a second extension of the project Closing Date was not approved. It was decided, however, only to extend the co-financing (DFID Grant) retroactively for a period of six months till June 30, 2010 to ensure continuity of critical services, namely (i) a successful harm reduction intervention for injecting drug users in Punjab; and (ii) financing of anti-retroviral pharmaceuticals (ARV) for treatment and care centers nationwide. The completion of the project and its post-completion phases were not positive. During 2010 the implementation of the DFID grant was adversely affected by management issues in Punjab and the National Program. The manager of the AIDS Program in Punjab terminated the contract of harm reduction services for IDU without requesting the Bank's no objection, and refused to make a final payment to the NGO. At the time of writing of this ICR, the matter remained under arbitration. US$1.32 million of the DFID Grant was still undisbursed on June 30, 2010, and based on a request from GOP the grant was extended retro-actively until March 2011 for one category only, i.e., the procurement of antiretroviral drugs (ARV). At the time of writing of this ICR, the GOP and the Bank were involved in discussions on the restructuring of the lending portfolio in response to the devastating floods of August 2010. It is unclear how the preparation of a second HIV/AIDS project, if any, would be affected by this dialogue. The Bank has offered technical assistance to NACP to adjust its HIV/AIDS strategy, given the funding constraints due to floods and involvement of new DP in this program. 3. Assessment of Outcomes3 3.1 Relevance of Objectives, Design and Implementation Objective: the objective of preventing the spread of HIV among high risk groups and within the general population was, and continues to be, highly relevant to Pakistan and consistent with MDG 6. The project remains highly relevant and consistent with Pakistan's development priorities, including the current HIV program PC-1 2008 ­ 2013. It is also consistent with the stated objectives of the current Country Partnership Strategy for period FY 2010 ­ 14, where under 3 Overall project values for indicators presented in the data sheet were taken from the ISR documents; however, these could not always be reconstructed from final IBBS or third party assessment reports and may have reflected data from preliminary analysis. Data for specific sites presented here were taken from final report documents. 13 "Strategic Outcome of Investing in Pakistan's Human Development Resources and Protecting the Poor", the Bank will extend support on HIV/AIDS among other Human Development priorities. Design: The focus on targeted interventions among high risk groups was in keeping with global best practice and is still relevant. However, Component 2 with its focus on BCC for the general population was, in retrospect, not strategic because of the concentrated nature of the HIV epidemic in Pakistan, Component 3 of the project; prevention of HIV/STI transmission through blood transfusion, proved overambitious, although it was a priority of many government officials. Implementation: The implementation of the targeted interventions through NGOs was an appropriate strategy, in principle since NGOs had an obvious advantage in dealing with the marginalized populations at highest risk of HIV infection. The GOP recognized the advantage of working through NGOs and not having to be directly involved in providing services to groups engaged in illegal activities. However, the level of mistrust between the GOP and NGOs was high and this coupled with lack of GOP familiarity with output based contracts resulted in: (i) delayed recruitment of the NGOs; (ii) delayed payments to NGOs; and (iii) an inappropriate focus on input and process indicators. The procurement and management firm failed to develop the appropriate management information systems needed to track progress, and could not sufficiently support the AIDS control programs in contract management. 3.2 Achievement of Project Development Objectives This section analyzes the achievement of the PDO of the project by examining available data for each of eight Key Performance Indicators listed in PAD. Indicator #1 - HIV prevalence among vulnerable groups (IDU) & #5 - Use of clean needles during last week by IDU: At the beginning of the project period, the HIV epidemic in Pakistan was believed to be marginal and concentrated, driven by vulnerable populations, particularly IDU, FSW, and MSM. Given the absence of data on these groups at the time of project design, the target was to maintain prevalence below 5% among these vulnerable populations. Once data became available in 2004 that indicated higher prevalence rates among IDU in Karachi, this target became inappropriate. At the same time, the use of HIV prevalence as a PDO indicator is controversial because it may simply be beyond the scope of any single project to influence and it is often difficult to interpret the meaning of changes in prevalence rates. (The use of HIV prevalence as a PDO indicator has been debated in many recent HIV projects.) The main source of data for tracking this indicator was the STI study conducted in 2004 (in Lahore and Karachi) and integrated biological and behavioral surveys (IBBS), conducted in three rounds (2005, 2006/7, and 2008) covering a majority of cities where prevention interventions were implemented. Data from the first round IBBS conducted in 2005 suggested that by the time the NGO had been contracted under the project, the epidemic among IDU had already begun to spread and exceeded the 5% threshold in cities targeted for intervention. In the absence of a proper baseline and a "without project" counterfactual, it is not possible to assess the exact impact of the project on the HIV epidemic among high risk groups. However, it is plausible that the IDU interventions had a positive impact on the spread of the epidemic. The evidence is strongest in the Punjab, in particular in Lahore and Faisalabad, where decreasing or stable HIV prevalence trends are consistent with increasing program participation and decreased 14 needle sharing. The picture in Karachi and Peshawar may be more complicated. While HIV prevalence trends in these cities also show declines or stabilization, increased needle sharing or decreasing program participation (although still above 70%) make definitive conclusions difficult (data from Sargodha, Peshawar, and Quetta are not presented due to lack of data and small population size.) It is important, however, to note that the results need to be interpreted with caution. First, NGO providing IDU interventions in Karachi, Lahore, Peshawar and Quetta had been in place with DFID support since 2001 (although at lower levels of financing) and were not taken over under the project until early 2005. Therefore, any evidence for impact in these cities is likely to be a combined effect of these two project periods. For example, by the earliest round of IBBS available in different cities, use of sterile injecting equipment at last injection was already relatively high among respondents ranging between 60-90%. Project supported interventions in Faisalabad, Sargodha, and Sialkot began in the Punjab in January 2005. In Lahore, Faisalabad, Peshawar, and Quetta, the prevalence of HIV did not rise as rapidly suggesting the plausibility that prevention interventions helped to curb the epidemic among IDU during the project period. Three of these cities (Lahore, Faisalabad, and Peshawar) showed large increases in program participation among IBBS respondents between Round 2 (2006/7) and 3 (2008), up from under 50% to over 80% in each site. Similarly, consistent use of clean needles was 59% in cities with interventions as compared to 27% in cities without large interventions (A Khan 2008). In Karachi and Sargodha, HIV prevalence levels among IDU were high (i.e. above 25%) in the early rounds of the IBBS, and then appeared to decline substantially in the third round (2008). These prevalence trends may reflect reductions in new infections consistent with impressive gains in program participation in these cities between Round 2 and 3 of the survey, but large declines in prevalence could be due to other factors, including potentially high levels of mortality among IDU; and/or high mobility among IDU caused sampling differences. In many cities (e.g. Karachi, Lahore) key characteristics of the sampled IDU, such as duration of being an injector, changed markedly from round to round, suggesting that the samples were capturing different IDU. This is consistent with the view of many NGO workers that the IDU populations are highly mobile. These and other factors make the interpretation of HIV prevalence trends very challenging. Even if the samples were comparable, in order to interpret prevalence trends must account for the underlying pattern of mortality among IDU, as they are at greater risk of death due to their drug habits and living conditions, and because there are significant barriers to accessing treatment for IDU in Pakistan. In more mature IDU epidemics, such as those in Karachi and Sargodha, substantial numbers of HIV-infected IDU will have progressed to AIDS, and barriers to ART may result in high levels of mortality. In Pakistan, those IDU wishing to be considered for ART must discontinue their illicit drug use and demonstrate full family support, which discouraged providers from promoting treatment and IDU from seeking services. No specific data on mortality among IDU were available to estimate the impact of death rates on HIV prevalence trends. Without epidemic modeling and complete and reliable program monitoring data for the entire project period, it would be difficult to attribute this stabilization of prevalence directly to the program efforts. However, experience of other IDU epidemics in similar low resources settings, such as in Manipur in India and in Central Asia, shows rapid increase in prevalence rates with little indication that they are decreasing, suggesting that interventions in Pakistan had had an impact. 15 Indicator #1 - HIV prevalence among vulnerable groups (FSW): Prevalence rates among FSW appears to have remained below 5% in the intervention areas based on IBBS surveys conducted in 2005 and 2007. Due to the low prevalence measured in the first two rounds of surveys, it was decided not to collect a third round of data near the end of the project period. Almost all sites included in these surveys detected close to zero prevalence. Given the low rates of "coverage", the continuing low HIV prevalence among FSW is more likely due to already prevalent behaviors and possibly the preventive effect of male circumcision, which is almost universal in Pakistan. Respondents of the FSW IBBS reported very low levels of participation in the project; no city exceeded 15% participation in any round of the survey. Consistent condom use also remained low to moderate among all FSW city samples, i.e., below 50%. These figures are consistent with the NGOs assessment conducted in mid 2008, comparing estimated condom need to the level of free condom distribution reported by the NGO contracted to provide services. Indicator #1 - HIV prevalence among vulnerable groups (MSW/HSW) & #4 - Condom use during last sex act among vulnerable populations (MSW/HSW): The project focused prevention interventions on the subset of MSM who identified as either male sex workers or hijra (transgender) sex workers (MSW/HSW). While this population is a subset of all MSM, they were believed to comprise a higher risk population because they had larger numbers of partners. A majority of the sites where interventions were implemented and IBBS surveys conducted suggest that HIV prevalence remained below 5%. In Karachi, HIV prevalence did exceed 5%; however, in the subsequent round, prevalence fell below this threshold. Although HIV prevalence measured among HSW in Larkhana was very high, at 14% in Round 1 and more than 25% in Round 2, this was not an intervention site, suggesting a missed opportunity to provide services for a community of known need. Condom use patterns among MSW and HSWs were low across all three rounds of the IBBS. Only in Lahore and Faisalabad did condom use at last sex exceed 30%. Participation in the program was similarly low, though in general higher among HSW than among MSW. Given the low level of program coverage, other explanations for the low HIV prevalence among FSW and MSW/HSW may include the lack of overlap between IDU and sex work networks and the preventive value of male circumcision which is very high in Pakistan. Indicator #2 - STI (syphilis) prevalence among vulnerable populations: Baseline data for this indicator come from a 2004 STI study financed by DFID. The survey covered five groups in Karachi and Lahore and used appropriate forms of probability sampling where possible. Current syphilis prevalence was particularly high among IDU and MSW/HSW (ranged from 23-60%), and was more moderate among FSW and truckers (~7% in both groups). Although a measure of syphilis prevalence was to have been included in the IBBS funded through CIDA, they were not. Thus, neither end-of-project data nor additional baseline data for other cities with large vulnerable populations are available for this indicator. Indicator #6 - percentage of blood transfusions in public sector screened for HIV in labs meeting quality assurance (QA) standard: The data available for this indicator comes from the third party assessment by the SOSEC consultancy firm. This assessment selected a sample of public sector blood banks. Results of screening for HIV were presented in aggregate using blood bank as the unit of measure, and could not be described in terms of units of blood transfused. The assessment of screening was based on written records provided by the different blood banks. Record keeping was cited as being poor in many sites which may result in an unreliable measure of coverage. Overall, the assessment by SOSEC found that 59% of public sector blood banks fully screened 16 blood products for HIV at baseline and showed some decline to 46% by the third round of the assessment. No explanation for this was given in NACP or project supervision document, but different sites were evaluated at baseline and end-of-project so the difference may be due to differences in the sample assessed rather than lower performance at all facilities. There were no data available to measure achievements at the end of the project for indicator #3 (HIV prevalence among women coming for ANC), indicator #7 (percentage of general population expressing positive attitude towards AIDS patients), and indicator #8 (Among general adult population admitting to sex with non-regular partner, proportion using a condom). The indicators 3 and 8 did not feature in the discussion of Bank team since it became evident quite early on during implementation that these would not be relevant in the context of the Pakistan epidemic, but the team did not formally revise the indicator framework. Indicator 7 was potentially relevant to assessing the impact of the BCC campaign for the general population. Some data are available in the DHS 2007 survey and can be included but there is no corresponding baseline from which to determine whether the program had an effect on knowledge and attitudes among the general population. Only 44% of female respondents were aware of HIV or AIDS. And only 5% had correct knowledge or HIV prevention and methods of transmission as defined by the standard UNGASS indicator. Among those women who were aware of HIV or AIDS, a majority expressed accepting attitudes towards caring for family members with AIDS (77%) and not wanting to keep secret that a family member was infected (62%). 3.3 Efficiency During the project preparation, a cost benefit analysis of the project was undertaken. The net present value (discounted benefits less discounted costs) of the project was estimated to be positive at US$647 million in the base case using a 10% discount rate for a 15 year time horizon. Economic benefits were in the form of medical costs averted by public health facilities (assumed to be half of total medical costs) and lost earnings averted due to a reduction in HIV infections as a result of the program. Costs used were the entire cost of the Program from all sources. As there was no surveillance data at the time projections of future incidence were based on the Indian experience assuming an increase in new infections of 50% per year. HIV cases (cumulative new infections) were projected to rise to 4.5 million without intervention and on the assumption that the program was moderately successful (reducing new infections to 45% annually in the first 5 years and 40% thereafter) it was expected to prevent 2.1 million new infections over the 15 year period. The model did examine alternate scenarios based on different assumptions of growth in new infections. NPV was estimated to be zero if increase in new infections was 20% without project and 15% with project. In fact, current projections by NACP undertaken in 2009 using the UNAIDS Estimation and Projection Package (EPP) show a much slower increase in new infections of roughly 19% per year. It is not feasible to carry out a further detailed analysis on the ex-post benefit cost ratio in the absence of data on preventive effectiveness that would enable a reliable estimation of infections averted due to project interventions. Cost effectiveness analysis. No formal cost effectiveness analysis was carried out for the project. However, evidence from other countries shows that preventive services for groups at the highest risk are the most cost efficient means of restricting transmission thereby greatly reducing AIDS treatment and other costs and providing a high return on investment (Disease Control Priorities by 17 Jamison et al. 2006). More resources under the project went to services for the most at risk for HIV infection such as sex workers, injecting drug users, transgender sex workers. Allocations for prevention for high risk groups accounted for 28% of project costs at appraisal. During implementation, the focus on these targeted preventive interventions was intensified with their share increasing to roughly 43% of project spending. Within the high risk groups, the largest share was spent on harm reduction interventions for IDU - - the group at the greatest risk and the main drivers of the epidemic in Pakistan. Needle exchange programs for IDU, which were almost entirely financed by the project, were by and large successful in promoting safe injecting practices in cities where they were in place. The coverage of these services was limited to 30 percent of the estimated population of urban IDU. The achievements of sex worker interventions were modest both in terms of behavior change and utilization. The quality of services suffered due an inadequate system of routine monitoring as well as delays in payments to NGO that lead to funding gaps that adversely affected field activities. The economic justification for government financing of the HIV/AIDS programs was also based on externalities and equity issues associated with the epidemic. The project supported activities that had significant externalities such as interventions for high risk groups and public goods such as advocacy and communication campaigns which would not be financed by the private sector in the absence of public financing. During the first phase of the Enhanced AIDS Control Program (2003-2009) a total of nearly US$83 million was spent jointly by the Bank, other bilateral such as DFID and CIDA, the UN system and GOP. During the period the government share of financing was 19% amounting to roughly US$3 million per year. HIV-AIDS spending comprised a relatively small share of government spending on health in 2008-09 of roughly 2 percent While there is little data for Pakistan on poverty impacts, research from other South Asian countries highlights the high cost of AIDS to affected households as well as the differential impact of the epidemic across wealth quintiles (World Bank, HIV and AIDS in South Asia, 2009). In the context of Pakistan and elsewhere, high risk groups particularly street-based IDU, sex workers and transgender people are among the most marginalized and socially stigmatized sections of the population. Surveillance data show that mean income of street based IDU and MSW/transgender is relatively low. Household survey data point to income disparities in access to health services that are likely to be further exacerbated due to the high stigma associated with MARP. With the benefit of hindsight, it can be said that generally allocations for TIs were not sufficient to scale up coverage to the extent required while some other components particularly BCC for the general population and blood transfusion were not likely to be cost-effective interventions in such a concentrated epidemic. 3.4 Justification of Overall Outcome Rating The PDO of preventing the increase of HIV among MARP and in the general population was and continues to be relevant to Pakistan. Existence of large high risk population along with persistence high risk behaviors, as confirmed by the STI survey in 2004 and three rounds of the IBBS, calls for continuous public health efforts to contain the HIV epidemic. This is also consistent with Pakistan's commitment to the Millennium Development Goals that include halting and reversing the spread of HIV. 18 The design of the project was in line with best practice for the concentrated HIV epidemics focusing on MARP. The public private partnership between the government and the NGO to implement this project was suitable taking into consideration the project development objectives, the population targeted and the aim of a decentralized implementation. The BCC campaign for the general population was not relevant to epidemiological scenario of the country. The inclusion of this component was partly a reflection of the priority of some government officials outside the AIDS control programs. While the contribution of HIV infections to HIV prevalence is probably negligible the inclusion of the component in the project is relevant given the importance of safe transfusion for the general health services. The project design suffered from inadequate assessment of risks, in particular those related to procurement of NGO services and payment to NGOs. The project also suffered from an inadequate routing monitoring systems and high staff turnover at GOP. Out of eight indicators there is data on only four; however these measure attainment of the primary PDO, i.e., containing the HIV epidemic. The MTR missed the opportunity to adjust the result framework with the ground realities. While some gains were made in scaling up prevention interventions for vulnerable groups (Component 1) for example the IDU interventions in Punjab were effective, overall the epidemic among IDU was not contained and coverage of the intervention for vulnerable groups was too low to be able to have a significant impact on the HIV epidemic in the country. Only modest progress was made in reaching the other groups at high risk, amongst whom a continued low HIV prevalence may be due to male circumcision and pre- existing behaviors. During project preparation, a cost benefit analysis of the project was undertaken. The net present value (discounted benefits less discounted costs) of the project was estimated to be positive at $647 million in the base case using a 10% discount rate for a 15 year time horizon. In fact, current projections by NACP undertaken in 2009 using the UNAIDS Estimation and Projection Package (EPP) show a much slower increase in new infections of roughly 19% per year. The PAD examined the NPV assuming different values in the rate of increase in new infections. The NPV would be zero if the rate of increase of new infections with program is 20 percent and rate of increase without program is 15 percent. It is not feasible to carry out a further detailed analysis on the ex-post benefit cost ratio in the absence of data on preventive effectiveness that would enable a reliable estimation of infections averted due to project interventions. Within the high risk groups, however, the largest share was spent on harm reduction interventions for IDU the group at the greatest risk and the main drivers of the epidemic in Pakistan. Needle exchange programs for IDU which were almost entirely financed by the project were by and large successful in promoting safe injecting practices in cities where they were in place. In conclusion, despite serious challenges which are inherent to projects which for the first time are providing services for marginalized populations, the project contributed to strengthening the national response to HIV: (i) the Government developed some capacity in the area of HIV surveillance, particularly a second generation IBBS; (ii) Pakistani NGO developed increasing expertise in working with IDU (and other most-at risk groups); (iii) treatment centers financed under the project have expanded and provided treatment to people living with HIV/AIDS; and (iv) the Government developed some experience in contracting and engaging civil society to 19 provide effective prevention interventions. However, most activities were carried out with considerable delays and not at the required scale. Rating: Moderately Unsatisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development See 3.6 below. (b) Institutional Change/Strengthening Institutional capacity has been an ongoing area of concern. The PAD points out that: "shortage of staff, particularly in the provincial AIDS Programs and the lack of managerial capacity of the government and NGO to implement a much larger program and limited experience of NACP and PACP with Bank procurement are the priority areas of capacity building to be considered for the successful implementation of the project". This situation did not change substantially throughout the project. The shortage of procurement and financial management staff, with exception of Punjab, continued in all provinces throughout the life of the project. The procurement of services was extremely lengthy and in the case of Baluchistan only one out of six contracts materialized. The capacity of NACP and PACPs to provide ongoing oversight and monitoring of NGO contracts improved only slightly. The management and procurement firm did not help to improve the situation. Despite the provision in the project document for exposure visits, few exposure visits between provinces were undertaken, although there was sharing of experience among IDU NGOs and some exposure visits to India were undertaken. The NGOs were supposed to receive support from other partners in terms of capacity building for better implementation of their contracts, however with exception of a few workshops conducted by the management firm, other partners support did not materialize. CIDA support for FSW interventions only materialized towards the end of the project. (c) Other Unintended Outcomes and Impacts (positive or negative) Even though the blood safety component was overambitious, and accomplished little, it contributed to the sensitization of the GOP authorities regarding how to improve transfusion safety. As a result GOP and the German government are now cooperating in a blood safety program. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops No beneficiary survey or stakeholder workshop was conducted for this core ICR. 4. Assessment of Risk to Development Outcome Rating: Significant Despite the contribution of the project to improvement of some of the risk factors, the risk of growth of the epidemic remains. Risk factors such as high number of IDU sharing injection equipments, high levels of MSM and commercial sex work with low condom use require appropriate action of high quality, high coverage targeted interventions. However, due to the 20 combination of the very high prevalence of male circumcision and probably low rates of concurrent multiple partners, the likelihood of a generalized HIV epidemic remains low in Pakistan. Deterioration in the security situation and new humanitarian priorities because of the recent floods along with other human development concerns such as high levels of malnutrition in Pakistan are among factors which overstretch Bank's capacity to finance HIV programs in Pakistan. GOP and the Bank are discussing how to restructure the lending portfolio in this context. The Global Fund to Fight AIDS, TB and Malaria recently approved a grant for HIV to Pakistan of around US $40 million; in addition it approved another US $10 million grant from its regional program for the MSM. CIDA is still financing surveillance until October 2011. Considering the current situation regarding HIV epidemic in the country and the available funding, there is a dire need for a highly focused, strategic HIV prevention program among IDU and MSM. Indications of GOP Commitment to AIDS program: The clearest indication of the Government commitment to the program is the fact that the PC-1s for the second phase of all programs except Baluchistan have been approved by the Executive Committee of the National Economic Council (ECNEC) which is the highest economic decision- making body in the country. These PC-1s are well prioritized with nearly 80-90 percent of Program costs in Sindh and Punjab allocated for preventive services for MARPs and the share of GOP is 20% of total cost. Staffing of all programs has been maintained (Baluchistan is the exception). Sindh continued with services for HRGs through its own funds for another six months till June 2010. The funding has been discontinued now due to the fiscal crunch following the floods. Punjab is planning to contract out services for MARPs through its own funds for the interim period till June 2011. The selection of consultants has been completed and the award of contracts is expected shortly. The Government through the Economic Affairs Division (EAD) has formally reiterated earlier requests to continue with the preparation of the proposed follow-on project. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory The project was prepared in a very participatory manner through consultations with all stakeholders. Intensive consultations with different groups of beneficiaries including FSW and truckers were conducted for social assessment and mapping exercises. The team also enabled DP to involve in the project preparation process, which increased DP' commitment to HIV/AIDS in Pakistan and the level of external financing. In addition, the possibility of public private partnership (PPP) was widely explored during the project preparation to assess existing NGO' implementation capacity. The team then reflected the results from the consultations into the project design and implementation arrangements. The design of the project responded to the low levels of procurement and contract management capacity by: (i) calling for advanced procurement 21 actions for the recruitment of NGO; (ii) financing the recruitment of a procurement and management firm; and (iii) insisting on third party assessment of NGO performance. Unfortunately, these turned out to be insufficient to mitigate the identified risk. The design of the project also under-estimated the difficulty of developing and implementing a satisfactory management information system for the NGO working with MARPs. (b) Quality of Supervision Rating: Unsatisfactory Besides attention to process issues (contracting of and payment to NGO), the Bank team had a focus on certain project outcomes measured by key indicators related to high risk behaviors of MARP which were of utmost epidemiological importance. Aide-memoires emphasize trends in key indicators and supervision missions focused on the latest available data from the IBBS. This contributed to dissemination of the results of the IBBS data as well. The team was also successful in securing additional resources through trust funds from DFID and CIDA to complement the project activities to expand the targeted interventions and secure TA for the implementing NGO and procurement of ARV. Although the reports produced during each Bank mission identified issues such as delayed payment and procurement process and made reasonable recommendations, the same issues were repeated in successive aide-memoires. For example, delayed payments to the NGO in Sindh and later on in NWFP significantly hampered project implementation. The IBBS 2008 showed a decline in the use of new syringes/needles, which is attributable to the reduced supply of new syringe/needles and outreach activities for IDU by the NGOs. The limited volume of the NGO outreach activities was clearly caused by persistent delays in payments from the government to NGOs working in these provinces. The issue was raised several times by Bank management including by the SM, CD and regional VP in dialogue with the Chief Minister/Governor and at the federal level with the Minister Health and even the Prime Minister, which resulted in client compliancein an ad hoc manner but no systematic and consistent solution was worked out. Given the protracted and serious problems, Bank management should have more seriously considered stronger remedies including partial cancelation. The Bank's limited pro-activity was also observed when the need for restructuring the project became evident in the second year of the project. At that time, the latest information about the size of the high risk groups, especially IDU and the level of HIV prevalence among the high risk groups became available from IBBS, however, the Bank team did not initiate restructuring of the project to adjust the scale of the original targeted interventions with the HIV epidemiological realities of the country, except for the Punjab IDU interventions. The "100-day campaign" organized by Bank team in August 2004 to expand coverage of contracts with NGO to deliver prevention services to IDU as response to STI survey in July 2004, showing a full-fledged IDU epidemic in Karachi, was commendable, and facilitated the adjustment of NGO contract in Punjab. Yet, while the team realized the implausibility of the blood safety component and irrelevance of any focus on HIV prevalence among pregnant women, the project was not restructured. And, although the need for additional financing was raised during the MTR in 2006, it never materialized. Part of the problem could be attributed to lack of flexibility on the GOP regulations 22 reflected in a very lengthy bureaucratic process required for revision of PC-1s, which were the binding constraint. The Bank's task team performance in terms of extension of the DFID trust fund is commendable. In order to make sure continuation of critical services and procurement of ARV, the team managed to extend the trust fund for six months and later on for another nine months, both retro- actively. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory Since quality at entry is Moderately Satisfactory, development outcomes and implementation support are Moderately Unsatisfactory, the overall rating for Bank performance is rated Moderately Unsatisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory The project benefitted from courageous and bold decisions on part of the GOP to focus on the marginalized and stigmatized population in the community and contract large scale services provision with the NGO. Hence, the difficulties that the project faced should be seen in the light of these ambitious undertakings. The project suffered from persistent and highly bureaucratic processes in the government that caused protracted delays in the procurement of the NGO services which together with the delayed payments to NGO adversely affected the project's success. With exception of Punjab, the government failed to bestow sufficient administrative power for the PACP managers, which could have solved many implementation challenges. (a) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory There were two levels of implementing agencies for the project: a) the NACP & PACPs, and b) the NGO. At the NACP/PACP level: The project was managed on behalf of the GOP by NACP at federal level and by PACP at province level. The NACP had overall responsibility for the program and would oversee implementation of the project, beside management of few contracts at federal level. PACP had the responsibility of overseeing the implementation of most of the activities at provincial level, including signing contracts, monitoring performance of the NGO and ensuring timely payment to the NGO. With exception of Punjab, where the PACP managed timely signing of the service contracts along with timely payment to the NGO, rest of the provinces, due to lack of capacity, faced serious implementation challenges both in the process of awarding and management of the contracts. Capacity at NACP and PACP for managing the project was 23 problematic throughout the project life. There was very high staff turnover especially procurement and financial management, leading to lack of institutional memory in the programs. At NGO level: The project service delivery was contracted out to NGO both at federal and provincial/AJK level. Despite the very lengthy procurement process and many payment delays, the NGO did manage to provide services to the high risk groups with variable coverage levels. Regarding IDU services, the coverage was the highest in four cities of Punjab. The success of IDU interventions in Punjab both in terms of behavioral outcomes and utilization is attributable to project support. The initial DFID support for IDUs in the province was limited to a small intervention in Lahore. Project financing over a 5 year period (Jan 2005-Dec. 2009) made it possible for the program to scale up coverage. At the end of the project nearly 14000 IDUs were receiving harm reduction services. This has been considered as one of the best programs in Asia on HIV prevention among IDUs, on which many other projects are modeled, where harm reduction for IDUs has been done effectively at such a scale. In early 2010, when a new Secretary of Health was assigned in Punjab, the IDU contract was cancelled by the PACP, without informing the Bank. The PACP terminated the contract using contract clause 2.6.1 (e) whereby "the Client in its sole discretion and for any reasons whatsoever decided to terminate the contract" with 60 days notice. No reason was specified in the termination letter. The Secretary Health had verbally provided a number of reasons for his action ranging from procedural deviation in award (PACP had reportedly not cleared the contract with the provincial law and finance departments), and PACP's weak supervision, to lack of funds after Jun 30, 2010. He however insisted that he wanted to play safe by terminating based on client's sole discretion. Following the termination the Secretary had stated that the Department would carry out a verification exercise that included verification of personal identifiers of IDUs. This exercise, to which the task team strongly objected, was not undertaken. Overall, the coverage of services to both male and female sex workers remained low, where the NGO could further increase the coverage of services through enhanced outreach activities. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory The government made the right decision to contract out services with NGO to reach out marginalized population in the society and to contain the HIV epidemic through strong public health measures. There was a strong focus on the main drivers of the HIV epidemic in the country, i.e. high risk behavior of IDU and sex workers. However, poor procurement, financial management, and non compliance with agreed Financial Covenants significantly impeded project implementation. Lack of proper political support and leadership prevented strong institution building which resulted in very high staff turnover throughout the life of the project. Hence, the government was not able to take full benefit of the available resources of the project. 6. Lessons Learned (both project-specific and of wide general application) NGO Contracts vs. Grants: The project financed large contracts with NGOs to deliver HIV prevention services. This was different from many other Bank-financed HIV projects where NGOs or community based organization (CBOs) are provided small grants to carry out HIV prevention activities. It is usually the NGOs/CBOs who decide where they will work, with which 24 MARPs and which services they will provide. The Bank's evaluation of these small grants programs has pointed out a number of shortcomings of the grant approach: (i) there is often limited coverage of the MARPs leaving many at risk people with no or limited access to services; (ii) there is limited evidence of effectiveness in terms of altering behaviors (sometimes the lack of evidence is the result of lack of data collection); (iii) often the services provided are not the ones that are most needed; (iv) NGO and CBOs tend to focus on relatively easy to reach groups ; (v) monitoring and evaluation is generally weak; (vi) grant management is very challenging because there are so many grants; (vii) there are many opportunities for corruption; and (viii) it is difficult to ascertain whether the government is getting value for money. The advantages of the grant approach are that it avoids the governments' and the Bank's procurement procedures, can disburse quickly, and is often politically attractive because it finances many small groups. Where the NGO contracts were large, the approach displayed a number of advantages as well as some serious challenges. The advantages included: (i) being able to focus on MARPs with a standardized package of HIV prevention services; (ii) being able to hold NGOs accountable for covering the MARPs within their contract area; (iii) facilitating monitoring and evaluation; (iv) providing economies of scale in terms of contract management, capacity building, and theoretically cost; and (v) allowing contractors to quickly respond to any "hotspots" that were detected without having to recruit or train new NGO/CBOs. However, the contracting approach also faced some serious challenges, including: (i) Setting Prices: Setting the prices of the NGO contracts was challenging particularly given the lack of competition. There were large differences between NGOs in terms of their price per beneficiary served with limited difference in performance. It probably makes sense to set the prices in advance using international comparators or the experience of previous contracts in the same country. (ii) Scale: The experience during the project suggested that larger NGO contracts had advantages in terms of the amount of time needed for recruitment. Punjab had one contract for IDU for the whole province and was signed in little more than 100 days while Sindh had five smaller contracts that did not cover the whole province and it took more than a year to sign the contracts. (iii) Procurement/Recruitment: Even advanced actions on NGO recruitment did not much reduce the very long time needed for contracting the NGO. Both the Government and the Bank need to consider more efficient ways of doing this kind of NGO recruitment. One option might be to link part of the payment to results in reaching the MARPs as judged by a third party. (iv) NGO Capacity: With the striking exception of the NGOs working with IDU, the other NGOs lacked the capacity, and often the will, to work with MARPs. It might have been helpful to insist that NGOs working with FSW or MSM demonstrate experience in that area. Timely TA to help the NGO would have been very helpful and should have been financed directly out of project funds. (v) Mapping MARPs: A lack of data on the size of the MARP in cities or provinces was an impediment in the project. However, there is a limit to the accuracy of such mapping and the results need to be seen as approximate. Nonetheless, a consistent approach to mapping using a third party would be valuable in judging progress. 25 (vi) While output-based contracting has in principle many advantages over input-based contracting, Government lack of familiarity with the method requires strong commitment from that side and much technical assistance throughout to implement it. Management Information Systems and M&E: The lack of well-functioning management information systems (MIS) and easy means for validating the data coming from the MIS created much distrust between the GOP and the NGOs. The GOP did not have sufficient assurances that NGOs were performing well. The IDU NGOs had an MIS developed by one of the NGO (NZ) and this functioned moderately well although there were limited opportunities to verify the quality of data. The difficulty of developing such an MIS should not be under-estimated and taking advantage of international experience through technical assistance will be very important. For example, there are a few NGOs in India who have very developed advanced MISs that could have been used in Pakistan. The Pakistani firm recruited to be the third party verifier of NGO performance lacked the required technical skills. This aspect of M&E could have been strengthened by technical assistance. Support for Procurement and Contract Management: The firm selected to provide procurement and management support not only lacked the qualifications for the job, it also lacked incentives to manage effectively. In future projects, TOR should be upgraded to take the above into account and contracts for such firms should specify that payments be at least partly linked to performance such as disbursements, contract signing, and timely payments to NGOs. Staff continuity with proper administrative authority: Most of the success in Punjab could be attributed to strong ownership of the provincial government that enabled managerial autonomy with continuity of staff, and flexibility in increasing allocations for priority interventions for IDU. The manager, FM and procurement staff continued to be the same individuals throughout the life of the project. Hence, staff continuity with proper administrative authority could be considered as preconditions for the relevant provinces for any future operation. In the meantime, NGO competencies and strong NGO/CBO leadership was another important factor for successes in Punjab. Flexibility: A project which is first of its kind, both in subject and in implementation modalities, must be designed in a flexible manner taking its pilot character into consideration. There should be prior agreement with the Government to make sure there is sufficient flexibility in the Government's planning process in terms of responding to a dynamic epidemic. Confidentiality of MARP records: The project suffered from lack of proper understanding of GOP officials regarding the importance of confidentiality of the high risk population groups and their HIV status. This requires special advocacy and orientation activities which should have been addressed in the project preparation phase. Difficulties of implementing nation-wide umbrella projects: the experience in Pakistan had highlighted difficulties of implementing umbrella projects covering multiple provinces. The project faced constraints due to complex process, particularly related to the multi-province umbrella design of the project where many processes required several layers of approval at the provincial and federal levels. For example the preparation of additional financing required the revision of the PC-1s of the National and Provincial Programs which took inordinate time of three 26 years to be completed. Hence, it would be advisable to consider separate projects with separate financial agreement with each province for future HIV projects. This is also in line with new PC-1 and the constitutional amendment 18, which decentralizes Health to provincial level. In built VCT for MARP intervention sites: The referral between the project sites and the VCT for HIV test did not work well. Third party assessments indicated that the organization and a follow-up on VCT had been weak. Most clients had been tested once only. Ideally, all those IDU and sex workers that remain HIV negative should be tested several times per year. Another limitation of VCT practices was that most NGOs had to send collected blood samples elsewhere for HIV testing. This led to delayed reporting of results which ended up with loss of clients. In light of the new WHO guidelines, on-site testing of target populations and same-day results should be considered to avoid loss in follow-up particularly for mobile populations. Accordingly, the HIV rapid test should be part of any SDP contract. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies See Annex 7 for the Borrower ICR and official Government comments on the ICR. (b) Co financiers N/A (c ) Other partners and stakeholders (e.g. NGO/private sector/civil society) N/A 27 Annex 1. Project Costs and Financing Project Cost by Component (in USD million equivalent) Components Appraisal Estimates Actual/Latest Estimate % of (USD million) (USD million) Apprai sal NAC Punja Sindh KPK B'tan AJK Total NACP Punjab Sindh KPK B'tan AJK Total P b Component 1: Expansion of Interventions Among Vulnerable Populations 1.1 Service 3.092 2.733 3.400 0.590 0 0.081 9.896 3.092 10.712 3.166 0.486 0 0.05 17.51 176.97 Delivery 7 3 % Contracts with NGO 1.2. Small Grants 0.133 0.154 0 0.025 0 0 0.312 0.130 0.147 0 0.025 0 0 0.302 96.79% Component 2: Improved HIV Prevention by the General Population 2.1 Behavior 4.693 1.383 0.901 0.787 0 0 7.764 4.678 1.031 0.069 0.203 0 0 5.981 77.03% change communication (BCC) aimed at the general adult Population 2.2 Advocacy 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2.3 Targeted 0 0.100 0 0 0 0.247 0.347 0 0.049 0 0 0 0.19 0.247 71.18% interventions for 8 youth, then police and formal sector workers 2.4 Improved and 0.547 0.282 0 0.056 0 0 0.885 0.507 0.165 0 0.049 0 0 0.721 81.46% expanded management of STI cases 28 Components Appraisal Estimates Actual/Latest Estimate % of NAC Punjab Sindh KPK B'tan AJK Total NACP Punja Sindh KPK B'tan AJK Total Apprais P b al Component 3: Prevention of HIV/STI Transmission Through Blood Transfusion 3.1 Establishing 0.863 0.062 0 0 0 0 0.925 0.691 0.023 0 0 0 0 0.714 77.18% and Building the capacity of provincial blood transfusion authorities 3.2 0.102 0.783 0 0 0 0 0.885 0.077 0.201 0 0 0 0 0.278 31.41% Implementation of a quality assurance system 3.3 Screening of 1.250 2.249 1.183 1.230 0 0.36 6.274 0.890 3.750 0.474 0.448 0 0.191 5.753 91.69% blood for HIV 2 and other STIs 3.4 Waste 0.383 0 0 0.020 0 0 0.403 0.382 0 0 0.001 0 0 0.383 95.03% management Component 4: Capacity Building & Program Management 4.1 Strengthening 5.589 1.794 0 0.550 0 0 7.933 4.692 1.024 0 0.679 0 0 6.395 80.61% of Federal AIDS Control Programs 4.2 NGO 1.767 0.015 0.594 0.030 0 0 2.406 1.572 0.008 0.737 0.021 0 0 2.338 97.17% Capacity development 4.3 Second 0 0 0 0 0 0.35 0.358 0 0 0 0 0 0.227 0.227 63.40% Generation HIV 8 surveillance and evaluation 4.4 Care for 0.533 0.176 0 0.145 0 0 0.854 0.414 0.015 0 0.063 0 0 0.492 57.61% people living with AIDS TOTAL 18.95 9.731 6.078 3.433 0 1.04 39.24 17.131 17.12 4.446 1.974 0 0.673 41.34 6 8 6 3 7 Table B: Financing US $ million components Type Appraisal Estimates Actual/Latest Estimate % of Co- (USD million) (USD million) of Financ App ing NACP Punja Sindh KPK B'tan AJK Total NACP Punja Sindh KP B'tan AJK Tota rais b b K l al World Bank Loan/G 16.112 12.13 7.311 2.747 0 0.891 39.19 13.401 13.71 3.696 1.4 0 0.547 32.8 83.7 rant 5 6 3 57 14 1% DfID Grant 0 1.785 0 0 0 0 1.785 0 1.785 0 0 0 0 1.78 100 TF93986 5 % Govt of Local 2.843 1.612 0.911 0.687 0 0.157 6.21 3.730 1.612 0.750 0.5 0 0.137 6.74 108. Pakistan Share 18 7 64% TOTAL 18.955 15.53 8.222 3.433 0 1.048 47.19 17.131 17.11 4.446 1.9 0 0.684 41.3 2 1 0 75 46 Source: Pakistan National Aids Control Program (NACP) - Baluchistan did not provide any financial information despite repeated request from both the Bank and the NACP . 29 (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) PREVENTIVE SERVICES FOR 10.53 9.23 87.65% VULNERABLE POPULATIONS IMPROVED HIV PREVENTION AMONG THE GENERAL 11.81 9.74 82.47% POPULATION PREVENTION OF HIV/STI TRANSMISSION THROUGH 9.21 7.68 83.39% BLOOD TRANSFUSION CAPACITY BUILDING AND PROGRAM MANAGEMENT 10.55 5.96 56.49% Total Baseline Cost 42.10 32.61 77.46% Physical Contingencies 2.10 0.00 0.00 Price Contingencies 3.57 0.00 0.00 Total Project Costs 47.77 32.61 68.26% Total Financing Required 47.77 32.61 Source: Pakistan National Aids Control Program (NACP) (b) Financing Appraisal Actual/Latest Type of Co Estimate Estimate Percentage of Source of Funds financing (USD (USD Appraisal millions) millions) Borrower 6.16 0.00 .00 CANADA: Canadian International 3.00 1.04 34.67 Development Agency (CIDA) UK: British Department for 1.50 4.12 274.67 International Development (DFID) International Development 27.83 24.88 89.40 Association (IDA) IDA GRANT FOR HIV/AIDS 9.28 9.24 99.57 30 Annex 2. Outputs by Component Component 1. Expansion of interventions among vulnerable populations (Total actual cost US$ 9.23 million) The objective of this component was to expand and strengthen the prevention interventions for vulnerable populations (IDU, FSW, and MSM). The primary activities included contracting NGOs to work with the vulnerable populations creating drop in centers, providing behavior change communication, peer-educator based outreach, distribution of free commodities (e.g. condoms and sterile needles/syringes), and provision of clinical services including counseling and testing and STI management. These interventions focused on vulnerable populations in cities selected according to what was believed to have the largest concentration of vulnerable groups. Under the project, between 2004 and 2006, 18 contracts were awarded to NGOs covering IDUs in 7 cities MSW/transgender sex workers in 6 cities, FSW in 5 cities, jail inmates in 5 cities and truckers nationwide. Estimates of the size of the vulnerable population in Pakistan suggest that project intervention cities comprise 44% of the IDU population; 49% of the FSW and 43% of the MSW/TSW population in the country. Table 1. Service Delivery Packages (SDPs) - Location and Date of Contract Signing IDU MSW/TSWs4 FSW Contract Contract Contract Cities NGO Cities NGO Cities NGO Date Date Date Karachi 1 Nai Zindagi June 05*Karachi ICS June 06 Karachi Amal April 04 Pakistan Karachi 2 June 05 Hyderabad ICS June 06 Lahore Contech April 04 Society Karachi 3 Al Nijaat June 05 Sukkur ICS June 06 Multan Rabta Nov. 05 Lahore Nai Zindagi January 05 Lahore Contech April 04 Peshawar ORA May 06 Faisalabad Nai Zindagi January 05 Faisalabad Contech Nov. 05 Hyderabad Greenstar May 06 Sargodha Nai Zindagi January 05 Peshawar ORA Dec. 053 Sialkot Nai Zindagi January 05 Dost Peshawar Foundation Quetta Legend April 06 * The NGO stopped operations since February 2006 Source: The World Bank Aide Memoire for HIV Financing, November 2007 In 2008, an assessment of NGO performance (Khan, 2008) reviewed program coverage as measured in terms of commodities distributed against the estimated number of injections per quarter by IDU and the number of commercial sex acts had by female and male sex workers. These results suggest that overall coverage for IDU populations was strongest at an estimated 30% of injections covered by free needle/syringe distribution across all intervention cities. This compares to only 4-5% of commercial sex acts among both FSW and MSW/TSW covered by free condom distribution. The projected size of the vulnerable population is much higher in the Punjab and Sindh, which comprise more than 85% of the IDU and SW in the country. There is some 4 Although Hijra and Transgenders are used interchangeably, the term Hijra narrowly defines transgenders with a female gender identity. Since, other transgenders such as transvestites are typically not distinguished from Hijras in the surveys that form data sources for this report; this report uses the term transgender rather than Hijra. 31 variation by province, with IDU coverage being higher in the Punjab (36%) compared to Sindh (16%); but SW coverage being higher in Sindh (10%) compared to the Punjab (2-3%). In addition to the interventions for IDU, FSW, and MSW/TSW the project supported prevention services for jail inmates in Sindh and NWFP and long distance truckers in 10 sites across the country. The long distance truckers program began in 2006 and was implemented by FHI. It has been characterized as well organized with strong MIS, and good coverage, making about 26,000 contacts on a quarterly basis with truckers and cleaners in the last quarter of 2007, and revised its registration targets from 40,000 to 70,000 in 2007. However, more than a third of the target represents a lower risk associated population of workers in truck stops who are not as mobile and less likely to engage in frequent commercial sex or other sexual risk behavior. Over time, the trucker program has adjusted its strategy to ensure a higher proportion of contacts are made with long distance truckers. Interventions for jail inmates were implemented in Sindh (in the cities of Karachi, Sukkur and Hyderabad) and NWFP (in 2 cities of Peshawar and Haripur). The Sindh targeted interventions covered roughly 15000 prisoners while the NWFP program covered 4000 jail inmates. The interventions included BCC through peer educators, VCT, medical services through the NGO own clinic as well as training of health providers in the jail dispensaries. While the peer educators provided demonstrations of condom use distribution of condoms was not allowed in the NWFP jails but in Sindh the NGO was able to supply condoms on a limited basis. Regular advocacy sessions were undertaken with jail management, district government officials and as well as with other NGO working in prisons such as legal-aid organizations. Jail inmates were not covered by IBBS so it is not possible to assess the effectiveness of these interventions. However, the third party evaluation report noted improvement in services for jail inmates in the last round in 2008 and found the overall quality adequate. Graph 1: IDU Project performance in Lahore, Faisalabad, Karachi, 2005-2008 Moderate to high program participation, high levels of safe injecting behaviour and slowly increasing HIV prevalence - 100 Lahore 90 80 70 60 50 40 30 20 10 0 2005 2007 2008 participated in program clean needle at last injection HIV prevalence 32 Large late gains in programme participation, declining clean needle use, and declining HIV prevalence - Karachi 100 90 80 70 60 50 40 30 20 10 0 2005 2007 2008 participated in program clean needle at last injection Large late gains in program participation, high levels of safe injecting behaviour and stable levels of HIV 100 prevalence - Faisalabad 90 80 70 60 50 40 30 20 10 0 2005 2007 2008 participated in program clean needle at last injection Source: IBBS, Round 1-3 Table 2: summarized the list of contracts planned versus contracts realized for each province during life of the project: Contract Geographic focus Duration Extension Remarks NACP Truckers Nationwide Jan 05-Jan 08 BCC firm national May 05-May 08 Third Party M&E assessment of all TIs Oct 04-Oct 08 Procurement contracted by NACP Management Firm to support all programs Oct 04-Oct 08 33 Punjab IDU 4 cities in Punjab Jan 05-Jan 07 Jan 07-Dec 08 FSW Multan Nov 05 FSW Lahore April 04-April 09 MSW Lahore April 04-April 09 MSW Faisalabad Nov 05-Nov 09 BCC Firm Punjab Aug 06 Planned but not contracted Jail inmates Lahore, Multan, Problems in procurement Rpindi process, delays MSW Gwala, Sialkot, rpindi Decision to go for province- wide proposals FSW Gwala, Sialkot, Fbad, Decision to go for province- Rpindi wide proposals Sindh IDU Karachi 1 June 05 -Jun 08 Services closed on Feb 2006 IDU Karachi 2 June 05-Jun 08 Jun 08-Dec 09 IDU Karachi 3 Jun 05-Jun 08 Jun 08-Dec 09 FSW Karachi April 04-April 08 FSW Dec 08-Dec Hyderabad May 06 -Dec 08 09 MSW/transgender Karachi, Hyderabad, Dec 08-Dec Sukkur Jun 06-Dec 08 09 Jail inmates Karachi, Hyderabad, Apr 08-Dec Sukkur Apr 04-Apr 08 09 Planned but not contracted BCC firm Problems in procurement process, huge delays-proposals received on May 31, 2004, TER June 2006 FSW Sukkur NWFP IDU Peshawar June 08 -Dec Dec 05-June 08 09 FSW Peshawar Office burnt down in Dec. Dec 05-June 08 2007 MSW Peshawar Work could not commence till July 2007-office burnt down in Dec 05-June 08 Dec. 2007 Jail inmates Peshawar Haripur March 06-June 08 34 BCC Firm The Program and Firm agreed April 07 to close the contract Planned but not contracted FSW Abboatabad MSW Bannu Baluchistan IDU April 2006-June Quetta 2008 Planned but not contracted FSW Huge variations in prices- decision to go for fresh proposals under the province- Quetta wide package Coal Miners 7 mines Same as above MSW Quetta Did not get a good response Jail inmates Quetta Did not get a good response BCC Firm Problems in procurement process, huge delays -Bank NoC for TE in March 2006- no response from Program-2007 procurement cancelled AJK BCC Firm April 07-April 09 Table 3: Summarizes the SDP coverage data: City Target NGO Size - Registered (%) Use services5 group HASP - NGO (HASP) Lahore IDU NZ6 3,350 7,7337 232% 89% Faisalabad IDU NZ 8,030 6,8008 80% Sargodha IDU NZ 2,450 5,8029 69% Sialkot IDU NZ 700 1,58110 Lahore MSM Contech 4,150 7,54011 182% 1% (7%HSW) Faisalabad MSM Contech 5,445 3,70012 68% 5% (8%HSW) Lahore FSW Contech 14,225 9,25613 65% 4% Multan FSW Rabta 2,725 2,54714 93% Multan MSW - - - 8% (13%HSW) 5 IBBS 2007 (2008 for IDU and MSW) 6 Note: in the Sept 2009 report, the following `regular' NSEP clients are mentioned: Lahore 3324; Fsbd 3103; Sargodha 3249; and Sialkot 1155. This is expectedly less than the total registered. 7 Khan report 2008 8 AM May 2008 9 AM May 2008 10 Khan report 2008 11 Final report Contech 12 AM May 2008 13 Contech final report 14 Khan report 2008 35 City Target NGO Size - Registered (%) Use services5 group HASP - NGO (HASP) 12 cities IDU NZ Tbd 0 Karachi 1 IDU NZ/MA Karachi 2 IDU PS 3150 3,36015 106% 72% Karachi 3 IDU Al Nijat 3000 3,30616 110% 72% Larkana IDU PS Na Na 17% Karachi MSM ICS 10,900 14,24617 130% 21% (31%HSW) Hyderabad MSM ICS 2,450 1,231 50% 1% (13%HSW) Sukkur MSM ICS 1,150 763 66% 1% (0%HSW) Karachi FSW Amal NA 11,48118 10% Hyderabad FSW Greenstar 2,300 1,71919 75% 2% 9 jails Sindh Jail SBDS 15,000 22,73720 Peshawar IDU DF 150 13821 80% Peshawar MSM ORA 1,100 52022 47% 3% (22%HSW) Peshawar FSW ORA 1,200 12023 10% 0.2% Peshawar/Haripur Jail DF 3,80024 Quetta IDU Legend 5% 25 10 sites Truckers FHI 70,000 62,825 Component 2. Improved HIV prevention by the general population (Total actual cost US$ 9.74 million) This component included four sets of activities: a behavior change communication (BCC) campaign aimed at the adult general population; advocacy among decision makers to support the AIDS control program; targeted communication activities among selected general population groups (e.g. youth, police, formal sector workers); and improvement and expansion of STI management in public sector facilities. During implementation the focus was mainly on BCC and advocacy. Behavior Change and Communication (BCC) and Advocacy: The Program invested substantially in BCC and communication for the general population and advocacy. The BCC campaign for the general adult population was to be conducted at both national and provincial levels, with the following behavioral objectives: (i) use of condoms with non-regular sexual partners, (ii) use of STI treatment services when they have symptoms of STI, and knowledge of the link between STIs and HIV; (iii) use of sterile syringes for all injections; (iv) reduction in the number of injections received, (v) voluntary blood donation (particularly among the age group 18-30), (vi) use of blood for transfusion only if it has been screen for HIV, and (vii) displaying tolerant and 15 Altaf report, September 2009 16 Altaf report, September 2009 17 Altaf report, September 2009, for all ICS coverage data 18 SoSec report 2008 19 Altaf report, September 2009 20 AM Dec 2008 21 AM May 2008 22 AM May 2008 23 AM May 2008 24 AM May 2008 25 AM dec 2008 36 caring behaviors towards people living with HIV/AIDS and members of vulnerable populations. Advocacy was aimed at mobilizing support for the program among key decision makers and opinion leaders. A qualitative review of the BCC component has pointed out most of the objectives particularly focusing on risk behaviors in the general population were not appropriate during a concentrated epidemic such as in Pakistan. Other objectives such as (iii), (iv) and (v) would be better if targeted to special populations such as health workers. Screened blood supply, particularly, needs to be addressed as an advocacy issue with government and health institutions. The communication and advocacy activities were implemented by private firms contracted by the National, Punjab, NWFP and AJK AIDS Control Program. In the absence of a common framework or strategy the campaigns were ad hoc and poorly coordinated. A wide range of media were used including TV and radio spots, sponsored talk shows and documentaries, print ads and articles, posters, brochures, stage productions and events. The messages focused primarily on modes of transmission and means of prevention while largely neglecting the important goal of mitigating stigma and discrimination. Due to official resistance to explicit mention of the protective effect of condoms, messages on safe sex were constrained and a disproportionate emphasis placed on blood safety and use of sterilized needles for therapeutic injections. The campaign's limited ability to convey accurate messages is borne out by recent evidence on knowledge of HIV risks and prevention. A survey of urban men in six large cities, a key target audience, found that overall 90 percent of the respondents had heard of AIDS and a large proportion 85 percent were aware that the disease was sexually transmitted (Population Council 2007). However, only 18 percent of those who had heard of the disease could respond without prompting that condom use was a means of prevention. Awareness of AIDS was also high among urban women; nearly 80 percent of ever-married women aged 15-49 living in major cities had heard of AIDS but only 30 percent knew (with prompting) that condoms can reduce the chances of getting HIV (Demographic and Health Survey 2007). Knowledge of risks and prevention methods varied by residence, economic status and education and as expected rural women in poor households were the least informed. Advocacy: The implementing agencies were able to mobilize broad support for the Program by key government officials, and other groups including religious leaders. As a result the Program did not face any opposition to interventions for highly stigmatized groups such as male sex workers and transgender sex workers even in provinces such as NWFP at a time when the provincial government was lead by a coalition of religious parties. However, advocacy activities lacked a strategic focus and clearly defined objectives and constraints to be addressed and many of the "advocacy" events undertaken by the media firms were general awareness raising workshops with audiences that did not necessarily include key decision makers. The objective of advocacy would be to facilitate, remove barriers to the implementation of the program, whether through the development of AIDS related policies, enactment of laws, modification of school curricula or standards of care, changes in the attitudes of the judiciary system, or the media. The review of the advocacy activities under Component 2 suggest that a number of advocacy events were carried out: seminars, workshops, meetings and trainings with groups such as policy makers, district authorities, religious leaders, journalists, media personnel, civil society organizations, police and others. However, the lack of a clear strategy, specific set of activities, 37 and targets or indicators of performance make the effectiveness of these events difficult to assess. The NACP representatives interviewed as part of the review indicated satisfaction with the efforts of the agency engaged to support the advocacy efforts, but not specific data exists which assess changes in the enabling environment to support the AIDS control program which resulted from these advocacy efforts. Targeted communication activities for youth, police and formal sector workers were conducted in the form of events for these groups accompanied by promotional/educational materials produced by the same agency which conducted the BCC campaign and advocacy efforts, for example a brochure on Youth and AIDS for youth events, a poster on universal precautions for hospital staff, and others. In addition, promotional materials such as t-shirts, mugs, key chains, red ribbons, male condom packs, female condom purses and bags were designed and produced for various groups. Much of this was done with support by the UN Agencies particularly work with formal sector workers and uniformed personnel. In addition Global Fund Grant Round 2 supported interventions for out of school youth. The review suggests that similar to the advocacy activities, targeted interventions for these selected general population groups were not part of a coherent strategy and difficult to assess in overall effectiveness. The sub-component of improving STI management among the general population was not strategic in the Pakistan context of a concentrated epidemic. Limited progress was achieved. Key outputs include the development of revised national guidelines for adopting syndromic management of STIs. These guidelines were intended to support enhanced clinical services at already established public sector STI clinics and recently trained providers. Other proposed activities such as improving recording and reporting and development of a check list for supervision of facilities were not implemented. The third party evaluation of quality of care and public hospitals and clinics (in 2006) suggested very poor levels of quality. Only 18% of the 57 clinics were assessed as providing satisfactory or highly satisfactory quality of care based on a broadly defined index. Patient satisfaction was low with only 36% of patients saying they were satisfied or highly satisfied with services during an exit survey. Only 25% of physicians could make a correct diagnosis when confronted with a standard scenario and only 11% indicated a correct treatment. This is not entirely surprising as only 21% of physicians surveyed indicated that they had been trained on the syndromic management approach to STIs. Component 3. Prevention of HIV/STI transmission through blood transfusion (Total actual cost US$ 7.68 million) The objectives under this component were to establish provincial level blood transfusion authorities to regulate private and public sector blood banks; implement a quality assurance system for screening blood products; and provide material support for screening blood products and addressing waste management issues at blood banks. Guidelines for standardization of transfusion equipment, diagnostic kits and supplies were developed through the project and supplies for public sector blood banks were provided with no real shortages. Project support for the procurement of hepatitis C kits was contingent on the establishment of an effective QA system. The project was delayed in developing quality assurance guidelines due to a poorly developed initial draft of the guidelines and the need to engage international technical assistance, before finally being adopted. 38 Legislation to regulate blood transfusion facilities had been passed before the project activities began, however, operationalizing these laws were an ongoing struggle in all provinces, except for Sindh. Toward the end of the project, Sindh province received 237 applications for certification from the approximately 400 existing private blood banks. Only 57 had been approved to continue operations, and 91 were given notice to be closed. These data are consistent with a study financed by the Global Fund which found that among private blood banks only 71% claimed to screen all blood for HIV. The actual observations of the blood banks show only 50% doing HIV screening and found very limited quality assurance. [Aide Memoire, May 2006] Midway into the project, the Bank and NACP recognized the need for a more comprehensive strategy and larger separate project to address issues in blood safety. Component 4. Capacity building and program management (Total actual cost US$ 5.96 million) To build the capacity of AIDS Control Programs for managing the enhanced HIV/AIDS control program, more staffs were deployed at both federal and provincial levels. However, with exception of Punjab, the retention of staff, especially procurement and FM staff was a continuous problem. Provincial manager were sent abroad for study tour. All programs were fully equipped and adequately housed with the exception of AJK, where the building was destroyed during an earthquake in 2005. The NACP recruited a management and procurement firm to assist the AIDS Control Programs in various aspects of management and procurement. The firm could expedite, to some extent, recruitment of NGO and procurement of goods in some provinces, but overall due to weak capacity of the firm and being overcharged with different tasks, the performance wasn't satisfactory. The Management Firm was also supposed to support contract management and organizational capacity of NGO, but not their technical capacity. The mid-term review in 2006 called for investment in capacity building. NACP organized some technical capacity building activities through consultancies (e.g. TA assessments for MSM and sex worker interventions and a workshop for MSM interventions). The Program benefited from the integrated biological and behavioral surveillance system (IBBS) among the vulnerable population established through CIDA support. The three rounds of IBBS Surveys conducted during the duration of the project were useful for tracking epidemic dynamics and assessing project outcomes. However, overall the IBBS results remained under-analyzed, under-utilized and suffered from lack of proper dissemination among partners. The reports were not made available expeditiously and the contents were not user-friendly. NACP recruited a third party firm to independently assess the performance of NGO providing HIV prevention services, the quality of blood bank services, and the quality of STI treatment services. The firm conducted three rounds of assessments during the project. The reports of blood transfusion were of reasonable quality but there was room for improvement in the methods used to assess targeted interventions. There were considerable delays in the submission of reports and the findings not discussed with relevant stakeholders. Monitoring of other project components such as treatment and care and BCC has also been inadequate. Treatment and Care: Antiretroviral (ARV) therapy was introduced in Pakistan in 2005. Standard Operating Procedures and the National Strategic Framework for HIV Treatment, Care and Support in Pakistan were developed in 2006. At present there are thirteen treatment centers established in tertiary care teaching hospitals across the country in both the public and private 39 sector. These centers provide free ARV, management of opportunistic infections, acute and chronic HIV care, STI management, counseling, in-patient admissions, and referral/specialty services to HIV positive people and their families. The project financed medicines for opportunistic infections and supported key staff positions. Support for procurement of ARV medicines and training has been provided by the Global Fund Round 2 Grant. As of December 2009 there were about 2900 people living with HIV/AIDS registered with these centers and of these 1,300 were receiving free antiretroviral therapy. The key challenges facing the treatment and care program include: (i) reducing barriers to access and increase utilization of the centers particularly be most at risk groups; for example those IDU wishing to be considered for ART must discontinue their illicit drug use and demonstrate full family support; this discouraged providers from promoting treatment or IDU from seeking services; (ii) Strengthening quality of care; and (ii) strengthening capacity and management including through the development of an effective monitoring system to track performance. 40 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Please see section 3.3 in the main text of the ICR. 41 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Benjamin Loevinsohn Sr. Public Health Specialist AFTHE Inaam Haq Sr. Health Specialist SASHN Hugo Diaz-Etchevehere Lead Operations Officer -- Muhammad Bashirul Haq Sr. Health Specialist -- Nancy Fee AIDS Specialist, UNAIDS Secondee -- Salim Habayeb Lead Public Health Specialist -- Maj-Lis Voss Operations Officer -- Mohammad Khalid Khan Program Assistant SASHD Javed Karimullah Team Assistant AFTHE Ahsan Ali Sr. Procurement Specialist EAPPR Arif Yaqub Sr. Financial Management Specialist -- Qazi Azmat Isa Sr. Rural Development Specialist SASDA Silvia Albert Program Assistant SASHD Akhtar Hamid Lead Counsel -- Nawaf Al-Mahamel Counsel -- Supervision/ICR Asif Ali Senior Procurement Specialist SARPS Agnes Couffinhal Sr Economist (Health) ECSH1 Shahnaz Kazi Sr Economist (Health) SASHN Benjamin P. Loevinsohn Lead Public Health Specialist AFTHE Riaz Mahmood Financial Management Analyst SARFM Hasan Masood Mirza Consultant SARPS Sandra Rosenhouse Sr Population & Health Spec. SASHN Furqan Ahmad Saleem Sr Financial Management Specialist AFTFM Hasan Saqib Sr Financial Management Specialist SARFM Ghulam Dastagir Sayed Health Specialist SASHN Tayyeb Masud Health Specialist SASHN Henri Aka Procurement Specialist SASHN Naoko Ohno Operations Officer SASHN Virginia Loo Consultant SASHN (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending 42 FY02 74 218.08 FY03 54 138.11 Total: 128 405.73 Supervision/ICR FY04 40 105.82 FY05 51 136.90 FY06 41 102.55 FY07 41 94.20 FY08 43 110.53 FY09 33 71.26 FY10 19 49.33 FY11 28 81.53 Total: 297 758.15 43 Annex 5. Beneficiary Survey Results (if any) Not applicable 44 Annex 6. Stakeholder Workshop Report and Results (if any) Not applicable 45 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR NATIONAL AIDS CONTROL PROGRAMME (National Institute of Health, Chak Shahzad, Islamabad) DRAFT IMPLEMENTATION COMPLETION AND RESULTS REPORT (Cr: 3376-PAK, Grant Ho44-PAK) Pakistan HIV-AIDS Prevention Project (2003 to 2008) Rs. 2.85billion Prepared by National AIDS Control Program Ministry of Health, Government of Pakistan Islamabad 2010 46 Table of Contents Abbreviations and Acronyms ................................................................................................. ii, iii 1. Project Context, Development Objectives and Design ...................................................... 1 1.1 Context at Appraisal .................................................................................................... 1 1.2 Project Objectives ........................................................................................................ 2 1.3 Beneficiaries of the Project........................................................................................... 3 1.4 Project Design............................................................................................................... 3 2. Implementation, Operational Experiences and Outcomes ............................................... 4 2.1 Component 1: Services for vulnerable population sub-groups ..................................... 4 2.2 Component 2: Improved HIV prevention by general population ................................. 6 2.3 Component 3: Prevention of HIV/STI transmission through blood transfusion .......... 7 2.4 Component 4: Safeguard and Fiduciary Compliance ................................................... 8 3. Performance of the Programme .......................................................................................... 9 4. Lessons Learnt and Changes Proposed for Future ......................................................... 11 47 ABBREVIATION AND ACRONYMS AIDS Acquired Immune Deficiency Syndrome AJK Azad Jammu and Kashmir BCC Behavior Change Communication CSW Commercial Sex Workers DoH Department of Health DFID Department for International Development FCA Federal Committee on AIDS GoP Government of Pakistan HMIS Health Management Information System HIV Human Immunodeficiency Virus IDA International Development Agency IDU Intravenous Drug User IEC Information Education and Communication IPC Inter-personal Communication KAP Knowledge, Attitude & Practice MCH Maternal & Child Health MOF Ministry of Finance MOH Ministry of Health MSM Men having Sex with Men NACP National AIDS Control Program NBTC National Blood Transfusion Committee NGO Non-Government Organization NIH National Institute of Health PBTA Provincial Blood Transfusion Authority PHC Primary Health Care PIHS Pakistan Integrated Household Survey PLWHA People Living with HIV/AIDS PACP Provincial AIDS Control Program PIP Project Implementation Plan QA Quality Assurance Rs Rupees RTI Reproductive Tract Infection STI/STD Sexually Transmitted Infection/Diseases TA Technical Assistance TOR Terms of Reference UNAIDS Joint United Nations Program on HIV/AIDS UNDCP United Nations Drugs Control Program UNDP United Nations Development Program UNFPA United Nations Population Fund UNICEF United Nations International Children's Emergency Fund VCT Voluntary Counseling and Testing WB World Bank WHO World Health Organization 48 Secretary, Ministry of Health, Islamabad: Deputy Director General Health, MoH, Islamabad: Senior Joint Secretary, MoH, Islamabad: Executive Director/National Coordinator, National Institute of Institute of Health: Manager, National AIDS Control program: Profile 1. Name of the Project Enhanced HIV/AIDS Control Programme 2. Sector Health 3. Location National Institute of Health, Islamabad. 4. Sponsoring Ministry/ Agency Ministry of Health 5. Executing Agency Ministry of Health 6. Name of Project Director Dr. Sajid Ahmad 7. Postal Address National AIDS Control Programme , National Institute of Health, Chack Shahzad, Islamabad. 8. Telephone Office 9255096, 9255241-42. 9255367-8. 9255378 Residence 9203703, 2823729. 9. Fax No. 9255214 10. E.mail address: info@nacp.gov.pk 49 1. Project Context, Goal, Objectives, and Design: 1.1. Context at Appraisal Since 1987, the number of reported HIV infections and AIDS cases had been steadily on the rise and were distributed in all geographical regions of the country. As of June 30, 2002, the total numbers of reported cases were 1940. However, the WHO/UNAIDS forecast model estimated HIV positive cases to be approximately 70,000 to 80,000 with 10,000 to 21,000 cases in youth i.e. 0.1% of the adult population. The mode of transmission of infection in HIV positive persons on cumulative basis (1997 ­ 2002) was as follows: heterosexual transmission (41.1%); contaminated blood or blood products (16.1%); injecting drug use (3.3%), homosexual or bisexual (2.1%), and mother to child transmission (1.4%). The mode of transmission in 34% of the reported HIV positive cases could not be established. The male to female ratio at that time was 6.5:1 and 6:1 in reported HIV positive and AIDS cases, respectively. The Reproductive Health and Family Planning Survey (RHFP) of 2001 by the National Institute of Population Studies (NIPS), Islamabad reported that 42% of married women had heard about HIV and AIDS. The majority of respondents knew about the major modes of transmission and prevention. According to the findings of a study conducted in 2001 to assess the KABP of people in response to AIDS awareness campaign, the overall awareness level in both rural and urban areas was 77% wherein respondents had heard the word AIDS, 74% of the respondents knew that AIDS is a disease and 72% of the respondents knew AIDS is a dangerous disease and has no cure. Television was identified by the respondents of the survey as the main source of information about HIV and AIDS. Government Strategy: The Government of Pakistan was fully aware of the growing challenge of HIV and AIDS and its implications for the population. The Ministry of Health (MoH) had established Federal Committee on AIDS (FCA) in 1987 soon after first case of AIDS was reported. The National AIDS Control Program (NACP) was established with a focus on establishment of HIV screening centers and launching awareness campaign. In its early stages, the programme focused on the laboratory diagnosis of suspected HIV cases, but progressively it began to shift its focus towards HIV prevention and control interventions. In 1994, the programme was brought under the multi-donor financed Social Action Program with a more pragmatic agenda including information education, blood screening and establishing provincial implementation units. The government's commitment is reflected by the fact that about 80% of all expenditures on HIV and AIDS have been financed by the Government of Pakistan (GoP). In 1999 - 2000, the GoP with the assistance of UNAIDS and other development partners undertook a strategic planning exercise with input from all stakeholders. This exercise resulted in development of a National Strategic Framework (NSF) in 2001, which provided strategic vision to the national response in the coming years. The framework envisaged a multi-sectoral response and development of partnerships and collective action, which was essential for decreasing the vulnerability of Pakistani population towards HIV and AIDS. This national strategic framework formed the bases of World Bank supported Enhanced HIV and AIDS Control Programme (EHACP) launched in 2003. Enhanced HIV and AIDS Control Programme (EHACP): Launched in 2003, The Enhanced HIV and AIDS Control Programme was aimed at improving human capital with specific reference to improving the health status of the population of Pakistan. It was envisaged to contribute towards preventing 49 mortality among the labour force by keeping the HIV/AIDS epidemic at the current low level of below 1 percent in the general adult population and below 5 percent in any of the high-risk population sub-group. Limited available research in early 2000s indicated that HIV prevalence was 1-2% in vulnerable/high risk population; which included, sex workers, migrant workers, injecting drug users, men who have sex with men, long distance truckers, blood and blood product recipients, STI patients, professional blood donors, jail inmates, and seamen. The programme focused on HIV prevention among most at risk groups via public-private targeted interventions. HIV prevention among IDUs and sex workers in provincial capitals and other major cities, implemented through private sector and civil society, formed the core of this project. Some interventions were also implemented for jail inmates and long distance truck drivers under the EHACP. The truckers' intervention was placed in 9 cities across the country. A major component of the Enhanced Programme was to raise awareness about HIV among the general public to mitigate stigma and discrimination against people living with HIV and AIDS. A comprehensive Treatment, Care and Support system was set up nationwide via 9 public and private centres and NGOs that provide counseling, logistic and nutrition support to People Living with HIV (PLHIV). About 600 PLHIV receive AIDS medicines (antiretroviral drugs - ARVs) nationwide and twice as many are registered and managed in anticipation of ARVs. In addition to that, HIV care of the children and Prevention of Parent to Child Transmission of HIV (PPTCT) were started by the NACP and PACPs. The NACP (MoH) set up the national HIV/AIDS Surveillance Project with support from the Canadian International Development Agency (CIDA). This project conducts annual rounds of internationally acceptable surveillance that forms the basis for much of the national data including monitoring and evaluation of the HIV response. 1.2. Project Objective The overall objective of the programme was to prevent HIV from becoming established in vulnerable populations and preventing its spread to the general adult population while avoiding further stigmatization of the vulnerable populations. The vulnerable populations included female sex workers (FSWs), injecting drug users (IDUs), men who have sex with men (MSM), prisoners, and migrant workers particularly long distance truck drivers. Given the inadequacy of available data on HIV prevalence and its spread patterns among the adult population, and the lack of substantial research on mostatrisk populations, coupled with insufficient experience and limited capacity of the AIDS control programmes and participating NGOs at the time of project initiation, the broad objectives of the project were most suitable to the programme needs. These objectives reflected the priorities outlined in the national strategic framework, which was developed with the participation of all stakeholders in 2002, and were in accordance with the epidemic trends in Pakistan (Asian Model) which called for targeted intervention among high risk groups. The objective of preventing the spread of HIV among high risk groups, and within the general population, was and continues to be highly relevant to Pakistan. Surveillance data indicated that prevalence rates are increasing among high risk group especially IDUs. Globally, the Millennium Development Goals established in September 2000 had established `halting and reversing the spread of HIV prevalence' as an explicit goal for all countries. 1.3. Beneficiaries of the Project 50 The project was aimed to provide direct benefit to male sex worker, female sex workers, Hijras, long distance truckers, intravenous drug users, coal miners and jail inmates through preventive and curative package of services. The estimated numbers of direct beneficiaries were as follows: Long distance truckers 40,000; Commercial Sex Workers (CSWs) 36,000; Injecting Drug Users (IDUs) 19,000; Men who have Sex with Men 14,000; Jail inmates 27,500; Coal miners 30,000; Youth12.0 million; and Public sector employees 250,000. In addition these, the entire population of the country was expected to benefit from Behavior Changes Communication (BCC), Health education, Blood transfusion services and Management of STI services. 1.4. Project Design The project design was developed through a participatory consultative process involving provincial actors and other stakeholders. Feedback was sought from key federal planners on the preparation and design of the project throughout the planning period. The detailed design of the project was based on the National Strategic Framework for HIV/AIDS control prepared by the NACP and adopted by the GoP At the core of the project design was the idea of public-private partnership. This concept was not entirely new to Pakistan as similar arrangements between GoP and NGOs/private sector had been in place in the past as well. The underlying assumption for the overall project design was that NGOs and the private sector are able to provide HIV prevention services to vulnerable populations more effectively and efficiently than the government. A distinguishing feature of the design of EHACP was the emphasis on a systematic approach to working with NGOs and the private sector rather than expansion of the piecemeal approaches of the past. The new approach included a focused and phased strategy towards implementation instead of trying to do everything at the same time. Most of the services provided to vulnerable populations and some other activities including BCC and advocacy were implemented directly by NGOs and private sector. The outcome based nature of the contracts with NGOs and private sector were designed to give more flexibility in implementation and room for innovative initiatives. Similarly, the small grants programme was incorporated to allow opportunity to smaller NGOs to come up with and implement new ideas for prevention and control of HIV and to build their own capacity for becoming competitors in the next rounds of biddings. The project design envisaged to bring about some key policy and institutional reforms. These included; (i) establishment of provincial blood transfusion authorities to implement existing legislation (ii) development and implementation of guidelines related to bio-medical waste management and (iii) development of technical guidelines on issues such as bio-technical reviews of proposed studies and contents of pre-and post-HIV test counseling. 2. Implementation, operational experiences and outcomes of the project There was no formal revision of the project components, nor were there any major changes made to the scope and design of the project during the implementation phase. In the 4th year of project, DFID allocated 15 million Euros for the provision of ARVs and strengthening the existing interventions. The 51 original date for project closing was June, 2008. A no cost extension was granted to the project from June 2008 to June 2009 and was later extended till December 2009. Subsequently a request was approved for further extension till March 2010 for the (i) ARVs procurement services, and (ii) Service delivery for IDUS in Punjab. 2.1. Component 1: Services for vulnerable population sub-groups (Total base cost: Pak Rs724.115 million) One of the major constraints during the design and implementation phase of this component was the lack of authentic data on sub-group populations and their behavioural patterns. A thorough analysis of existing mapping studies of most-at-risk populations was undertaken and it was agreed that the existing studies were sketchy and presented either underestimates or overestimates about the numbers. Consequently, comprehensive mapping of vulnerable sub-groups was undertaken in 12 major cities of Pakistan. Targets for service delivery contracts with NGOs and private sector were set in accordance with findings of the mapping study. A management consultancy firm was hired for building the capacity of AIDS control programmes to contract out the services to NGOs and private sector firms which were selected through an open competitive process. The bid evaluation committees in each province consisted of members from a cross-section of their respective health departments and representatives of the NACP and the management consultancy firm. Since the outcome-based service delivery contracts was a new concept for almost all involved, a major problem faced during the first few months of implementation of these contracts was a lack of sufficient understanding of the nature of contracts, which lead to contracting parties assuming certain expectations which were not actually part of the contracts. For example, the quarterly monitoring and evaluation in some instances resembled financial auditing of the contracting NGOs or firms, where the monitors were more interested in the inputs and channels of service provision rather than the quality and output of services. Moreover, the constitution of the monitoring committees was such that it sometimes took weeks to arrange monitoring visits. On top of this, the contracting NGO or firm had to be certified by the management consultancy firm before the release of quarterly payments. A further complication was that, apart from Punjab, the financial powers were not delegated to programme managers and the payments could not be made with a clearance by health department/ministry. This resulted in substantial delays in payments to the contracting NGOs. Though the NGOs and firms claimed that they did not stop service delivery and made for delays in payments from their own resources, it certainly left a bad feeling between NGOs and PACPs during implementation process in at least the initial phase of the project. Nevertheless, the problem of delayed payments was gradually overcome and after initial difficulties the implementation of this component in provinces went on smoothly. The service packages for Most at Risk Populations (MARPs) included information provision, skill development, condom distribution, syringe exchange, drug harm reduction inclusive of detoxification and preventive/curative care in a setting that could also cater to the VCT needs. According to HASP round III surveillance, the utilization of SDPs among MARPs, particularly the IDUs increased considerably over the duration of the project. Service delivery interventions with IDUs was the main success of the programmes as shown by flattening of epidemic trajectories in cities were SDPs for IDUs are in place--Lahore and Karachi. On the other hand, the trajectory continues to soar in other towns and cities where no service provision was contracted 52 under the project. As shown by behavioural surveys, the project has contributed significantly toward adoption of safer behaviors among IDUs where the service delivery packages were in place. Always used new syringe in Shared a needle during last IBBS rounds last month injection All SDP Non-SDP All SDP Non-SDP Round 1 22% 26% 8% 35% 15% 56% Round 2 41% 59% 27% 40% 28% 12% Round 3 46% 60% 23% 23% 14% 38% Note: IBBS round 1, 2, and 3 were undertaken in 2005, 2006 and 2008. Though the results of the bio-behavioural surveys were encouraging, the coverage of HIV prevention services remained low among venerable groups, both in terms of numbers and types of target groups reached. For instance; the coverage of harm reduction services for IDUs was only 15.5% compared with the planned coverage of 45%, the FSWs service delivery packages focused mainly on brothel-based sex workers and the MSM interventions were not effective as most of the projects working with MSM targeted transgender and transvestites, which are comparatively easier to reach. The quality of service provision in provinces was not very satisfactory in some instances due to capacity issues of some of the participating NGOs whereas others NGOs especially those working among IDUs in Punjab and Karachi are very well-experienced. However, the project did not clearly outline how the gaps in technical capacity would be filled if the programme were to expand to include newer cities, introduce newer interventions, and/or reach newer target groups. The NACP was responsible for implementation of a service delivery package for long distance truck drivers in 9 cities of Pakistan. This project was contracted out to FHI through competitive bidding. Around 50000 truckers were provided services over the project duration of 3 years. The project developed strategic behavioral communication strategies, provided condom education, distributed water-based lubricants and condoms, and monitored the organization and locations of sex work. It also provided primary healthcare services for truckers, including diagnoses and treatment of STIs and access to voluntary counseling and HIV testing. Under the small grants project, which was also the responsibility of the NACP, a number of NGOs and CBOs were given small grants over the course of the project. The NACP's contract with the management consultancy firm for capacity building was successfully concluded. Achievements of NACP under objective-1 of the enhanced programme Sub-Components Achievements Mapping of Vulnerable population groups in Mapping completed in 10 cities 10 cities (12 cities also covered through HASP) Delivery of Service Packages for Long 50,000 truckers provided services in 10 distance truckers major cities Pilot testing innovative approaches (10 small All small grants awarded and completed grants) 53 Provincial capacity building in procurement & The firm completed its contract and contract management by a firm evaluation under progress One of the significant outcomes of this component of the project was that the networks of PLHIV took root during this period and an endeavor on legislations dealing with their rights was embarked upon. 2.2. Component 2: Improved HIV prevention by the general population (Total cost: Pak Rs. 801.331 million) For BCC activities and advocacy for general population under this component, a media communication firm MIDAS was selected by NACP through open competition. MIDAS launched its BCC campaign through print, TV, Radio, CCTV and cable and managed to deliver more media insertions than committed at the time of the contract. In addition, MIDAS also implemented as part of the BCC and advocacy campaign, outreach prorgrammes for schools and colleges, sporting events, red ribbon campaigns and street theatres and helpline for questions and queries regarding HIV and AIDS. The contract between NACP and MIDAS was successfully completed. However, some issues regarding the contents of the BCC messages arouse during the implementation of MIDAS contracts with KPK and AJK PACPs, which affected the working relations between the media firm and the provincial programme/departments of health and resulted in problems with the timely release of payments to the media firm. As part of targeted interventions with youth, labour and uniformed persons NACP developed a National Strategy for youth and conducted various other activities in this regard. To improve and extend the management of STIs as one of the objective of component 2, a National STI Management Guideline was prepared and NACP conducted a large number of trainings on syndromic management of STIs. Despite significant efforts at advocacy, much remains to be done to provide an enabling environment in which implementing agencies can work effectively and in which target populations can be more readily accessed and encouraged to adopt safer behaviours. Work is underway to develop legislation on HIV/AIDS but a lot of groundwork remains to be done in terms of advocacy and lobbying to pave the way for a rights-based anti-discriminatory law. Achievements of NACP under objective-2 of the enhanced programme Sub-Components Achievements BCC & advocacy for general public through The firm completed its contract and its a firm evaluation completed Targeted interventions for Youth, Labour & National HIV strategy for youth developed Uniformed personnel and various activities conducted Improved and expanded management of · > 1600 health care workers trained in STIs through trainings and provision of STIs management medicines · STI medicines (Rs. 7 Million) distributed in public sector hospitals · National STIs management guidelines 54 developed A key lesson learnt during implementation of this component was to avoid singular focus on mass media and change the strategy from merely awareness raising to communication and advocacy. Nevertheless, The Enhanced Programme has succeeded in raising the awareness among decision makers in the country through its advocacy efforts. Government officials are increasingly favourable to HIV programming and HIV control work has even involved religious leaders in strategic locations. Component 3: Prevention of HIV/STI transmission through blood transfusion (Total base cost Pak Rs. 621.131 million) The objective of this component was to reduce the transmission of HIV and other STIs through transfusion of blood and blood products. The target was to ensure 100 percent screening of blood and blood products against HIV and Hepatitis B and C. Following measures were taken to meet this objective. The NACP developed Quality Assurance guidelines and Standard Operating Procedures (SOPs) for transfusion of blood and blood products; HIV and HBV screening kits were made available in all public sector blood banks; capacity building of blood bank staff was done, universal precautions for blood transfusion were promoted, blood transfusion authorities were established; and a national blood transfusion strategic framework was developed through a consultative process. A National Blood Transfusion Safety Ordinance was developed and promulgated to protect the general public from HIV transmission through blood and blood products. This was done through an extensive mass media BCC campaign for the general population, key policy/decision makers, political leaders, faith-based organizations, and the ministries of education, narcotics, and religious affairs. Achievements of NACP under objective-3 of the enhanced programme Sub-Components Achievements Development of QA guidelines QA guidelines and SOPs developed Provision of HIV & Hep. B screening kits HIV=280,000 & HBV=295,000 screening (390,000 each) & consumables in public kits and consumables supplied to BBs in sector BBs Federal Govt. hospitals Establishment of National and provincial · Blood transfusion authorities blood transfusion authorities established · National Blood transfusion strategic framework developed 2.3. Component 4: Capacity building and progamme management (Total cost: Pak Rs. 711.844 million) Deficiency of staff particularly for financial management and procurement had been a key challenge for NACP from 2003 to 2006. To meet the project objective under component 4, more than 60 percent of the professional programme staff was recruited during the course of the project, and capacity building of the staff was carried out at federal and provincial levels. 55 Monitoring of NGO interventions was steady feature during the implementation of project. It was done through the provincial AIDS Control Programmes, Ministry of Health and the consultant firm as well. For this purpose, monitoring manuals and toolkits were developed and quarterly monitoring reports of the SDPs were fed into a monitoring MIS. NGOs submitted their quarterly report at the provincial level. Towards the later stage of project a system of regular meetings with NGOs was evolved to share findings of monitoring reports. Three annual rounds of 3rd party evaluation were conducted to evaluate the performance of various projects. The HIV and AIDS Surveillance Project (HASP) was established and it conducted one pilot and three full annual rounds of surveillance in 12 major cities. There has been an increase in HIV-related research in the past three years. These include the mapping conducted by HASP as well as mapping studies undertaken by two HIV intervention projects for CSWs and MSWs, and IDUs; DFID supported STI/RTI studies; and various assessments carried out with migrant workers, youth and adolescents. Data from the surveillance and behavioural research have been made available to provide a much clearer picture than previously available of HIV prevalence and risk behaviors and practices among the key populations. Nevertheless, some gaps in monitoring and surveillance system still remain and to address these gaps a comprehensive M&E framework has been evolved but it is still not operational. Against a target of establishing 5 HIV treatment centers, 13 ART and 7 PPTCT centers were established which are providing free VCT, medical consultation, ART and diagnostics. The providers in these centres have been trained locally and internationally. The main limitation in provision of services was, however, the limited access to care for all HIV infected individuals. This limitation is being addressed by enhancing case detection mechanisms in the SDPs. Another limitation was sub-optimal quality of care provided at some centres leading to treatment failures, resistance to treatment and death. This weakness will be rectified by enhancing M&E of the programme and with improved training of providers. Achievements of NACP under objective-4 of the enhanced programme Sub-Components Achievements Recruitment and capacity building >75% of professional staff recruited and ongoing capacity building III party evaluation for various projects (3 All rounds of evaluation studies and annual rounds) operational research studies completed Establishment of HIV/ AIDS Second HASP established and four rounds of generation surveillance with CIDA surveillance conducted in 12 major cities Establishment of 5 HIV/AIDS treatment 12 centers established, providing free VCT, centers medical consultation, ART and diagnostics 3. Performance of the programme Before the start of Enhanced programme, the NACP was severely understaffed, had very limited financial resources, a sketchy database on HIV prevalence, its transmission patterns and only rough estimates of vulnerable populations. Likewise, the provincial implementation units were severely constrained in human resource and technical capacity. There was very limited service delivery to most-at-risk 56 populations and the BCC campaign was also under resourced. Many of these weaknesses of the programme were adequately addressed under the Enhanced Programme. To address the gaps in national response identified in a mid-term review in 2007, a multi-sectoral approach and a reorganization of the national and provincial programmes were recommended. This required creating an enabling environment, strengthening the institutional framework, building the right capacity, and scaling up programme delivery. The MTR was, therefore, used as an opportunity for multi sector collaboration and therefore some progress has been made towards achieving a multi-sector response with projects conducted in partnership with the Ministry of Education (MOE), the Ministry of Labour, Manpower and Overseas Pakistanis (MOL), and the Ministry of Religious Affairs, Zakat and Ushr. The capacity of provincial programmes and participating NGOs has considerably improved. More NGOs/private firms are eager to participate in HIV prevention and control initiatives and there is a greater understanding of the modalities and working of the outcome based contracts introduced under the enhanced programme. There is now more acceptance of HIV prevention efforts among general population and opinion leaders and a willingness among the policy makers in government sector to take initiatives and own HIV prevention efforts. The salient achievements of the project are summarized as under: Substantial expansion in the number and scope of HIV prevention interventions for high-risk groups and vulnerable populations through public sector financing. Strengthened role of public-private partnerships in service delivery i.e. partnership with over 350 NGOs under the umbrella of national and provincial AIDS consortia. Inclusion of condom promotion as an integral component of service delivery packages for high-risk and vulnerable populations. Introduction of an extensive mass media campaign to raise awareness among general adult population about the methods of HIV transmission and its prevention. Promoting active involvement of other ministries and departments like Ministry of Education, Ministry of Narcotics, Ministry of Religious Affairs and others. Development and implementation of protocols and operational guidelines in a number of areas like VCCT guidelines, guidelines for laboratory diagnosis of HIV and AIDS, infection control guidelines, and guidelines for syndromic management of STIs and others. Development and promulgation of National Blood Transfusion Safety Ordinance to prevent HIV spread through transfusion of blood and blood products. A number of research studies to understand biological and behavioural trends of STIs and HIV among high-risk and bridge populations. 57 Development of National Monitoring and Evaluation (M&E) Framework to feed into policy and programme planning. Establishment of National Network of PLHIV and preparation of legislation dealing with human rights. Formulation of National HIV and AIDS Policy and Legislative Framework. A major achievement during this period was the development of the National HIV/AIDS Policy Document' 2005 with an aim to provide and maintain an enabling environment for HIV/AIDS prevention and care programs and services through a consistent multi-sectoral process at all levels of government and community. Likewise, the `HIV/AIDS Prevention and Treatment Act, 2007' was designed to establish National and Provincial AIDS Coordination Committees to provide a multisectoral forum for coordination of the national and provincial response. This act also supports the government in providing HIV-specific information, care, support, equitable access to treatment and stigma/ discrimination reduction for PLHIV, vulnerable populations that have quasi-legal status and the bridging populations of families and clients. Both the documents await approval from the government. 4. Lessons learnt and changes proposed for future The EHACP completed its five years of implementation in June 2008; however, it was extended till December 2009. Although the HIV response in Pakistan has evolved to a coordinated and multisectoral level, in order to meet the MDG Goal 6 of "Halting and begin to reverse the spread of HIV/AIDS" by the year 2015, urgent scaling-up the program interventions is needed. Therefore, the PC-1 of the EHACP was revised in 2008. It has a total budget of US$ 99.4 million and was approved in 2008 by Provincial Development Working Parties (PDWP), Central Development Working Party (CDWP) and Executive Committee of the National Economic Council (ECNEC). Rapid geographical expansion of the HIV epidemic from large cities to smaller towns/cities demanded immediate scaling-up of prevention and control services to various new locations. Although the overall total of MARP groups showed a nearly 2-3 fold increase over the original estimates, only about 40% of the IDUs and less than 15% of the M/F/HSWs are being currently accessed. The remaining would have to be targeted to achieve the National Universal Access target of 60% coverage for F/MSWs and 80% for IDUs. ARV treatment till 2008 had been provided by the GFATM grant R-2. Currently, the provision is being facilitated through Continuation of Services (CoS) proposal of Global Fund R-2 grant. In order to ensure uninterrupted supply and strengthening of this vital service, a complete HIV and AIDS care component was needed to be incorporated in the revised PC-I. Increasing prevalence of HIV among spouse/female partners and families of IDUs prompted the need for comprehensive services provision to all MARPs including provision of primary health care, STI medicines, behavior change communication services as well as Voluntary Counseling and Confidential Testing (VCCT) and Prevention of Parent to Child Transmission (PPTCT). Safe blood transfusion needs an enhanced scope to cover Hepatitis B and C screening in all public sector blood banks. Programme 58 Management of the all AIDS control programmes needs further capacity building, especially in the fields of research, surveillance, monitoring and evaluation and procurements. The main challenges faced during the Enhanced AIDS Control Programme include: 1) limited coverage of the most at risk groups in most locations across the country, 2) limited quality of many of these interventions, 3) limited mechanisms to understand the contextual nuances, that impact the interventions and limited flexibility in the plans to incorporate lessons from ongoing experience; 4) limited mechanisms for appropriately addressing the interventions for different targets including the general public; 5) the focus of the communication campaign on mass media which did not allow exploration of the best modalities to deliver optimal messages; and 6) monitoring and evaluation based on inputs and channels rather than impact or outcomes; thus leading to insufficient understanding of the lessons from experience to be assimilated into day to day operations. Going forward, the NACP seeks to rectify many of these limitations. The main thrust is to expand coverage of targets that are considered important based on an understanding of the trajectory of the epidemic. The project design of the revised PC1 has the ability to add additional cities or alter the content of the interventions based on real time experiences. The contracts for SDPs will now be given for a whole province. This will allow flexibility to the SDPs to go where the programme is needed (based on mapping and surveillance). Some weaknesses were identified regarding procurement and financial management during the project implementation. NACP and PACPs did not have appropriate procurement systems in place. Computerised management information systems (MIS) to track commodities or inventories were also lacking. Moreover the procurement procedures of the programmes were lengthy and contorted. This resulted in scanty or inconsistent data. The procurement was reactive rather than active. In contrast, the procurement from donors such as condoms from DFID and ARVs from Global Fund was less complicated. A major complicating factor for financial management of the programme was that the national and provincial programme managers could not release contract payments beyond a certain limit as they were designated as Category II officers under the Delegation of Financial Powers Rules 2001. The relatively slow utilization of committed expenditures was due to: (i) substantial delays occurring in procurement of services due to reasons like long time taken to evaluate proposals, (ii) delays in payments to the contracted firms and NGO's (iii) capacity limitations of the programmes and NGOs that limited the pace of implementation, and (iv) frequent changes in programme management. At the time of annual budget preparation, PACP and NACP had the flexibility to raise a budget demand, keeping in view their absorptive capacity. However, NACP exercised limited authority to introduce the implementation of an intervention that was not included in the original approved programme (PC-1). Also, there was no flexibility to internally re-adjust the costs of various budget categories without the approval of the finance department. In a situation where the epidemic may change direction or new issues arise within the five-year period, the perceived inflexibility of the PC-1 for responding to such changes has proved to be a real hindrance in executing work plans. The Government of Pakistan is committed to controlling HIV and is signatory to a number of international conventions including the International Conference on Population Development (ICPD), the Millennium Development Goals (MDGs), United Nations Special Session on HIV/AIDS (UNGASS). These commitments are manifest by the depiction of HIV/RTIs/STIs as a priority area in policy 59 documents including: the National Policy for Development and Empowerment of Women, the Population Policy of Pakistan, the Reproductive Health Services Package 1999, the National Health Policy 2001 and the Poverty Reduction Strategy Paper, Medium Term Development Framework 2005-10 and Vision 2030. WB-ICR Comments by NACP 1. Report states that BCC for general populations and blood transfusion as design components were not appropriate for the concentrated phase of epidemic and were perhaps included due to political influence. This inference is partially correct as it was not fully clear at the stage of EHACP inception that Pakistan is in concentrated phase of HIV epidemic and also inclusion of BCC for general population and blood transfusion were well identified priority areas in the NSF-I. Also, the program design was finalized in consultation with WB and other stakeholders in the country. 2. Involvement of too many implementing agencies was again due to program design and was intended to build the capacity of civil society in Pakistan and promote Public-Private Partnership concept. This is very well acknowledges achievement of the program. 3. The report recommends that the focus in the design should have been on Sindh and Punjab only; the provinces with highest concentration of risk populations, but again this was not well known at the time of the project development. However, during the course of implementation, maximum utilization of funds was observed in Punjab and Sindh. 4. The report very correctly identifies that slow procurement of services, recruitment problems in contract management, capacity issues of APCs and NGOs, limitation of coverage of targeted populations were the main challenges in implementation. 5. It is correctly inferred that delayed start of the project was due to lengthy government procedures, but for many of the other instances especially in relevance to procurement of services, delays were also observed due to lengthy procedures at World Bank. 6. The comment on payment delays to NGOs in Sindh, NWFP and at federal level owing to clearance required from health ministry/department is partially correct. Such delays were only observed towards the end of project especially at federal level and there were multiple reasons for these delays. 7. The comment that additional financing was not processed by the Bank because of delays in approval of the PC1s is not correct. All PC-1s were approved well within time by the PDWPs, CDWP and ECNEC, but delay was observed at World Bank level due to 60 multiple reasons. PC-1s were approved from ECNEC in November, 2009, while the agreement for this project ended on 31st December, 2009. 8. All PDO indicators were selected in consultation with World Bank and NACP or PACPs. The report gives data on 5 PDOs rather 4 and data on PDO 7 can be obtained from End Project Evaluation report of BCC component. 9. The comment that the target of below 5% prevalence among vulnerable populations became inappropriate after 2004 when higher prevalence rate was discovered among IDU in Karachi validates the fact that project design was not very appropriate for concentrated epidemic. 10. Not clear why report states that inclusion of BCC component was a priority of some government officials outside AIDS control programmes and what evidence was shared with ICR team to draw this conclusion. 11. The comment that the need for additional financing that was raised during the MTR never materialized because of the inflexibility of GOP regulations is also not fully correct. 12. The lesson learnt that larger NGO contracts have advantages in terms of amount of time needed for recruitment is partially correct because such contracts also create monopolies with a number of disadvantages, as they were observed in Punjab. 13. The inference that the project suffered from the lack of proper understanding of GoP officials regarding the importance of confidentiality of the risk population groups and their HIV status is not fully correct. This issue was observed only in one province towards the end of project i.e. in 2010 and was also linked to a number of other issues as well. 61 Annex 8. Comments of Co financiers and Other Partners/Stakeholders N/A 62 Annex 9. List of Supporting Documents 1. Review of Component 2: "Improved HIV Prevention by the General Population" of the HIV/AIDS Prevention Project for the World Bank, Pakistan, Barbara A.K. Franklin, PhD 2. AIDS in South Asia (understanding and Responding to a Heterogeneous Epidemic), HNP Series, Stephen Moses et al, 3. Effectiveness and Coverage of HIV Interventions in Pakistan: Insights from Triangulation of Program, Field and Surveillance Data, 2008, Adnan A. Khan, 4. World Bank (2003 ­ 2009), Pakistan HIV/AIDS Prevention Project: Aide-memoires 5. World Bank (2003 ­ 2009), Pakistan HIV/AIDS Prevention Project: Implementation Status and Results Reports (ISR) 6. HIV Second Generation Surveillance in Pakistan National Reports Round I-III (2005, 2006-7 and 2008), Canada-Pakistan HIV/AIDS Surveillance 7. The end of project evaluation of the National AIDS Control Program's BCC component - 2008, Kamran Sadiq 8. Third Party Evaluation Reports for National AIDS Control Program, 3 rounds, SoSec, Consulting Services, Pakistan UK 9. National Study of Reproductive Tract and Sexually Transmitted Infections, Survey of High Risk Groups in Lahore and Karachi, 2005, FHI 63 MAP 64