Document of The World Bank Report No: ICR2094 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H1400) ON A LOAN IN THE AMOUNT OFUS$21.0 MILLION TO THE GOVERNMENT OF ANGOLA FOR A HIV/AIDS, MALARIA AND TUBERCULOSIS CONTROL PROJECT (HAMSET) December 29, 2011 Human Development Sector Health, Nutrition and Population (AFTHE) Country Department 1 AFCS2 Africa Region CURRENCY EQUIVALENTS Exchange Rate Effective August 12, 2004 Currency Unit = Kwanza Kwanza 82.7 = US$ 1 Kwanza 1 = US$ 0.0121 FISCAL YEAR ABBREVIATIONS AND ACRONYMS ACT Artemisinin-based Combination Therapy AIDS Acquired Immune Deficiency Syndrome NCB National Competitive Bidding AM Aide Memoire KAP Knowledge Attitude and Practice ARV Anti-Retroviral Therapy LQAS Lot Quality Assurance Sampling BCC Behavior Change Communication TF Trust Fund CDC Centre for Disease control MAP Multi –Sectoral AIDS project GDP Gross Domestic Product M&E Monitoring and Evaluation DGA Development Grant Agreement MCP Multiple concurrent Partnership DO Development Objective MDG Millennium Development Goals DOTS Directly Observed Treatment Strategy MHSS Municipal Health Service Services Project GEPE Gabinet de Estudos, Planeamento e Estatstica. MICS Multiple Indicator Cluster Survey (Office of Studies, Planning and Statistics) MIS Malaria Indicator Survey GFATM Global Fund to Fight HIV and AIDS, TB and MPLA Movimento Popular de Libertação de Angola Malaria MTR Medium Term Review GOA Government of Angola NGO Non Governmental Organizations GHAP Global HIV and AIDS Programme NMCP National Malaria Control Program EC European Commission PAD Project Appraisal Document EMRP Emergency Multi-Sector Rehabilitation Project PCU Project Coordinating Unit EU European Union PDO Project Development Objective FAA Forcas Armadas Angolanas.(Angolan Armed PPF Project Preparation Facility Forces) PMI Presidential Malaria Initiative (USG) FBOS Faith Based Organizations PMTCT Protection of Mother-to-Child Transmission HAMSET HIV/AIDS, Malaria and Tuberculosis control PNCM Plano Nacional de Controlo da Malaria. (National Project Malaria Plan). HIV Human Immuno-Deficiency Virus SDR Special Drawing Rights HR Human Resources SSA Sub-Saharan Africa IBEP Inquerito de Bem Estar (Household Expenditure STI Sexually Transmitted Infection Survey) TB Tuberculosis IBRD International Bank for Reconstruction and TSS Transitional Support Strategy Development UN United Nations ICB International Competitive Bidding UNDP United Nations Development Program ICR Implementation Completion Report UNICEF United Nations Children‘s Emergency Fund IDA International Development Association UNITA União Nacional para a Independência Total de IEC Information, Education, and Communication Angola INE Instituto Nacional de Estatistica (National USAID United States Agency for International Institute of Statistics) Development INLS Instituto Nacional da Luta Contra o SIDA (NAC) USD United States Dollars IP Indigenous People USG United States Governmnet IPT Intermitant Preventative treatment QCBS Quality and Cost Based Selection ISN Interim Strategy Note QEA Quality at Entry ISR Implementation Supervision Report QSA Quality of Supervision ITN Insecticide Treated Net VCT Voluntary Counselling and Testing JUCA Juntos Unidos Contra Anopheles. (United WB World Bank together against Anopheles) WHO World Health Organization MOH Ministry of Health Vice President: Obiageli Ezekwezili Country Director: Mr. Laurence Clarke Sector Manager: Jean J. De St Antoine Project Team Leader: Humberto Albino Cossa ICR Team Leader: Katie Bigmore ANGOLA HIV/AIDS, MALARIA AND TUBERCULOSIS CONTROL PROJECT (HAMSET) TABLE OF CONTENTSA. Basic Information ................................................................ i B. Key Dates .................................................................................................................... i C. Ratings Summary ........................................................................................................ i D. Sector and Theme Codes ........................................................................................... ii E. Bank Staff ................................................................................................................... ii F. Results Framework Analysis ...................................................................................... ii G. Ratings of Project Performance in ISRs ................................................................... vi H. Restructuring (if any) ............................................................................................... vii I. Disbursement Profile ................................................................................................ vii 1. Project Context, Development Objectives and Design ............................................ 1 1.1. Context at Appraisal ......................................................................................... 1 1.2. Original Project Development Objectives (PDO) and Key Indicators ............. 2 1.3. Revised PDO .................................................................................................... 2 1.4. Main Beneficiaries ............................................................................................ 3 1.5. Original Components ........................................................................................ 3 1.6. Revised Components ........................................................................................ 3 1.7. Other significant changes ................................................................................. 3 2. Key Factors Affecting Implementation and Outcomes ........................................... 4 2.1. Project Preparation, Design and Quality at Entry............................................. 4 2.2. Implementation ................................................................................................. 5 2.3. Monitoring and Evaluation (M&E) Design, Implementation, and Utilization. 8 2.4. Safeguard and Fiduciary Compliance............................................................... 9 2.5. Post-completion Operation/Next Phase .......................................................... 10 3. Assessment of Outcomes ....................................................................................... 10 3.1. Relevance of Objectives, Design and Implementation ................................... 10 3.2. Achievement of Project Development Objectives (Efficacy) ........................ 10 3.3. Efficiency........................................................................................................ 13 3.4. Justification of Overall Outcome Rating ........................................................ 14 3.5. Overarching Themes, Other Outcomes and Impacts ...................................... 15 4. Assessment of Risk to Development Outcome ...................................................... 15 5. Assessment of Bank and Borrower Performance .................................................. 15 5.1. Bank Performance .......................................................................................... 15 5.2. Borrower Performance ................................................................................... 16 6. Lessons Learned..................................................................................................... 17 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners........ 18 Annex 1. Project Costs and Financing .......................................................................... 19 Annex 2. Outputs by Component ................................................................................. 20 Annex 3. Economic and Financial Analysis ................................................................. 32 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 36 Annex 6. Stakeholder Workshop Report and Results................................................... 37 Annex 7a. Summary of Borrower's ICR (Translated from Portuguese to English) ..... 38 Annex 7b. Summary of Borrower's Comments on ICR ............................................... 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 49 Annex 9. List of Supporting Documents ...................................................................... 50 A. Basic Information HIV/AIDS, Malaria and Country: Angola Project Name: TB Control Project (HAMSET) Project ID: P083180 L/C/TF Number(s): IDA-H1400 ICR Date: 12/28/2011 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: ANGOLA Original Total XDR 14.10M Disbursed Amount: XDR 13.26M Commitment: Revised Amount: XDR 14.10M Environmental Category: B Implementing Agencies: Ministry of Health Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 10/09/2003 Effectiveness: 10/20/2005 10/20/2005 Appraisal: 09/20/2004 Restructuring(s): 06/22/2010 Approval: 12/21/2004 Mid-term Review: 06/30/2007 04/18/2008 Closing: 06/30/2010 06/30/2011 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 50 Health 9 Other social services 34 Sub-national government administration 7 100 Theme Code (as % of total Bank financing) HIV/AIDS 40 Malaria 20 Participation and civic engagement 20 Tuberculosis 20 100 E. Bank Staff Positions At ICR At Approval Vice President: Obiageli Katryn Ezekwesili Gobind T. Nankani Country Director: Laurence C. Clarke Michael Baxter Sector Manager: Jean J. De St Antoine Dzingai B. Mutumbuka Project Team Leader: Humberto Albino Cossa Jean J. De St Antoine ICR Team Leader: Katie Bigmore ICR Primary Author: Katie Bigmore F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project‘s development objectives are to: (i) reduce the spread of HIV/AIDS in the Angolan population through a multi-sector approach that strengthens institutional capacity and increases access and utilization of health services for prevention, diagnosis, care, and support; (ii) strengthen the capacity of the health sector to detect new cases of TB, improve treatment compliance, and increase the completion rate; and (iii) strengthen the capacity of the MOH for effective case management of malaria. ii Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : HIV/AIDS prevalence in pregnant women aged 15-49 Value 2.8% Dec 04 quantitative or 19% Less than 5% 2% 11% (PAD/DGA) Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Achieved. Current HIV prevalence for 15-49 year olds is 2% (INLS 2010) achievement) Percentage of young people aged 15-24 who can both correctly identify ways of Indicator 2 : preventing the sexual transmission of HIV and reject major misconceptions about HIV transmission and prevention Value 39.9% (KAP 09) 60% (PAD) quantitative or 85% 28.5% (IBEP 22.7% (KAP 06) Qualitative) 08/09) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % Target not met. (40% achieved) achievement) Percentage of men and women aged 15-49 reporting the use of a condom in their Indicator 3 : last act of sexual intercourse with a non-regular sexual partner during the previous 12 months Value 10% (PAD) quantitative or 80% 60% 50.5% (KAP 09) 31.9% (KAP 07) Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Target not met. (84% achieved) achievement) Proportion of the 59 priority municipalities that have implemented TB control Indicator 4 : activities Value quantitative or 4% (PAD) 100% 100% Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % Achieved achievement) Indicator 5 : TB Case Detection Value NA NA 70% (WHO) 77% iii quantitative or Qualitative) Date achieved 06/30/2011 06/30/2011 04/18/2008 06/30/2011 Comments Achieved. 77% case detection rate against the WHO target of 70%. (TB program (incl. % 2010) achievement) Indicator 6 : TB Success Rate Value quantitative or NA 85% (WHO) 72% Qualitative) Date achieved 10/31/2005 04/18/2008 06/30/2011 Comments Target not met. (85% achieved). (incl. % 72% (TB program 2010) achievement) Indicator 7 : Institutional mortality due to malaria among children under five years Value quantitative or 53% (PAD) 15% 9.9% Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments Achieved. (incl. % 9.9% (Malaria Program 2009) achievement) Indicator 8 : Institutional maternal mortality due to malaria Value quantitative or 57% 10% less than 5% 3.2% Qualitative) Date achieved 06/01/2005 12/22/2004 04/18/2008 06/30/2011 Comments Achieved. (incl. % 3.2% (Malaria Program 2009) achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Number of VCT centers functioning per year Value (quantitative 35 200 547 or Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % Achieved achievement) Indicator 2 : Number of counselors trained per year Value (quantitative NA 400 249 or Qualitative) Date achieved 10/31/2005 12/22/2004 12/31/2009 iv Comments (incl. % Target not met (62% achieved). achievement) Indicator 3 : Number of people being tested for HIV Value (quantitative NA 300,000 462,680 or Qualitative) Date achieved 10/31/2005 12/22/2004 12/31/2010 Comments (incl. % Achieved (INLS 2010) achievement) Indicator 4 : Number of sentinel surveillance sites in antenatal clinics functioning per year Value (quantitative 26 180 44 36 or Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Target not met (82% acheived) achievement) Number of health facilities providing HV infected pregnant women and their Indicator 5 : new born with Mother to child transmission intervention services, per province Value (quantitative 8 100 150 200 or Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Achieved achievement) The number of condoms distributed through the public sector and NGO Indicator 6 : programs (million) Value (quantitative 2.6m 200m NA or Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % No data available achievement) Number of participating line ministries which have and are implementing their Indicator 7 : respective HIV/AIDS workplans Value (quantitative 0 All 8 7 or Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Target not met (87.5% achieved) achievement) Number of decentralized government entities that are implementing integrated Indicator 8 : HIV/AIDS workplans Value 0 18 6 8 (quantitative v or Qualitative) Date achieved 12/22/2004 12/22/2004 04/18/2008 06/30/2011 Comments (incl. % Achieved achievement) Indicator 9 : Amount of funds channeled to civil society initiatives per year (US$million). Value (quantitative NA 3m NA or Qualitative) Date achieved 10/31/2005 12/22/2004 06/30/2011 Comments (incl. % No data available achievement) Indicator 10 : Number of staff trained in DOTS in each priority municipality Value (quantitative 105 NA 700 or Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % 700 trained in DOTS in 2010 (TB program) achievement) Number of priority municipalities that have laboratories with adequate Indicator 11 : equipment and trained technicians to do sputum smear microscopy Value (quantitative NA NA or Qualitative) Date achieved 12/22/2004 06/30/2011 Comments (incl. % No data available achievement) Number of Malaria Control Program coordinators trained in operational Indicator 12 : management, budgeting and supervision. Value (quantitative NA 55 450 or Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % Achieved (Malaria program 2010) achievement) Indicator 13 : Number of trainers trained in malaria case management per province. Value (quantitative NA 50 1007 or Qualitative) Date achieved 12/22/2004 12/22/2004 06/30/2011 Comments (incl. % Achieved (Malaria program 2010) achievement) G. Ratings of Project Performance in ISRs vi Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 04/07/2005 Satisfactory Satisfactory 0.00 2 06/24/2005 Satisfactory Satisfactory 0.00 3 02/28/2006 Satisfactory Satisfactory 1.44 4 06/15/2006 Satisfactory Satisfactory 1.65 5 12/22/2006 Satisfactory Satisfactory 2.64 6 06/25/2007 Satisfactory Satisfactory 3.78 7 10/17/2007 Satisfactory Satisfactory 4.83 8 05/01/2008 Satisfactory Satisfactory 6.63 9 11/10/2008 Satisfactory Moderately Satisfactory 8.77 10 06/30/2009 Satisfactory Moderately Satisfactory 11.47 11 12/30/2009 Satisfactory Moderately Satisfactory 14.40 12 06/30/2010 Satisfactory Moderately Satisfactory 17.82 13 03/26/2011 Satisfactory Moderately Satisfactory 19.00 14 06/29/2011 Satisfactory Moderately Satisfactory 19.80 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 06/22/2010 S MS 17.82 I. Disbursement Profile vii 1. Project Context, Development Objectives and Design 1.1. Context at Appraisal 1. Country Context1. In April 2002 Angola emerged from 30 years of war with 65 percent of the health units destroyed, with limited health equipment and significant human resource capacity constraints. Gross Domestic Product (GDP) per capita in 2005 was US$ 1,857 and subsequently increased to US$ 4,420 in 2010. At the time of appraisal it was estimated that HIV could reduce GDP by up to 1.0 percent a year and malaria by a further 1.3 percent per year. Life expectancy at birth in 2005 was 37 years for men and 40 years for women (increasing to 48 years in 2009). In 2002 the country faced massive developmental challenges including rebuilding its infrastructure, improving institutional capacity, governance, public financial management systems, as well as improving the human development indicators and the living conditions of the population. 2. Health Sector Context. At appraisal stage of the project the health status of the Angolan population was amongst the worst in the world. The maternal mortality ratio in 2005 was estimated at 1,400 per 100,000 live births 2 however higher estimates had also been reported earlier 3 ). In 2001, infant mortality was 150 for every 1,000 child under one year while the mortality of under 5 year olds was estimated at 250 per thousand4. HIV/AIDS, Tuberculosis (TB) and malaria accounted for 75 percent of all infectious diseases.  In 2005 there were an estimated 2.5 million cases of Malaria and 23,000 deaths a year. Malaria was the main cause of morbidity, with 40 percent of children dying in the first five year of life.  In 2003 21,000 TB cases were registered and around 7,000 new cases were diagnosed each year. Out of the 1,463 health facilities, only 4 percent were implementing Directly Observed Treatment Strategy (DOTS).  The HIV prevalence in 2004 was estimated at 5.5 percent, with higher levels reported amongst sex workers reaching 23 percent (2006). HIV service availability, including VCT, PMTCT and ARV was extremely limited. 3. Health Structures and Strategy. At appraisal the government did not have a well defined health strategy5. At the time, the Ministry of Defense staffing, hospitals and lower level health posts were more developed than the Ministry of Health (MoH). The network of national NGOs with capacity to provide health care was limited and focused more on HIV awareness raising. Faith based organizations were beginning to rehabilitate their network of rural hospitals, and some support was being provided by international Non Governmental Organizations (NGOs). 4. Health Financing. In 2004 government expenditure on health was 2 percent of GDP, and represented 4 percent of government expenditures. Total health spend was US$ 26 per capita, with 76 percent met by government and 24 percent met out of pocket. The budget for the health sector significantly increased over five years of the project, and even doubled between 2005 and 2006. At appraisal, there was limited experience in the Ministry of managing external funding, 1 In 2005 Angola‘s population was estimated at 13.9m, today it is estimated at 18m. The first population census is due to take place in 2012. 2 World Health statistics, 2008 3 1,850 per 100,000 live births in 2002.Ministry of Health, Plano Estrategico Nacional de Saude Reprodutiva, Luanda, 2002 4 INE, Resultados dos indicadores Multiplo, MICS2,2001, Folha de InformapLo rapida, April 2002 5 A comprehensive health sector strategy – Plano de Desenvolvimento Económico e Social de Médio Prazo, 2009-2013 was developed in 2007/2008. 1 but this changed during project implementation. For example, by 2004 government had secured $US 35m from Round 3 (Rd3) of the Global Fund For AIDS TB and Malaria (GFATM) for Malaria and the activities selected by HAMSET were those not financed by GFATM. 5. Rationale for Bank involvement. At the time of appraisal there were few development agencies supporting health in Angola, and having emerged from 30 years war the health indicators were amongst the worst in the world, demanding a huge boost in funding and technical assistance to support the Government to respond to these challenges. The absence of a comprehensive health strategy also meant there was an opportunity to engage and support strategic planning and policy setting within the sector. 6. Links with Country Assistance Strategy. At the time of the design the Bank had a Transitional Support Strategy (TSS) for Angola6 which included expanding service delivery to war-affected and other vulnerable groups. Within this context HAMSET support was designed to help the government control the HIV/AIDS, malaria and TB epidemics. The project remained in support of the Bank‗s Interim Strategy Note (ISN) for 2005-06 and 2007-2009, as well as the key areas of the World Bank‗s Africa Action Plan (2006-08) which focused, amongst other things on, Building the Capacity of Post-Conflict States to Deliver Essential Services. The Project was aligned to contribute to the Millennium Development Goals (MDGs) 4, 5 and 67. Complementary support to the health sector was provided through the Emergency Multi-Sector Rehabilitation Project (EMRP, US$ 50.7 million 2005-11)8. 1.2. Original Project Development Objectives (PDO) and Key Indicators 7. The projects development objectives as defined in the Project Appraisal Document (PAD) are to: (i) reduce the spread of HIV/AIDS in the Angolan population through a multi- sector approach that strengthens institutional capacity and increases access and utilization of health services for prevention, diagnosis, care, and support; (ii) strengthen the capacity of the health sector to detect new cases of TB, improve treatment compliance, and increase the completion rate; and (iii) strengthen the capacity of the MOH for effective case management of malaria. It should be noted that the PAD and the Grant Agreement articulate the PDO slightly differently, although the substance is essentially the same. In the grant agreement the objective of the Project is to assist the Recipient to organize a response to the HIV/AIDS, Malaria, and Tuberculosis epidemics through a multi-sectoral approach by: (i) reducing the spread and mitigating the effects of HIV/AIDS through the increasing access to prevention services diagnosis, care and support for PLWHA; (ii) strengthening the capacity of the health sector to detect new cases of Tuberculosis, improve treatment compliance, and increase the completion rate; (iii) strengthening the capacity of the MOH to effectively manage cases of malaria. 1.3. Revised PDO 8. The project objective was not revised. 6 The TSS included three pillars (i) enhancing transparency, efficiency and credibility of public resource management; (ii) expanding service delivery to war-affected and other vulnerable groups; and (iii) preparing the ground for broad based pro-poor economic growth. 7 MDG4: Reduce child mortality; and MDG 5: Reduce maternal mortality MDG6: Combat HIV, malaria, and other diseases; It was recognized that only those aspects of maternal and child mortality that relate directly to combating of HIV/AIDS, malaria and TB would be addressed under the project. 8 The EMRP had a Health component of US$ 8 million targeted to 4 provinces (Bié, Kwanza Norte, Malange, and Moxico). 2 1.4. Main Beneficiaries 9. The project original design was to support 59 priority municipalities, which span all 18 provinces and cover 70 percent of the population9. The primary target group identified in the Project Appraisal Document (PAD) is the Angolan population, particularly people with or at risk of HIV, TB or Malaria. While the project supported direct service delivery it also financed some health systems improvements, particularly training of health professionals, including staff of the HIV, TB and Malaria programs. 1.5. Original Components Component 1. Public Sector Response Component PAD: US$ 2m (9.5%), Revised budget10: US$ 2.9m (14%) 10. This component aimed to support a multi-sector response from government ministries and local government agencies. The project would help: (i) build capacity of the line ministries to reduce the impact of HIV/AIDS, malaria and TB on ministry staff and their dependents; and external clients. Priority activities within this component would be information, education, and communication (IEC), behavior change communication (BCC), social marketing of condoms, and access to Voluntary Counseling and Testing (VCT) and Protection of Mother-to-Child Transmission (PMTCT). Component 2. Health Sector Response Component PAD: US$ 7.2m (34.3%), Revised budget: US$ 8.8m (41%) 11. This component aimed to provide funding to the MOH for the prevention and treatment of TB and malaria, as well as for HIV/AIDS activities for which the MOH is directly responsible such as training of MOH staff, VCT and Sexually Transmitted Infections (STIs) management, PMTCT, and treatment of opportunistic infections. Activities included in HAMSET were designed as a complement to those supported by the GFATM. Component 3. Community Response Component PAD: US$ 7m (33.3%), Revised budget: US$ 4.3m (20%) 12. This component aimed to provide funding and technical assistance to civil society organizations, communities, NGOs, and faith-based organizations (FBOs) that would present subprojects to prevent and mitigate the effects of the three diseases. They would cover preventive activities for HIVIAIDS focused on behavior change, particularly among high-risk groups, social marketing of bed nets, and community treatment of malaria, and support to the implementation of DOTS for TB. Component 4. Project Coordination PAD: US$ 4.8m (22.9%), Revised budget: US$ 4.9m (23%) 13. This component would help: (i) support the establishment of Provincial AIDS and Endemic Diseases Commissions; (ii) contract professionals for the Project Coordinating Unit (PCU); (ii)contract professional services including financial and procurement management; and (iii) develop information systems to monitor and evaluate progress in controlling the epidemics. 1.6. Revised Components 14. The components were not revised. 1.7. Other significant changes 9 PAD page 11 10 The revised budget was agreed during the project restructuring and extension in June 2010 3 15. Cost and Financing. The cost of the overall original grant was US$ 21 million. The total appreciated to US$ 21.9 million as a result of the appreciation of the SRD against the US Dollar. The project disbursed US$ 19 million or 90.5 percent of the total grant amount11. 16. Dates and Amendment. The project became effective in October 2005, without experiencing delays. The MTR was held in April 2008, 10 months later than planned. A minor amendment to the Financing Agreement was done in June 2010 to extend the project closing by one year to end June 2011 and reallocate funding across disbursement categories. By June 2010, a total of US$ 17.8 US$ (82%) of the funds had been disbursed and US$ 3.5m remained. In effect, the one year extension allowed the government to fully utilize the grant and to complete procurement processes. Neither the Project Development Objectives (PDOs), nor the indicators or the targets were changed with the restructuring and one year extension. 2. Key Factors Affecting Implementation and Outcomes 2.1. Project Preparation, Design and Quality at Entry Preparation 17. Project preparation began in 2004, following a request from Government in 2003, at which time Angola had only recently emerged from 3 decades of war. The project preparation team recognized the key challenges in the sector at the time, including (i) weak capacity of government and civil society partners, (ii) importance of focusing on the provinces most in need (iii) need to work with multiple stakeholders including government, civil society and the private sector to promote a multi-sectoral response as well as the (iv) limited track record and experience of the public sector line ministries in working on HIV. A number of NGOs were working in Angola in 2005, but most had limited experience of working on communicable diseases. National NGOs were rightly identified as in need of capacity building. Design 18. The project design was quite ambitious given the post conflict situation, the general challenging environment and capacity constraints of implementing partners. It followed the first generation MAP design, but was innovative to the extent that it also aimed to support Malaria and TB more broadly12. The Project Coordination Unit (PCU) was set up in the Ministry of Health, not the National AIDS Control Council, in order to develop strong links with the respective programs (HIV, TB and Malaria) and to provide them both technical and institutional support. 19. With limited demographic information and limited capacity of the MoH to produce routine service data it meant that some assumptions had to be made about the disease burden. For example, at the time of design the HIV prevalence was considered to be around 5.5 percent13. With the increased post war population movements, combined with low contraceptive use (7%) and low awareness of HIV/AIDS amongst the general population it was assumed that prevalence was set to increase over the 5 years of the project. However it was later revealed that the HIV epidemic is concentrated, and the prevalence was a lot lower than was originally assumed. The latest prevalence data is 1.9 percent in 2009 amongst 15-49 year olds. Quality at entry 11 The final spend figures are taken from the Project Portal at the time of writing this report in November 2011. 12 The combination of HIV, TB and Malaria meant that this was not a typical MAP project, the same combination of three diseases was only supported in Eritrea and Burkina Faso 13 PAD page 2 (2004, 5.5%), and page 35 (2005, 11%), reference is also made to a 2001 prevalence rate of 8.6% p 23)) 4 20. The preparation team correctly judged the overall risk to be high, identified the major critical risks, and proposed appropriate measures. Two critical risks flagged as high were: (i) societal denial of the threat of HIV and low priority given to it and (ii) Line Ministries lack of capacity and commitment to carry out sector programs. The other more modest risks included the capacity of implementing entities (particularly outside the cities) and slow disbursement of funds because of limited financial and procurement capacity. Like other MAPs, the M&E and the results framework had some major weaknesses at entry. These are discussed under section 7.3. Disbursement overview. 21. By the end of 2006 the project disbursed 11 percent of the credit. Subsequent years saw accelerated rates of disbursements reaching 13 percent in 2007 and 19 percent in 2008. By June 2009 54 percent of the total grant had been disbursed which rapidly increased up to 82 percent in June 2010, at the time of the amendment and extension. Total disbursements reached 90.5 percent by the end of the project. 2.2. Implementation Factors that contributed to successful implementation or gave rise to problems, included: 22. Policy Environment. By 2005 when the project became effective the government had begun to take measures to rebuild the health sector. The strategic context for the project was strengthened with the development of a comprehensive health sector strategy – Plano de Desenvolvimento Económico e Social de Médio Prazo, 2009-2013. This was supported by the National Health Policy of 2010, which gave priority to : (i) Restructure and develop the National Health Service, prioritizing access for the entire population to primary health care. (ii) Reduce infant and maternal mortality and morbidity. (iii) Promote and preserve and overall context and environment conducive to health (iv) Empower individuals, families and communities in promoting and protecting health. 23. Government expenditure. During the implementation of the project government commitment to health has been strong, which was partly reflected in increasing or generous levels of spending on health. According to the latest National Health Accounts data from 2009, 5 percent of GDP was invested in health which equals to 8 percent of general government expenditure 14 . This translates into a total of US$ 204 per capita, with 89 percent met by government and 11 percent met out of pocket. Particularly noteworthy is the government‘s support to HIV/AIDS. In 2007 the National Institute to fight AIDS (INLS) had a budget of US$ 39 million, of which 88 percent was from the state. 24. External resources. During the implementation of the HAMSET project there were also increasing external resources available to support health, particularly HIV, TB and Malaria 15. From 2005 onwards it is estimated there was approximately US$ 80 million to support TB, HIV and Malaria annually (mainly from the GFATM, the USG, the UN and EU) 16. The support from 14 NHA data from WHO on Angola 15 With increased government allocations the external funds declined in proportion to the overall funding available (from 8% in 2004 to 3 % in 2009). 16 GFATM support started from 2005 onwards, with the successful applications to Rd 3 (Malaria - US$ 35m), Rd 4 (TB - US$ 10m and HIV- US$ 86m) and Rd 7 (Malaria - US$ 57m). Between 2005-10 the GFATM committed US$ 178m and disbursed US$ 134m to Health in Angola. The US also began to increase its support to health from 2006 onwards, reaching a total of US$ 50m a year in 2010, and providing roughly US$ 154m over the same 5 year period. Health support from the UN also increased with the largest support 5 other comparatively larger funding sources meant that the project needed to maintain a degree of flexibility, to compliment the funding of others and respond to gaps where they emerged. All accounts suggest that the project did this well and maintained strong coordination with the programs and other donors. 25. Geographical targeting. In recognition of the increasing presence of other players in 2004 the Government and the Bank correctly decided to better focus the Bank support on: (i) 6 provinces: Luanda, Cabinda, Benguela, Kuando Kubango, Cunene and Lunda Sul,17 and (ii) 8 key line ministries. The public sector support also provided support to provincial governments in Kuando Kubango, Cunene, Lunda Sul and the municipality of Rangel (Luanda).The six provinces cover a total population of approximately 3.2 million, which was a much more manageable target group. While the geographical focus on the ―most at need‖ provinces is positive, the selection of the 6 priority provinces could have been better documented. Figure 1. Map of Angola showing the 6 priority provinces, the number of sub projects supported and the HIV prevalence*. Cabinda 7 projects (HIV 1.9%) Luanda LundaP Sul 3Projects 20 projects (HIV 2.5%) (HIV 3.9%) Benguela 3 projects (HIV 4.4%) Huambo* 3 projects 20 subprojects 20 subprojects Kuando Kubango Cunene 4 sub projects (4.2%) 5 sub projects (HIV 4.4%) Map from the Final Evaluation Report (June 2011). HIV Prevalence (from INLS, Aug 2010) *Humabo was later added to the list of priority provinces. 26. Implementation constraints. The Project Coordination Unit faced numerous fund disbursement issues throughout the implementation period of the project. While the PCU team tried to manage and resolve each of these, it is clear that these constraints had an impact on implementation and the overall results of the project. They affected the start date for support to different components, the duration of support and the amount of support. Three interrelated factors that particularly affected the implementation are: (i) slow and delayed disbursements to implementers (ii) high turnover and limited finance and procurement capacity of PCU staff (ii) weak reporting from implementers. It should be noted that these capacity constraints were systemic; they were not necessarily specific to this project and affected other WB supported projects in Angola. These three constraints are described in more detail below: coming from UNICEF with US$ 25m for child survival plus US$ 7m to Malaria and US$ 1.5m to HIV a year today. Another major donor in Health is the EU. 17 In addition to supporting the priority provinces a number of project activities operated at national level. Huambo* Province was also later added to the provinces supported, as shown in the map. Huambo has a population of 1.9m) 6 Slow and delayed disbursements to implementers 27. Financial and procurement management. The project design included the contracting of a separate Financial and Procurement Management Agent (for at least the first 2 years) to be fully integrated within and report to the PCU director18. A private firm was selected in 2006, but did not deliver on any of its functions, before the contract was ended in late 2007. It was difficult to obtain qualified procurement and financial management specialists in the Angola market, and there were also work permit restrictions on hiring international experts. After looking into alternative options the PCU eventually took on these functions which resulted in a slower than planned start up and disbursement of the project funds, as well as delays in starting procurement of items by National Competitive Bidding (NCB) and International Competitive Bidding (ICB)19. 28. Disbursement delays. From 2008 onwards the project started reporting disbursement constraints due to the low ceiling on the special account which created liquidity issues. This was resolved in March 2009 when the limit was increased from $US 800,000 to US$ 1,500,000. Then from 2009 onwards there were reports of a shortage of US dollars in the commercial bank which limited the amount of funds that could be withdrawn from the project account, posing additional constraints on the flow of funds, particularly in the last 12 months20. 29. Government counterpart contribution. The project was designed with a 85 percent contribution by IDA in USD and 15 percent counterpart contribution by the government in Kwanza. Delays in the provision of the counterpart funds by the Ministry of Finance led to nonpayment of contractors and significant delays in payment of contract obligations. 21 The amounts released were often at reduced levels and delayed. When the Bank sought to remove this requirement and increase the IDA contribution to 100 percent the government argued to maintain its counterpart funding as a demonstration of its commitment. High turnover of PCU staff and implications on procurement. 30. The PCU was headed by an ―Assistant Project Coordinator‖ who was supported by three coordinators, one for each component (Health, Community, Public sector) plus specialists for M&E, Procurement and Finance. The project faced challenges recruiting and retaining skilled procurement specialists22, and by the time of the Mid Term Review (MTR) in April 2008, there were still US$ 4million worth of contracts that needed to be processed (88% of goods to be procured). It was only in 2009 that the project managed to proceed with procuring goods by International Competitive Bidding, however once started they took on average 24-26 months to be completed. As a result it was only at the end of the project extension in 2011 that a number of the goods were delivered23. One example is the 11 mobile VCT units, which cost US$ 2.3m and made up 10 percent of the project spend (or 70% of goods procured by ICB). Capacity constraints and weak reporting from implementers. 18 The initial assessment report showed that MoH would not be able to implement the project without this. 19 The challenges the project faced on procurement were similar to that experienced by other projects in Angola. 20 All implementing partners complained that the services provided by the commercial bank (ESA) were very slow and delayed disbursements to partners. 21 This was particularly in relation to the providers of goods and services, but also the public and private sector project support, payment of consultants, payment of customs duties of goods imported, as well as non payment of taxes and national insurance contributions of project staff. 22 One agency and 5 procurement specialist were hired during the five years of the project 23 One procurement process was not completed and cancelled (solar panels) 7 31. As was the case for other MAP operations, it was found that the programs had little experience of managing and reporting on external funds, as a result it took time for the funding for certain components to begin to flow effectively, particularly for the NGOs contracted under the community sector. Likewise the Line Ministries had little existing capacity to report on their finances and performance. Mid Term Review (MTR) 32. The projects MTR was due in June 30 2007, but took place 10 months late in April 2008. It happened two and a half years into the project when 28 percent of funds were disbursed. At the time the PCU was fully staffed but it was still having difficulties recruiting and retaining its own finance and procurement specialists. 21 NGOs and 14 public sector institutions had been contracted but only 10 percent of the public and community components funds had been disbursed and engagement with the private sector had not yet taken off. By contrast, the health component was on schedule with 55 percent of funds disbursed. One of the main achievements by the MTR was the production of a Knowledge Attitude and Practice (KAP) survey which gathered data from 2006 and was published in 2007. This survey generated the baseline data for the national HIV and Malaria programs as well as the HAMSET project itself. Other KAP surveys were also undertaken among employees of the participating public sector line ministries. A routine information system and manuals for monitoring the HAMSET project inputs and outputs were also developed. By the MTR the project had also made important contributions to the development of national strategies and guidelines on HIV, TB and Malaria. On HIV, it was recognized that Angola has a concentrated epidemic which required greater targeting at risk populations however it seems that the project missed the opportunity to adjust the design at this stage and instead continued to support general awareness raising and general service delivery programs such as VCT. Amendment 33. An amendment took place in June 2010 to extend the project until the end of June 2011. The changes between the original budget (in the PAD) and the revised budget following the restructuring are discussed in Annex 3. The main difference is the decreased support to the community sector and increased support to the public and health sectors, plus the inclusion of support to the private sector. One of the reasons for the reduced budget to the community sector, was the late start of subproject funding which was constrained by the low ceiling of the special account and the completion of the guidelines for subprojects. In effect the extension allowed a second phase of support of approximately $US 900,000 to a select number of sub projects24. The extension also covered the remaining procurement processes and final project evaluation by the PCU. 2.3. Monitoring and Evaluation (M&E) Design, Implementation, and Utilization (a) M&E design 24 The contract extensions for the sub projects implementation varied between 3-7 months for 10 NGOs and a longer period for 7 public sector partners, plus funding was initiated to private sector agencies. 8 34. The main strength of the Results Framework was that the M&E design included support to establish the baseline and end line25 for the project as well as collecting and strengthening the reliability of national routine data. The project also helped to conduct KAP studies, to set in place national M&E frameworks and also supported population based surveys (eg MICS). The key weaknesses included: (i) lack of consistency between the activities financed (inputs) and outcome indicators, (ii) measurability and large number of indicators as well as (iii) overambitious targets. (b) M&E implementation 35. In 2005-06 the health programs did not have an agreed list of indicators for which there were regular updates. As a consequence the intermediate indicators that were initially selected, particularly for TB and Malaria were subsequently changed to use the nationally agreed indicators, others were dropped where data was not routinely collected (eg TB/HIV)26. The project‘s original 30 indicators were reviewed in 2008 (at the time of the MTR). 11 indicators were revised and 5 were dropped and a number of new indicators were proposed. It was recommended that a final LQAS survey should take place in 2009 to measure the impact on behavior change. (c) M&E Utilization 36. The project started reporting on performance in 2008. The M&E function focused on gathering activity level indicators, but did not systematically review these against the higher level PDO targets that the project aimed to achieve. This is discussed in Annex 2. 2.4. Safeguard and Fiduciary Compliance 37. Safeguards. The project triggered Operational Policy 4.01 due to the potential negative environmental impacts of medical waste management.27 An assessment carried out in October 2003 showed that the state of health care waste management and disposal systems were inadequate. To address these concerns the MoH prepared and disclosed a National Medical Waste Management Plan (financed by a Project Preparation Facility). The plan included the establishment of a new centralized incineration centre for infectious health care waste and a new sanitary dump site, and aimed to bring on board private waste management service providers to broaden the coverage of services in a safe and timely fashion. In spite of some early delays in the implementation of the waste management plan it was successfully implemented by recruiting an agency to deliver on the plan, which focused on establishing waste management services in Luanda. 38. Covenants. There were four conditions of effectiveness and four covenants The covenants were generally complied with, albeit with some delays in the timing of (i) the completion of the Financial Management Manual, (ii) the adoption of the operations manual, and (iii) timing of the Mid Term Review which was due to be conducted by June 30 2007, and took place in April 2008. 39. Fiduciary compliance. The project generally provided timely financial management reports. There was a general compliance with the financial management covenants. The project complied with the Financing Agreement and all audit opinions were unqualified. 25 The project used Lot Quality Assurance Sampling (LQAS) to collect baseline (2006) and endline (2009) data from the 6 focus provinces. The 2006 data was reported in 2007. 26 The PDO indicators dropped include one on high risks groups, one on the percentage of diagnosed TB patients covered by DOTS in the priority municipalities, and another on the percentage of VCT clients who test positive for HIV who are screened for symptomatic TB. 27 The safeguards environmental screening category was B 9 2.5. Post-completion Operation/Next Phase 40. Angola continues to provide high levels of budgetary allocations to the health sector which augurs well for a continuation of the activities supported under the project. However, improvements in efficiency and effectiveness of services are necessary along with greater prioritization of interventions that are known to deliver results and more support to decentralized levels outside the major cities. The HIV/AIDS response in particular would benefit from more evidence based resource allocation and more targeted approach on high risk groups, where new infections are occurring. 41. At the end of the project some concerns were raised about prospects of sustainability given the disbanding of the PCU. First, it is worthwhile noting that the project provided significant capacity building support for implementation, in total it allocated US$ 5 million to training and provided assistance to improve the planning, priority setting and management of funds by the HIV, TB and Malaria programs. Second, the project extension included support by the HAMSET PCU to help prepare the new Municipal Health Sector Services project (MHSS) worth US$ 70.8m which was approved by the Board in May 2010. The PCU team helped put together the operational manual and the procurement plan, sharing lessons learned from HAMSET. Against these positive aspects, it should be noted that while the PCU developed procedures and administrative systems (procurement, finance and HR) to manage and channel funds to partners more could have been done to mainstream these activities within the Ministry of Health. 42. The second issue relates to the future of the support for the public sector institutions. The project helped set in place HIV policies and plans for key line ministries. This gained high level support of various Ministers. The project provided funding for HIV focal points, recruited assistants to the line ministries/departments and financed their work plans. It was expected that the line ministries would take over and mainstream HIV activities in their sector plans, however there was no systematic follow up to secure a firm budget allocation from the state for this work, or to get a commitment to take on these staff functions. As a result not all line ministries will be in a position to continue these activities at the same level of effort. 3. Assessment of Outcomes 3.1. Relevance of Objectives, Design and Implementation 43. Relevance of Objectives and Design. The relevance of all three objectives and components are rated substantial based on (i) Angola‘s epidemiological situation and program priorities: and (ii) the Bank‘s overall policies and country strategy. The project design is also rated substantial given that the project was both innovative and ambitious for its time and included arrangements for the multi-sectoral response and engagement with NGOs, the public and private sectors. 3.2. Achievement of Project Development Objectives (Efficacy)28 44. More than half of the PDO targets were met29. Progress is measured using the 8 PDO indicators and 13 Intermediate Output Indicators (IOIs). By the end of the project five of the eight 28 A full assessment of progress against each of the PDO and intermediate output indicators is presented in Annex 2. This includes those indicators identified at the start of the project, those dropped, and the new ones introduced at the MTR. The list also highlights those achieved, those not achieved and those where there is no updated information available. 29 For the purposes of the ICR we have used the PDO objectives in the PAD (and not the Grant Agreement, which are worded slightly differently but essentially mean the same thing). 10 PDO targets were achieved (62%). A total of three targets were not met, of which two were 85% achieved. These results are very positive given the challenging implementation environment combined with the significant capacity constraints in government and amongst the partners during project implementation from 2005-2011. 45. PDO I: Reducing the spread of HIV/AIDS in the Angolan Population (See Annex 2, p21). The HIV prevalence target of below 5 percent was achieved, but essentially the target was already met at the start of the project as it has been below 3 percent since 2004 until today. What is important is that there was no increase in HIV prevalence nationally, but particularly amongst the youth. The two other HIV PDO targets for the project were not met. Behavior change trend data suggests that there has been a slow but steady increase in HIV knowledge and modest improvements in behavior change, including increased reported condom use 30 . The target for correct knowledge on HIV amongst the youth was particularly overambitious, set at 85 percent against a baseline of 23 percent. 46. HAMSET project activities included the capacity building and expansion of VCT and PMTCT services in five provinces through training and supervision. The project supported awareness raising, behavior change and impact mitigation interventions implemented by NGOs in the community and HIV awareness raising through the public and private sector support. The project also provided 11 mobile VCT units for different partners31and funding to the centre for blood transfusion (training and 8 vehicles).While the project recognized and promoted the need to rapidly scale up the number of condoms distributed this received little attention and the initial plans to support condoms social marketing did not go ahead. In summary, the HIV prevalence remained low, but more could have been done to target the youth and most at risk populations. 47. PDO II: Strengthening the capacity of health services to detect and treat Tuberculosis (See Annex 2, p22). Two out of the three TB PDO targets of the project were met. The achievements include (i) the new smear positive case detection rate reached 77 percent in 2010 (surpassing the WHO target of 70 percent) and (ii) the percent of priority municipalities implementing TB control activities (lab and DOTS) which increased from 4 to 100 percent between 2005 and 2007. The treatment success rates of 85 percent were not met. Between 2005- 2008 the level remained around 70 percent and the latest data shows little change (72% in 2009). 30 Monitoring behavior trends on HIV has proven difficult, as data from surveys have been contradictory. The 2007 KAP reported 22.7 % of young people aged 15-24 can correctly identify ways of prevention the sexual transmissionof HIV and reject major misconceptions. The end line data from the 2009 KAP showed an increase to 39.9% but the IBEP data from 2008/09 reported 28.5%. 31 The 11 mobile VCT units were only delivered at the end of the project. They were allocated as follows: 8 to the health sector, 1 to community sector and 2 to the public sector 11 Figure 2: Case Detection and Treatment Success Rates under DOTS: 2005-2010 100 DOTS new smear positive 80 treatment success rate 60 (%) DOTS new 40 smear positive case detection 20 rate (%) 0 2005 2007 2008 2009 2010 Source: WHO 2010 TB report (some data is not available for 2007 and 2010) 48. HAMSET Project activities included support to policy guidelines (including TB/HIV), combined with capacity building efforts and the expansion of DOTS, through supervision and logistical training as well as the provision of vehicles and microscopes32. The project supplied US$ 500,000 worth of TB drugs and supported training to improve TB diagnostics through the national reference laboratory. While the project recognized the need to integrate TB and HIV, there was slow progress and little to demonstrate these results 33 . The project also missed an opportunity to support the implementation of DOTS for TB through the community component. In summary, the TB interventions partially contributed towards the PDO targets of strengthening TB detection and treatment. 49. PDO III. Strengthening the capacity of MoH for effective case management of Malaria (see Annex 2, p23). The two malaria PDO targets of the project were met, these are on maternal and under five malaria mortality. Overall, while the numbers of reported malaria cases have steadily increased there has been a rapid drop in institutional maternal mortality due to malaria from 57 percent in 2005 to 3.2 percent in 200934. Under five mortality has now dropped to 161 per 1,000, but remains one of the highest in the world Figure 3: Malaria Cases and Deaths between 2000-2010 (translation – casos=cases, obitos=deaths) Tendencias de Casos e Óbitos 2000-2010 4,000,000 45,000 3,500,000 40,000 Casos de Malária 35,000 3,000,000 Casos de 30,000 Malária 2,500,000 25,000 Óbitos 2,000,000 por 20,000 malária 1,500,000 15,000 1,000,000 10,000 500,000 5,000 0 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 32 The health sector support included 30 microscopes to the Malaria and TB programs 33 The PAD included on PDO indicator on HIV/TB integration another was introduced at the MTR. One on the percentage of VCT clients who test positive for HIV the other on the percent of VCT clients who are screened for symptomatic TB. Both indicators were dropped due to lack of information available to report on progress. 34 2010 Malaria Report. It should also be noted that more than 54% of births occur at home 12 50. HAMSET project activities to strengthen MoH‘s capacity for case management of malaria were designed to compliment and fill gaps in the Rd3 GFATM support. The project funded operational research which informed the national choice of interventions on key issues from diagnostics, to treatment, to community outreach35. It also supported the Malaria communication strategy/package (JUCA), which in turn will have increased the demand for and use of malaria services. Support was also provided for Artemisinin based Combination Therapy (ACT) roll out through training and the provision of 4 vehicles and 18 motorbikes. The project also helped create the national partnership forum on Malaria and provided funding to the national reference laboratory to improve diagnostics. ITN distribution was not supported by the project despite it being part of the original design36. In summary the Malaria interventions supported will not have achieved the PDOs on their own, but they did help strengthen capacity and provide evidence which contributed to national policy setting and also guided subsequent resource allocation from Government, USG and GFATM (Rd3 & Rd7). 51. Half of the Intermediate Outcome Indicator (IOI) targets were met. While 5 out of the 8 (62 percent) of the higher level PDO targets were achieved, 7 of the 13 (54 percent) intermediate output targets were achieved. Three were not achieved, a further 2 have no data. Annex 2 presents the project results chain 37 and includes a discussion on the causal linkages between these intermediate outputs and outcomes organized under each of the three PDOs and the four project components. The project supported a range of capacity strengthening activities and outputs which translated into positive service delivery outcomes. On the basis of the evidence presented above, and in Annex 2, the efficacy is rated substantial. 3.3. Efficiency 52. Overall efficiency is rated modest partly due to limited evidence as there was no Economic Rate of Return (ERR) carried out at appraisal, nor were there any cost efficiency analysis or technical audits conducted during the project. It was also not possible to carry out a review of spending by PDO objective or disease outcome area. 53. HIV/AIDS. Efficiency is rated modest. The project was efficient in that it selected only the relevant line ministries in HIV prevention and contracted NGOs and the private sector to focus on certain at risk geographical areas. However the activities implemented on HIV could have been more cost effective if they had focused more on the youth and the most at risk populations that are key to preventing HIV in a concentrated epidemic. While the project did make a significant contribution to establish a baseline on HIV, with hindsight the project could have done more to identify the key drivers of the epidemic and support a strong national prevention strategy based on this. For example, the provision of 11 mobile VCT units worth over US$ 2.3m was not particularly cost effective (at roughly US$ 200,000 each) and they were only delivered at the end of the project. 54. Tuberculosis. Efficiency is rated modest. The project was partially efficient in that it assisted in setting guidelines and supported facility based disease control. The project‘s focus on training and supervision served to strengthen capacity of the TB program. While the case detection rate 35 The Operational Research included studies on Malaria diagnostics (sensibility test study), Coartem efficacy, Community level treatment pilot, Efficacy of TIP policy for pregnant women and a study on insecticide resistance. 36 The original PAD included support to the social marketing of bed nets through the community component, but this did not go ahead. ITN ownership has shown good progress, but reported ITN use particularly amongst under fives is very low (16.4% MICS Dec 09). 37 The Results Chain that is presented has been retrofitted to establish the links between the project inputs, capacity outputs and the service delivery outputs and coverage outcomes. It is fair to say that some of these links are stronger than others and that if the results chain had been established early on in the project then the contribution to the results would have been easier to establish. 13 surpassed the WHO target, the treatment success rate showed no change which suggests challenges with treatment access and adherence. Given this the project could have done more to strengthen outreach in the communities. 55. Malaria. Efficiency is rated substantial. The project was efficient in that with a small amount of resources it provided catalytic support to a series of operational research studies which served to inform national policy and guided subsequent plans and funding allocations. The project‘s malaria support was well coordinated through the Malaria forum and filled critical funding gaps in improving case management in areas with high malaria risk. 3.4. Justification of Overall Outcome Rating Relevance Efficacy Efficiency Outcome Rating PDO 1 To reduce the spread of Substantial Modest Modest Moderately HIV/AIDS Unsatisfactory PDO 2 Strengthen the capacity Substantial Substantial Modest Moderately of the health sector to detect Satisfactory new cases of TB, improve treatment compliance and increase the completion rate. PDO 3 Strengthen the capacity Substantial Substantial Substantial Satisfactory of MOH for effective case management of malaria Overall Rating Substantial Substantial Modest Moderately Satisfactory 56. The project‘s achievements need to be viewed within the post war context of reconstruction, a period during which the national policy direction was weak, data availability was poor and there were huge capacity constraints in government and amongst implementing partners. While there were a series of project management and disbursement constraints, the project did keep on track and by the time of project closure more than half of the project objectives had been successfully met. On the basis of substantial relevance, substantial efficacy and modest efficiency, the overall project outcome rating is moderately satisfactory. 57. It should be noted that the PDO ISR ratings were rated satisfactory throughout the project implementation. The main reason for the differing rating of the last ISR and the ICR is that performance against the PDO was primarily assessed according to relevance because final progress data was not yet compiled and available. During implementation performance was being reviewed according progress on the different project components and disbursements and less so on against the specific project outcome targets. Another reason for the positive ISR rating, compared to the ICR, is that while Angola sits in the epicenter of the HIV epidemic the HIV prevalence has essentially remained the same, and did not increase, unlike neighboring countries. Also during the 5 years implementation there was a scaling up of HIV services, particularly PMTCT and VCT, which was considered significant given the post conflict context, the poor infrastructure and health services when the project started. The argument put forward by the country management unit is that if the ICR rating criteria took into account the enormous challenges of working in a post-conflict country like Angola it could be rated satisfactory. 58. Achievement Summary. HAMSET was instrumental in supporting Angola at a crucial time in the country‘s development process. It helped build basic administrative capacity and arrangements and enabled the government to seek and receive additional funds of its own, but also from external partners like the GFATM and USG. The project succeeded to establish 14 baselines, it strengthened M&E and supported the development of national strategies and guidelines to inform priority setting. It also funded a range of activities which contributed towards the PDO targets. The project despite its financial challenges managed to successfully spend 90.5 percent of the funds by the time it closed in June 2011. Much was accomplished despite difficult circumstances. 3.5. Overarching Themes, Other Outcomes and Impacts 59. Poverty Impacts, Gender Aspects, and Social Development. The projects PDO targets were set at the health sector level and implementation did not focus explicitly on the wider themes of poverty reduction, gender and social development. Despite this many of the activities supported in health, will have reduced vulnerabilities to ill health and contributed to development. For example, the reduced institutional mortality rates among pregnant women and children due to malaria is a clear contribution by the project to poverty reduction. 60. Institutional Change/Strengthening. While the project did not intend to address the major institutional or long-term reforms within the sector, it did provide capacity building and technical support to (i) the HIV, TB and Malaria programs in the MoH, (ii) the line ministries and provincial offices, and also (iii) the civil society and private sector implementing partners. The approach included strengthening national planning and priority setting and the promotion of a multi-sectoral partnership between government, civil society and the private sector. 4. Assessment of Risk to Development Outcome 61. Internal Risks. The government‘s commitment to HIV, TB and Malaria was strong and clear from the outset and has remained so. Although the national HIV prevalence remains low, the risk of there being an increase in some parts of the country remains, and there is a need to better understand and halt the drivers of the epidemic. Also the institutional arrangements for the multi- sectoral response and engagement with NGOs, the public and private sectors are unlikely to continue in the same form in future. Some discontinuity is to be expected as the PCU is disbanded, and there is also a degree of uncertainty regarding the mainstreaming functions created under the project. It is expected that some of the Ministries will incorporate some key activities in their plans, but there has been no firm commitment to do so by government. 62. External Risks. The delays in disbursements from the GFATM in 2010 and 2011 may have a negative effect on the programs, particularly activities that are earmarked to receive this support. Nevertheless Angola is less likely to be affected from reduced external resources related to the financial crisis, as it has large resources of its own which it allocates to health. 63. The overall risk to the development outcome is rated as moderate. 5. Assessment of Bank and Borrower Performance 5.1. Bank Performance 64. (a) Quality at Entry. The project objectives were aligned with the Governments priorities and selectively focused on three diseases. The project preparation team did well to identify the critical risks and capacity constraints and made a good decision to establish the PCU in the MoH. The M&E and the results framework, as in many MAP projects, had some major weaknesses 15 including as discussed earlier. Quality at entry is rated as moderately satisfactory, bearing in mind the post war context and problems of availability of information at the time. 65. (b) Quality of Supervision. The task team conducted regular supervisory missions of the project (every 7 months on average) during which it was proactive in identifying the threats to the relevant development outcomes, it was moderately effective in resolving them. Among the challenges faced by the Task Team was the high turnover of project staff and capacity constraints in implementing partners. The lack of capacity in Angola was particularly problematic, especially at the start of the project. As a result a lot of time was spent early on to help strengthen the fiduciary arrangements especially in procurement and financial management which troubled the project throughout its implementation. Attempts were also made by the task team to amend the Development Grant Agreement to allow for 100 percent IDA financing, to address the problems in obtaining the 15 percent counterpart contribution, however no agreement was reached with government. Despite it being recognized early on that two of the HIV targets set at the start of the project were overambitious, they were not changed. As a result progress on the HIV PDO was not met and is rated unsatisfactory. Had the targets been changed and the project PCU been tasked with supporting a more targeted HIV response then the project could have made better progress. On a positive note the project recognized the need to focus on specific geographical areas and target 6 provinces. Quality of supervision is rated moderately satisfactory. 66. (c) Overall Bank Performance. Based on the moderately satisfactory quality at entry and moderately satisfactory quality of supervision, the overall Bank performance is rated moderately satisfactory. 5.2. Borrower Performance 67. (a) Government Performance. Throughout the project there was strong government ownership and commitment to achieve the development objectives. There were four conditions of effectiveness and four covenants which were met albeit with some delays. Despite the high level support the project suffered from delays relating to the 15 percent counterpart allocation. The delays in making available the counterpart funding was most problematic when it was required to allow the customs clearance of the 11 mobile VCT units. It resulted in a one year delay which meant that the VCT units were only released for use at the end of the project. By the end of the project Governments payment of its contribution only reached 55 percent of what was due. The problems regarding the counterpart contribution was a portfolio wide issues and was experienced by many WB funded projects in Angola. 68. The decision to establish the PCU in the MOH was positive and was the right choice in 2004- 2005. However over time having a separate unit in the MoH which was tasked with multi- sectoral support did create some coordination challenges and complaints. While the set up was critical to get the project up and running, some of these institutional arrangements became less appropriate over time. For example it was argued that direct funding to the public sector and NGOs on HIV should go via INLS. Government performance is rated moderately satisfactory 69. (b) Implementing Agency or Agencies Performance. The implementing agencies for this project included (i) the MoH‘s HIV, TB and Malaria programs (ii) the public sector line Ministries and provincial offices (iii) Private sector agencies, and (iv) national and international NGOs. Together these implementing partners were responsible for the bulk of projects overall performance against outcomes. A common challenge for all partners was the weak capacity to manage and report on funds combined with poor M&E. Project implementation was generally adequate although there were a number of delays in fund disbursement and the contracts for 16 support to the NGOs were for limited time periods, also there were long gaps between the first and second phases of support. While some partners performed well and other performed not so well, overall implementing agency performance is rated moderately satisfactory. 70. (c) Overall Borrower Performance. Based on the moderately satisfactory government performance and moderately satisfactory implementing agency performance, overall Borrower performance is rated moderately satisfactory. 6. Lessons Learned 71. Over the period 2005-2011 the Bank contributed US$ 21 million to the financing of the sector, and this ICR has documented evidence of outputs and outcomes associated with this financing. There have been a number of achievements, but also challenges, and there are several lessons that emerge from this experience. 72. Capacity Building for Results: The lack of capacity in Angola during the period of project implementation needs to be recognized and a review of performance and results achieved needs to take this firmly into account. Capacity building is critical in post conflict situations and the project was instrumental in providing support for the HIV, TB and Malaria programs at a challenging time. Over 24 percent of project funds ($US 5m) were spent on training, which was combined with technical and strategic support, plus guidance from the various project coordinators. 73. Implementation arrangements: The establishment of a PCU in the MoH, and not the National AIDS Council (as with other MAP projects) was positive and relevant at the time. It allowed the PCU to develop strong links with each of the three disease programs. Looking back, the PCU was essential to get the project up and running but also to manage, channel and account for funds used in the various project activities. Having specialized staff working on this was key. Future use of PCU‘s in Health in Angola, should carefully consider how government procedures and administrative systems (procurement, finance and HR) can be strengthened by the project. 74. Integration and Efficiency: Looking ahead Angola should focus more on integration of disease specific programs and do more to address system wide constraints. Also further attention should be given to the efficiency and cost effectiveness of services delivered, to maximize value for money. The projects support to the three main communicable diseases provided an opportunity to adopt a patient oriented approach and to achieve efficiencies and economies of scale. However the opportunity to truly integrate HIV, TB and Malaria through systems strengthening and joint activities was missed38. 75. Geographical Targeting: Any geographical targeting of most in need populations by the project should be documented and accompanied by a clear rationale. The project‘s decision in 2004, to focus on specific geographical areas and key line ministries covering approximately 18 percent of the Angolan population was positive. Luanda province received the bulk of the sub project support, which may well have been deliberate as 25 percent of the population are now estimated to live in Luanda today (4.6m). However the selection of the 6 priority provinces was not very well documented, for example Huambo province (pop 1.9m) was later added. 38 The project dropped the two intermediate indicators on TB/HIV which further suggests that this integration was a challenge. 17 76. Flexibility but with a focus on desired results: The project provided a good degree of flexibility to respond to the rapidly evolving needs and also provided support to fill critical gaps in the national plans. However in supporting new activities the project gave up and missed opportunities to fund some critical interventions (in the original plans) which would have provided a more direct contribution to the projects desired results 39 . The project would have benefitted from documenting the results chain early on to ensure consistency between the activities financed (inputs) and intermediate outcome indicators, and desired results. 77. Monitoring and Evaluation. The Angolan context requires that special attention and support be given in all projects to M&E, especially to establish the baseline and endline data for the project. In this respect the project M&E design was good practice in that it allowed the collection of both baseline and endline data through conducting its own surveys in 2006 ands 2009. While positive, weaknesses continue to exist with the routine data collection and health information systems, which are in need of support. The project also encountered challenges in accessing official reports and obtaining annual progress data in a timely manner. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies: The borrower has prepared a comprehensive final HAMSET evaluation report in Portuguese including a separate short summary. The ICR task team and the borrower discussed the evaluation report findings at the time of the ICR mission. (b) Cofinanciers: not applicable. (c) Other partners and stakeholders: The ICR task team discussed the borrowers evaluation findings with NGOs/civil society and the public sector partners at the time of the ICR mission. 39 For example the initial plans to support social marketing of bed nets, to implement of DOTS for TB and to scale up the distribution of condoms, through the community component did not go ahead. The total budget for the community component was cut by 50 percent (from $US 6.8m to 3.1m). 18 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Actual/Latest Percentage of Components Estimate (USD Estimate (USD Appraisal millions) millions) Total Baseline Cost 0.00 0.00 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 0.00 0.00 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00 Total Financing Required 0.00 0.00 (b) Financing Appraisal Actual/Late Type of Estimate st Estimate Percentage Source of Funds Cofinancing (USD (USD of Appraisal millions) millions) Borrower 3.70 0.00 .00 IDA GRANT FOR HIV/AIDS 21.00 0.00 .00 Bilateral Agencies (unidentified) 14.90 0.00 .00 19 Annex 2. Outputs by Component I. INTRODUCTION 1. This annex presents the latest end line progress data on the outcome indicators for the project and analyzes the outputs by component using a results chain. The annex is divided into five sections: (I.) an introduction, (II.) a summary list of all indicators used by the project, (III.) a review of progress against the project development objectives (PDOs), (IV.) a review of progress against the intermediate output indicators (IOIs) by component, and (V.) the results chain for the project. Appropriate evidence and data has been collected and included from surveys and program reports. However, to better understand and interpret the data, it is important to point out the following caveats, weaknesses and issues. 2. Firstly, it should be noted that the results chain has been developed for the ICR (and was not available before). Activities supported under each component have been retrofitted to link up with their relevant project Development Objectives. The Results Chain includes four columns: the first (I) lists the key activities implemented or inputs by component, the second (II) lists the institutional and capacity strengthening support as capacity outputs. It then takes the indicators in the PAD and DGA, and presents the project outputs in the third column as the (III) service delivery outputs. The PDO level outcome indicators are listed under level (IV) as coverage outcomes. It is fair to say that some of these links are stronger than others and that if the results chain had been established at the start of or early on in the project then the connection with the results would have been easier to establish. A particular challenge is to demonstrate the links between the inputs and capacity outputs (level I and II), and the service delivery outputs (III). The projects M&E system and the Governments Final Evaluation report focuses on the inputs and capacity outputs and there has been less systematic monitoring and data available on all of the Service Delivery Indicators. 3. Secondly, there was limited baseline data for a number of the indicators and targets set in the PAD. This was more a reflection of the context at the start of the project, given that Angola had only just emerged from 30 years of civil war, and there was little up to date quality information available on disease prevalence, nor were there any systems in place to collect routine health information county wide. 4. Thirdly, for the purposes of this end of project review progress is assessed against 8 PDO indicators and 13 intermediate output indicators. The original PAD, included 9 PDO level indicators and 15 intermediate output level indicators. At the time of the MTR in 2008, it was recognized that a number of these indicators were not appropriate or data was not available, and a number of changes were proposed40. However subsequent to the 40 A full list of indicators used by the project, including those identified at the start of the project, those dropped, and the new ones introduced at the MTR are provided in Section II below. This table also provides a useful summary of the indicators achieved, those not achieved and those where there is no updated information available. 20 MTR not all revised or new indicators were systematically reported in subsequent ISRs or the annual reports. Therefore this end of project review is limited to those PDO and intermediate indicators where data is available and comparisons can be made from 2005- 2011. 5. Fourthly, there continues to be a challenge of availability, reliability and potential generalization of data in Angola today. Factors contributing to this are: (i) The last census was in the 1970‘s and there are no denominators for many of the indicators. Today Angola has an estimated population of 18m, and the country is preparing for its next census in 2013. (ii) A comparison of survey data over the years is complicated by a large movement of people, both coming into the country from Zambia and Congo, but also movement around the country, from Luanda to provincial capitals and from the provincial capitals into the surrounding areas. This flux of people means it is difficult to draw comparisons over the last few years, be it use of condoms or bed nets or other indicators. (iii) National level indicators also hide significant variations by income quintile. For example, population aged 15-24 aware of HIV prevention measures varied from 40 percent in the fifth wealthiest quintile, to 10 percent in the first poorest quintile (MICS). (iv) Added to this access by the population to health services is still around 50% and varies significantly by location. 6. Fifthly, the restructuring and extension of the project from June 2010-11, was not accompanied by a revision of the targets, despite the increased duration of the project by one year. Related to this it is important to note that the majority of end line data is from 2009, some is from 2010, but there is no data from 2011. 7. Finally, it should be noted that since 2005, support to HIV, TB and Malaria in Angola has been funded from many funding sources. In this respect the HAMSET project helped to fill gaps and to scale up interventions, contributing to the observed trends in outcomes discussed below. On average donors provide US$ 80m annually. SUMMARY OF ALL INDICATORS AND PROGRESS 8. This table shows the full list of indicators used by the project. It includes those identified at the start of the project, those dropped, and the new ones introduced at the MTR. The table also highlights those achieved, those not achieved and those where there is no updated information available. PDO level Progress Indicator Status Notes PDO To reduce the spread of HIV/AIDS in the Angolan 1 population through a multisectoral approach that strengthens institutional capacity and increases access and utilization of quality health services for prevention, diagnosis, treatment care and support #1 Achieved HIV HIV/AIDS prevalence in pregnant women aged 15-49 prevalence was below 21 originally assumed #2 Percentage of young people aged 15-24 who can both Not 40% correctly identify ways of preventing the sexual Achieved achieved transmission of HIV and reject major misconceptions about HIV transmission and prevention #3 Percentage of men and women aged 15-49 reporting Not 84% the use o f a condom in their last act of sexual Achieved achieved intercourse with a non-regular sexual partner during the previous 12 months X Percentage of people in high-risk groups (commercial Dropped sex workers, truckers, uniformed services, excombatants), who can cite at least two major modes of HIV/AIDS transmission and at least two methods of protection PDO Strengthen the capacity of the health sector to detect 2 new cases of TB, improve treatment compliance and increase the completion rate. #4 Proportion o f the 59 priority municipalities that have Achieved implemented TB control activities #5 TB Case Detection Achieved Intro at MTR #6 TB Success Rate Not 85% Achieved achieved X TB Abandonment Dropped Intro at MTR X Number and % of TB patients tested for HIV Dropped Intro at MTR X Percentage of diagnosed TB patients covered by Dropped DOTS in the 59 priority municipalities X Percentage o f VCT clients who test positive for HIV Dropped who are screened for symptomatic TB PDO Strengthen the capacity of MOH for effective case 3 management of malaria #7 Institutional mortality due to malaria among children Achieved under five years #8 Institutional maternal mortality due to malaria Achieved Output level Intermediate Indicator PDO 1 1 Number o f VCT centers functioning per year Achieved 2 Number o f counselors trained per year Not 62% achieved achieved 3 Number of people being tested for HIV Achieved 4 Number of sentinel surveillance sites in antenatal Not 82% 22 clinics functioning per year achieved achieved 5 Number of health facilities providing HV infected Achieved pregnant women and their new born with Mother to child transmission intervention services, per province 6 The number of condoms distributed through the public No sector and NGO programs (million) update 7 Number of participating line ministries which have and Not 87.5% are implementing their respective HIV/AIDS achieved achieved workplans 8 Number of decentralized government entities that are Achieved implementing integrated HIV/AIDS workplans 9 Amount of funds channeled to civil society initiatives No per year (US$million). update PDO 2 10 Number o f staff trained in DOTS in each priority No municipality update 11 Number of priority municipalities that have Dropped laboratories with adequate equipment and trained technicians to do sputum smear microscopy X Number of facilities providing VCT services for HIV Dropped testing to TB patients X VCT sites implementing protocols to screen Dropped symptomatic TB in HIV positive persons PDO 3 12 Number o f Malaria Control Program coordinators Achieved trained in operational management, budgeting and supervision. 13 Number of trainers trained in malaria case Achieved management per province. Additional Indicators Y % of children under five who slept under an ITN the Target previous night Not achieved Y % of households with at least one ITN Target Not reached NOTES Indicators reported in final ISR a) PDO = 8 targets (5 achieved, 3 not achieved) b) Intermediate Output = 13 targets (7 achieved, 3 not achieved, 2 no data). c) The numbering of the indicators in this table corresponds to those in the results chain. Indicators numbered X have been dropped, and are not being measured as part of this ICR 23 III. PDO - OUTCOME INDICATORS 9. PDOI: Reducing the spread of HIV/AIDS in the Angolan Population. The understanding of Angola‘s HIV epidemic and its spread, has increased during the period of project implementation. Between 2005-2010, the number of epidemiological surveillance sites went from 26 to 36 allowing a better assessment of the number of cases of HIV amongst pregnant women. In contrast to what was expected at the start of the pro ject the HIV prevalence has remained low (i.e. under 3 percent), although rates are higher amongst high risk groups. There are also variations in HIV prevalence between different provinces, Cunene (one of the priority provinces targeted under the project), for example has a prevalence of 4.4 percent. Today there are an estimated 166,500 PLWHA, with 18,140 new infections annually41. Nationally, between 2005 and 2010, the number of health facilities providing PMTCT to prevent the spread of HIV from mothers to children increased from 8 to 20042. Of the 203,463 pregnant women tested in 2009, 4,780 were positive and approximately 64 percent received treatment. The percentage of children born positive from HIV positive mothers was 2.3 percent (18/733)43. Over the years other HIV services have steadily increased, including the availability of VCT services and ARV44. Nevertheless correct identification of prevention methods and rejection of misconceptions is still very low, particularly amongst the higher risk youth groups aged 15-24. Stigma associated with HIV also continues to be high.45 Outcome indicator #1: HIV/AIDS prevalence in pregnant women aged 15-49 (or 15-24) Achieved 10. Current HIV prevalence for 15-49 year olds is 2 percent. The end target was 19 percent which was higher than the assumed baseline of 11 percent because there was no trend data at the time of the appraisal. It was later revealed that HIV prevalence amongst pregnant women was 2.6 percent in 2007 and increased to 2.8 percent in 2009. Despite being set too high this target was not changed at the MTR. Prevalence amongst 15-24 yrs old is 0.6 percent amongst male and 1.6 percent amongst females remains low. Outcome indicator #2: Percentage of young people aged 15-24 who can both correctly identify ways of preventing the sexual transmission of HIV and reject major misconceptions about HIV transmission and prevention – Not Achieved 11. The initial baseline was set at 60 percent in the PAD, but the survey conducted by the project in 2006 reported it as 22.7 percent. The target was set at 85 percent, and was recognized as too ambitious at the MTR, but it was not changed. The end line data from the survey conducted in 2009 reported 39.9 percent but the IBEP of 08/09 reported 28.5 percent (39% urban and 12% rural). The variations may be due to survey methodology 41 National Strategic Plan to control STI‘s HIV and AIDS 2011-14, Ministry of Health INLS 42 Between 2006-09 the number of women who were tested and received results for HIV tripled from 22% to 65% (KAP 2007, KAP 2009) 43 National Strategic Plan to control STI‘s HIV and AIDS 2011-14, Ministry of Health INLS 44 The number of functioning VCT sites increased from 35 to 54 with a total of 462,680 people tested for HIV. ARV provision began in 2004 and by 2009 a total of 27,520 PLWHA (25,456 adults and 2,064 children) were on treatment 45 56% of 15-24 year olds demonstrated discriminatory attitudes towards PLWHA (KAP 09) 24 and other factors. Either way the target was not met and knowledge on HIV transmission remains very low. Outcome indicator #3: Percentage of men and women aged 15- 24/49 reporting the use o f a condom in their last act of sexual intercourse with a non-regular sexual partner during the previous 12 months – Not achieved 12. The initial baseline was set at 10 percent and the target for 2010 was 80 percent. The KAP survey in 2006 reported 31.9 percent. This target was revised down at the MTR to 60 percent as it was considered too optimistic. The end line data was 50.5 percent (from KAP 2009), showed the target was not achieved. This suggests a positive increase in behavior change, and a near success against the target. There are large variations in reported condom use by income, gender and location (ISR progress only reports on the 15-24 age group). 13. PDO II Strengthening the capacity of health services to detect and treat Tuberculosis (TB). TB is a major public health problem in Angola. According to the World Health Organization‘s (WHO‘s) Global Tuberculosis Control Report 2011, the number of new sputum smear positive (SS+) cases notified increased almost threefold, from approximately 7,379 to more than 21,146, between 1999 and 2010. WHO estimated that there were 44,655 new TB cases in 2010. An analysis by the Ministry of Health (MOH) in 2007 found that the national program was implemented in only 8.6 percent of all health units. In addition, TB drugs were in irregular supply and 40 percent of clinics have experienced stockouts. Challenges include the low coverage of diagnostic services and slow progress on the integration of TB and HIV. While the 2010 WHO report provides a national estimate of 5 percent TB-HIV co-infection, data on HIV prevalence among TB patients show a range from 2.8 (Cabinda), to 15.6 percent (Benguela Province), and 6 percent in the province of Luanda (2008 data). Treatment of TB and HIV co infection is at 20.4 percent. Fewer than 1,000 cases of multidrug-resistant (MDR) TB have been detected in Angola; however, the actual prevalence is not known. Outcome indicator #4: Proportion o f the 59 priority municipalities that have implemented TB control activities (lab + DOTS) – Achieved 14. The baseline was 4 percent and 2010 target was set at 100 percent. The 100 percent coverage of the 59 priority municipalities was reached in 2007. Outcome indicator #5: TB Case Detection Rate – Achieved 15. This indicator was introduced after the MTR following the revision of the National TB strategy. The targets adopted are those set by WHO at 70 percent. The DOTS new smear positive case detection rate was 82 percent in 2005. This increased to 85 percent in 2008, then back down to 75 percent in 2009 and reached 77 percent in 2010 (WHO 2011 report). The WHO target of 70 percent was achieved. Outcome indicator #6: TB Success Rate – Not achieved 25 16. This indicator was introduced after the MTR following the revision of the National TB strategy. The targets adopted are those set by WHO at 85 percent. In 2010 72 percent of all new smear positive TB cases registered for treatment that were successfully treated (WHO 2011 report). The trend shows little change. The rate was 73 percent in 2006 and increased to 74 percent in 2007 before dropping back to 72 percent in 2009. There is still a significant gap in meeting the WHO cure rate of 85 percent. 17. PDO III:. Strengthening the capacity of MoH for effective case management of Malaria. Malaria remains the principal cause of morbidity and mortality throughout the country46. It is responsible for 35 percent of demand for curative services, 20 percent of hospitalization, 40 percent of perinatal deaths and it also contributes to 25 percent of maternal mortality47, which stands at 610 per 100,000. Malaria is the main contributing factor to Angola‘s alarmingly high under 5 mortality rate (161/1000 live births)48. Trend data shown in Figure 4 suggests that malaria morbidity and mortality was over reported at the start of the project, especially in 2003-2004. Since then case management of malaria has improved with better diagnostics and training of staff accompanied by 100 percent treatment availability by ACT 49 . Between 2008 and 2010 ACT availability increased from 541 to 2,240 facilities. In 2010 52 percent of pregnant women accessed the first dose of intermittent preventative treatment (IPT) but only 39 percent accessed the second dose. Between 2005-2009, there were approximately 10,000 deaths reported a year, in 2010 this reduced to 8,100 (2010 Malaria Program). Despite the increased efforts to control malaria in the last 5 years, the number of malaria cases reported continues to be high50. However in 2010 only 46 percent of cases diagnosed with malaria were confirmed by microscopy and rapid tests. Increased ITN coverage will also have contributed to decreased maternal and under five mortality, however despite it being part of the original design the project did not fund these activities51 Outcome indicator #7: Institutional mortality due to malaria among children under five years – Achieved 18. The 2005 baseline was 53 percent, with a target of 15 percent by 2010. This target was reached with just 9.9 percent of deaths amongst under fives reported in 2009. Malaria remains the main contributing factor to under five mortality. Outcome indicator #8: Institutional maternal mortality due to malaria – Achieved 19. A baseline of 57 percent maternal mortality due to malaria is very high. The trend data suggests there has been a significant reduction in institutional maternal mortality down to 3.2 percent in 2009. While the data is questionable there has no doubt been a 46 Transmission varies along three geographical areas (i) northern provinces high transmission (ii) central provinces where transmission is moderate and stable and (iii) the south where transmission is moderate and unstable. 47 Ministry of Health. 2007. Report of Survey on Nutrition in Angola. 48 World Bank. 2011. World Development Indicators Databank. http://data.worldbank.org/indicator/ 49 The National Treatment Policy is 100% ACT. 50 Reasons for this are: increased access by the population to health services, poor diagnostics which favors diagnosis of fever as malaria, weak service level data, high availability and subsequent high use of Coartem (2010 PNCM report) . 51 The original PAD included support to the social marketing of bed nets through the community component, but this did not go ahead. ITN ownership has shown good progress, but reported ITN use particularly amongst under fives is very low (16.4% MICS Dec 09). 26 series of improvements as a result of a combination of factors such as the introduction of Coartem, but also training, laboratory support and improved diagnosis (rapid tests) accompanied by ITNs and improvements in sanitation.52 IV. INTERMEDIATE OUTPUT LEVEL INDICATORS BY COMPONENT 20. While 5 out of the 8 (62%) of the PDO targets were achieved, 7 of the 13 (54%) intermediate output targets were achieved and 3 were not achieved, a further 2 have no data. Please refer to the results chain (Section V.) for an overview of the groupings of the service outputs indicators by component and their links to the coverage outcomes. Please note that activities supported under the different components may have contributed to the same service delivery output achievements (which is why they may appear twice below). The indicator numbers have been kept the same as in the Results Chain for ease of reference. Component 1: Public Sector Table 1: Baseline data, 2010 targets, midterm target changes, and end of project service delivery outputs for component 1 Output Indicators Baseline Target Mid End of Notes 2010 Term project 3. Number of No data 300,000 462,680 Achieved (INLS data). people being tested 28,826 people were for HIV tested for HIV through the public sector (p 78), 55,000 through the community sector (p91) totals 83,826 6. The number of 2.6m 200m x Not achieved. 2.9million condoms distributed condoms were through the public distributed by the project sector and NGO through the public sector programs (million) (p77), plus 443,694 by the private sector 7. Number of 0 8 7 Not achieved. participating line ministries which have and are implementing their respective HIV/AIDS 52 In 2005 there were approximately 4m (not confirmed) cases of malaria, with just 2.5m actual notifications. In 2008 there were 3.7m accurate fever cases, but the current diagnostic capacity is around 30% suggesting about 1m real cases a year. 27 workplans 8. Number of 0 6 8 Achieved decentralized government entities that are implementing integrated HIV/AIDS workplans This component received US$ 2.9m or 14% of funding53. 4 indicators (2 achieved, 2 not achieved) Component 2: Health Sector Table 2: Baseline data, 2010 targets, midterm target changes and end of project service delivery outputs for component 2 Results Indicators Baseli Targe Mid End Notes ne t Ter of 2010 m proje ct Number of VCT centers 35 200 547 Achieved. INLS baseline functioning per year and end line Number of counselors trained No 400 X Not Achieved. 249 out of per year data 340 target reached in 2009. No data on 2010. Also not clear if target it cumulative. Number of people being No 300,0 462,6 Achieved (INLS 2010 tested for HIV data 00 80 report) Number of sentinel 26 180 44 36 Achieved. It was agreed surveillance sites in antenatal 26 sites would provide a clinics functioning per year sufficient sample, yet target revised to 44. Number of health facilities 8 160 150 200 Achieved providing HV infected pregnant women and their new born with Mother to child transmission intervention services, per province 10. Number o f staff 105 x 700 Achieved. Baseline from trained in DOTS in each TB program. In 2010 priority municipality approximately 700 staff were trained in DOTS. 53 The spend figures used are the revised approved budget at the time of restructuring. There is no spend data by component available on the project. 28 Results Indicators Baseli Targe Mid End Notes ne t Ter of 2010 m proje ct 11. Number of priority No 59 52 Not achieved. 52 priority municipalities that have data municipalities were laboratories with adequate covered in 2009. equipment and trained technicians to do sputum smear microscopy 12. Number of Malaria No 55 450 Achieved. In total 450 Control Program coordinators data health professionals were trained in operational trained on Malaria by the management, budgeting and project (p53 & Malaria supervision. Program 2010) 13. Number of trainers No 50 1007 Achieved. In 2010 the trained in malaria case data malaria program trained management per province. 1007 health professionals (p53) Additional Indicators 14. % of children under five 24 60 16.4 Not achieved. 08/09 who slept under an ITN the MICS. previous night 15. % of households with at 22 80 66.7 Not achieved. 08/09 least one ITN MICS This component received US$ 8.8m 41% of funding. 11 Indicators (7 achieved, 2 not achieved, 2 no data).Two additional indicators (14-15) were proposed at the time of the MTR. Although the project is not being assessed against these they are still relevant and are reported for information below: Additional output indicator 14: % of children under five who slept under an ITN the previous night 21. The first KAP conducted by HAMSET in 2007 reported 24 percent of children sleeping under an ITN the previous night. The end line data from the MICS from 08/09 showed 16.4 percent of under fives. The drop from the baseline may be as a result of the different survey methods and samples sizes of LQAS, and MICS, with LQAS being less precise. A 2010 MIS is currently being analyzed and is due to be made public in a couple of months time. This should allow a better comparison with the 2006 MIS. Additional output indicator 15: % of households with at least one ITN 22. There has been good progress on ITN ownership from 22 percent in 2006 to 66.7 percent in 2009, despite this the target was not reached54. 54 Approximately 95 % of all LLINs in Angola are procured by UNICEF. UNICEF is the procurement agent for PMI, and is also the sub recipient of the GFATM Rd 7 Malaria grant 29 Component 3: Community Sector Table 3. Baseline data, 2010 targets, midterm target changes and end of project service delivery data for component 3 Results Indicators Baseline Target Mid End of 2010 Term project 3. Number of people No data 300,000 462,680 Achieved (INLS 2010 being tested for HIV data). 28,826 people were tested for HIV through the public sector (p 78), 55,000 through the community sector (p91) totals 83,826 6. The number of 2.6m 200m x No end of project target - condoms distributed Not achieved. 2.9million through the public condoms were distributed sector and NGO by the project through the programs (million) public sector (p77), plus 443,694 by the private sector. 9. Amount of funds No data 3 m No progress data channeled to civil available society initiatives per year (US$million). This component received US$ 4.3 m, or 20 % of funding. 3 Indicators (1 achieved, 1 not achieved, 1 no data) Component 4: Project Coordination 23. Baseline data, 2010 targets, midterm target changes and end of project outputs for component 4 There were no indicators or targets for this component. The component aimed to (i) support the establishment of Provincial AIDS and Endemic Diseases Commissions; (ii) contract professionals for the Project Coordinating Unit (PCU); (ii) contract professional services including financial and procurement management; and (iii) develop information systems to monitor and evaluate progress in controlling the epidemics. End Note. The project M&E mechanism that was set up included the collection of the input and output level information instead of focusing on the above intermediate output indicators. As a result there was little ongoing monitoring of the above indicators, instead the project focused on measuring progress in the following way:  Public Sector results were measured using 27 activity level indicators,  Private sector results were measured using 10 indicators (similar to public sector)  Health sector results were measured by comparing the number of activities funded to those in the respective program work plans. 30  Community sector results were measured using 16 activity level indicators, but these focused only on HIV and did not capture the support to TB and Malaria. V. HAMSET Results Chain – Showing links between Inputs, Outputs and Outcomes by component I Inputs by Component II Capacity Outputs III Service Delivery IV Coverage Outcomes Outputs Strengthened capacity of government Reduced impact of HIV, Malaria and TB on Public Sector Response (USD) ministries and local government staff and external clients agencies to tackle AIDS, TB and Malaria in the workplace 8 PS line Ministries and 5 Provincial Directorates 6. Number of condoms distributed through the public Establish workplace programs on AIDS 1: To reduce the spread of HIV/AIDS in the funded (for approx 18-24 months) PDO Angolan population through a multisectoral - - Office set up costs sector - X Ministries and X provincial Offices funded (for + - Provision of Educational materials HIV, TB and 7. Number of participating line ministries which have approach that strengthens institutional 9months) - Malaria ] and are implementing their respective HIV/AIDS capacity and increases access and utilization 7 cars provided to X and Y in (2010 ?) - Training and Awareness raising amongst staff workplans of quality health services, Supply of office equipment and clients 8. Number of decentralized government entities that Funding -of annual operational plans - are implementing integrated HIV/AIDS workplans Private Sector Support : 3. Number of people being tested for HIV - #1 HIV/AIDS prevalence in pregnant women aged 15- 49 Increased prevention and treatment of TB, Malaria #2 Percentage of young people aged 15-24 who can Health Sector Response (USD) Strengthened MoH capacity for and HIV both correctly identify ways of preventing the sexual prevention and treatment of TB, HIV transmission of HIV and reject major misconceptions Malaria and HIV , 1.Number o f VCT centers functioning per year about HIV transmission and prevention TB, Malaria, AIDS, Blood Bank, National P 2. Number o f counselors trained per year # 3 Percentage of men and women aged 15-49 - Health Lab, and Armed Forces Programs 3. Number of people being tested for HIV - reporting the use o f a condom in their last act of - funded (for approx 1-2 years) Support to elaborate Strategic Plans, and 4. Number of sentinel surveillance sites in antenatal clinics functioning * sexual intercourse with a non-regular sexual partner establish national norms and guidelines per year during the previous 12 months Funding of Operational plans Delivery of training and workshops 5. Number of facilities providing HIV infected pregnant women and their - TA, to support the programs & conduct studies Strengthening M&E and Surveillance new born with Mother to child transmission intervention services per Printing of IEC material, Guidelines Funding of Operational Research (Malaria) province - Purchase of lab equipment IEC Communication support TB Provision of HIV test kits Diagnostics and Treatment, plus 10. Number of staff trained in DOTS in each priority municipality X microscopes for Malaria Community DOTS and Community 11. Number of priority municipalities that have lab-oratories with X microscopes for TB Integrated management Malaria) adequate equipment and trained tech-nicians to do sputum smear - microscopy Purchase TB drugs [ Malaria 8 4WD cars in 2007 6 Cars provided to X and Y (2011) 12. Number o f Malaria Control Program coordinators trained in PDO 2: Strengthen the capacity of health services 30 Motorbikes provided to the TB program (2007) operational management, budgeting and supervision. to detect new cases of TB, improve treatment 18 Motorbikes for Malaria 13. Number of trainers trained in malaria case management per compliance and increase the completion rate Provision of X mobile VCT vans (2011) province. 14. % of children under five who slept under an ITN the previous night 15. % of households with at least one ITN # 4 Proportion o f the 59 priority municipalities that have implemented TB control activities # 5 TB case detection # 6 TB success Rate - - - * Community Response Community Response (USD) - Capacity strengthening of local NGOS Increased coverage of HIV, TB and Malaria prevention and (USD) treatment services in the community - 25 NGO/FBOs contracted/funded (for 2 approx years) Large Umbrella NGOs strengthen 3. Number of people being tested for HIV 9 NGOs given a 9 month extension from 2010-11 capacity of 5 local NGOs each 6.Number of condoms distributed through NGO programs Funding of NGO workplans Training and Supervision 9. Amount of funds channeled to civil society initiatives per year PDO 3: Strengthen capacity of MoH for effective 1 Car provided to JAR -- Peer education (USD million) case management of Malaria 1 mobile VCT unit - - * # 7 Institutional mortality due to malaria among Project -Coordination (USD) * Improved Monitoring, Coordination children - and Management # 8 Institutional mortality due to malaria - X no of HAMSET posts contracted/funded for 5 Support to National Coordination mechanisms (Govt and UN) years -- Participation in National Forums on HIV, TB and Malaria 1 Car provided to for Project Coordination Support to the development of National Strategic Plans and M&E Hire of PCU office space Development of HAMSET work plans - Budgeting, Financial Office equipment, running and maintenance costs Management and Procurement Other costs Ongoing M&E and review of progress against HAMSET targets, Annual Reporting, MTR and End of Project report 31 Annex 3. Economic and Financial Analysis Comparison of planned versus actual resource allocation (a) Costs by Category Original Budget Actual Final (PAD) Spend* Works 80,527 1,200,000 Goods 4,390,164 4,500,000 Training 5,080,457 3,200,000 Services 4,915,565 5,200,000 Subprojects 3,137,240 6,800,000 Operations 821,505 100,000 PPF 590,568 Total 19,016,026 21,000,000 *Final Spend as received from the Task Team on 7th November (data from project portal) Comparison of planned vs actual resource allocation by category 8,000,000 7,000,000 6,000,000 5,000,000 4,000,000 Original Budget 3,000,000 2,000,000 Actual Spend 1,000,000 - A comparison of the budget and actual allocation by category shows a large increase in spending to Training (US$ 3.2m – 4.9m) which was accompanied by a significant reduction in allocation to Subprojects (US$ 6.8m - 3.1m). Due to some of the disbursement and implementation constraints the support to sub projects reduced by half from 32 percent of the original budget to 16.5 percent of the actual spend. This in turn allowed the project to increase its support to the Health component which contributed towards the increased funding of training activities from 15.2 percent of the original budget to 26.7percent of the final spend. 32 (b) Costs by component (million) Component Original % Revised % Final Spend Budget Budget June 2011* % (PAD) (June 10) Public Sector 2 9.5 2.9 14 Health Sector 7.2 34. 8.8 41 3 Community 7 33. 4.3 20 Sector 3 Private Sector 0 0 0.4 2 Project 4.8 20 4.9 23 Coordination Total 21 10 21.3 10 0 0 *Final Spend by component is not yet available at the time of writing this report. Comparison of Original budget and Revised Budget by component US$ million (June 2010) 10 9 8 7 6 Original budget 5 (PAD) 4 3 Revsied Budget 2 (June 10) 1 0 The table and graph above show that in the initial budget (PAD) largest proportion of funds is allocated to the Health sector component (34.3%), followed by the Community sector (33%), Project coordination (20%) and the public sector (9.5%). In June 2010 the project was restructured and the allocations to each component were changed. The main difference is the decreased support to the community sector and increased support to the public and health sectors. (c) Costs by Disease Area The project has three development Objectives, one on HIV, one on TB and another on malaria. Budget allocation broken down by disease is not available at the time of writing this report 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Lawrence Barat Consultant SASHD Maria C. Correia Sector Manager SASDS Jean J. De St Antoine Lead Operations Officer AFTHE Mary Green Temporary AFTED Christy Hanson Consultant HDNHE Isabella Micali Drossos Senior Counsel LEGES Nadeem Mohammad Senior Operations Officer OPCRX Francesco Sarno Consultant AFTEN Iraj Talai Consultant HDNHE Katherine Anne Tulenko Public Health Spec. AFTHE Morag N. Van Praag Senior Finance Officer CTRDM Supervision/ICR Slaheddine Ben-Halima Consultant MNAPR Eduardo Brito Senior Counsel LEGAF Antonio L. Chamuco Senior Procurement Specialist AFTPC Humberto Albino Cossa Sr Health Spec. AFTHE Inguna Dobraja Advisor to Executive Director EDS20 Joao Carlos Duarte Consultant AFTHE Pacheco Blasques d Isabel Duarte A. Junior Program Assistant EASIN Elisabeth Maier Consultant AFTCS Amos Martinho Malate Procurement Analyst AFTPC Esteves Mbangu E T Consultant AFMAO Maria Isabel Nhassengo- Procurement Asst. AFCS2 Massingue Jonathan Nyamukapa Sr Financial Management Specia AFTFM Jenni Amanda Pajunen E T Consultant LCSPS Francesco Sarno Consultant AFTEN Monica Sawyer Country Officer AFCMZ Joao Tinga Financial Management Analyst AFTFM Joseph J. Valadez Consultant SASHD 34 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY03 22.16 FY04 269.49 FY05 153.59 FY06 0.00 FY07 0.00 FY08 0.00 Total: 445.24 Supervision/ICR FY03 0.00 FY04 0.00 FY05 70.24 FY06 129.22 FY07 97.27 FY08 123.11 Total: 419.84 35 Annex 5. Beneficiary Survey Results No Survey took place 36 Annex 6. Stakeholder Workshop Report and Results No workshop took place 37 Annex 7a. Summary of Borrower's ICR (Translated from Portuguese to English) OFFICE OF STUDIES, PLANNING, AND STATISTICS HAMSET PROJECT PROJECT COORDINATION UNIT HIV/AIDS, Malaria, and Tuberculosis Control Project IBRD Map No. 24288 DRAFT 1 RELATÓRIO FINAL SUMMARY OF FINAL PRE-APPRAISAL JUNE 2011 COODINATOR, HAMSET PROJECT ____________________________ Dr. DANIEL ANTÓNIO 38 I. THE HAMSET PROJECT This project is a grant for sectoral investment within the context of MAP,55 a program of the International Development Association (IDA) for the Africa Region, in the amount of US$ 21 million over a five-year period in alignment with the World Bank HIV/AIDS Regional Strategy ―Intensifying Action against the HIV/AIDS in Africa: Responding to a Development Crisis.‖ The project is also part of the World Bank Transition Support Strategy (TSS) for Angola56 and is aimed at assisting the government in controlling the HIV/AIDS, malaria, and tuberculosis epidemics, thus helping to reduce the devastating effects of these diseases on the Angola economy.57 The operation also supports the following Millennium Development Goals: Goal 6: Combat HIV/AIDS, malaria, and other diseases; Goal 4: Reduce child mortality; and Goal 5: Improve maternal health. With regard to Goals 4 and 5, only those aspects of maternal health and child mortality that relate directly to combating HIV/AIDS, malaria, and tuberculosis are addressed within the scope of this project. The HAMSET project, funded by the International Development Association (IDA) in the amount of US$ 21 million, was launched on October 20, 2005, and is scheduled to close on June 30, 2011. The objectives of HAMSET are to:  Reduce the spread of HIV/AIDS in the country through a multi-sector approach that strengthens institutional capacity and increases access to and utilization of health services for prevention, diagnosis, care, and support;  Strengthen health sector capacity to reduce the incidence of tuberculosis, improve treatment compliance, and increase the rate of treatment completion;  Strengthen Ministry of Health (MOH) capacity for effective malaria case management. The project focuses on six provinces in Angola—namely: Luanda, Cabinda, Benguela, Cuando Cubango, Cunene, and Lunda Sul—although some of its activities are implemented nation-wide. Closure of the project was extended from June 30, 2010, to June 30, 2011, to allow for utilization of the entire grant and to give the Angola Government time to wind down the project‘s activities. 55 Multi-Country HIV/AIDS Program for Africa, with activities in 28 countries. 56 The TSS includes three pillars: (i) increasing the transparency, efficiency, and credibility of public resource management; (ii) expanding service delivery to war-affected groups affected and other vulnerable groups; and (iii) preparing the ground for broad-based pro-poor economic growth. 57 HIV/AIDS, tuberculosis, and malaria were responsible for 75% of all deaths from infectious diseases in Angola. Studies have shown that an HIV/AIDS epidemic can reduce GNP by about 1.0% a year, while malaria has been estimated to reduce GNP by about 1.3 % a year [footnote].2 39 Project components: The project has the following components: Public Sector Response to HIV/AIDS This component provides funding for: (1) training personnel in the participating sectoral ministries and government agencies at the national, provincial, and municipal levels; (2) assisting the participating line ministries and local government agencies in reducing the impact of HIV/AIDS, malaria, and tuberculosis on their staff and dependent families at all levels; and (3) supporting line ministries in reducing the impact of AIDS, malaria, and tuberculosis on their external clients. Health Sector Response to HIV/AIDS, Malaria, and Tuberculosis This component provides funding to the MOH and the Armed Forces for the prevention and treatment of tuberculosis and malaria, as well as for HIV/AIDS-related activities under the MOH such as training of health human resources, voluntary testing and counseling (VTC) services, interruption of mother-to-child transmission, management of sexually transmitted diseases, and treatment of HIV/AIDS opportunistic diseases. Community Response This component provides funding and technical assistance to civil society organizations (CSOs), nongovernmental organizations (NGOs), faith-based communities and organizations (FBOs) that present subprojects for preventing and mitigating the effects of HIV/AIDS, malaria, and tuberculosis. These include preventive activities for reducing HIV/AIDS-related stigma and discrimination through information, education, and communication (IEC) campaigns, promotion of safe sexual behavior and adequate condom social marketing (CSM), advocacy for defense of the human and legal rights of persons living with HIV (PLHIV), promotion of malaria prevention and distribution of mosquito nets, training for civil society organizations, and support for the expansion of directly observed treatment, short course (DOTS) for tuberculosis patients. Project Coordination This component is intended to support: (1) hiring professionals to work with the Project Coordination Unit; (2) contracting professional services for financial and procurement management; (3) developing an information system to monitor and evaluate progress in controlling the three endemics; and (4) supporting the implementation of Provincial Commissions on Combating AIDS and Major Endemic Diseases. Financial Resources Provided by the Project The funding provided for five years amounted to US$ 24,700,000 (twenty-four million seven hundred thousand United States dollars), of which US$ 3,700,000 (three million seven hundred thousand dollars) was contributed by the Angola Government. The breakdown of funds allocated to the project components was as follows: 40 Table 1. HAMSET project budget, by components Amount Amount initially allocated after Component allocated conversion to (US$) quoted SDR value (US$) Project Coordination 5,363,154 5,363,154 Health Sector Response (INLS, Malaria, 9,149,920 9,617,045 Tuberculosis, Armed Forces) Public Sector Response 2,617,045 2,617,045 Community Response (NGOs, CBOs, 7,569,881 7,569,881 FBOs) Total 24,700,000 25,167,125 World Bank **21,000,000 21,397,151 Angola Government 3,700,000 3,769,974 Source: Governo de Angola, Manual de Operações do ANGOLA HAMSET, 2005. **Because of the depreciated SDR value,58 the US$ 21 million allocated to HAMSET in 2005 currently corresponds to US$ 25 million. In the tables on the project‘s execution phase, the SDR value applied is the value quoted on the date of performance, hence the differences in the figures reported. The cumulative variation is 12.268%. II. PURPOSE OF THE APPRAISAL The Project Coordination Unit (UPC) conducted the final pre-appraisal of the HAMSET Project over a three-week period in August 2011 with the participation of Dr. Paula Figueiredo, a national consultant hired for the purpose. The main objective of this evaluation was to facilitate an analysis of the progress made toward attaining the project‘s stated objectives, as well as the relevance of these objectives vis-à-vis the results achieved by the project. The methodology was an open, participative approach, and the instrument was based on a protocol that combined several different means of collecting data and analyzing the information available. The performance appraisal, which included integrating and analyzing the information obtained from the UCP, monitoring and evaluation, procurement and financing records, and the public and private health sectors, was done by the consultant, working together with the UCUP team. 58 Special Drawing Rights (SDR) refers to an international reserve asset created by the IMF. 41 The results of this appraisal were consolidated in a basic document, which was used by the Government in discussions of the HAMSET Final Appraisal conducted by the World Bank last August. III. Relevance of the results achieved to the objectives of the HAMSET project and to its contribution to the strategic national plans of the HIV/AIDS, malaria, and tuberculosis programs The HAMSET project was the World Bank‘s first and primary commitment in the Angola health sector to support the fight against the epidemics of AIDS, malaria, and tuberculosis. During the evaluation of the project, government and development partners recognized that HAMSET needed to respond to emerging needs, despite limited access to many regions, land mines, weak program management capacity, and the crying need to strengthen institutional capacity and rebuild partnerships. As we saw in the descriptions of the components, the project‘s contribution toward strengthening the capacity of national programs in the implementation of national strategic plans has been very positive. The activities provided for in the health action plan are consistent with the national strategies for HIV/AIDS, malaria, and tuberculosis. All the components of the national strategic plans have benefited, and priority has been given to operational research, mobilization of partnerships, diagnosis and treatment, and institutional strengthening (Figure 1). Figure 1. HAMSET health component support for national malaria, AIDS, and tuberculosis plans59 Malária SIDA Tuberculose 95 90.9 77.8 78.6 77.8 69.6 70 61.9 61.7 51.7 46.2 35.0 reforço Diagnostico e pesquisa e advocacia, IEC e institucional tratamento vigilancia parcerias epidemiologica 59 See end note for translation of text contained in Graphs 42 The average overall HAMSET performance in health sector activities was 63%. The performance of the Malaria Program was best, at 68.2% (Figure 2). Figure 2. Average performance of activities under the health sector programs 70.0 68.0 66.0 68.2 64.0 61.4 62.0 59.2 60.0 58.0 56.0 54.0 malaria sida tuberculose  HAMSET financing, with support for training, enabled the National Institute for the Fight Against AIDS (INLS) to expand its voluntary counseling and testing (AVT) programs, as well as those for the prevention of mother-to-child transmission (PMTCT).  The project also supported the expansion of PMTCT to an additional five provinces: Namibe, Cuanza Sul, Malange, Cuanza Norte, and Bengo. About 52 service providers have familiarized their personnel with techniques for voluntary counseling and testing in general hospitals and prenatal clinics in seven provinces: Lunda Sul, Cuando-Cubango, Zaire, Uige, Malange, Cuanza Norte, and Cuanza Sul.  HAMSET supported creation of the National Malaria Control Forum in April 2007, which brings together more than 40 partners of the Ministry of Health and its National Malaria Control Program (PNCM) that are engaged in the financing, planning, and implementation of measures to combat malaria in Angola. These partners include United Nations agencies such as UNICEF, the World Bank/HAMSET, and the World Health Organization (WHO), as well as the Global Fund, the President‘s Malaria Initiative (PMI), the Japan International Cooperation Fund (JICA), the Armed Forces, oil companies and other partners, national and international nongovernmental organizations (NGOs), churches, professional associations, community organizations, and enterprises in the public and private sector.  The Armed Forces Health Service received microscopes and reagents that have made it possible to improve diagnostic capacity for malaria, as well as tuberculosis, syphilis, and HIV (Table 33). It also received support in the training of peer 43 educators, which is important for the prevention of these endemic diseases among the military. The Armed Forces also provided specific training and orientation on voluntary counseling and testing activities within military units.  Of special note is the project‘s contribution to improving the tuberculosis program‘s medicine supply and management system, with logistic training support provided to 75 technicians in the country‘s 18 provinces, as well as technical supervision and training in the provinces of Zaire, Cuanza Norte, Cunene, Namibe, Huila, and Benguela.  In general, we can say that the HAMSET Project focused on providing support to the Institute for the Fight Against AIDS for implementation of the National Strategic Plan (PEN). In all the components, activities were for the most part directly related to HIV/AIDS (see Annex 1: Summary Matrix of PEN III Performance).  Through the public, private, and community sectors, HAMSET has supported the promotion of a multi-sector approach and the involvement of a variety of stakeholders and sectors of society, especially in HIV/AIDS-related activities (Table 2).  All the public sectors identified in the National AIDS Strategy as being key to expanding the response to HIV/AIDS, malaria, and tuberculosis have been involved in the project—specifically: agriculture and rural development, education, youth and family, women‘s development, civil administration, and internal security (police).  Advocacy for the prevention of stigma and discrimination was a priority in all the strategies. Other priorities have included, inter alia: preventive biosafety (Ministry of Health) and positive empowerment of seropositive women, counseling, and human rights (Ministry of Family and Promotion of Women). Table 2. Public and private sector contributions toward the targets of the National Strategic Program III (PEN-III) Actions taken under HAMSET HAMSET the National Strategic Target for 2010 Contribution Contribution Total % Plan-LS (private) (public) 3.1. Drafting and 60 company members of implementation of the Business Committee to 35 0 35 58.3 projects at work sites Fight AIDS 3.2. Identification of focal points in the 60 25 13 ... Ministries and companies 44 3.3.Drafting of strategic ... 2 7 9 ... plans in the Ministries 3.4. Distributing condoms at work sites 760,000 443,694 2,944,638 3,388,332 763.7 (10% of total provided for) 3.5. Drafting and production of 60 companies (100%) with IEC/community IEC/MC materials 45 13 58 96.7 mobilization (MC) produced and disseminated materials for work sites 3.6. Training of workers 60 enterprises (100%) with to serve as peer 45 13 58 96.7 trained peer educators educators 3.7. Promotion of 20 enterprises and/or implementation of VCT ministries with operational 7 7 35.0 centers at work sites VCT centers IV. CONCLUSIONS The following main conclusions can be drawn from the appraisal: 1. The participation and engagement of key partners was decisive in achieving the project‘s objectives. 2. Sustainability and the possibility of replicating and applying the subprojects in additional institutions (in the public, private, and other sectors) is ensured because: a. Awareness has been raised among authorities in the Ministries involved, and the subprojects have space and personnel already designated to carry out the activities; b. The peer educators are officials in the Ministries; c. A majority of the Ministries and provincial governments have already allocated funds for HIV/AIDS-related activities; d. The peer educator strategy is having an impact through the acceptance and ease of spreading messages within the work site. 3. In designing the HAMSET project, consideration was not given to the management modalities and capacities of the national programs. According to the statutes of the National Directorate of Public Health (DNSP), the national programs are considered only technical in nature and financial management is the responsibility of the secretary-general of the DNSP. 45 The programs did not have enough experience or personnel to enable them to meet World Bank standards for requesting and justifying the use of funds. 4. The way in which procurement was managed under the HAMSET project was a major bottleneck for the Program Coordination Unit. Difficulty in recruiting and retaining procurement specialists and the lack of local capacity were basic factors contributing to delays in implementing the project, especially in the case of large purchases. 5. HAMSET absorbed about 88% of the total funds allocated in the program budget, which means that approximately US$ 3,045,366 (three million forty-five thousand dollars) failed to be disbursed. 6. The World Bank disbursed 93.43% of the programmed amount, leaving a total of approximately US$ 1,500,000 (one million five hundred thousand dollars) in undisbursed IDA funds as of June 30, 2011. 7. The Angola Government spent 55% of its programmed amount. The Government‘s total debt to its partners and suppliers as of June 30, 2011, was estimated at AKZ 64,085,448, equivalent to US$ 685,405 (six hundred eighty- five thousand four hundred and five US dollars). 8. The various missions to evaluate financial management of the project concluded that the audits would be acceptable to the World Bank. V. LESSONS LEARNED 1. The coordination, communication, and sharing of information between the principal implementers and beneficiaries of the project are crucial to the success of its activities, techniques, financial planning, and procurement. 2. Training in the standards and procedures of procurement and financial management for the component coordinators and partners is indispensable in order to improve the capacity to render accounts. 3. The formation of national teams associated with the Project, especially in the area of procurement, is fundamental to having a consistent base for the procurement process and to strengthening this capacity, which will be of benefit to the country in the future. 4. Monitoring and evaluation tools for financial management and procurement for use by the UCP and subprojects should be identified from the outset and kept up to date in the database itself throughout the life of the project, which would make it possible to compare indicators over the course of this period. 46 Ends Note: Translation of Text in Graphs Portuguese English malária malaria SIDA AIDS tuberculose tuberculosis Reforço institucional Institutional support Diagnostico e tratamento Diagnosis and treatment Pesquisa e vigilancia epidemiologica Research and epidemiological surveillance Advocacia, IEC and parcerias Advocacy, IEC, and partnerships 47 Annex 7b. Summary of Borrower's Comments on ICR REPÚBLICA DE ANGOLA MINISTÉRIO DA SAÚDE COMENTARY ON THE IMPLEMENTATION COMPLETION RESULTS REPORT (IDA-H1400) Relatório Nº: ICR00002094 December 5, 2011 We noted that under the Basic Information Data Sheet there was no mention of the Government Contribution to the Project. We added a row (highlighted in yellow) indicating the amount in SDR of the Government Contribution and the amount disbursed. A. Informações Básicas Controlo do VIH/SIDA, Malária e País: Angola Nome do Projecto: Tuberculose (HAMSET) Identidade do Número(s) do P083180 IDA-H1400 Projecto: E/C/TF : Data do ICR: 09/19/2011 Tipo de ICR: Core ICR Instrumento de GOVERNO DE SIL Mutuário: Empréstimo: ANGOLA Compromisso Montante XDR 14,10M XDR 13,27M Original IDA: Desembolsado: Montante Revisto: XDR 14,10M Compromisso Montante XDR 2,50M XDR 1,6M Original GOA: Desembolsado: Categoria Ambiental: B Agências de Execução: Ministério da Saúde Co-financiadores e Outros Parceiros Externos: 6.6 Revised Components We noted that there was no mention of the establishment of the private sector component. We suggest that it should be mentioned that after the Mid Term Review, a Private Sector Component was established. Previously the activities of the private sector were under the responsibility of the Deputy Coordinator but it became clear that due to the work load of 48 the deputy coordinator little attention had then been given to this sector. A new coordinator for the private sector response component was subsequently recruited. 7. 2 Factors that contributed to successful implementation or gave rise to problems, included: We propose the following text under the section below: High turnover of PCU staff and implications on procurement. The PCU was headed by a Coordinator assisted by a Deputy Project Coordinator who was supported by four coordinators, one for each component (Health, Community, Public sector, and Private Sector) plus specialists for M&E, Procurement and Finance. In Annex 2, there should be a short description of the Private sector component as follows: Component 5: Private Sector Component This component was established after the MTR. It implemented activities manly in the area of advocacy, training of peer educators in the construction industry and funded a sub project for truckers in the province of Benguela. A growing number of private construction companies showed interest in promoting education campaigns for their workers. Bank Team Response In relation to the Private Sector component it should be noted that very few activities were implemented with limited impact. The Coordinator of this component was not pro- active and he ended up resigning before the closing date of the project. Efforts were made by the Project Coordinator assisted by an assistant but not much was achieved. In reality most of activities were limited to training of peer educators and funding of a subproject for truckers in Benguela Province. Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders No cofinanciers 49 Annex 9. List of Supporting Documents Global Fund for AIDS, TB and Malaria (GFATM). Angola Grant Portfolio, Website. Approved Grants for HIV, TB and Malaria in Angola. Instituto Nacional de Estatistica (INE). 2002. Resultados dos indicadores Multiplo, MICS2,2001, Folha de InformapLo rapida, Macro International. 2007. Angola Malaria Indicator Survey (MIS) 2006-2007 Ministry of Health. 2002. Plano Estrategico Nacional de Saude Reprodutiva, Luanda Ministry of Health. 2007. Inquerito sobre Conhecimentos, Atitudes e Practicas sobre Malaria, ITS e VIH/AIDS na Populacao com 15-49 anos (CAP/KAP) Ministry of Health. 2007. Report of Survey on Nutrition in Angola. Ministry of Health. 2008a. Rervisao de Meio Termo – Programa de Controlo do Apoio ao HIV/SIDA, Malaria e Tuberculose. Mid term report (HAMSET) Ministry of Health. 2008b. Plano de Desenvolvimento Económico e Social de Médio Prazo, 2009-2013 Ministry of Health. 2009a. Inquerito Integrado Sobre o Bem Estar da Populacao IBEP 2008/09. Household Expenditure Survey Ministry of Health. 2009b. Inquerito sobre Conhecimentos, Atitudes e Practicas sobre Malaria, ITS e VIH/AIDS na Populacao com 15-49 anos. (CAP/KAP) Ministy of Health. 2010a. Consolidated report on the evaluation of HAMSET sub projects. Independent Consultant. Ministry of Health. 2010b. National Malaria Program Report. January-December 2010 Ministry of Health. 2010c. National Tuberculosis Program Report. National Health Accounts. 2009. Angola National Health Accounts Report. Ministry of Health. 2010d. Avaliacao Final – Programa de Controlo do Apoio ao HIV/SIDA, Malaria e Tuberculose. Final Evaluation Report (HAMSET) Ministry of Health. 2010e. National Strategic Plan to Control STI‘s HIV and AIDS 2011-14. Instituto Nacional de Luta Contra o SIDA (INLS) Ministry of Health. 2010f. Resumo da Avaliacao Final – Programa de Controlo do Apoio ao HIV/SIDA, Malaria e Tuberculose. Summary Evaluation Report (HAMSET) 50 Ministry of Health. Various. Relatorio Annual – Programa de Controlo do Apoio ao HIV/SIDA, Malaria e Tuberculose. Annual Project Reports - Jan-Dec 2008, Jan-Dec 2009, Jan-Sept 2010, Jan-Dec 2011. UNICEF. 2010. UNICEF Angola Annual Report 2010. A better Angola for ALL children World Bank. 2003. Transitional Support Strategy (TSS) for AngolaWorld Bank. 2004. Project Appraisal Document: Angola HIV/AIDS, Malaria and Tuberculosis Control project (HAMSET). World Bank. 2004. Emergency Multi-Sector Rehabilitation Project 2005-2011 (EMRP1). World Bank. 2004. Bank‗s Interim Strategy Note (ISN) for 2005-06. World Bank. 2005. Development Grant Agreement: Washington DC World Bank. World Bank. 2005. World Bank‗s Africa Action Plan for 2006-08. World Bank. 2006 Bank‗s Interim Strategy Note (ISN) for 2007-2009 World Bank. 2010. Project Appraisal Document: Angola Municipal Health Sector Services Project (MHSS). World Bank 2010. Restructuring Paper on a proposed project restructuring of HIV/AIDS, Malaria and TB control Project (HAMSET). World Bank. 2011. World Development Indicators Databank World Bank. Various. Aide Memoires from project Supervision Missions dating from September 2004, until December 2011. World Health Organization. 2011. Global Tuberculosis Control. 51 IBRD 33361 15°E 20°E GABON A N GOL A SELECTED CITIES AND TOWNS PROVINCE CAPITALS ANGOLA NATIONAL CAPITAL C ON G O RIVERS MAIN ROADS RAILROADS PROVINCE BOUNDARIES 5°S 5°S INTERNATIONAL BOUNDARIES CABINDA To To Mbanza-Ngungu Kimpese Cabinda Soyo Congo M’banza Congo M’banza DEMO CRAT IC 0 40 80 120 160 200 Kilometers ZAIRE REPUBLIC 0 40 80 120 Miles U�GE O F CO NGO To Tshikapa Bembe N'zeto To Kikwit Uíge Uíge Chicapa Ambriz Camabatela Caxito Cua D an de LUNDA Lucapa ng CUANZA o LUANDA NORTE NOR TE NORTE LUANDA N’dalatando N’dalatando CASSA Malanje Saurimo C u a nza BENGO MALANJE 10°S 10°S Cacolo LUNDA UL SMuconda CUANZA To Kolwezi Porto Porto Amboim Luau C uba SUL ATLANTIC Sumbe l e ez To mb Solwezi OCEAN Luena Za Camacupa HUAMBO Lobito B í e Plateau Bíe Kuito Lucusse Benguela Lumbala Huambo BIÉ BENGUELA MOXICO To Lusaka Chitembo Caconda Lumbala N'guimbo Cubango HU�LA Matala Menongue Lubango 15°S Huíla Plateau 15°S Namibe Chibia Cuí Cuíto Cuíto Cuanavale ZAM B I A Cu an Tombua Tombua Mavinga Chiange do NAMIBE CUANDO N a m i b CUNENE CUBANGO ne e ne Cu n une Ondjiva b b C Chitado Cuangar D e To Cubango To Otjiwarongo Tsumeb s e s s This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information NAM I B I A shown on this map do not imply, on the part of The World Bank r t Group, any judgment on the legal status of any territory, or any Etosha Pan endorsement or acceptance of such boundaries. BO T SWA N A 15°E 20°E SEPTEMBER 2004