Document of The World Bank FOR OFFICIAL USE ONLY Report No: 37558 IMPLEMENTATION COMPLETION REPORT (IDA-34440 PPFI-Q2210) ON A CREDIT IN THE AMOUNT OF SDR 31.4 MILLION (US$40 MILLION EQUIVALENT) TO THE STATE OF ERITREA FOR A HIV/AIDS, MALARIA, STD & TUBERCULOSIS (HAMSET) CONTROL PROJECT October 6, 2006 Human Development 1 Country Department 5 Africa Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective April 30 2006) Currency Unit = Eritrean Nakfa Nakfa 15.0 = US$ 1.0 US$ 1.512 = 1 SDR FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immune Deficiency Syndrome ANC Antenatal Care ARV Anti-Retroviral drugs BCC Behavior Change Communication BIDHO Eritrean Association of People Living with HIV/AIDS CCEP Community Counseling Enhancement Process CMHRP Community-Managed HAMSET Response Program CPAR Community Procurement Assessment Report CQ Chloroquine CSW Commercial Sex Worker DA Deaths Averted DCA Development Credit Agreement DDT Dichloro-Diphenyl-Trichloromethane DHS Demographic and Health Survey DOTS Directly Observed Treatment Short Course ELISA Enzyme-Linked Immuno Solvent Assay FMR Financial Monitoring Report GDP Gross Domestic Product GOE Government of Eritrea HAMSET HIV/AIDS, Malaria, STD and TB HeaLY Healthy Life Years ICB International Competitive Bidding ICR Implementation Completion Report IDA International Development Association IEC Information, Education and Communication IECD Integrated Early Childhood Development IMCI Integrated Management of Childhood Illnesses ITN Insecticide-Treated Nets. LIB Limited International Bidding LQAS Lot Quality Assurance Sampling M&E Monitoring and Evaluation MAP Multi-Country AIDS Program MIS Management Information System MOD Ministry of Defense MOI Ministry of Information MOH Ministry of Health MOLG Ministry of Local Government MOTC Ministry of Transport and Communications MTCT Mother-to-Child Transmission of HIV MTR Mid-Term Review NATCoD National HIV/AIDS/STD and TB Control Division NGO Non-Governmental Organization NMCP National Malaria Control Program NUEW National Union of Eritrean Women NUEYS National Union of Eritrean Youth and Students OPD Out-Patient Department OVC Orphans and Vulnerable Children PAD Project Appraisal Document PLWHA People Living With HIV/AIDS PMP Pesticides Management Plan PMTCT Prevention of Mother-to-Child Transmission of HIV PMU Project Management Unit QER Quality Enhancement Review RH Reproductive Health RRI Rapid Result Initiatives SP Sulphadoxine-Pyrimethamine SSA Sub-Saharan Africa STD Sexually Transmitted Diseases UNAIDS Joint United Nations Program on HIV/AIDS USAID United States Agency for International Development VCT Voluntary Counseling and Testing WHO World Health Organization Vice President: Gobind T. Nankani Country Director/Manager: Colin Bruce / James Christopher Lovelace Sector Manager: Dzingai Mutumbuka Task Team Leader/Task Manager: Son Nam Nguyen ERITREA HIV/AIDS, Malaria, STD & TB (HAMSET) Control Project CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 5 5. Major Factors Affecting Implementation and Outcome 17 6. Sustainability 18 7. Bank and Borrower Performance 19 8. Lessons Learned 21 9. Partner Comments 24 10. Additional Information 24 Annex 1. Key Performance Indicators/Log Frame Matrix 25 Annex 2. Project Costs and Financing 27 Annex 3. Economic Costs and Benefits 30 Annex 4. Bank Inputs 35 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 38 Annex 6. Ratings of Bank and Borrower Performance 39 Annex 7. List of Supporting Documents 40 Annex 8. Government Confirmation for Acceptance of HAMSET I ICR 41 Map IBRD33403 Project ID: P065713 Project Name: HIV/AIDS, Malaria, STD & TB (HAMSET) Control Project Team Leader: Son Nam Nguyen TL Unit: AFTH1 ICR Type: Core ICR Report Date: October 6, 2006 1. Project Data Name: HIV/AIDS, Malaria, STD & TB (HAMSET) L/C/TF Number: IDA-34440; PPFI-Q2210 Control Project Country/Department: ERITREA Region: Africa Regional Office Sector/subsector: Health (83%); Other social services (8%); Central government administration (7%); General education sector (2%) Theme: Population and reproductive health (P); HIV/AIDS (P); Participation and civic engagement (P); Health system performance (S); Conflict prevention and post-conflict reconstruction (S) KEY DATES Original Revised/Actual PCD: 02/24/2000 Effective: 03/01/2001 03/01/2001 Appraisal: 11/08/2000 MTR: 05/03/2004 11/22/2004 Approval: 12/18/2000 Closing: 03/15/2006 03/31/2006 Borrower/Implementing Agency: GOVERNMENT OF ERITREA/MINISTRY OF HEALTH Other Partners: STAFF Current At Appraisal Vice President: Gobind T. Nankani Callisto E. Madavo Country Director: Colin Bruce Oey Astra Meesook Sector Manager: Dzingai Mutumbuka Arvil Van Adams Team Leader at ICR: Son Nam Nguyen Eva Jarawan ICR Primary Author: Khama O. Rogo 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: S Sustainability: L Institutional Development Impact: SU Bank Performance: S Borrower Performance: HS QAG (if available) ICR Quality at Entry: S S Project at Risk at Any Time: No 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The overall objective of this project was to reduce the mortality and morbidity of the Eritrean population due to HIV/AIDS, malaria, sexually transmitted diseases and tuberculosis (HAMSET) by increasing utilization of quality, effective and efficient health services for HAMSET prevention, diagnosis and treatment, supported by healthy practices. The project would benefit all regions of Eritrea, with some interventions covering the whole populations (especially those in the reproductive age groups) while others would target specific groups. The project also included poverty targeted interventions designed to benefit the poor, especially in rural areas. The objectives and design were appropriate for Eritrea, a country whose economy had been devastated by war and drought. Eritrea, being one of the poorest countries in the world, with a current per capita income of less than US$200 per annum, has few natural resources. Most of the 4 million population live in rural areas and engage in subsistence farming as the primary means of livelihood. During the design of HAMSET, the country was emerging from a border conflict that had resulted in the mobilization of up to 20% of adults and left a third of the population without water, sanitation or shelter. Communicable diseases were rampant with 62% of the burden of disease attributed to maternal/perinatal causes, malaria, diarrhea, acute respiratory infection and tuberculosis (TB). Morbidity and mortality from preventable diseases had risen significantly as the result of the border conflict; the HIV/AIDS epidemic was already a threat and STI and TB were on the rise. Heavy rains in 1998/99 had been accompanied by a major malaria epidemic that had an impact on rural agriculture. And although the national malaria program had taken off, with support from the IDA-funded Eritrea Health Project (P043124, US$21.1 million, effective May 1998, closed December 2004) and technical assistance from USAID, additional funds were needed urgently to expand and consolidate its activities. It is in this context that HAMSET was born as a project under the Multi-Country AIDS Program (MAP) but also covered malaria and TB. Thus it was a MAP project with a difference - a multi-disease control program. 3.2 Revised Objective: The original objectives were not revised. 3.3 Original Components: The project was designed with the following four major components. A large share (47.2%) of IDA financing went to component C which supported the major health sector interventions: diagnosis, care and counseling for HAMSET. A. Collect and Analyze Information on HAMSET to Facilitate Evidence-Based Decision Making and Rapid Response: This component aimed to strengthen the government's capacity to collect comprehensive information on HAMSET in a timely and efficient manner, to analyze and use the information for planning and towards responding appropriately to changes in disease trends. That capacity would exist at the central, Zoba* (regional), and sub-Zoba levels and would allow a quick response to epidemics. Activities supported by the credit would: (i) improve HAMSET surveillance techniques; (ii) establish an epidemic forecasting and preparedness system; (iii) improve the country's capacity to carry-out operational research for identifying changes in HAMSET; (iv) introduce methods to link the results of research and M&E to policy formulation, annual planning and budgeting; and (v) strengthen management of communicable diseases at the Ministry of Health. _______________________ * Eritrea has six administrative regions or Zobas. - 2 - B. Multi-Sectoral Control of HAMSET Transmission: This component aimed to bring together all the country's institutions for communication and build on existing mechanisms to promote healthy and safe behavior. It would provide high risk groups with means to prevent HAMSET diseases. It was made up of the following sub-components: B.1. Promote healthy behaviors through multi-level communication. Activities would enable the MOH's Health Promotion Unit to coordinate the communication activities of all implementing partners and build capacity at Zoba and sub-Zoba levels. Further, it would enable Zoba-level IEC staff to: (i) conduct formative research to gain a better understanding of target audience attitudes and beliefs about benefits and barriers to adoption of desirable behaviors; (ii) develop a communication strategy and conduct communication activities to support project objectives and promote healthy behavior among target populations; (iii) coordinate and supervise the work of partner agencies at the Zoba and sub-Zoba level; and (iv) develop a system to track changes in knowledge, attitudes, beliefs, and behavior among target audiences reached by communication campaigns. Orientation of leaders of NGOs, and senior officials of other line ministries as well as national advocacy efforts would also be supported under this sub component. B.2 Promote healthy lifestyle through the education system. These activities aimed to promote good health and prevent the spread of HAMSET diseases through the Ministry of Education (MOE) school health program in both the formal and non-formal systems, by: (i) strengthening central and regional skills in school health programming; (ii) promoting in students and teachers healthy practices and behavior change; (iii) establishing school based support and health services; and (iv) promoting healthy practices and behavior change in adults. B.3 Enhance access to preventive, diagnostic and treatment services for conscripts. The sub-component focused on: (i) promotion of healthy behaviors through multiple channels of communication; (ii) strengthening health care services available to conscripts (including the availability of voluntary counseling and testing); (iii) promoting the increased use of condoms for HIV and STD prevention and insecticide treated nets for malaria control; and, (iv) establishing a program to address HAMSET concerns in the context of demobilization. B.4 Promote environmentally sound and cost-effective techniques for Malaria vector control by implementing a pesticide management plan (PMP) that would: (i) identify, test, validate and introduce safe, cost-effective chemicals to replace DDT; (ii) test and validate malaria biological vector control; (iii) develop a strategy for pesticide use and control; (iv) test community acceptance of validated methods and techniques; and (v) replicate socio-environmentally validated malaria vector control methods. C. Strengthen HAMSET Diagnostic, Health Care and Counseling Services. The objective of this component was to reduce morbidity and mortality from HAMSET by improving access to quality health services. This was to be accomplished under the following sub-components: C.1 Establish safe blood banks in Zoba hospitals. Support the establishment of four blood banks for Zoba hospitals, to complement the Health Project which was to establish two blood banks in the remaining Zobas, as well as a central blood bank in Asmara. C.2 Improve diagnostic, treatment and counseling of HAMSET through integrated in-service and on-the-job training on HAMSET prevention and detection, case management, syndromic and laboratory diagnosis of HAMSET, as well as pre- and post-HIV-voluntary counseling and testing - 3 - (VCT). The primary strategies for HAMSET management include: directly observed therapy, short-course (DOTS) for TB; rapid detection and treatment of malaria in health facilities and in the community, including IMCI for children less than 5 years of age, with laboratory confirmation when available; management of severe malaria at referral facilities; voluntary counseling and testing for HIV; management of opportunistic infections in HIV-infected persons; syndromic management of STDs, with laboratory confirmation when available. C.3 Improve availability of basic medical materials and drugs required to diagnose and treat HAMSET in health facilities. Support to procurement and distribution of basic essential drugs and diagnostic materials to treat HAMSET (only opportunistic infections in the case of HIV/AIDS), surveys to assess the availability of drugs and medical materials, in-service and on-the-job resource management training for MOH staff especially at facility level, and transport for drugs, medical materials and to provide supervision. D. Community-Managed Response Program: Through this component, the project was to identify community-managed affordable, effective mechanisms for minimizing the transmission and impact of HAMSET and have them ready for replication nationwide. The component consisted of two sub-components: D.1 Community Counseling and Support Groups: This aimed to strengthen community support services provided by the MOLHW and to provide counseling and establish support groups for HIV/AIDS affected people by (i) providing home-based care and support for orphans; (ii) community managed efforts to identify malaria, anemia, STDs and TB symptoms and signs, treat and refer; and (iii) consulting services to publish manuals workshops to improve skills and strengthen community support. D.2 Community-Managed Response: This aimed to test the capacity of the communities to use their own structures and socio-cultural fabric to: (i) respond to technical information about the HAMSET for their prevention, care and cure; (ii) organize their internal mobilization, discussion and decision mechanisms on the support they deem necessary to assess and otherwise manage the diseases; (iii) identify and input their grassroots and socio-cultural contribution to HAMSET messages, prevention, care and cure methods, and available support services; and (iv) identify, decide on and implement sub-projects to prevent or mitigate the diseases and related impacts in the community. Lessons learned during the initial phases were to be incorporated into the plans for subsequent phases. The component was to be linked with the IDA-financed Eritrean Integrated Early Childhood Development (IECD) as appropriate. The component would cover all Zobas and sub-Zobas, two in each cultural area, including semi-nomads but one Kababi (village) each in Year 1 and 2 and expand to other Kababis in Year 3 and 4. This phasing would allow for internalizing of community-based processes among field staff and community-based workers and the testing of a number of assumptions in the initial phases. E. Project Management and Evaluation: This component was to support overall project management by strengthening the existing PMU (located in the MOH) to be responsible for planning and budgeting, procurement and financial management. There would be an emphasis on evaluation and monitoring of activities. . 3.4 Revised Components: The original components were not revised. - 4 - 3.5 Quality at Entry: A quality Enhancement Review (QER) was conducted on July 5, 2000 and was enriched by the participation of the Ministry of Health, led by the Minister himself. The Panel strongly supported the objectives and approach of the project, especially the integration of TB, malaria and HIV/AIDS control, but highlighted the challenges of implementation, due to the prevailing limitations on borrower capacity and lack of knowledge of the epidemiology of the diseases. The Panel suggested that special attention be given to the following issues: · information and data gathering, including establishment of project baselines; · specific project objectives, milestones and measurements; · roles, responsibilities and accountability of various stakeholders; · communication and the need to develop a unified strategy; implementation planning and execution of activities and supervision strategy. The suggestions were very pertinent as most of the shortcomings identified by this ICR are closely linked to these issues. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The overall project objective of reducing mortality and morbidity in Eritrea due to HAMSET diseases through increases in utilization of quality, effective and efficient services for prevention, diagnosis and treatment, supported by healthy practices has been largely achieved. The overall performance of the project is therefore rated as satisfactory. The most significant achievement has been in malaria control. Malaria-related morbidity and mortality declined by as much as 60% between 2000 and 2005. In 1999, 179,501 cases of malaria were identified at health facilities compared to 16,749 cases in 2005. Malaria morbidity dropped from 55 per 1,000 population to 10 per 1,000 in 2002, with current figures putting the decrease in total morbidity at 85%. At the same time, mortality among under-five children decreased from 10.6 per 1,000 in 2001 to 0.84 per 1000 in 2004, while case fatality rate for the same group dropped from 5.97% in 1999 to 0.21% in 2005. As discussed in subsequent sections, Eritrea has been able to achieve good coverage of effective interventions for malaria control as recommended by the Roll Back Malaria Partnership. For example, Eritrea is the only Sub-Saharan African (SSA) country which was able to achieve the Abuja target for insecticide-treated nets (ITN) coverage. HAMSET supported all the major malaria interventions. HIV sero-prevalence among antenatal care (ANC) clients was 2.8% in 2001, 2.4% in 2003 and 2.38% in 2005. The 2003 surveillance data revealed that prevalence in unmarried women of the 15-24 age group was 7.5%, compared to the 2.4% overall prevalence. A similar study in 2005 confirmed a 7.2% sero-prevalence among unmarried ANC clients in urban areas. These indicate possible stabilization of the epidemic. Hospital data also showed decline in mortality rates due to HIV/AIDS from 12.9% in 1997 to 9.7% in 2002. The STI situation was more uncertain due to lack of reliable prevalence data. Facility-based statistics, however, confirmed a reducing trend in the number of cases treated. The HAMSET project also sought to increase the proportion of diagnosed and successfully treated new smear-positive TB patients and made some good progress in the implementation of the DOTS strategy. This led to decrease of the burden of disease and improvement in cure rates which reached 80% in - 5 - 2005. With HAMSET support, the country conducted a national TB prevalence survey (the first of its kind in Sub-Saharan Africa over the past 45 years) which indicated a much lower prevalence of smear positive TB than previously estimated by WHO. It should be noted that these achievements were made in a difficult macro-economic environment, the "no war, no peace" situation due to the border tension and under severe human resource constraints. HAMSET was also among the first IDA-funded projects which successfully piloted the Rapid Results Initiatives (RRIs), an implementation tool which emphasizes results in project activities. It should also be noted that besides HAMSET, the Global Fund also provided financial support for malaria and HIV/AIDS control. Significant technical assistance in HAMSET diseases came from agencies such as WHO and USAID. 4.2 Outputs by components: A. Collection and Analysis of Information on HAMSET to Facilitate Evidence-Based Decision Making: This component aimed to strengthen the government's capacity to collect comprehensive information on HAMSET in a timely and efficient manner, to analyze it and to use the information for planning and responding appropriately to changes in disease trends. That capacity would exist at the central, Zoba, and sub-Zoba levels and would allow a quick response to epidemics. Achievements were uneven in this component, notable ones being: (i) improvement of HAMSET surveillance techniques and strengthening of an epidemic forecasting and preparedness system; (ii) improvement of the country's capacity to perform operational research for identifying changes in HAMSET; and (iii) strengthening management of communicable diseases at the MOH as described in the next section. Overall, a lot of data were collected by various implementers at all levels, but they were not always analyzed or used for decision-making. Monitoring and evaluation of the HAMSET diseases was uneven with the exception of the malaria program which was able to develop a good M&E system. Given the importance of M&E for a project that was supposed to `learn by doing', and despite considerable input into training of staff, renovation of structures and provision of computers, the performance of this component was less than anticipated. This is further complicated by the discrepancies between the various data sources in the MOH. Some key deficiencies were: · Lack of a M&E conceptual framework and operational plan in the design of the project and in the implementation manual. Although the M&E framework was finalized, this happened towards the end and did not therefore fully benefit the project; · Until the recent delegation of the responsibility to coordinate M&E to the Regulatory Department, no M&E unit existed either at the PMU or at the MOH; · There was no computerized MIS for most HAMSET activities (these include health facilities, BCC, CMHRP, line ministries and partner agencies); · The PMU lacked M&E capacity and was not able to report effectively on HAMSET activities. In addition, both information technology and human resources for M&E were insufficient at central and Zoba levels; · Data for different disease control programs were sent vertically to their respective national programs without collation at the Zoba level, thus, they were unavailable for decision-making at that level; - 6 - · Data from some line ministries and agencies were also sent directly to the PMU without notification of respective focal persons and therefore, they were unavailable for monitoring sectoral progress and by the Ministry of Health; · Methods to link the results of research and M&E to policy formulation, annual planning and budgeting were not systematically introduced; and · The MTR was done rather late and therefore did not benefit the project in a timely manner. However, the capacity of the MOH for surveillance techniques, data analysis and operational research has undoubtedly been enhanced. In addition, the successful RRI pilots provided significant lessons on how simple, practical and results-oriented monitoring can provide regular feedbacks for project implementation. The component is rated as marginally satisfactory. B. Multi-Sectoral Control of HAMSET Transmission: Major activities were undertaken in the following areas: advocacy, community sensitization, BCC, condom promotion, life-skill education in schools, targeting of military staff and families, the youth, orphans, commercial sex workers and workplace interventions. The overall rating for this component is highly satisfactory because much has been done and there are measurable achievements as per the Lot Quality Assurance Sampling (LQAS) survey of 2006. The star performers include the Ministry of Defense (MOD), Ministry of Education (MOE), Ministry of Labor and Human Welfare (MOLHW), Ministry of Information (MOI), National Union of Eritrean Women (NUEW), BIDHO (Eritrean Association of People Living with HIV/AIDS) and the Catholic Secretariat. The Ministry of Local Government (MOLG) also did a commendable job of coordination at the Zoba level. The MOLHW spearheaded the work on CSW and orphans and vulnerable children (OVC) but was unable to link HAMSET activities with those under the IECD project. B.1. Promote healthy behaviors through multi-level communication: The main activities covered were advocacy, community sensitization, BCC, life-skills education, activities targeting military staff and families, youth, CSW and workplace interventions. The project succeeded to a very large extent in promoting healthy behavior using multiple channels of communication. A communication strategy was developed and formative research conducted. Eritrea has a very well structured peer-led BCC program with peer educators, peer facilitators in model communities down to the village level which is not commonly seen in other SSA countries. A system has also been put in place to track changes in knowledge, attitudes and practice. The consultant report titled Mid-Term Evaluation of BCC on HIV under HAMSET Program in Eritrea: Final Report December 2005 summarizes these findings and provides good recommendations on the way forward. According to the LQAS survey, males have higher knowledge of HIV/AIDS prevention compared to females. The most frequently mentioned way to prevent HIV transmission was the use of condoms (females 47%, males 67%). Forty-five percent of females and 58% of male know abstinence. Forty-five percent of females and 53% of males know being faithful. A high proportion of women (88%) and men (92%) knew at least one benefit of VCT but only 31% of women and 32% of men knew the place to be tested for HIV, and only 12% of women and 20% of men had taken the test. Although 82% of men, 76% of women and 71% of mothers of children 0-11 months know about MTCT, only 48% of men, 49% of women and 32% of mothers knew the risk during pregnancy. Only 34 % of men, 26% of women and 32% of mothers knew that the risk of MTCT can be reduced. Both men and women have significant misconceptions about how the HIV/AIDS virus is spread, for example: - 7 - · 35% of males and 45% of females believe that HIV could be transmitted by eating chicken or goat that has eaten a condom used by an infected person. · 28% of men and 37% of women believe HIV could be transmitted through mosquito bites. · Other misconceptions included - touching an infected person (males 6%, females 8%), sharing utensils with an infected person (males 11%, females 15%), sharing toilets with an infected person (males 15%, females 23%), and sharing of food with an infected person, (males 9%, females 14%). Such misconceptions would contribute to the prevalence of stigma against HIV/AIDS in Eritrea: 63% of men and 46% of women were willing to buy food from a shop owner who is HIV positive; while only thirty-nine percent of men and 37% of women would accept a teacher with HIV to continue teaching. Regarding malaria, the LQAS found that 96% of the mothers of children 0-11 months old reported that they have ever heard of malaria and 92% of them mentioned mosquito bite as the cause of malaria. Eighty-four percent of the mothers of children 0-11 months and 75% of mothers 12-59 months reported that they have bed nets in their household. From the owned bed nets, 72% of the mothers of children 0-11 months and 66% of the mothers of children 12-59 months reported that their children slept under a bed net the night which preceded the survey. The mixed LQAS findings point to the need for more focused attention to behavior change communication. Nonetheless, HAMSET already laid a solid foundation and this sub-component is rated as satisfactory. B.2 Promote healthy lifestyle through the education system: The education sector was one of the very good performers under HAMSET. A special unit was established in the Ministry to oversee the implementation of HAMSET activities, with clear roles at every level, including support to schools. Working through the school health program and in conjunction with the Focusing Resources for Effective School Health (or FRESH) initiative, the program supported the development of an action plan, school curriculum, training materials and training of teachers. The project also invested in radio transmission and has been transmitting health messages to all schools in the country. This sub-component succeeded in establishing a credible base for school health and information on HAMSET diseases, which promoted behavior change in both teachers and pupils. However, more could have been done to integrate the HAMSET program with other school health initiatives as well as integrate HIV with other HAMSET diseases. A valid concern raised by MOE staff during the ICR was sustainability of the programs. They also acknowledged a missed opportunity to link with IDA- funded IECD activities that were being implemented by the sector. Despite this reservation, this sub-component is rated as highly satisfactory. B.3 Enhance access to preventive, diagnostic and treatment services for conscripts: The project succeeded in strengthening health care services available to conscripts (including the availability of voluntary counseling and testing). The project also promoted the use of condoms and insecticide-treated nets (ITNs). Although HIV/AIDS activities in the military had started before HAMSET (with the support of the U.S. Department of Defense), the project expanded the activities and helped institutionalize the interventions by putting in place: (i) the active involvement of the top army hierarchy; (ii) HIV screening of all new conscripts, repeated on discharge two years later; (iii) development of a special `condom pouch', which every army personnel must load and carry whenever on duty; (iv) use of treated bed nets by all conscripts, especially when in the field, and by their families; - 8 - and (v) a special outreach program undertaken by army personnel in their own villages, with full support by the army. In addition, free ARVs are provided to all eligible army personnel and their families. HAMSET also supported TB detection and management among conscripts and created link with HIV/AIDS program so that all TB patients are referred for CT services, while all HIV-positive conscripts receive TB prophylaxis. The impact of the program has not been assessed due to army restrictions, but the range of activities put in place and the rigor of application should have led to significant changes. This is a very comprehensive program for the military in the region. The `condom pouch' developed with HAMSET support is a brilliant innovation that should be replicated everywhere in Africa. Mandatory military service by all youth aged 18 years in Eritrea means universal conscription for both boys and girls. Eritrea therefore has excellent longitudinal data on HIV/AIDS in this age group. The data were discussed with the ICR author but they are not available for general release. However, the MOD is satisfied with the sero-prevalence trends (remaining below the 4.6% recorded in 2001) and with HAMSET support. This sub-component is rated as highly satisfactory. B.4 Promote environmentally sound and cost-effective techniques for malaria vector control by implementing a pesticide management plan (PMP): Some studies were carried out on alternative insecticides to DDT. They were found not to be as cost-effective as DDT and it was beyond Eritrea's capacity to come up with cost-effective insecticides to replace DDT when the rest of the world was still debating the issue. Known alternatives to DDT are relatively expensive and unaffordable for most African countries, including Eritrea. In this context, HAMSET supported the prudent use of DDT for indoor residual spraying for malaria control. Other environmental control activities have been undertaken at community level, including larvicidal activities (using chemicals and biological agents), source reduction for thousands of mosquito-breeding sites, research on vector control and the planting of Mim trees in some regions. This sub-component is rated satisfactory. C. Strengthen HAMSET Diagnostic, Health Care and Counseling Services: The objective of this component to reduce morbidity and mortality from HAMSET by improving access to quality health services was only partially realized, largely due to insufficient human resources and poor health infrastructure. The rehabilitations undertaken by both the Eritrea Health Project and HAMSET were not completed in time due to shortage of building materials and high inflation. They included two hospitals (Barentu and Mendefera), health centers, wards and laboratories. In this context, HAMSET's success with Malaria control could be due to its stronger links to primary care interventions, than HIV/AIDS and TB that require more technical expertise, drugs and equipment. Overall, the component is rated satisfactory. C.1 Establish safe blood banks in Zoba hospitals: The intent of this sub-component to ensure that all blood being transfused in Eritrea is safe has largely been realized but not strictly through the establishment of four blood banks for Zoba hospitals as originally envisaged. The Central Blood Bank in Asmara was constructed and equipped by the Health Sector Project. HAMSET supported the expansion of its operations and implementation of the National Blood Transfusion Policy through staff training, supply of supplementary equipment, consumables and laboratory reagents. Plus, HAMSET supported the establishment of `satellite banks' in all Zobas providing some small capacity for storage and screening. Due to human resource constraints, however, a decision was made to retain the full - 9 - functions of a blood bank in Asmara and add another unit in Barentu. The Blood Transfusion Center in Asmara has established high standards that are demanding of the regional hospitals. The new Regional hospitals have catered for this service in their construction plans. Eritrea has made major steps towards assuring universal blood safety, although recruiting blood donors is still a problem. The blood transfusion system is working effectively by sourcing safe blood from the center. As 100% of all blood units are screened for HIV and other common blood-borne diseases, the objective of universal blood safety is achieved (although not exactly through the way originally planned at project appraisal). This sub-component is therefore rated as satisfactory. C.2 Improve diagnostic, treatment and counseling of HAMSET. This sub-component was to support integrated in-service and on-the-job training on HAMSET prevention and detection, case management, syndromic and laboratory diagnosis of HAMSET, as well as pre- and post-HIV voluntary counseling and testing (VCT). In total, 201,828 persons utilized VCT services and got their HIV test results. HIV testing of couples before marriage has become a routine practice The cumulative number of mothers receiving PMTCT was 18,581. A considerable amount of in-service training took place but not in an integrated manner for all HAMSET diseases. For HIV/AIDS, the number of VCT sites increased from 6 in 1999 to 84 in 2005 and 91 in mid-2006. HAMSET also supported the procurement of 2 ELISA machines as well as rapid test kits. In addition, HAMSET supported the training of clinicians in management of HIV/AIDS, PMTCT, nursing and treatment of opportunistic infections. The first round of ARV procurement was initiated with HAMSET funding in 2004. With the support from both HAMSET and the Global Fund, 1,277 PLWHA are now on antiretroviral therapy. The impressive data on malaria is summarized in Table 1. Good coverage of effective interventions for malaria control have contributed to the reduction in malaria mortality and morbidity. Progress in HIV/AIDS is summarized in Table 2. - 10 - Table 1: Progress in malaria control in Eritrea (1998-2005) 1998 1999 2000 2001 2002 2003 2004 2005 Malaria morbidity (cases 254,152 179,501 119,232 125,746 74,861 65,518 27,783 24,192 registered in out patient departments or OPD) Number of fever cases 192,697 114,170 120,245 132,807 93,611 82,545 88,324 89,885 managed by malaria agents Cases at in-patient 6,651 8,973 5,478 7,207 3,543 3,517 departments (< 5 yrs) Cases at in-patient 1,297 1,913 1,337 1,588 835 519 departments (> 5 yrs) Registered mortality 533 199 105 133 86 79 16 32 Number of ITNs 117,863 67,708 276,038 187, 709 214,752 132,353 distributed Cumulative number of bed 65,000 127,000 245,000 387,000 472,000 531,000 746,000 878,353 nets distributed % of population owning at 34% 81% least 1 bed net % of population owning at 73% least 1 ITN ITN reimpregnation rate 17.2% 53.3% 58.5% 76.5% 835% 93.3% % of children < 5 sleeping 76.2% 12% 58.6% under a net % of pregnant women 7% sleeping under a net % of children < 5 sleeping 65.4% 4% 48.3% under an ITN % of pregnant women 40.1% 3% 50.4% sleeping under an ITN Number of 15,988 23,810 25,355 15,539 27,494 39,586 mosquito-breeding sites eliminated Number of 11,691 7,690 12,547 67,684 33,442 46,772 mosquito-breeding sites treated % of pregnant women 0.9% 5% 4.2% receiving prophylaxis Malaria case fatality rate 5.97% 2.31% 2.35% 2.39% 1.76% 0.72% 0.21% (under 5) Malaria case fatality rate 3.63% 1.07% 1.19% 1.25% 0.80% 0.38% 0.45% (total) Mortality attributed to 10.8% 6.5% 1.0% 0.40% malaria (under 5) Source: National Malaria Control Program. - 11 - Table 2: Progress in HIV/AIDS control in Eritrea since the start of HAMSET Indicator Total Number Number of women enrolled in PMTCT 18,581 Total number of VCT sites in the country 91 Number of VCT sites established with MAP contributions 76 Number of persons who used VCT services and received their test results afterwards 201,828 Number of male condoms distributed 47,171,048 Number of female condoms distributed 47,000 Number of IEC or BCC events 1500 Number of blood units that have been tested for HIV 23,480 Estimated number of persons that have been treated for STD 13,761 Number of sites providing ART (including PMTCT) 52 Cumulative number of persons on ARVs 1,277 Number of PLWHs receiving prophylaxis for Opportunistic Infections 3,258 Source: NATCoD. HAMSET supported a new category of community health worker called "malaria agents" whose roles include: (i) providing antimalarial treatment to suspected malaria cases in the communities; (ii) facilitating ITN distribution and ITN impregnation, and (iii) guiding communities in other vector control activities such as source reduction, larviciding, etc. Around 60% % of fever cases are now treated by malaria agents at the community level. The effectiveness of this cadre and the strong community focus could to a significant extent account for the good outcome of the malaria activities. Microscopy for malaria diagnosis is available in health centers. Where there is no microscopy (e.g. health stations), rapid diagnostic tests are used. Chloroquine (CQ) and sulphadoxine-pyrimethamine (SP) combination therapy is still the first line treatment as resistance stands at a low 3%. Prophylactic treatment for pregnant women is only recommended for selected areas where malaria is endemic; and therefore the coverage is only 5%. All women attending antenatal clinics in high prevalence areas are currently given free ITNs. Entomology laboratories supported by HAMSET are at various levels of completion. For TB, HAMSET intended to increase case detection, introduce directly observed therapy short-course (DOTS) and improve cure rates. The TB program was much slower in taking off due to staff shortages. This was partly solved by the formation of NATCoD in the MOH, combining the management of HIV/AIDS and TB programs and accelerated diagnostic and treatment services. According to the Eritrea TB Prevalence survey (the first national TB prevalence survey conducted in Africa in the past 45 years) Eritrea has a significantly lower TB prevalence than earlier estimated by WHO. The prevalence of new smear positive tuberculosis was 90/100,000 (95%, Confidence Interval 35-145/100,000) in individuals above 15 years of age and 50/100,000 (95%, Confidence Interval 19-80/100,000) in the total population. By 2005, DOTS was available in all hospitals and in 70% of health centers where smear microscopy was available. The National TB Plan was also finalized and disseminated in 2004. The MTR in 2004 reported a case detection rate of 68.6% and cure rate stood at 82%. It also found a lack of trained technicians to be a major constraint to effective case detection and DOTS management. Quality of smear microscopy at local levels was a concern as shown by the high percentages of smear negative slides and the proportion of tests not done due to lack of reagents. HAMSET tried to address this by building capacity for fluoroscopic microscopy in every region. It has also supported the training of 500 TB promoters, but they lack incentives and technical back up. - 12 - Although the initiative is still in an early stage, it seems TB promoters are not as effective as the "malaria agents" due to the lack of incentives and technical backup. In addition, the creation of different community agents dealing with different diseases is not optimal. HAMSET also supported the STI program through training and promotion of syndromic management and provision of drugs. However, there are minimal data in this area. A recent study showed 32% HIV prevalence in clients who reported STDs twelve months prior to the survey compared to 7% prevalence in those who did not, emphasizing the need for more attention to be given to the problem. In conclusion, the performance is rather mixed. The impressive performance of malaria control and, to some extent, of the HIV/AIDS program outweigh the not-so-stellar performance of the TB and STI programs. In addition, staff shortage is a major constraint and is beyond the scope of this project. The sub-component is therefore rated as satisfactory. C.3 Improve availability of basic medical materials and drugs required to diagnose and treat HAMSET in health facilities: The project funded procurement and distribution of basic essential drugs and diagnostic materials to treat HAMSET diseases, and in-service and on-the-job resource management training for MOH staff at the facility level and under supervision. About US$15 million (75% of the budget for this component) was allocated for the procurement of drugs, condoms, bed nets and lab supplies. Over 47 million male condoms were distributed through clinics and social marketing channels. Nearly 0.9 million treated bed nets were distributed in the country. For HIV, although it was originally envisioned that only drugs for opportunistic infections were funded, HAMSET kick-started the procurement of ARV in Eritrea. Apart from some delays in procurement, this investment alleviated the problem of drug stock outs which is now very rare in public facilities. Extension of anti-malarials at community level was a big success for the program since over 60% of cases are now treated by "malaria agents" as discussed above. The sub-component is rated as highly satisfactory. D. Community-Managed Response Program: There was some overlap between this component and the multi-sectoral one. It focused on empowerment of communities to fight against HAMSET diseases through awareness creation, behavior change communication and support of various community-generated and led projects. They were supported by zonal PMUs in needs-assessment and design of the small projects which communities would run on their own. Community facilitators were hired to support this process. In the original budget, nearly US$ 10 million was allocated for this component but actual expenditure had risen to over US$ 16 million by the end of 2005. Although the component did well eventually, it had a late start due to the time needed to introduce the concept to communities. In addition, the operations manual was not ready until late 2002 when the CMHRP coordinator and Zoba facilitators were also recruited. The Ministry of Local Government was in charge of the component but its management was in the hands of Zoba PMUs. Despite the delays, the component increased its pace from 2003, with the facilitation by NUEW, NUEYS and different ministries. By December 2005, 1059 sub-projects had been submitted (with 11% rejection rate) of which, 892 had been completed. - 13 - GRAPH 1: SUBPROJECTS IMPLEMENTATION STATUS PER ZOBA 350 300 250 Promotted STATUS Approved 200 Rejected 150 Not yet started Under implementati on 100 Implemented MPLEMENTATIONI 50 0 Anseba Debub G/Barka Maekel N/R/Sea S/R/Sea ZOBAS It is estimated that over 3 million people had been reached through this component by the end of 2005. This component has played an important role in raising awareness, changing behavior, and providing community-level care and support for HAMSET diseases. By introducing the Community Capacity Enhancement Process (CCEP), the project demonstrated commitment to improve performance. In addition to its success in mobilizing communities to contribute resources and play such a significant role in malaria diagnosis and treatment, this component is deemed as the core reason why HAMSET did so well and is rated as satisfactory. D.1 Community counseling and support groups: This subcomponent funded counseling and the establishment of support groups for AIDS-affected people by providing home-based care and support for orphans, community managed efforts to identify malaria, anemia, STDs and TB symptoms, and capacity-building to improve skills and strengthen community support. It is rated as satisfactory. Some of the highlights include: · 2500 malaria agents and 500 TB promoters are working in communities to inform, counsel and reduce the burden of HAMSET diseases; · Over 8,283 people have been trained under this initiative and are involved in running HAMSET activities at the community level; · Through NUEYS, many youth groups were supported in various ways in activities to control HAMSET diseases and to empower them; numerous sports fields have been constructed to provide recreation for the youth; · HIV testing of couples before marriage has been popularized as discussed earlier and is now promoted by all religious groups and is widely accepted; · Through NUEW, gender issues were addressed in multiple ways: Income generating activities with a special focus on the economic empowerment of women, sustainability of some of the projects notwithstanding; increased participation of women in project committees and in decision-making processes; and education on gender rights in marriage and divorce; · Increased attention to high risk groups; 81% of sex workers interviewed in Asmara had condoms in their possession compared to 30% in Masawa and 58% in other towns; - 14 - · Provided support to a proportion of the over 90,000 children orphaned by war and HIV/AIDS in Eritrea. HAMSET supported 3,105 OVC, providing Nakfa 2,000 per orphan to assist families in establishing income-generating activities. Through BIDHO, support provided to 1,625 PLWHA and home-based care programs started all over the country, in conjunction with major religious groups. In addition, special nutrition support programs for TB patients were implemented. D.2 Community-managed response: This subcomponent built the capacity of communities to use their own structures and socio-cultural fabric to respond to the challenges of preventing HAMSET diseases, mobilize internally, identify and use their own socio-cultural resources and implement sub-projects to prevent or mitigate the diseases. The intention to link with the IDA-financed Eritrean Integrated Early Childhood Development (IECD) did not materialize. Small projects that were to cost between US$1,500 and US$30,000 were first defined by the communities and reviewed by the Zoba technical committee. Only projects costing more than US$30,000 were selected by the Zobas and sent to Asmara for review by the National Technical Committee. There was an overall rejection rate of about 11%, mainly due to inclusion of construction. Communities contributed 20% of the cost, although there was considerable variation. Each community formed a leadership committee with a team leader, treasurer and technical coordinator. The whole process took an average of two months and the duration of a project was seven months on average. Although allocation criteria based on demography and disease profiles had been set in the operations manual, this did not work effectively. Out of the 892 sub-projects completed by end of 2005, 70% were on malaria, accounting for 79% of the total costs. HIV received 20% and TB only 6% (see Graph 2). In 2003, 89% of TB and only 3% of malaria related sub-projects were in the two coastal regions. GRAPH 2: IMPLEMENTEDSUBPROJEC TS PER HAMSETDISEASES 180 160 140 120 Malaria numberst 100 HIV/AIDS 80 T B 60 Others Subprojec 40 20 0 Anseba Debub G/Barka Maekel N/R/Sea S/R/Sea Zobas Malaria projects were predominantly in Debub (40%), Gash-Barka (36%) and Maekel (14%) Communities received a total of US$ 7 million and raised US$1.4 million (Nakfa 17, 189 per project) from their own resources. The number of primary beneficiaries per project varied from 10 to a few hundreds while secondary beneficiaries were more difficult to assess. The M&E system originally designed to evaluate these sub-projects did not work but recent analyses indicate that they have been effective. However, it is difficult to estimate the cost-effectiveness of sub-projects. Apart from this, the - 15 - two major concerns reported by both the MTR and the final government reports are the considerable overlap of activities and funding from different components of HAMSET (especially 2 and 4) and the disproportionate expenditure on food and refreshments. Food was a big incentive for participation, given the drought situation in Eritrea. The program should have explored different forms of non-monetary incentives and recognition in order to sustain interest and participation. The 2003 recommendation that IEC strategies for HAMSET diseases be jointly implemented via community sub-projects, hence requiring coordination from the HAMSET community coordinator and the technical programs (NATCoD, NMCP) was not done. Despite the problems, this subcomponent is widely appreciated by communities and they are asking for more. In addition, the introduction of the BCC should place appropriate focus on behavior change. The shortcomings are outweighed by the benefits, hence the rating of the sub-component as satisfactory. E. Project Management and Evaluation: This component supported overall project management by strengthening the current PMU (located in the MOH) to undertake responsibility for planning and budgeting, procurement and financial management. There was to be emphasis on evaluation and monitoring as part of the `learning by doing' approach. The HAMSET project implementation and coordination arrangements are unique and different from other MAP programs in the region. Most MAP programs set up National HIV/AIDS Coordination Councils chaired by the President or the Prime Minister. Some countries have also created Ministers of HIV/AIDS in addition to a large secretariat led by an Executive Director. In Eritrea, high-level oversight is provided by the HAMSET Steering Committee, which is multisectoral and chaired by the Minister of Health. The PMU was inherited from the previous IDA-funded Health Project and also sits in the Ministry. Staffing of the PMU remained stable and averaged 28 persons until 2005 when the number rose to 49 due to additional recruitments for Zoba PMU offices. The level of staffing was deemed adequate and commensurate with the shortage of professional staff in Eritrea. Turnover of Zoba coordinators has however been higher than desired, thus affecting continuity of oversight. Despite this shortcoming, oversight structures and the PMU have provided excellent management of the program. There has been commitment in these structures but planning, monitoring and evaluation were weak. This was partly due to lack of essential technical skills (e.g. the TB program) but also insufficient attention to M&E in the project design and implementation. Although some targets were set in the PAD, baselines were unknown and the plan to put in place a strong evaluation framework was not undertaken after inception. In addition, although a lot of information was collected, most of it remained in silos within technical programs (e.g. HIV/AIDS, Malaria, etc), in the Zobas or in the various sectors. There was limited medium for integration, dissemination or sharing of this information even within the MOH. Neither was any attempt made to integrate HAMSET information with the HMIS, probably due to poor integration of vertical programs' M&E and HMIS. The ICR team considers this to have been the weakest aspect of an otherwise successful project and the Bank supervision team should share the blame. Stakeholders also pointed to delays in procurement and financial disbursement. These two issues and the gaps between budgets submitted and approved and actual expenditure are addressed elsewhere in this document. Zonal PMUs concentrated more on community activities whose budgets they were aware of and had assisted in planning. They were less concerned about multisectoral and MOH activities, probably a factor in the overlaps mentioned above. Zoba coordinators also expressed interest in receiving more feedback and appraisal of their performance, comparison with regions and overall performance of the - 16 - whole program on a regular manner. Some shortcomings in management were beyond the scope of the PMU and the Steering Committee of HAMSET. The overall rating for this component is satisfactory. 4.3 Net Present Value/Economic rate of return: The HAMSET project for all intents and purposes was essentially an investment project. It sought to invest in Eritrea's main asset: its people. The benefits of HAMSET, although they already are in view, were designed to materialize far into the future. These long-term capital investments explain why the costs of the project seem excessive, five years after as compared to similar disease specific projects as detailed in Annex 3. A thorough evaluation of the economic benefits was not feasible, as the disbursements did not discriminate between the component diseases with the exception of the CMHRP component. It would aid future analysis if this were addressed in the subsequent HAMSET II. 4.4 Financial rate of return: N/A 4.5 Institutional development impact: The impact of HAMSET on institutional development in Eritrea has been substantial and is not limited to the health sector. The project succeeded in harnessing high level commitment to fight HAMSET diseases across many sectors. The impact is particularly strong in the MOH where HAMSET resources accounted for a significant proportion of the annual budget. The planning and successful implantation of HAMSET, especially the achievements in malaria control, have raised the profile of MOH in the eyes of the public and strengthened the confidence of staff in their ability to improve the health of Eritreans, despite the poor economy. The project has also strengthened the decentralization process through direct support to the Zobas to plan and implement community activities and work with NGOs. The capacity of the Ministry of Local Government to coordinate multisectoral activities at that level was also put to severe test. Additional capacity was built through training of personnel and supply of equipment. All stakeholders have benefited from this, improving planning, management and monitoring of programs across the board. Finally, under CMHRP, the capacity of communities to identify their HAMSET problems, prepare proposals, manage funds and implement activities has been significantly enhanced under the project. Through this process, communities have been greatly empowered to voice their concerns, demand services and take actions to improve their own health. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: Three factors affecting HAMSET implementation and outside MOH control were: (i) limited human resources; (ii) country macro-economic situation and the border tension with Ethiopia; and (iii) the unpredictability of donor resources. The second factor affected availability of construction materials and labor, which delayed civil works, while the third led to the premature exit of critical technical inputs. 5.2 Factors generally subject to government control: There are a few special factors that were subject to government control. The government enacted policies that limit participation of NGOs and the private sector. The hard economic realities have also made it difficult for communities to work voluntarily. The low remuneration of field staff has also made it difficult to recruit and retain qualified staff in the Zobas. Finally, the government could have sought assistance from UN technical agencies more aggressively concerning the TB program that went - 17 - for a lengthy period without proper leadership. This may account for its relatively poor performance. The MOH could have done more to integrate the different disease programs. They still in operate vertical silos despite intentions to the contrary declared in the PAD. 5.3 Factors generally subject to implementing agency control: Savings could have accrued from the reduction in overlaps of activities supported by different components of the project. All this could not happen, however, without effective planning, monitoring and evaluation. A lot of information has been collected but has not been analyzed to inform planning and budgeting. The information has not adequately disseminated and shared with stakeholders. Additional staff and expertise would be needed for this to happen. 5.4 Costs and financing: The original cost of the project was US$50 million, including an IDA credit of U$40 million, US$ 8.2 and US$1.8 from the government and communities respectively. The project was to be implemented over five years. Component C (strengthening HAMSET diagnosis, health services) originally accounted for 41% of the budget. Around US$ 21 million or 30% of the project cost was allocated for the procurement of drugs, condoms, ITNs and other goods. In line with the intention of the project, 57% of the budget was allocated to activities that can be regarded as `investment' in accounting terms. In this context nearly 30% of the total HAMSET budget went into capacity building (training and workshops) as part of component D. Although there were significant delays in disbursements, activities were never interrupted. Expenditure increased steadily over the five years, with accentuation over the last two years. For example, only US$ 1.4 million (3%) was spent in the first year compared to US$15.2 million (nearly 30%) in 2005. Most of the expenditure in 2005 was by the MOH. By December 2005, the project had disbursed US$ 45,627,763.48 (99.3%) of the IDA grant and US$ 5,658,253.50 (99.9%) from the government expected total contribution of US$ 5,665,562.80 (99.88%) during the life of the project. The balance of US$ 4.7 million was earmarked for spending during the last three months of the project life. As shown in Annex 2, credit utilization by expenditure category also demonstrates efficient disbursement by the project. There were significant differences between real expenditure by component compared to the original budget. In addition US$5 million was used to addressed the cost overrun of the construction of the two new completion of two new regional hospitals whose construction had started under the IDA funded Health Sector Project. All IDA funds were fully disbursed by project closing. 6. Sustainability 6.1 Rationale for sustainability rating: Overall sustainability is rated as likely. The rating takes into account multiple aspects of the project - financial, institutional and technical. The rating is complicated by the poor state of the economy, the public resources available for health and levels of poverty in Eritrea. Other factors affecting sustainability are political commitment and availability of human resources for both managerial support and clinical services. There are also many activities that are part of routine, technically sound health care, which will continue. Such activities would more likely be sustained. Eritrea has few donors, which ironically could be an advantage for sustainability. The country has a severe shortage of trained personnel but distribution is not a big problem as most of them accept their posting irrespective of the location. Migration is still not a major factor and the new medical school will soon be producing doctors. HAMSET II is also accelerating training of nurses and other providers. Sustainability is very likely with community-based activities that are responsible for the success of the malaria and HIV - 18 - programs and are built in the historically strong social fabrics of communities in the country. This also will also help with the sustainability of behavioral change which needs regular reinforcement. 6.2 Transition arrangement to regular operations: HAMSET was largely implemented by regular government staff, except for the PMU and the community component. Very few consultants have been used in the project. HAMSET II and the Global Fund are already in place and have taken up many activities related to this project, with special attention being given to the integration of activities within the different sectors. The human resource problem in Eritrea was beyond the scope of HAMSET but is now being addressed under HAMSET II. Sustainability of community-level activities will continue to be a challenge, given the level of poverty in Eritrea. However, the strong sense of patriotism and self-sufficiency of the Eritrean people is likely to help significantly. 7. Bank and Borrower Performance Bank 7.1 Lending: The Bank's performance during project preparation is judged to have been satisfactory. The project addressed key diseases that were high priority for the country (as reflected in its health and development strategies) and the Bank (as reflected in the Interim Support Strategy and Bank's HIV/AIDS and Health Nutrition and Population strategies). The financial and technical aspects of project preparation were well handled, including the QER, and technical assistance was provided to the government during preparation by the Bank and other development partners (especially USAID). Appraisal, risk analysis, the lending instrument and safeguards were in line with prevailing MAP guidelines. The project design was based on local needs and priorities. Deficiencies other than that related to national capacity include: (a) some overlaps among the project components; (b) lack of clarity about activities and responsibilities of some stakeholders; and (c) inadequate focus on M&E, especially the quality of data, collation and utilization to inform planning and implementation. This was particularly weak at the lower levels. Given that this was the first multisectoral project in Eritrea and the Health Sector Project was not doing that well, coordination by the MOH was expected to be challenging but were rather well considering the circumstances. 7.2 Supervision: Technical support by the Bank was made through regular supervisory missions and some targeted consultancies. Missions were generally fielded regularly every six months and overall sound support was provided despite the absence of any technical staff in the World Bank country office. Details of the missions and their composition are provided in Annex 4. Financial: HAMSET finances were well-managed. However, utilization of resources was sometimes affected by the limitation in the budget outturns which did not match the past project budgetary estimates. The Bank supported installation of enhanced computer-based accounting system through upgrading FINPRO software to TOMPRO, which had a significant effect on the PMU's ability to maintain excellent and reliable financial data and information. The shift from traditional disbursement to the FMR disbursement in the final year benefited the project by increasing flexibility and improved efficiency in credit utilization and the liquidity situation. Procurement: HAMSET procurement provided special challenges both at the center and at community level. Stakeholders felt that the Bank's procurement procedures were not robust enough and created delays that sometimes affected implementation. The PMU carried out the procurement process in - 19 - accordance with the agreed procurement schedule in the Development Credit Agreement (DCA) including fast tracking of emergency and other urgent purchases. Large purchases were carried out centrally under International Competitive Bidding, assisted by specialized procurement agencies like PHARMECOR for technical evaluation. Participating agencies, however, carried out minor procurement actions at the local level in accordance with the manual of procurement procedures. The satisfactory performance of HAMSET is directly related to a realistic procurement implementation plan approved and closely monitored by the Bank, especially in the first year of project implementation. 7.3 Overall Bank performance: Performance of the Bank is rated as satisfactory. Borrower 7.4 Preparation: The borrower seems to have prepared adequately for HAMSET. In this context, the experience from the IDA funded Eritrea Health Project proved invaluable. The PMU and core staffs were inherited from the Health Project. The slow start of HAMSET was not due to lack of adequate preparation. Rather, the project had underestimated what it would take to get such a complex project off the ground, especially the community component and multi-sector components. The management structure of HAMSET, though different from other MAP projects, was appropriate for Eritrea and has served the project well at all levels. The borrower also seems to have prepared well for its contribution to the project. The government has continued to honor its commitments despite hard economic times. 7.5 Government implementation performance: Implementation by the government was highly satisfactory for the following reasons: (i) high level support and commitment to HAMSET; (ii) successful oversight and coordination by the HAMSET Steering Committee made up of several ministers and chaired by the Minister of Health; (iii) effective leadership at the decentralized level by both zonal governors and zonal medical directors; and (iv) effective mobilization of communities and their very positive response. Good governance and transparency in the context of the project was another important factor. For example, interviews with communities implementing CMHRP sub-projects often showed that community members were well aware of the funding amount made available to the communities and its purposes. 7.6 Implementing Agency: The HAMSET project, as indicated in the PAD, would be implemented by various agencies, including the MOH, MOLG, MOE, MOT, MOTC, MOLHW, MOI, NUEYS, NUEW, NCEW, etc. The PMU staff had gained experience of implementing Bank projects and therefore had a head start. The PMU staff exhibited extraordinary commitment throughout the project and were highly responsive to other stakeholders. PMU performance was, however, constrained by lack of adequate focus and technical expertise in several areas: · Planning and Budgeting: The budgeting process was to a large extent static once the allocations were fixed for each Zoba and on the basis of the HAMSET components. As a result of this, the MOH/PMU was loading and reflecting mostly project costs in its HQ budget. The budgets were not regularly reviewed and revised during the year to reflect any changes and the reality on the ground. - 20 - · Internal Audit: The internal audit unit was operated by a single staff member. The unit's day to day activities were purely of an accounting nature and have had nothing to do with the internal audit function. Fortunately, because HAMSET was well managed, with no known incidence of fraud or loss of project assets, the arrangement sufficed. · External Audit: The annual audit reports for HAMSET were all submitted in time in compliance with the Financial Covenant on audit. The auditor also provided unqualified opinions on the annual financial statements for the project since inception. · Procurement: Overall, despite the complexity due to the number of ministries/agencies involved, HAMSET procurement went well. This can be attributed to: (i) adequate capacity of the PMU to carry out international and national competitive bidding and shopping. This provided adequate coordination and management structure, leaving to the individual ministries the responsibility for the implementation of their respective subcomponents; (ii) strong commitment by all participating agencies and satisfactory staffing at the central level for most leading executing agencies such as MOLG, MOF and MOE; and (iii) medium-term procurement planning. Procurement of major inputs such as condoms, mosquito nets, pesticides, drugs, laboratory reagents, and test kits, were based on medium-term (three years) requirements of items and expenditure plans. This involved the establishment of a resource envelope, including contributions from the GOE and the major donors. The Mid Term Review (November 22 ­ December 11, 2004) recommended: (i) the need to include an additional procurement method such as Limited International Bidding (LIB) in the DCA; (ii) the evaluation of multi-items lots for malaria-related drugs and laboratory supplies; (iii) the use of an "item by item" evaluation method to increase the number of contracts needed for the full procurement of all items under one lot. These recommendations were timely as well as essential and the DCA was amended to include the LIB procurement method and the two other evaluation criteria were also effectively used thereafter. By January 2006, all procurement activities were fully accomplished and the general impression is that the institutional arrangement set up for HAMSET, the staffing as well as the procurement systems put in place have been adequate and worked effectively. In addition, compared to the Country Procurement Assessment completed in June 2002 which found that there was lack of procurement capacity in Eritrea, a joint Country Portfolio Performance Review exercise conducted in December of 2004 confirmed that while the overall procurement capacity in the country is still very weak - especially at the Regional level, due to lack of skilled procurement staff - there were four well-performing IDA funded projects, and HAMSET is one of them. Thus the capacity in procurement under HAMSET was significantly strengthened over time. 7.7 Overall Borrower performance: Based on (i) the above points, (ii) the impressive performance of HAMSET (especially in malaria and HIV/AIDS as discussed earlier) in a difficult macro-economic environment and the "no war, no peace" situation, borrower performance is rated as highly satisfactory. 8. Lessons Learned HAMSET provides several important lessons regarding the Bank's MAP projects. The lessons are particularly important since HAMSET is one of the first MAP projects that has closed and been immediately followed by a second one. In addition, HAMSET design, with its multi-disease approach, was different from most MAP projects which focus only on HIV/AIDS. The lessons, varying from stewardship of MAP projects to their sustainability, have relevance beyond Eritrea to other countries. - 21 - 1. Project design should be flexible and tailored for the country: HAMSET was different from other MAP projects in other ways too. The HAMSET Steering Committee (equivalent to the National AIDS Council in other countries) was chaired by the Minister of Health and the PMU based in the same Ministry. This arrangement neither denied the project high level support nor diminished the interest of non-health sectors. The other sectors were content with technical input by the health sector into their interventions. Unlike other MAP programs where the health sector is often a poor performer, HAMSET obtained the best from the sector by challenging it directly to provide the leadership. It is this arrangement that facilitated concurrent coverage of the other two diseases. Community facilitators explained to the ICR team that most communities had broader health concerns than HIV/AIDS and working with other health issues such as malaria made it easier for them to penetrate the communities and discuss HIV/AIDS. It is clear that HIV/AIDS in Eritrea is not widespread with a high prevalence concentrated on a few special groups and regions. With awareness already very high among the public, future investment should be concentrated on the special groups and regions, where behavior change is greatly needed and the returns are likely to be more substantial. The focus on BCC is a move in the right direction. 2. Mobilizing communities and relevant sectors is key: As evident from the report, the multi-sectoral and community components conferred great benefits to both malaria and HIV control. Communities and the multi-sectoral approach have a critical role in the success of malaria and HIV programs in Eritrea, as (i) most of the key effective interventions (BCC, ITN, care and support, community-based management of fever, etc.) require the active participation of communities and (ii) other sectors have their comparative advantages in reaching certain populations. These are important lessons not only for malaria and HIV/AIDS programs but also for the control of many other communicable diseases in the region. 3. Effectively integrate activities of vertical disease programs, reproductive health and support health systems: The PAD for HAMSET highlighted the importance of integration of the activities addressing the three diseases but very little happened during implementation. This was due to a lack of interest by program managers who had no incentives to integrate. Integration is an active process that requires a strategy, resources and time. Both the Bank and its clients need to acknowledge this and ensure that the steps are clearly spelled out at appraisal, including measurable targets and resources. A well-designed integrated project should limit the overlap of activities seen between different components of HAMSET and will inevitably address key health sector issues, an important consideration for sustainability. Systemic issues that plagued the project include weak HMIS and the shortage of human resources. HAMSET II is taking these issues into account and has introduced a new reproductive health component. 4. Clarify policy on the use of DDT: The environmental aspect of Malaria control has been complicated by the mixed messages over the use of DDT among international organizations and governments. This is not only in Eritrea but the whole Region. It is clear that most alternatives are costly and unlikely to be used in large scale in Eritrea and DDT may be the only option. As the country consolidates its successful campaign against Malaria to avoid new epidemics, the Bank and other development partners need to speak with one voice to provide clear direction on their policies and how far they are willing to support the government on its decisions. 5. Build local capacity for planning and budgeting: The ICR agrees with the government report that planning was a major weakness of HAMSET. Consolidated work plans lacked clear narratives and had - 22 - no logical linkage with budgets and expenditure at all levels, explaining the wide gaps noted between submitted and approved budgets as well as the actual expenditure. Strategic planning in the budgeting process needs to be introduced in HAMSET II to allow a multi-year year roll-over budget cycle that is updated annually at the time of preparation of the annual budget estimates. This should help the PMU in making informed decisions on finances, assist the Zobas to strategically prioritize and plan their activities on a longer time frame and rationalize their financial requirements. In addition, internal audit capacity at the PMU should also be strengthened to enable it to perform the required functions. In this regard, the role of the Bank and effectiveness of supervision missions in supporting planning and budgeting under a program of this complexity should be re-examined. As stated in the independent report commissioned by the government, `weakness of the project had more to do with oversight than lack of M&E systems or tool. Lack of oversight that otherwise would have helped adjust planning and budgeting is accountable to both the country's stakeholders and the World Bank back-up teams.' 6. Procurement can be facilitated by innovative thinking: HAMSET procurement worked rather well, albeit with some delays. The institutional arrangement to allow the PMU to handle all foreign procurement centrally proved to be very effective and therefore, should be further strengthened. Procurement of items such as laboratory equipment and reagents need to be procured from specific suppliers through direct contracting. This is because (i) the amount involved may not be significant while the time it takes to process it could be very long; and (ii) as a result of standardization and specialization, especially for laboratory equipment, other procurement methods may not be necessarily appropriate. 7. Give M&E higher priority in design and prepare early for collection of data for analysis of cost effectiveness: M&E was treated as an afterthought in HAMSET and was supposed to be strengthened during implementation. This did not happen because the attention of management was taken up by more immediate financial and procurement considerations. In addition, the PMU lacked technical expertise in M&E. This shortcoming has been addressed in HAMSET II, where M&E has been given special attention. Simple and participatory monitoring and evaluation which incorporates regular feedback at local level should be adopted to assist in decision making and sub-project management. This is also in line with prevailing emphasis in the Bank on strengthening the result framework. Although the overall performance of HAMSET is very positive, lack of appropriate data has rendered comprehensive analysis of the economic impact impossible. This analysis is complex and requires data that is not routinely collected. Attention should be given to this requirement during project design. It is recommended that HAMSET II takes up this particular point. 8. Address the discordance between knowledge, attitude and practice on HAMSET diseases: Results of interviews and focus groups discussions with targeted populations (women, CSWs, youth, truck drivers, etc.), as well as results from HAMSET impact assessment studies at Zoba level, from the 2005 Knowledge, Attitude and Practice study, the final MTR report of the BCC strategy and the LQAS report consistently show significant difference of the impact of the sensitization and awareness campaigns on knowledge and attitude on one hand (which is excellent), and practice and behavior change (e.g. condom use, VCT) on the other hand. This calls for greater attention to BCC which, although initiated under HAMSET, still need to be taken to full scale through the expansion of model communities. It is important that HAMSET II pays maximum attention to this approach. 9. Address the challenges of sustainability, starting with integration and community ownership: Like other MAP projects, sustainability remains a major challenge. Luckily for Eritrea, HAMSET II has already commenced and has taken up from where HAMSET ended, with a strong focus on integration. - 23 - The Global Fund should also help bridge the immediate funding gap, with the hope that the country's economic prospects will improve and peace will prevail. In the case of CMHRP, the Zobas reported that most of the sub-projects particularly those carried out by youth, religious leaders, BIDHO and women organizations are sustainable. This is due to the fact that these organizations were in place before HAMSET and will continue to exist after HAMSET. Moreover, the organizations have communication mechanisms, continuous demonstration activities, panel discussions and other related activities that are on going. Integration and community ownership should be given greater emphasis under HAMSET II, much as additional external support will still be required in the foreseeable future. Integration of activities, especially at the Zoba level and below should be encouraged whenever appropriate, as this would help improve efficiency and, consequently, sustainability. For this reason, appropriate activities to be integrated need to be identified and factored in the planning and budgeting process with the collaboration of different programs (for example, integrated in-service training of health workers, integrated M&E and supervision, multiple roles of community health agents, etc.). 10. Address the shortage of human resources for health: Such a shortage was a major constraint for a more successful implementation of HAMSET activities within the health sector. This however is a cross-cutting issue and needs to be addressed at higher levels of governance. It is worth noting that the subject being addressed to some extent under HAMSET II and that it is identified as priority by the government of Eritrea. 11. The positive effects of the report-based disbursement system: The report-based disbursement system (which was adopted in the second half of the project) allowed more predictable cash flow, more flexibility and facilitated project implementation. Building the fiduciary capacity of the PMU to enable them to switch to such a disbursement system was key. 9. Partner Comments (a) Borrower/implementing agency: The Ministry of Health confirmed its agreement with this ICR and the ratings. (Please see Annex 8). (b) Cofinanciers: N/A (c) Other partners (NGOs/private sector): N/A 10. Additional Information - 24 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate % pregnant women, aged 15-24, at antenatal N/A 2.46 clinics that test positive for HIV. % conscripts, aged 18-27 that test positive N/A N/A for HIV. Malaria death rate - children < 5 and 2.8% Proportional malaria mortality: 0.4% among pregnant women. under five 2.7% among > 5 Prevalence of severe anemia in women of N/A N/A child-bearing age. % women, aged 15-24, at antenatal clinics 1.2% 1.6 that test positive for syphilis. Proportion of diagnosed and successfully 70% case detection rate, 75% cure rate 23% case detection rate, 79% cure rate treated new smear - positive TB patients. Output Indicators: 1 Indicator/Matrix Projected in last PSR Actual/Latest Estimate Number of actions in annual work program N/A 15 directed at expected, high-priority HAMSET problems. % community members who report HAMSET N/A N/A brochures/posters available. % respondents with correct knowledge of 95% 88% HIV transmission and prevention methods. % of retail outlets and service delivery points N/A N/A with condoms in stock. % respondents reporting use of condoms at 100% 36% last sex encounter with non-cohabiting partner within last 12 months. Median age of first sex among young women N/A 19 aged 15-24 % children <5 and pregnant women sleeping N/A 58.6 under insecticide-treated nets in malaria infected areas % patients with uncomplicated malaria who 7.2% (under-five) 30.2 receive correct treatment at a health facility or in the community within 24 hours of onset of symptoms. % communities with access to N/A 90% environmentally validated, socially acceptable vector control methods % secondary school children, teachers and N/A 98% adult learning participants able to describe cause and means of prevention of HAMSET diseases. % secondary schools with HAMSET N/A N/A counseling services. % schools equipped with segregated latrines. N/A 60% % schools equipped with latrines where the N/A 60% latrines are clean and working. - 25 - % conscripts that reports easy access to N/A 100% condoms. % blood units transfused that have been N/A 100% screened for HIV according to national guidelines. % Zoba hospitals with blood donor N/A 83% recruitment program. % Zobas that have at least one center staffed N/A 100% with trained HIV counselors. % health care professionals trained in 70% 75% diagnosis of HIV/AIDS and care of common opportunistic infections. % health care facilities reporting disruption in 10% 0% stock of anti-malarial drugs for more than one weeks during the previous 3-month. 1End of project Source: MOH - 26 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million A. Collect and analyze information on HAMSET to 4.84 2.88 60 facilitate evidence-based decision making and rapid response. B. Multi-sectoral control of HAMSET transmission. 10.16 6.75 66 2.1 Promote healthy behaviors through multi-level communication; 2.2 Promote healthy life styles through the education system; 2.3 Enhance access to preventive, diagnostic, and treatment care for conscripts; 2.4 Promote environmentally sound cost-effective techniques for malaria vector control. C. Strengthen HAMSET diagnostic, health care, and 20.18 22.04 109 counseling services. D. Community-managed HAMSET response. 8.36 16.93 203 E. Project Management 2.50 2.79 112 F. PPF 0.40 Total Baseline Cost 46.44 51.39 Physical Contingencies 1.39 Price Contingencies 2.17 Total Project Costs 50.00 51.39 Total Financing Required 50.00 51.39 - 27 - Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) ProcurementMethod1 Expenditure Category ICB NCB Other2 N.B.F. Total Cost 1. Works 0.00 3.31 0.00 0.00 3.31 (0.00) (2.70) (0.00) (0.00) (2.70) 2. Goods 4.69 0.00 2.37 0.00 7.06 (3.30) (0.00) (2.00) (0.00) (5.30) 3. Drugs & HAMSET devices 14.34 0.00 0.00 0.00 14.34 (15.00) (0.00) (0.00) (0.00) (15.00) 4. Consulting services, studies 0.00 0.00 9.14 0.00 9.14 (0.00) (0.00) (9.60) (0.00) (9.60) 5. Special funds 0.00 0.00 9.35 0.00 9.35 (0.00) (0.00) (5.80) (0.00) (5.80) 6. Project supervisions & 0.00 0.38 1.22 0.00 1.60 management (0.00) (0.30) (0.30) (0.00) (0.60) 7. Recurrent costs 0.00 0.00 0.00 4.87 4.87 (0.00) (0.00) (0.00) (0.00) (0.00) 8. PPF 0.40 0.00 0.00 0.00 0.40 (0.40) (0.00) (0.00) (0.00) (0.40) Total 3 19.43 3.69 21.96 4.89 50.00 (18.65) (3.05) (18.30) (0.00) (40.00) Notes: 1. Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2. Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. 3. The unallocated amount was US$ 0.6. - 28 - Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) ProcurementMethod1 Expenditure Category ICB NCB LIB Other2 N.B.F. Total 1. Works 4.58 0.36 0.00 2.80 0.00 7.75 (4.32) (0.34) (0.00) (2.64) (0.00) (7.30) 2. Goods 3.95 0.07 0.00 3.97 0.00 8.00 (3.69) (0.07) (0.00) (3.71) (0.00) (7.47) 3. Drugs & HAMSET devices 8.78 0.00 0.62 0.49 0.00 9.89 (8.77) (0.00) (0.62) (0.49) (0.00) (9.88) 4. Consulting services, studies 0.10 0.09 0.00 13.98 0.00 14.17 (0.10) (0.09) (0.00) (13.94) (0.00) (14.13) 5. Special Funds 0.00 0.00 0.00 6.18 0.00 6.18 (0.00) (0.00) (0.00) (4.95) (0.00) (4.95) 6. Project supervisions & 0.00 0.00 0.00 1.11 0.00 1.11 Management (0.00) (0.00) (0.00) (0.90) (0.00) (0.90) 7. Recurrent costs 0.00 0.00 0.00 0.00 4.30 4.30 (0.00) (0.00) (0.00) (0.00) (0.00) (0.00) Total 17.42 0.52 0.62 32.83 4.30 51.39 (16.89) (0.50) (0.62) (26.62) (0.00) (44.63) Notes: 1. Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2. Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate IDA Govt. CoF. IDA Govt. CoF. Bank Govt. CoF. A . Collect and analyze 3.70 1.44 2.09 0.79 56.5 54.9 information on HAMSET to facilitate evidence-based decision making and rapid response B. Multi-sectoral control of 8.74 2.42 5.65 1.10 64.6 45.5 HAMSET transmission C. Strengthen HAMSET 18.90 1.79 20.15 1.89 106.6 105.6 diagnostic, health care and counseling services D. Community-managed 5.98 3.91 14.86 2.07 248.5 52.9 response program E. Project management 2.28 0.44 1.88 0.91 82.5 206.8 and Evaluation F. PPF Refinancing 0.44 0.00 0.0 0.0 Total 40.04 10.00 44.63 6.76 111.5 67.6 - 29 - Annex 3. Economic Costs and Benefits This annex summarizes the findings of the economic analyses conducted by the ICR task team. The purpose of this analysis is to better understand the benefits, effectiveness and impact of the program. The HAMSET project was essentially an investment project. It sought to invest in Eritrea's main asset: its people. HAMSET directly affects the social and economic stability of its people, as its target diseases collectively constitute the highest burden on the productivity of Eritrea's people. Some of the benefits of this investment, both social and economic, are apparent and are expected to proliferate far into the future. Hence, it is important that the structures invested in be maintained and in some key areas, improved upon. This underpins the importance of the new HAMSET II project. While successes in malaria and to a lesser extent, HIV need to be maintained, there is significant room for further strengthening of the TB and STD facets of the project. Though appropriate at the time, the benefit analysis done in the PAD does not reflect the current situation. This is based on the fact that some of the assumptions made do not currently hold. It was assumed that the economies of scale would contribute greatly to the overall economic and effectiveness picture. However, because of varying lag times in implementation, certain components (malaria and HIV) were far ahead of others and severely limited the economies of scale that could have been realized. Secondly the malaria analysis did not take into consideration the decline in malaria epidemiological indices as at the commencement of HAMSET in Eritrea, hence minimizing the possible short-term economic returns. Malaria: The malaria component of the HAMSET project has been successful in achieving its set objectives, even surpassing them in some cases as for children under 5; morbidity has decreased by 84%, while mortality has decreased from 10.6 per 1000 in 2001 to 0.84 in 2004. Methodology: Economic evaluations seek to clarify the linkages between inputs, outputs and outcomes in health interventions. The malaria component is the only component that can be reasonably defined for its linkage between input (project investments) and outcomes (reduced malaria morbidity and mortality). Based on the recommendation from the Bank's Handbook of Economic Analysis of Investment Operations (1998, OPR), cost-effectiveness and cost-utility analysis are more appropriate for health projects when compared to cost-benefit analysis for two reasons. One is to clarify the objective of a health intervention, which is to improve health outcomes and not just to generate economic benefits. The second is the difficulty encountered in assigning monetary value to health outcomes and hence the difficulty of conducting a cost-benefit analysis. Hence effectiveness, which is a non-monetary term that measures the contribution of a health project to the improvement of health status, is the metric of choice. One possible way to grossly estimate the economic impact of HAMSET when it comes to malaria control would be by applying the cost of the malaria component directly against the mortality and morbidity conserved. As seen in Table 1, the total mortality averted was 291 deaths between 2000 and 2005. The approximated total expenditure on malaria was US$ 6.8 million. Hence, costs per malaria death averted (DA) would amount to US$ 23,368.00. This looks excessive compared to cost-effectiveness data available elsewhere. A 1999 study estimating the cost per malaria DA in - 30 - Gambia, Ghana and Kenya, found those values to be US$ 219 and US$ 665 in Gambia, US$ 2,112 in Ghana and US$ 2,985 in Kenya. Cost per malaria DA also seems excessive when compared to the cost of halving the malaria burden in SSA by 2015 (an MDG), which is US$ 21 per person. Another study (Graves, 1998) which compared the cost-effectiveness of using ITNs against the spf66 vaccine, found that the cost per DA for spf66 was US$ 252 and ITNs was US$ 711. Why does the HAMSET project seem so expensive? In considering this, it is important to consider three things: · Prior to HAMSET, there was already a steep decline in malaria mortality, having dropped from 533 in 1998 to 105 in 2000 (a 79% decline). This in part was due to investments by USAID and to malaria support by another Bank funded project, the Eritrea Health Project which lasted from 1998 to 2003 and hence ran in parallel with the HAMSET project for its first 3 years. Hence, HAMSET was continuing a process in active motion, in a different epidemiological phase and hence reducing the potential impact. At the same time, there was the need to keep the system to maintain the low level of malaria mortality and morbidity. · While the other studies included either uni- or bi-modular intervention projects (e.g. ITNs, chemoprophylaxis, etc.), the HAMSET malaria component was all encompassing, involving multiple substantive interventions (e.g. Entomology laboratories, community projects, training, institutional strengthening, BCC, and diagnostic improvement) as well as ITNs and chemoprophylaxis. Strictly speaking, the results are not comparable. · Other projects analyzed, built on the existing infrastructure of health systems already on ground e.g. trained staff, laboratories etc. HAMSET to a large extent had to build and create these systems from the ground up as systemic infrastructure was either insufficient or unavailable for the level of the malaria intervention needed. · HAMSET, by nature was an investment project and is structured to yield benefits into the future such as knowledge, practices and culture that have been promoted by HAMSET. As a result of the capitalization of the investment, many more lives will be saved. · HAMSET Malaria component is widely regarded as an exceedingly successful program in comparison to others world wide, making Eritrea one of the few countries that have already achieved the Abuja targets of the Roll Back Malaria program. The morbidity picture is similar. If this cost were measured against the total number of cases averted after HAMSET commenced, the cost per case averted came to US$ 31.61 using the morbidity figures recorded as are shown in Table 2. This seems high when compared to the comparative study mentioned above which showed that spf66 averted 50,502 attacks at a cost of US $ 3.71 while ITNs averted 69,415 attacks at US$ 15.75 each. The Goodman and Mills study showed that two separate interventions in Gambia averted cases at US$ 6.58 and US$ 1.35 a case respectively. The explanations mentioned above also apply to morbidity. Following the same arguments, the long-term effects of the program will save many more lives, thus improving the capitalization (human and social capitals) of that investment. Attitudes and practices have changed; communities know how to organize themselves in order to control the vector. These are the long-lasting effects of the program. If approximately 100,000 cases were averted in 2005 at the OPD level alone (and about 150,000 altogether), one may speculate that this trend would continue for several years (with decreasing numbers), even in the absence of HAMSET. Future lives saved and future reduction in productivity loss, thanks to fewer cases, would have to be discounted in order to weigh them against a financial cost that is fixed over a definite period of time (2001-2005). To do this, we used a discount factor of 10% and discounted for a period of 10 years into the future which came to US$9,008.00 per mortality averted and US$12.19 per morbidity averted. - 31 - Table 1: Malaria mortality in Eritrea (1998-2005) 1998 1999 2000 2001 2002 2003 2004 2005 Registered 533 199 105 133 86 79 16 32 mortality Table 2: Cost effectiveness of malaria control in Eritrea 2002 2003 2004 2005 Cases averted* 94,152 112,581 148,954 151,326 Cumulated 94,152 206,733 355,687 507,013 HAMSET's share of NMCP costs (2001-2005) 43% Number of cases averted attributed to HAMSET 216,456 (in 2002-2005) HAMSET's financial contribution to NMCP (2001-2005, in Nakfa) 93,229,519 Cost per case averted (2001-2005) in Nakfa 431 Cost per case averted (2001-2005) in Nakfa 31.61 *Computed on the basis of the 2001 level of morbidity in health facilities and communities; namely, the number of cases registered in 2001 minus the number of cases registered in the given year. HIV/AIDS: The main impact of HAMSET on HIV has been the stabilization of the pandemic, which could have otherwise spiraled out of control. Economically, the cost of managing such an epidemic would have been enormous. The majority of HAMSET funds were channeled towards HIV/AIDS although the exact amount cannot be determined due to the lack of provisions for attribution in the HAMSET program. The overall economic assessment on HIV/AIDS carried out under the Multi-country HIV/AIDS program (MAP) indicates significant net economic benefits of investing in HIV/AIDS with an internal return rate of around 30% (World Bank 2000). However, as the HIV component of HAMSET is immensely important, a quantitative analysis has been carried out on the CMHRP component as this sole component was able to discriminate funds according to disease. Though this is in no way representative of the total economic impact of HAMSET on HIV (as the whole component utilized just under 20% of the funds), it serves as a surrogate template that will be used for comparison with HAMSET II. Impact of the HIV component of the Community Managed HAMSET Response Program (CMHRP) a) The HeaLY approach was also used to calculate the impact of the CMHRP on HIV control in Eritrea. This is necessary to give some form of attribution of the HAMSET project to HIV control. Reasons are two-fold: a) the majority of HAMSET funds were spent on HIV linked activities and significant results were achieved in the HIV program such as the reduction of the prevalence from 2.8% to 2.4% as well as the proliferation of VCT sites. The HeaLY method, while not widely used, - 32 - provides a simplistic way of accounting for both morbidity and mortality, presenting a single figure which objectively guides decision makers as regards the level of commitment necessary to achieve a certain range of health outcomes. b) CMHRP is used because it was the only project component able to discriminate disbursements by disease. Also out of the 892 completed projects in the CMHRP component, 626 (70%) of them were HIV/AIDS related. In estimating the HeaLYs saved, we used data available in the final evaluation report as well as from UNAIDS (extent of disability, average age of death). Table 3: Cost effectiveness of malaria control in Eritrea Parameters 2000 2001 2002 2003 2004 2005 Total AIDS Cases 2,759 2,462 2,223 1,815 2,030 1,979 AIDS Deaths 339 239 216 204 227 151 Life Expectancy 52.2 52.2 52.2 52.2 52.2 52.2 Average age of death 28 28 28 28 28 28 Extent of disability due 0.32 0.32 0.32 0.32 0.32 0.32 to HIV HeaLY lost to deaths 8,407 5,783 5,227 4,937 5,493 3,654 HeaLY lost to disability 883 788 711 581 650 633 Saved HeaLY to Deaths 2,624 3,180 3,470 2,914 47,531,69 41 Saved HeaLY to 95 172 302 233 2,501,052 disability Total HeaLY saved 17,993 (years) Total HeaLY saved 6,567,445 (days) Total funds committed to HIV/AIDS in CMHRP component was 68,008,640.00 Nakfa (US $4,857,760.00). Ratio of HeaLY /project cost for the CMHRP component is 1.35 day per US$. The CMHRP component was allocated 19.8% of the funds used for the HAMSET 1 project, so again this value is not representative of the total present benefit derived from the HAMSET project. In addition, the period of time considered in this calculation is too short to accurately quantify HIV interventions as many of the effects are long-term. Tuberculosis: As has been noted in the document, TB morbidity has reduced, although there is some epidemiological confusion in the trends. Since we are unable to attribute cost to the TB component of the program, the economic impact of HAMSET on TB is difficult to assess. This would have been especially relevant given the evidence that the potential cost of lost productivity due to TB is 4-7% of GDP globally. In macroeconomic terms, WHO estimates that every dollar invested in the TB DOTS program would lead to a US$ 55 return to the country due to increased productivity. The TB program has been plagued by inadequate capacity, weak supervision, lack of quality assurance as well as a limited information base - 33 - for decision making, hence the failure to make provisions for adequate completion analysis. We, however, know that more patients are being detected and treated successfully that can now return to economic productivity if there are no other complications (e.g. concurrent HIV/infection). It takes, on average, approximately two years of suffering before a patient dies of TB. These two years are now reduced to two months of intensive treatment during which the patient is taken care of, fed, and saved in about 93-94% of cases. This has to be compared to an estimated 70% fatality rate when TB is not treated. Finally, it should be noted that while it would have been desirable to assess the impact of the project on all the HAMSET diseases, this was not possible because: · Allocations of funds were not disease specific; hence it was not possible to directly link the inputs of the HAMSET project with disease specific outcomes. The exception was with Malaria, which is estimated to have received approximately 16% of total IDA funds disbursed. This was reached by adding the amount disbursed to the NMCP and the expenditure on the Malaria component of the CMHRP program. · It was not possible to determine disease specific disbursements for any of the project components, save the CMHRP program, as available records show that disbursements covered more than one HAMSET disease (e.g. condoms for STIs and HIV/AIDS; drugs; consultants and operational expenses). The HAMSET project has no doubt contributed to Eritrea's development process. It trained thousands of people that are involved in various aspects of its implementation. It was successful in containing the spread of HIV, in itself a stellar achievement in the prevailing circumstances. The Malaria component has produced outstanding results which are significantly contributing to overall development. It is also clear that these effects will be long-lasting, long enough to repay its economic costs as well as yield dividends. This is, however partly conditional on the maintenance and strengthening of key interventions. TB and STD components are lagging behind and need to be supported to catch up with Malaria component and HIV in order to match epidemiological trends and effect o the economy. - 34 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 03/16/2000 8 TEAM LEADER (1); HD ECONOMIST (1); SCHOOL HEALTH SPECIALIST (1); EHP, MALARIA SPECIALIST (1); ENVIRONMENTAL SPECIALIST (1); COMMUNICATIONS SPECIALIST (1); MALARIA & TB ASSISTANT (1) 05/20/2000 5 TEAM LEADER (1); IMPLEMENTATION SPECIALIST (1); PROJECT COSTING & DESIGN SPECIALIST (1); COMMUNICATIONS SPECIALIST, EXTRO (1); PROGRAM ASSISTANT (1) 09/30/2000 9 TEAM LEADER (1); SR. SCHOOL HEALTH SPECIALIST (1); HIV/AIDS SPECIALIST (1); ECONOMIST (1); ENVIRONMENT & COMMUNITY PARITICIPATION SPECIALIST (1); IMPLEMENTATION SPECIALIST (1); PROJECT COSTING & DESIGN SPEC., CONSULTANT (1); SR. DEVELOPMENT COMMUNICATIONS SPECIALIST (1);WHO EXPERT IN COMMUNICABLE DISEASES (1) Appraisal/Negotiation 05/31/2001 8 TEAM LEADER (1); S S MALARIA SPECIALIST (1); LEAD SCHOOL HEALTH SPECIALIST (1); FINANCIAL MANAGEMENT SPECIALIST (1); HR ECONOMIEST (1); - 35 - COMMUNICATIONS ADVISOR (1) M&e CONSULTANT (1); HIV/AIDS PECIALIST, UNAIDS (1) Supervision 11/08/2001 8 TEAM LEADER (1); HR S S ECONOMIST (1); FINANCIAL SPECIALIST (1); MALARIA SPECIALIST (1); LEAD SCHL HLTH SPEC. (1); HIV/AIDS SPECIALIST (1); COMMUNICATIONS CONSULT (1); M&E CONSULTANT (1) 06/07/2002 10 LEAD HEALTH SPECIALIST S S (1); HR ECONOMIST (1); MALARIA SPECIALIST (1); COMMUNITY PARTICIP. (1); TB SPECIALIST (1); COMMUNICATION (1); HIV/AIDS (UNAIDS) (1); M&E (1); ARCHITECT (1); ENVIRONMENT (1) 11/22/2002 10 TEAM LEADER (1); SR. HD S S ECONOMIST (1); HEALTH SPECIALIST (1); COMMUNICA. CONSULTANT (1); LEAD OPER. SPECIALIST (1); SR. HEALTH SPECIALIST (1); CONSULTANT (2); INST. ASSESS. AND M&E (1); LEAD HEALTH SPECIALIST (1) 02/27/2003 6 TEAM LEADER (1); HEALTH S S PLANNER, CONSULTANT (1); HEALTH FINANCING SPEC., CONSULTANT (1) SR. ECONOMIST, M&E (1); MTL AFRICA COORDINATOR (1) RRI COORDINATOR, CONSULTANT (1); CONSULTANT IN M&E 06/26/2003 7 LEAD HEALTH SPECIALIST S S (2); SR. HD ECONOMIST (1); HEALTH SPECIALIST (1); SR. HEALTH SPECIALIST (1); COMMUNIC. CONSULTANT (1); SR. ECONOMIST (1) - 36 - 02/06/2004 7 TASK TEAM LEADER (1); S S HEALTH SPECIALIST (1); SENIOR HD ECONOMIST (1); CONSULTANT (COMMUNICAT (1); LEAD EDU. SPECIALIST (1); SR FM SPECIALIST (1); TB CONSULTANT (1) 06/26/2004 2 HEALTH SPECIALIST (1); RRI S S CONSULTANT (1) ICR 08/19/2005 7 TASK TEAM LEADER (1); S S HEALTH SPECIALIST (2); OPERATIONS Officer (1); JUNIOR PROF. ASSOCIATE (1); PROCURMENT SPECIALIST (1); CONSULTANT IN M&E (1) 02/17/2006 8 TASK TEAM LEADER (1); S S HEALTH SPECIALIST (2); LEAD HEALTH SPECIALIST (1); SR. FINANCIAL MANAGEMENT SPEC. (1); SR. OPERATIONS Officer (1); PROCURMENT ANALYST (1); TB CONSULTANT (1) (b) Staff: Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation 15.57 65.54 Appraisal/Negotiation 36.33 152.91 Supervision 112.67 381.09 ICR 34.14 118.40 Total 198.71 717.94 - 37 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 38 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 39 - Annex 7. List of Supporting Documents 1. CA GOODMAN and AJ MILLS: "The evidence base on the cost effectiveness of malaria control measures in Africa". Health Policy and Planning and planning; 1999:14 (4): 301-312. 2. HAMSET Project Appraisal Document. November 27, 2000. 3. Eritrea HAMSET Aide- Memoires 2001-2005. 4. Eritrea Health Project ICR. June 16, 2005. 5. HAMSET ICR Mission Aide-Memoire February, 17, 2006. 6. HAMSET Mid-Term Review Documentations. November 22-December 11, 2004. 7. Socio-economic Impact Analysis for HAMSET. February 28, 2006. 8. HAMSET Implementation Status Reports 2000-2005. 9. Eritrean Catholic Secretariat HIV/AIDS report, January 2003-December 2005. 10 Updated and Consolidated HAMSET Diseases Situation and Response Analysis March 7, 2006. 11. Eritrea: HAMSET, IECD and ESIP projects, School Health and HIV/AIDS Components Aide- Memoire. September 29, 2005. 12. Final Report Cultural Affairs Bureau for the HAMSET Control Project, December 2005. 13. Final Report on the HAMSET Project Activities of HAMSET by BIDHO (Eritrean Association of PLWHA). November 10, 2005. 14. Final Implementation Completion Report for HAMSET by Independent Consultants. December 2005. 15. CMHRP Final HAMSET Report. March 20, 2006. 16. Eritrea Integrated Childhood Development Project Appraisal Document. May 15, 2000. 17. Quality Enhancement Review. July 2000. 18. The World Bank's Handbook of Economic Analysis of Investment Operations. 1998. - 40 - Additional Annex 8. Government Confirmation for Acceptance of HAMSET I ICR - 41 - 36°E 38°E 40°E ERITREA 18°N 18°N To Port Sudan Karora SELECTED CITIES AND TOWNS REGION CAPITALS NATIONAL CAPITAL RIVERS Hagar Nish Sala Plateau MAIN ROADS SUDAN RAILROADS Erghershatu (2576 m) Nakfa R REGION BOUNDARIES Barka A N S E B A NORTHERN E INTERNATIONAL BOUNDARIES D Anseba RED Gulbub SEA (SEMIEN-KEIH- Dahlac 42°E This map was produced by 16°N BAHRI) the Map Design Unit of The Archipelago S World Bank. The boundaries, Keren E colors, denominations and any other information shown A on this map do not imply, on Akurdet Massawa the part of The World Bank Jemahit Group, any judgment on the To CENTRAL Ingal legal status of any territory, Kassala Sebderat (MAEKEL) or any endorsement or a c c e p t a n c e o f s u c h G A S H - B A R K A ASMARA Alighede boundaries. Barentu Teseney Dekemhare Gash Areza S O U T H E R N Buia Mendeferas Adi Keyh Fatuma (DEBUB) Tio REP. OF Adi Quala D en a YEMEN Um-Hajer k il Tekeze To S O U T H E R N Adirgat 14°N R E D S E A To Idi 14°N Adi Abun (DEBUB-KEIH- BAHRI) E T H I O P I A ERITREA Beylul Assab 0 20 40 60 80 100 Kilometers SEPTEMBER Andale IBRD 0 20 40 60 80 100 Miles To To Logiya 33403 Djoboiti 2004 36°E 38°E 40°E 42°E DJIBOUTI