Document of The World Bank FOR OFFICIAL USE ONLY Report No: 20566 IMPLEMENTATION COMPLETION REPORT (39810) ON A LOAN N THE AMOUNT OF US$ 24.8 MILLION TO THE REPUBLIC OF INDONESIA FOR THE HIV/AIDS AND STDS PREVENTION AND MANAGEMENT PROJECT June 16, 2000 Human Development Sector Unit East Asia and Pacific Region This document has a restricted distribution and may be used by recipients only in the perfornance of their I official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective As of April 2000) Currency Unit = Indonesia Rupiah (Rp) Rp. 1 million = US$ 125.87 US$ 1 = Rp. 7,945 FISCAL YEAR April 1 March 31 ABBREVIATIONS AND ACRONYMS AIDS Acquired Immune Deficiency Syndrome KfW Kredietanstalt fur Wiederaufbau APBN National Development Budget KPKN Ministry of Finance Treasury Office AusAID Australian Agency for International Development Localizatio Regulated CSW complex n Bappenas National Development Planning Board MCH Maternal and Child Health BLK Provincial Laboratory MOF' Ministry of Finance BPKP Central Audit Bureau MOH Ministry of Health CHE Center for Health Education NAC National AIDS Commission CSW Commercial Sex Worker NGO Non-governmental Organization DG of Directorate General of Communicable Diseases PAC Provincial AIDS Commission CDC-EH Control and Environmental Health Dinas Local Government Health Office Pimpro Project Officer Kesehatan DIP Project budget allocation PIU Provincial Implementation Unit DKI Special region of Jakarta PMU Project Management Unit Jakarta ELISA Enzyme Linked Immunosorbent Assey SAR Staff Appraisal Report GOI Govemment of Indonesia SOP Standard Operating Procedures HIV Human Immunodeficiency Virus STDs Sexually Transmitted Diseases ICB International Competitive Bidding UNAIDS The Joint United Nations Programme on HIV/AIDS IEC Information, Education, Communication USAID United States Agency for International Development Kanwil Ministry of Health Provincial Office UJNFPA United Nations Population Fund Kesehatan Keppres Presidential Decree WHO World Health Organization Vice President: Jemal-ud-din Kassum, EAPVP Country Manager/Director: Mark Baird, EACIF Sector Manager/Director: Alan Ruby, EASHD Task Team Leader/Task Manager: Samuel S. Lieberman, EASH) FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT INDONESIA (LOAN 3981) HVAIDS AND STDS PREVENTION AND MANAGEMENT PROJECT CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 1 4. Achievement of Objective and Outputs 4 5. Major Factors Affecting Implementation and Outcome 9 6. Sustainability 10 7. Bank and Borrower Performance 11 8. Lessons Learned 13 9. Partner Comments 15 10. Additional Information 19 Annex 1. Key Performance Indicators/Log Frame Matrix 20 Annex 2. Project Costs and Financing 23 Annex 3. Economic Costs and Benefits 25 Annex 4. Bank Inputs 26 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 27 Annex 6. Ratings of Bank and Borrower Performance 28 Annex 7. List of Supporting Documents 29 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not be otherwise disclosed without World Bank authorization. Project ID: P039643 Project Name: ID-STD/AIDS Team Leader: Samuel S. Lieberman TL Unit: EACIF ICR Type: Core ICR Report Date: June 14, 2000 1. Project Data Name: ID-STD/AIDS L/C/TFNumber: 39810 Country/Department: INDONESIA Region: East Asia and Pacific Region Sector/subsector: HT - Targeted Health KEY DATES Original Revised/Actual PCD: 01/10/95 Effective: 05/15/96 05/15/96 Appraisal: 08/15/95 MTR: Approval: 02/27/96 Closing: 09/30/99 09/30/99 Borrower/Implementing Agency: GOI Other Partners: STAFF Current At Appraisal Vice President: Jemal-ud-din Kassum Russell J. Cheetham Country Manager: Mark Baird Marianne Haug Sector Manager: NA-pls edit Roles in SAP Himelda Martinez Team Leader at ICR: Puti Marzoeki Samuel S. Lieberman ICR Primary Author: Puti Marzoeki 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: UN Institutional Development Impact: N Bank Performance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: S Project at Risk at Any Time: Yes 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: The project's objective was to develop institutional mechanisms and interventions for reducing the transmission of STDs and AIDS by using intensive pilot efforts which would be monitored and evaluated for the purpose of identifying the optimal interventions. The proximate objectives included enhancing awareness about WHV and STD transmission and prevention strategies in specific groups; promoting behavior change among those participating in risky sexual activities; increasing the knowledge and skills of health providers who deal with STDs and HIV; improving the capabilities of the health laboratory system as regards STDs and HIV; and establishing effective surveillance mechanisms for STDs and WHV, and monitoring and evaluation systems to gauge the impact of various interventions. The project was to be implemented in DKI Jakarta (Special Region of Jakarta) and Riau provinces for a period of three years. GOI acknowledged Indonesia's vulnerability to IHV and the negative economic impacts of a possible epidemic. As early as 1986, MOH had established an AIDS working group which later became the Control Committee that drew up short and medium term plans for HIV/AIDS prevention control. A National AIDs Commission (NAC) was created later through a Presidential Decree dated May 30, 1994. The Committee was chaired by the Coordinating Minister for People's Welfare, and involved 17 cabinet ministers including the Minister of Health, Minister of Religious Affairs, Minister of Social Affairs and the State Minister of Population as Vice Chairmen. An accompanying Ministerial Decree was released on the same year providing guidelines in canying out HIV/AIDS prevention and control programs. While project objectives were consistent with the direction of the national strategy, the intention to accomplish them in a short term framework was ambitious, particularly since some of them involved behavior change targets. Moreover, most of the tools to be used in the comprehensive program were still to be developed. 3.2 Revised Objective: Project objectives were not revised throughout the project. 3.3 Original Components: The project consisted of provincial level components which addressed WV/AIDS and STDs interventions in DKI Jakarta and Riau Province, and complementary central level activities. The provincial components included: 1) IEC and behavioral interventions directed at various groups engaging in high risk behavior, health providers and technicians, workplace and school audiences, and community organizations; 2) biomedical services to support the training of staff in the syndromic approach and infection control procedures, training of laboratory staff and purchase of equipment and materials needed to perform tests, and stepping up of HIV and STDs surveillance activities; 3) management and support through the establishment of a province level management unit and coordination functions to be carried out by the provincial AIDS commission. Central support was to include guidelines, modules, standards and materials development for both IEC and biomedical interventions; technical assistance; and research activities. The whole approach was seen as a pilot since when the project was designed, HIV/AIDS control was a relatively new program in Indonesia and there was no model for a comprehensive approach. -2 - Components Cost (Appraisal) US$ million Jakarta: IEC and Behavioral Interventions 4.9 Biomedical services 4.3 Management and Support 0.9 Riau: IEC and Behavioral Interventions 2.9 Biomedical Services 4.5 Management and Support 0.8 Center: IEC and Behavioral Interventions 3.5 Biomedical services 11.7 Management and Support 1.7 Total 35.2 The pilot was intended to produce a cost effective and sustainable intervention package to slow transmission of HIV and STDs in Indonesia. The components reflected the need to comprehensively address various behavior change and service delivery measures, accompanied by intensified surveillance. More importantly, they also recognized the synergistic relationship between HIV and STDs. Considering that the project was a pilot expected to yield short term results, a separate monitoring and evaluation component would have given stronger emphasis on the need to draw lessons from the project. Having it as a separate component would ensure resource allocation and full attention from the implementing agencies. To ensure learning from non-biased information, an independent monitoring and evaluation unit could have been considered. Assessment of project achievements, especially in pilot projects, would have been more appropriate if the evaluation design was not limited to "pre and post" as was intended for the project, but also included "with and without". Although design of the project indicated some process indicators, no tracking system was designed to evaluate trials and methodologies. The absence of a plan to merge monitoring data that tracks implementation of the project with impact assessment data would have made it impossible to say whether the project failed to produce the intended changes or the intended changes failed to produce the desired outcomes when the impact assessment indicated no difference between project and no project. 3.4 Revised Components: During the last year of the project (1999-2000), only the surveillance component continued to be implemented and was limited to the dissemination of surveillance SOPs to eight provinces. Two new activities were introduced: the External Assessment of HIV/AIDS and STDs in Indonesia, and the preparation of HIV/AIDS and STDs Masterplans for Irian Jaya and Central Java. 3.5 Quality at Entry: The project recognized the importance of combining prevention and management efforts and carefully identified key factors influencing HLV transmission in Indonesia and the elements required to control the spread of the disease. Consultants were recruited to develop important components including the laboratory and surveillance systems. IlV/AIDS experts from CDC Atlanta were involved, as well as NGOs which were experienced in high risk group interventions in Indonesia. The MOH preparation team was led by the Bureau of Planning and consisted of all units related to HIV/AIDS control within MOH. The project provinces participated actively during preparation by developing province specific proposals. However, the - 3 - Bank had overestimated the capacity of the implementing units and the DG of CDC-EH as the managing unit, when conducting the institutional assessment. The capacity of the units might be adequate for a long term project with enough time for capacity improvement during the first few years. But this was not the case for this project. The project was of limited duration. It was also intended to plan and implement numerous activities, involving multiple groups of people at numerous agencies as well as individuals in different levels and locations. Due to the project's complex nature, it would be difficult to make effective changes during implementation. The weak institutional assessment might have contributed to the difficulties the project faced during implementation. More attention should also have been given to readiness for implementation. The baseline surveys, consultants' TORs and procurement plan, modules and guidelines should have been developed during preparation and they should have been available by negotiations or at the latest by effectiveness. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The assumption underlying the project design was that HIV/AIDS was a real threat to Indonesia, although it was not quite certain when the country would reach a stage where the number of HIV and AIDS cases were going to increase at an exponential rate. In reality, until the end of the second year of the project, reported AIDS cases were still below 300. This phenomenon had affected the level of government commitment and ownership of the project which was reflected in the poor project performance. Baseline data collection and needs assessment were not done during the first year as intended because guidelines were not ready and no budget was allocated for the task. The activity was delegated to the provinces and the NGOs. Results of the baseline, as shown by the provincial reports, did not reflect the performance indicators agreed for the project. Final data collection was never scheduled and was not conducted until the end of the project. Considering that the project was a pilot, measurement of project outcomes were irnportant to assess effectiveness of the interventions. In the absence of a comprehensive baseline and final data collection, project outcome was immeasurable. Based on these findings, the rating for project outcome was unsatisfactory. 4.2 Outputs by components: A. LEC and Behavior Change Interventions The IEC interventions were targeted toward 5 main groups: the high risk groups, individuals in the workplace and schools/universities, the community and the general public, and health providers. Numerous media materials consisting of electronic as well as printed channels were produced during the project. Electronic media items included radio spots, TV fillers and 16 mm films, while printed media materials included posters, leaflets, T-shirts, handbooks, stickers, calendars, booklets, flipcharts, billboards and banners. No data were documented on the number of people who received training or participated in sessions to increase awareness or change the behavior related to risky sexual practices. Monitoring of training activities focused more on the training process. The effectiveness of the training sessions and the media used in the training was inassessable due to the absence of post training evaluation. The project recognized the high risk group as an important link in STDs and HIV transmission. IEC interventions were intended to target these groups, particularly in regards condom use. Despite the emphasis in the SAR on behavior change targets, the govemment's IEC approach, developed by the Center for Health Education, chose to follow a national strategy aimed at increasing the awareness of the general public. The difference of perception was a source of continuing methodological debate between GOI and -4 - the Bank, and contributed to reduced focus on the interventions. The need to focus on high risk groups was also stressed by the consultant hired to assist CUE in developing IEC methodology and implementing an effective IEC strategy. The concern was that government formal institutions and the NGOs tended to concentrate on high risk groups within the "localizations" because they were easy to identify and approach, while in reality the majority of high risk groups operated outside the "localizations". The project was able to identify and list the high risk groups in DKI Jakarta and Riau which ranged from street CSWs, drink and snack peddlers, street children, night club/bar/karaoke visitors, tourist guides, pimps, massage parlour and beauty parlour workers to transvestites, ABGs (junior high and high school students who gathered at malls and disco places looking for dates), sailors, fishermen, truck drivers and about a dozen more groups. Unfortunately, until the project ended little IEC efforts were directed to these specific groups. The peer group approach was considered effective for CSWs in localizations, but little was learned about how to approach the other groups. The CHE was not providing assistance to the provinces in reaching the so called "unreached" groups. The role of NGOs in delivering the IEC component was very significant. They conducted CSW peer group training, as well as training in schools and workplaces. Some NGOs were also involved in training health providers on the syndromic approach and universal precaution. The NGOs proved effective in reaching the high risk groups. Many CSWs and other high risk groups participated in NGO organized training sessions. Other noted achievements including development of a network for patient referral among health centers and hospitals in Jakarta as a follow up to the universal precaution training, involvement of an increasing number of facilitators in universities and schools, and development of a supporting program for the adolescents, and a CSW organization working on reproductive rights protection. The project realized that despite their potential, many of these NGOs lacked adequate management capacity which made sustainability of their activities a challenge. An NGO capacity building category was created to address such concerns. Indeed, some capacity building efforts were tried during the project, including proposal development training; management training; provision of grants as stimulants; pairing between smaller, inexperienced NGOs and larger, well established NGOs; and establishment of an NGO communication forum. Unfortunately, these efforts were not well planned and were more of a one time activity. Only about 17% of budget allocated in this category was utilized. The number of NGOs working on HIVIAIDS increased dramatically during project implementation, but only the NGOs which were already active prior to the project could maintain their activities after the project ended. Despite their positive contribution, involvement of NGOs in the project was not easy. Although an NGO involvement procedure was agreed during project negotiations, in practice, the Ministry of Finance Treasury Office (KPKN) and the BPKP insisted that the NGOs were to be contracted according to Keppres 16, 1994, which regulates national procurement of goods, works and services. The NGOs, particularly the smaller ones, had difficulties in fulfilling the requirements of Keppres 16. Such rules were debilitating for the project, which by design intended to intensively involved the NGOs in delivering project activities. Not until late 1998, was a Bappenas-MOF joint circular finally released to rectify the financial mechanism for NGO involvement in GOI financed projects. Although the circular was produced much too late to be of use for the project, the project succeeded in enabling fornal support for future NGO involvement. The USAID funded HAPP (HlV/AIDS Prevention Project) launched condom social marketing activities among others in North Jakarta at roughly the same time as the project. The activities were aimed at increasing condom use among sex workers and their clients, increasing access to condoms and increasing sales of condoms. The project reported respectively 10% and 6% increase in the number of sex workers and clients reporting condom use in project areas after almost three years (1996-1999), and respectively 18% and 26% increase in the sales of commercial brand condoms during the first quarter of 2000 compared to -5- the same period last year. Although not remarkable these were positive results. There were three things that might have contributed to the relative success of the HAPP interventions compared to those of the Bank funded project. First, the campaign was contracted to a professional private company; second, an effective marketing strategy was implemented including distribution, promotion, advertising, public relations, and media advocacy in collaboration with condom producers, non traditional condom selling outlets, the media and NGOs; and third, research and solid monitoring and evaluation were used effectively to support the campaign. B. Biomedical services The Biomedical services included development and testing of procedures relating to the syndromic approach and universal precaution, enhancing surveillance activities, and improving laboratory support for HIV and STDs testing. Syndromic approach and universalprecaution training Training modules for the syndromic approach training were not ready when the project started, and were only made available to the provinces during the last year of the project. Since budget for the training was already allocated during the first year of the project, the provinces had to use draft modules for the earlier training batches. The training was done in single sessions with no follow up through refresher training or continuous supervision efforts. No evaluation on health providers' knowledge and skills was conducted six months post training as was required in the SAR. The capacity to procure drugs to support implementation of the syndromic approach was not considered when planning the number of providers to be trained which created discouragement for using the approach. Counseling, another important aspect of the syndromic approach, was not constantly given to clients, particularly in relation to the importance of examining sexual partners. Health providers in hospitals and health centers also received universal precaution training. Aside from the number of trained providers, little information is available on post training follow up. Four studies were implemented during the project. They were: Study on Infection Control, Study on Microbiological Resistance, Study on STDs among pregnant women, and Study on STD Management using the Syndromic Approach. These studies provided valuable inputs not so much for the project, but for developing future policies and programs. Surveillance The surveillance component had similar problems. The Standard Operating Procedures (SOPs) were only finalized during the last year of the project. From the 8 SOPs initially drafted, only 2 SOPs, i.e., the SOP for AIDS surveillance and the SOP for sentinel surveillance, were declared by the Bank ready for trial and dissemination. The trial was later conducted using WHO funds. Assessment by the external expert team hired by the project found some uncertainties at the provincial level about sampling frames, especially in relation to non-localization sex workers and the lines of reporting. Sentinel Surveillance was among others conducted in sex worker localizations. There were some disadvantages in using those localizations. In DKI Jakarta, for example, the decision of the local government to close down all sex worker localizations had disrupted surveillance activities in this province. Another disadvantage was the high turn over of sex workers in the localizations, and this was true not only for Indonesia. Perfornance of the surveillance component was affected by the institutional set up within the DG of CDC-EH. Beside a surveillance subdirectorate under the Directorate for Epidemiology and Immunization, -6 - there were separate surveillance units under subdirectorates of specific diseases, including one under the subdirectorate of HfV/AIDS and STDs. This situation influenced division of responsibilities and affected the working environment between the two subdirectorates. This was a sensitive issue and might have been beyond the project to resolve, but the condition had contributed to delays in finalizing the SOPs and poor quality of the initial drafts. Overall, the surveillance component still needed a lot of strengthening. Behavioral surveillance was the least developed, little information was available on this. On HIV serosurveillance, the guidelines should be strengthened to ensure that implementation is clear on the type of target groups, sampling procedures and lines of reporting from district to province to national level while still allowing some flexibility at the provincial level. The protocol for surveillance in large STD clinics and in tuberculosis patients should be introduced and implemented. Collaboration with NGOs in reaching the marginalised population groups to participate insurveillance activities needs to be explored. Surveillance on transvestites and injecting drug users is still to be strengthened. Laboratory component The laboratory component consistently followed project design. Centers of excellence were established in 4 cities: Jakarta, Medan, Surabaya, and Ujung Pandang. Two staff from each center were trained at the National Reference Laboratory at the Cipto Mangunkusumo hospital with the assistance of an international consultant in 1997. From these four centers, only Surabaya fully functioned as a center of excellence in a sense that it conducted confirmation tests, served as an anchor in quality assurance, carried out training and gave guidance to provincial laboratories in its catchment area. The other three centers lacked the Western Blot equipment to do the confirmation testing, due to the failure in completing the second ICB package. Surabaya got its Western Blot equipment from JICA. Eight provincial laboratories (BLKs), six hospitals in Riau and five hospitals in DKI Jakarta received equipment and their staff were trained to improve capacity in HIV and STD testing. Some staff from private health laboratories also participated in the training. Quality assurance activities were ongoing and the results were satisfactory. The main constraint for this component was that not all intended equipment, including the Western Blot, was procured due to problems in following ICB procedures. And this influenced the performance of the laboratories particularly that of the centers of excellence. Five million dollars worth of reagents were not procured because national needs for reagents were already fulfilled through APBN allocations for the DG of CDC-EH. Had this been disclosed during project preparation, the loan size could have been reduced by 20%. This was a surprising finding , particularly because the project was implemented during the time when the country was in the middle of the economic crisis and rupiah resources were very limited. GOI could have taken advantage of the resources made available by the project. What happened indicated inadequate coordination and communication between the Center for Health Laboratories and the DG of CDC-EH. Most of the testing carried out by the provincial BLKs was for surveillance purposes. Their use by the general public for HIV testing is still limited. Supply of kits for both rapid tests and ELISA for the BLKs prior to decentralization was the responsibility of the central government. With decentralization, assuring regular supply is a potential problem. Despite the advocacy from the central level, there are risks that the provinces put less priority on allocating money to sustain HIV/STD testing at the BLKs. Ongoing quality assurance system for IRV antibody testing appeared to function well. Proficiency panels were sent regularly to the BLKs, and results of the evaluation were reviewed and feed back was given by the Center for Health Laboratories as well as the international consultants. More attention should be given to variation in test performance caused by storage, transportation and operator competency. The national guidelines and systems for external quality assurance of HIV test kits and testing procedures may need to be strengthened to accommodate those concerns. -7 - Despite the failure to purchase the reagents and uncertainty about funding sources for maintaining the functions of BLKs, in general the laboratory component has fulfilled the intended project objectives satisfactorily. C. Management and support Over the course of the project, project management unit functioned ineffectively, resulting in frustration for all parties involved. The continuing problems in coordination, DIP planning, NGO contracting, lack of management control systems and delays in SOP/module development that hampered the logical sequence of activities, were symptoms that signaled more fundamental problems. At the time the project was designed, MOH was expected to take proactive leadership to assume the new tasks and responsibilities imposed by the threat of the AIDS epidemic. This was reasonable justification for institutional adjustrnents. Moreover, the prevention and management notion of the project, and the desire to develop institutional mechanisms to support the program were ambitious goals to be achieved in a three year period. Such endeavor required a strong project management support, more than what was customary for traditional projects. Under such standpoint, the Project Management Unit (PMU) at the central level was established to have Echelon II functions. Its scope of work included providing technical support to central and provincial structural units, coordinating the activities of four echelon two level units at the central level, i.e. the Directorate of Directly Transmitted Diseases, Directorate of Epidemiology and Immunization, Center for Health Education and Center for Health Laboratories, and finally, coordinating all HIV/AIDS donor funded projects in Indonesia. In practice, this arrangement was the source of conflict for most of the project life time because the tasks of the PMU duplicated those of the STD subdirectorate under the Directorate of Directly Transmitted Diseases. The problem was enhanced by the appointment of a person with no structural position to become the Project Manager who reported directly to the Director General of CDC-EH. The person had difficulties establishing his authority in coordinating Echelon II level officers involved in the project. The organization structure was a deviation from more traditional projects in which the PMU was headed by an Executive Secretary who was responsible for project administrative matters only, and reported to the Project Director through a structural Echelon II Project Manager. Project planning, particularly budget allocation, was a problem from the start. In FY 1996/1997 (project first year), counterpart budget was missing for all components except for the biomedical component at the DG of CDC-EH. DIP revision procedures were complex and lengthy. When the budget was finally made available, only about 3 months were left for implementation. Adjustrnents were hardly done during the following year. Despite the activities carried over from the previous year, a full program as outlined in the project cost tables was entered into the DIPs, bringing heavy workload for the implementing units. The importance of sequencing of activities was lost on the project. The central level allocated fund for module/SOP preparation and training of the trainers during the first year of the project, while the fund for the training itself was allocated at the provincial DIPs for the same year. When central activities were delayed, and there were only 3 months to complete non-carried over activities, the provinces had no choice but to conduct training without proper guidelines. During implementation, consultants, international as well as national, were hired to support the PMU as well as the implementing units. Consultant perforrnance, when assessed against the utilization of products, varied across implementing units. Most of their products were recommendations. Visible products were quality assurance systems for the laboratories and SOPs for surveillance and STDs and HIV/AIDS services. For most of the national consultants, participating units could not state clearly whether the - 8 - consultant services were satisfactory. Unclear outputs from these consultants reflected their unclear scopes of work and lack of control from the users on the quality of their work. Since the project was a pilot effort, monitoring and evaluation were discussed lengthily in the project documents. However, throughout the project there was almost no systematic monitoring to track the process of carrying out activities for program implementation, or evaluation to measure progress of the project towards its objectives, i.e., outcome objectives which specified changes in behavior or services as a result of project activities; and impact objectives which described expected changes in STD/H1V incidence and prevalence. Although a draft guidelines for baseline data collection was available, only a few people were aware of the need to collect baseline information, and no final data collection was scheduled or carried out until the end of the project. As the result, the effectiveness of most interventions was not measurable. Project management at the provinces was too dependent on central guidance and was lacking initiatives to cover for central weaknesses. In Riau, the PIU functioned more effectively because its head was the head of Dinas Kesehatan and therefore guaranteed project implementation. The executive secretary was at Kanwil Kesehatan and Pimpros of the project were also at Kanwil Kesehatan. The mixture of Kanwil and Dinas personnel at the PIU facilitated linking between project technical and financial aspects. In DKI Jakarta, on the other hand, all PIU members including the executive secretary were at the Dinas Kesehatan office while all Pimpros were at Kanwil Kesehatan. This arrangement created some coordination problems between project implementers at Dinas Kesehatan and the Pimpros at Kanwil Kesehatan.. 4.3 Net Present Value/Economic rate of return: In the design, project benefit was calculated based on the direct (treatment costs) and indirect (the product lost to the economy due to the death of adults of working age) costs saved by averting a certain number of deaths from AIDS. The number of HIV and AIDS cases was predicted using the iwg-AIDS projection model. Based on the model, the predicted cumulative figures for AIDS cases in Jakarta alone was 55,000 by 2005, and 123,000 by 2010. In reality, the DG of CDC-EH reported only 253 AIDS cases throughout Indonesia by August 1998. The actual number of reported cases is far too low for applying the scenario of calculating the economic rate of return based on savings from direct treatment costs of AIDs cases as well as indirect economic benefits as was intended. Therefore, the ERR was not calculated for this ICR. 4.4 Financial rate of return: Not applicable. 4.5 Institutional development impact: Institutional development impact is negligible except for improvement of the capacity of the four center of excellence laboratories, six provincial health laboratories, eleven hospital laboratories, and some private laboratories. As explained previously under the Management and Support paragraphs, the goal to develop institutional mechanisms to support HIV/AIDS management efforts was not achieved. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: The project was designed and implemented at the time when the possibility of H1V/AIDS epidemic in Indonesia was a major concern for a large number of donor agencies. The Bank project was implemented at the same time with at least 7 other donor funding AIDS-Related Activities. These donors were: UNAIDS, WHO, AusAID, USAID, KfW, UNFPA and the European Union. Although each worked on different programs and geographical areas, the demand for time of the GOI counterpart at the central level was substantial. Combined with the poor GOI coordinating role, this adversely affected the quantity and quality of time given by the central units to this project. Delays in finalizing guidelines and modules were partly -9- related to the excessive workload of central staff. The economic crisis which hit the country in 1997 also had a negative impact on the project. Some activities could not be implemented due to lack of counterpart budget. ICB was delayed in part because Letters of Credit from Indonesia were rejected by foreign banks. The crisis also forced GOI to make savings from development budget allocations. In FY 96/97, the APBN budget for the whole country was frozen for about four months pending the results of negotiations between GOI and IMF. 5.2 Factors generally subject to government control: Strict government regulation and procedures based on Keppres 16, 1994 were applied to all NGOs, despite the guidelines and supporting legal documents agreed during project negotiations. The difficulties of the NGOs in complying with those regulations caused significant delays in implementing activities contracted to the NGOs. The problem was not resolved until late 1998, when a Bappenas-MOF joint circular was finally released rectifying financial mechanisms for NGO involvement in GOI financed projects. 5.3 Factors generally subject to implementing agency control: The project organization at the central level included a unit responsible for technical as well as administrative matters. Inclusion of technical responsibilities proved to be counter productive. There was constant conflict with the subdirectorate of STDs and HIV/AIDS within the DG of CDC-EH. The weakness of project management structure was magnified by staffing problems. The decision to change the structure of the PMU and its staffing was made very late in the project The weak PMU deprived the project of the strong management support needed to guide and coordinate the various implementing units involved in implementing a three year project. The missing counterpart budget during the first year of the project, lack of guidance in conducting baseline data collection, the absence of a systematic project monitoring and evaluation, lack of coordination among the implementing units, and NGOs contracting problems were some of the main problems causing poor project performance. 5.4 Costs and financing. The project disbursed very little while it was being implemented. US$ 19.8 was canceled on August 26, 1999, a year before project closing date. Each component absorbed less than 20% of the allocated budget. Factors affecting low disbursement among others were overly ambitious allocation during project preparation, very limited activities during the project first year due to the missing counterpart budget, unsuccessful ICB procurement, unrealized reagent procurement, and the huge increase of the exchange rate during the economic crisis. 6. Sustainability 6.1 Rationale for sustainability rating: Sustainability for this project is unlikely due to high dependency on project resources. Activities have not been sustained since the project ended. The current CDC budget allocation in central DIPs is limited for conducting serosurveys and sentinel surveillance. Some funds are available from the local government at the provincial level, mostly to fund meetings of the PAC (Provincial AIDS Commission). There is also concern over the continuity of reagents and testing kits provision for the provincial laboratories. Despite possible problems in acquiring laboratory supplies, the achievement of the laboratory component as a whole is relatively more sustainable than that of the other project components. The unit is committed to conducting the external quality assessment by maintaining contact with the international consultant recruited by the project to assist in the development of the STD and IV/AIDS testing system. The consultant sends the proficiency panel regularly to monitor the performance of the centers of excellence. Results of the proficiency panel evaluation are satisfactory. - 10- The sustainability of the IEC interventions is affected by the characteristics of the target groups. The peer group activities, for example, although considered effective, are prone to high turn over of facilitators as well as the CSWs. These peer group activities were practicable because of the existence of the CSW localizations. The DKI Jakarta policy to close down CSW localizations in Jakarta has made these activities hard to sustain. Such policy has also affected the utilization of CSW localization as sentinel surveillance areas. Currently, sentinel surveillance is carried out in 14 Indonesian provinces and CSW localizations are consistently chosen as the primary sentinel areas. Although sentinel surveillance is also carried out in a number of other population groups, the range of groups chosen across Provinces and in different years makes it somewhat difficult to use these results to establish national trends or patterns. Continuing pressure from religious groups throughout the country might force other provincial governments to close down CSW localizations in their areas, and thus affecting sustainability of surveillance data. Many NGOs involved in implementing the behavior change component in the project have discontinued their activities due to lack of other funds. This is attributable to two things; first, the project triggered the setting up of new NGOs which were less experienced and had less credibility with funding agencies; second, the capacity building efforts tried in the project had limited success in making these NGOs self sustaining. Sustainability of funding for overall STDs and HIV/AIDS program is strongly affected by the reduced budget resources for health caused by the economic crisis in the country. Transition to decentralization has given the authority to allocate central budget resources to the provincial level. HIV/AIDS has to compete for resources with other more prevalent diseases such as malaria, tuberculosis and dengue. 6.2 Transition arrangement to regular operations: The most valuable output for this project is perhaps the recommendations resulting from the external assessment conducted at the end of the project. Based on the recommendations, GOI prepared a plan consisting of main activities and supporting activities related to the prevention and management of HI V/AIDS and STDs. The SOPs, training modules, IEC materials and methods, and the laboratory system developed during the project are useful resources to support the plan. This plan has been introduced to the provinces as guidelines for developing local specific plans. Eight provinces met in Batam island during the last year of the project to discuss among others future operations after the project ended. It was agreed that the provinces would adopt the holistic prevention and management model developed in Riau and DKI Jakarta consisting of STDs and HIV/AIDS surveillance, behavior surveillance, IEC activities on high risk groups and general population, establishing STD clinics, counseling, laboratory activities including blood screening, care and treatment. But as was previously mentioned, it is up to the provinces whether to regard HIV/AIDS control as their priority and whether they are ready to allocate money to support a comprehensive STDs and HIV/AIDS intervention. 7. Bank and Borrower Performance Bank 7.1 Lending: Bank performance at the lending stage was satisfactory. The designed project comprehensively addressed crucial aspects influencing STDs and HIV/AIDS transmission. Although seven other donor funding HIV/AIDS related activities were already or about to be implemented at the time the project was prepared, each worked in different geographical areas, and none was design as a broad HIV/AIDS and STDs approach. The SAR recognized the need to coordinate with the other donors and arranged for the PMU to play the coordinating role. The importance of having a strong management team was flagged during the preparation as well as appraisal missions. Potential problems for recruiting NGOs was anticipated early, - 1 1 - resulting in the attachment of guidelines and procedures for NGO involvement in the minutes of negotiations as well as the loan agreement. Unfortunately, the guidelines and procedures were insufficient to facilitate NGO involvement in the project. What agreed was during negotiations was not communicated to MOF offices (KPKN) at the field level. 7.2 Supervision: Bank supervision was unsatisfactory. The first supervision mission although carried out during the first year was late for a short term project. When problems were identified, and some corrective actions were taken, there was little time left in the first year to complete planned activities. The mission was inundated by superficial irregularities, while missing more fundamental problems concerning overly ambitious expectations, unrealistic targeting, and unrealistic assumption of the course of the -V/AIDS epidemic that should have resulted in adjustments of project objectives. Bank missions were also less aggressive in pushing for changes in project management structure and staffing. Immediate appointment of a senior project advisor in response to ineffective management arrangements, for example, might have been able to improve project management performance. The project was rated a problem project around midterm and remained that way until it was closed. In view of the problems the project was facing, GOI was requested by the Bank to prepare a new proposal with a fresh outlook on HIV/AIDS status in Indonesia and the appropriate interventions. This effort was not successful in reviving the project, GOI's proposal was rejected, and during the last year of the project, the Bank only approved the implementation of very limited activities including the dissemination of surveillance SOPs, implementation of an external assessment on WHV/AIDS and development of masterplans for Irian and Central Java. Suspension or cancellation of the project was considered as an unpopular alternative because of the sensitivity of the HIV/AIDS issue. When project supervision responsibilities were moved to the field, supervision missions became rare. Closeness of the task manager to day to day project undertakings did not strengthen supervision, as a matter of fact, it made conducting regular supervision missions seemed unimportant. The project received even less attention when the country was hit by the economic crisis. The Bank as well as the Government were preoccupied with crisis response efforts and little time was available for supervising this project. The Bank was unsuccessful in developing a conducive atmosphere with the govemment. The govermuent felt intimidated by the Bank, and interactions were rather tense. Available records indicated that the Bank was often late in releasing no objection letters. 7.3 Overall Bank performance: Overall performance of the Bank particularly during supervision and completion was less than satisfactory. The project received little attention from the sector unit even after it remained a problem project for a long tine. The situation should have triggered collective efforts from the headquarters as well as the field office to salvage the project. The Bank for example had failed to request for or to bring additional support to the project as it became apparent that the project was in jeopardy. The PSRs (Project Status Reports), although prepared regularly, were more of an administrative requirement rather than a tool to control the quality of the project. Borrower 7.4 Preparation: Borrower performance on preparation was satisfactory. When the project was prepared, consciousness about the threat of an AIDS epidemic in the country was high, resulting in strong commitment from the borrower to project design and objectives. The teamwork between the government, consultants and the World Bank was excellent. Both project provinces participated actively in preparing specific provincial proposals. Local NGOs were involved particularly in designing the behavior change component of the - 12 - project. 7.5 Government implementation performance: Government implementation performance was unsatisfactory. Project management did not take advantage of the project steering committee which comprised of intersectoral high level officials. The committee was chaired by the Secretary General of MOH, and comprised of relevant echelon two staff from the Ministry of Health, the National AIDs Commission, Bappenas, Ministry of Finance, Ministry of Home Affairs and the Project Manager. They hardly ever convened. The first meeting did not happen until the second year of the project. In the occasional meetings held, attendance was often delegated to staff. The influence of the National AIDs Commission (NAC) on the project was minimum. The initial high level commitment on the national strategy was not supported by budget allocation to facilitate coordination. After the project launching workshop, interactions with the NAC tended to be passive. Despite its coordinating function, no regular reports were submitted to the commission for review and follow up. Allocation of counterpart budget was neglected during the first year of the project, causing lengthy revisions. The second year of the project coincided with the height of the economic crisis in the country. Limited government resources, and government policy to freeze project spending throughout the country for several months effected the timeliness and completeness of the steps required to carry out planned interventions. 7.6 Implementing Agency: Performance of the implementing agency was unsatisfactory. Except for the project's last year, the weakness of the Project Management Unit at the central level had significantly affected project performance. Poor performance was reflected in very low disbursement, problems in procurement, and delayed submission of audit reports. Lack of management monitoring and evaluation tools had prevented the PMU from anticipating and making timely response to implementation problems. 7.7 Overall Borrower performance: Overall Borrower performance was unsatisfactory. Aside from the weak advocacy on the part of MOH, commitment made at the echelon one level did not reach implementers at the ground level. Commitment to the project was questionable as seen in the failure to put a strong project management team in place. Support from other agencies and sectors, e.g., MOF for enabling NGO involvement, was minimal. The intersectoral steering committee gave very limited attention for the project even when the project faced serious problems that led to cancellation of a large part of the loan. The waning commitment to the project was not only caused by the government preoccupation with the crisis response but also by the realization that an intensive HIV/AIDS intervention might be premature since Indonesia was not experiencing an exponential increase in HIV/AIDS cases as predicted would happen during the life time of the project. 8. Lessons Learned The project was successful in developing a laboratory system to support HIV/AIDS and STD testing and surveillance, unfortunately support from the other components was negligible. As the result, the project was not able to produce a comprehensive model of proved effectiveness for the prevention and management of HIV/AIDS and STDs. Bank supervision was ineffective in pushing attainment of project goals. Project Design The project was ambitious in its design and objectives. It planned to address all aspects of HIV/AIDS prevention and management from the supply as well as the demand side. However, the three year project period was too short for achieving behavioral change targets, particularly since most guidelines, manuals, - 13 - modules and IEC materials were still to be developed during the first years of the project. The project was also weak in its monitoring and evaluation design. Project Management The decision to establish a Project Management Unit (PMU) led by a Project Manager who was responsible not just for project administration but also for technical matters proved to be counter productive. Appointment of a person with no high structural background in MOH's organization to become the Project Manager had created further problem, because the person had no authority over echelon two officers, who by their structural positions, were responsible for technical matters. There was constant conflict and miscommunication about technical issues between the PMU and the units responsible for HIV/AIDS and STDs management within the DG of CDC-EH. Future projects should consider maintaining the traditional arrangement in which the PMU is led by an Executive Secretary who is responsible for overseeing project administrative matters only and reports to the Project Manager. However, the unit should also be responsible for facilitating communication among the technical units at the central level, and between the central and the regional implementing units. The Project Manager should be someone with a high enough structural level within the MOH organization structure, and therefore have the authority to coordinate the work of the implementing units which most of the time are headed by echelon II officers. Appointment of the Project Manager and Executive Secretary should be agreed by both the Bank and the government at negotiations at the latest Monitoring and Evaluation The project was a pilot that had to deal with numerous activities in a short time frame, involving multiple groups of people, numerous agencies and individuals in different locations. The challenge to the monitoring and evaluation component was to quickly detect and respond to problems of implementation, and also to be able to give information on what worked or did not work. This important component was neglected by GOI as well as by Bank supervision missions. Poor monitoring and evaluation of the project contributed significantly to poor project performance and to difficulties in assessing effectiveness of project interventions. While project monitoring should be kept within the responsibilities of the project management unit, delegating project evaluation to a third party should be considered, particularly for pilot projects. Role of NGOs NGOs were effective for reaching the high risk groups, but most of them were wary about administrative requirements. Therefore, regulations such as Keppres No. 16, 1994 could not and should not be applied to NGO contracts. NGOs, especially the smaller and newer ones, had difficulties in meeting contracting requirements that were used in the regular bidding process. Special procedures were needed to facilitate NGO involvement in govermnent financed projects. The NGO guidelines agreed during negotiations were ineffective, because unlike circulars and decrees, minutes of negotiations are not widely distributed and are not considered as a binding document. It was proved that the Bappenas-MOF circular on NGO recruitment released during the end of the second year was helpful in moving the project. Another alternative such as awarding grants to NGOs based on review of their proposals was not tried in the project but can be considered for future projects. The project correctly put an NGO capacity building component in the project, which was directed particularly at smaller NGOs with strong ties to particular groups of interest. However, the government who controlled the funds was inexperienced in working with NGOs, and lacked creativity in designing capacity building activities, hence the limited disbursement of the NGO capacity building category. In view of this experience, future projects involving NGOs might need to consider recruiting NGO specialists attached to the provincial irnplementing units. The project had triggered the development of new NGOs. However, many were not able to maintain their work when the project was over. To sustain the activities, - 14 - future projects should be more selective in involving NGOs. The NGO should be credible in that it is self sufficient financially, and has good management capacity. It should also be experienced in its field which is proven by its track records, and has an adequate number of qualified staff. 9. Partner Comments (a) Borrower/implementing agency: The HIV/AIDS and STDs Prevention and Management Project financed by the World Bank was implemented for three years (1996 to 1999) in two provinces, i.e., DKI Jakarta and Riau. The project was designed using accurate HIV/AIDS management principles but some problems occurred in implementation due to improper assessment of local condition including lack of capable personnel especially at lower administrative levels, lack of NGO both in number and capability, weak coordination among those involved in the project, and excessive workload of personnel of the implementing units. Delay in releasing project funds (DIP) had made the project unable to complete the activities as planned and to optimally absorb the allocated funds. In addition to these constraints, Indonesia was hit by the monetary crisis that forced the project to severely reduce its budget. In the beginning of the three-year project, US$ 24.8 million was planned for Bank financing, or 70% of the total costs estimated to run the project. US$ 10.4 million was to be allocated by the Indonesian Government as project counterpart budget. Total project cost was US$ 35.2 million or equal to Rp. 41 billion (exchange rate US$ l=Rp. 2,370). One year after the start of the project, Indonesia was hit by the monetary crisis and project fund was reduced. With the drastic decline in the value of rupiah (the lowest rate was Rp. 10,600 to the dollar in 1998/1999), US$19.8 million was canceled. Until the end of the second year, the project had only absorbed about Rp. 25.5 billion. About 7.5 billion rupiah was allocated for conducting activities during the last year of the project (April I to September 30, 1999). Various government regulation (Keppres 16/1994) and other bureaucratic procedures, both from the Indonesian government and the World Bank, had slowed down the implementation of the project. On the other hand, the project team did not give prompt responses to the critics and suggestions given by the Bank evaluation missions. Early correction and improvement would have yielded better achievement. Inspite of the constraints and obstacles, the project had made a number of positive results for HIV/AIDS and STDs management in Indonesia. Some noted achievements were: 1. Concrete involvement of the local government and the AIDs management committees in Riau and DKI Jakarta in the prevention and management of HIV/AIDS. 2. Establishment of eight SOPs for H[V/AIDS and STDs surveillance, STD treatment using the syndromic approach, microbiological tests, AIDS management, AIDS and TB co-infection, AIDS treatment at home, and STD and MCH integrated service. 3. Development of training modules and IEC materials (posters, pamphlets, filler, films, stickers, and so forth) for HIV/AIDS and STD prevention. 4. Increased number of NGOs involved in the management of HIV/AIDS in Jakarta (19 NGOs) and Riau (41 NGOs). 5. Increased number and management capacity of many laboratories in Indonesia in the HIV and STD testing, particularly in the pilot project sites. 6. Training of health workers in DKI Jakarta and Riau on the treatment of STDs using the syndromic - 15- approach and universal precaution. 7. Prevalence studies and mapping of high risk groups in Jakarta and Riau. 8. Overseas fellowship for two laboratory technicians and two IEC specialists. 9. An integrated clinic in Riau specializing in STD treatment. 10. Fresh assessment of HIV/AIDS and STDs in Indonesia. 11. Trainers and laboratory technicians to implement the SOPs on HIV Sentinel and AIDS surveillance in 8 provinces (North Sumatera, Riau, South Sumatera, DKI Jakarta, West Java, Central Java, Maluku and Irian Jaya). Project personnel as well as implementers had gained many useful experience in canying out this project. Working with staff from different levels of administration in different departments and with the NGOs was a very useful experience. Experience in carrying out surveys, developing SOPs, designing training modules and IEC materials, and conducting laboratory tests for STDs, had contributed to the improvement of managerial and technical skills of project personnel and program staff. Even the severe budget cut had served a positive purpose, in that it stimulated the creativity and motives of all parties involved in the project to achieve optimal results. Given the positive experience and the availability of laboratory equipment and technical skills for conducting HIV/AIDS tests, project activities should be maintained. Financing can come from international donor agencies or from the Indonesian government through the APBN or APBD. At least, the IEC, counseling skills and laboratory skills for STD testing should be maintained and if possible improved. The laboratory equipment purchased during the project should also be maintained for long lasting use. In addition, a study needs to be done to fmd out about behavior change among people who had received I-V information, training and counseling. Additional IEC activities should be provided if necessary to reinforce the newly adopted behavior and to guarantee its sustainability. The Ministry's performance during project implementation: 3 Unavailability of counterpart budget had delayed project implementation during the first year, both at the central and regional levels. The delay had reduced implementation period to only 3 months, forcing the implementing units to move the larger part of the activities to the second year. * DIP revision was very slow causing delays to project implementation. * Due to the complex organizational structure, personnel recruitment for the project was difficult and took a long time, preventing the organization from functioning optimally as expected. Personnel recruitment was not based on competency as required by each job and had sacrificed professionalism. Many project staff at the regional offices did not work full time as was requested by the Bank. * Project staff was often changed, hampering the continuation of project activities. * Several project teams did not function optimally and had failed to provide the PMU with requested advice and guidance. * MOH high level officers did not elaborate further the PMU's extensive tasks and functions as described in the Staff Appraisal Report (SAR). This had caused organizational conflicts, overlaps and problems in decision making. * The project did not promptly respond to the findings and recommendations of the World Bank evaluation team. Early correction and improvement would have yielded better project achievements. * MOH high level officers often made decisions without clarifying the reasons to the PMU and hence created problems and questions. * Some policies were made verbally, not in writing, e.g., the prohibition for using fellowship for short courses, and termination of SOP pre-testing without a clear explanation. * The project and MOH were not strong enough in giving arguments to defend their standpoints during negotiations with the Bank and other institutions. - 16 - * The project faced numerous obstacles since early in the project resulting in inability to absorb funding. In the second year, the country was hit by the monetary crisis that became worse during the lifetime of the project. Funding allocated in the first two years grew while absorption capacity declined. Upon request the PMU developed and submitted a new proposal for the project that was later rejected by the Bank. * Inspite of the difficult condition, PMU at the Center and PIU in the regions and the entire program implementers had done their best to carry out the project. The hard work had resulted in the achievement of a number of crucial activities. One such activity was hiring consultants as well as domestic consultants to carry out a fresh assessment on HIV/AIDS and STDs in Indonesia. Meanwhile, the salaries of the project staff were drastically cut during the last year of the project, e.g., the project manager's salary was reduced from Rp. 3,500,000 to Rp. 150,000, while a driver's salary was cut from Rp. 350,000 to Rp. 42,000. These salaries were unreasonably low particularly for non active civil servants working full-time for the project. The project regretted that MOH had failed to negotiate this matter with Bappenas. The Bank's performance during project implementation: * In general, the Bank and related government organizations such as Bappenas and MOF were very supportive in assisting the project to overcome various difficulties. Meetings were organized to solve problems particularly during the first two years of the project. * Nevertheless, the Bank had shown little understanding on the problems of the project, especially on those caused by external factors. * The Bank was too strict in assessing project performance. Some of its decisions were questionable and had discourage the moral of project staff. Some staff thought that the Bank intervened to much on project matters. = The Bank's team paid little attention to the cause of the problems. People doubted the expertise of Bank team members. The project would be better if supervised by a team which members had expertise in different areas e.g., biomedical services, IEC, laboratory and project management. Future arrangement of HIV/AIDS prevention and care in Indonesia: When the project ended in September 1999, the country was at a transition towards decentralization which reduced the role and power of the central government. Most foreign assisted HIV/AIDS project in Indonesia will be closed in year 2000, although some will be extended to 2003/2004. With decentralization, the planning of the future arrangement for HIV/AIDS prevention and care will be in the hands of the regional government. Activities conducted will be subject to approval by the local legislative body (DPRD). However, the central government has established a national plan consisting of main and supporting activities to be used by the provinces as guidelines in developing local specific plans. The design of the national plan draws heavily on the results of the External HIV/AIDS Assessment and the Consensus Workshop held in Jakarta in September 1999. Activities in the national plan include: A. Main Activities: 1. Prevention: * Screening of 100% blood donation and provision of HNV test kits for blood screening. * Condom promotion aimed at high risk groups and conducted in cooperation with the NGOs. * Infection control and universal precaution: distribution of existing SOPs, training and socialization of universal precaution among health workers, provision of supplies and equipment by the local government, development of guidelines and training package by the HAPP project (USAID funded HV/AIDS project). * Prevention of mother to child transmission: implementation of a WHO supported pilot project. - 17 - * Harm reduction among IDUs (Injecting Drug Users): conducting rapid assessment and interventions among IDUs using WHO, USAID and AusAID resources. 2. Care: * STD case management in hospitals and health centers. * Integrated STD case management and MCH/FP services. * STD outreach services. * STD clinics. * AIDS case management in hospitals. * Community based AIDS care. * Provision of logistics (drugs, supplies and equipment). * Increase coverage, access, and equity of testing and counseling services throughout the country. 3. IEC: * Targeted intervention towards high risk groups. * Systematic, and wide coverage of IEC to the general population, using effective, tested and evaluated methods, taking into consideration the target audience's tradition, religious beliefs and socio-culture. * Special attention to moral and ethics aspects, promoting family resilience and welfare. 4. Surveillance: * WHV and syphilis sentinel surveillance conducted in 15 provinces, as hoc surveillance conducted in 26 provinces, printing and distribution of SOPs, and training of health workers using local government resources. 3 AIDS surveillance, training, printing and distribution of SOPs using local government resources. i STD surveillance, development of guidelines, pretesting and printing of guidelines, and training using local government budget. 3 Behavioral surveillance survey, development of tools and incorporating activities in biomedical sentinel sites. i Gonnorhoea Antimicrobial-resistence Surveillance, extension of laboratory sites. * Periodic projection and estimation of H[V infection loads. 5. Monitoring and evaluation: * Supervision for HIV/AIDS and STDs prevention and control at provincial, district and health center levels. * Review meeting for planning and evaluation at all levels. - Annual national meeting on H1IV/AIDS and STDs prevention and care in the health sector. 6. Research and Development by government as well as private institutions, and universities. 7. Training and Education: * Pre-service training for medical and paramedical education on WIV/AIDS and STDs case management, universal precaution, and counseling. * In-service training for government and privately employed health workers on HIV/AIDS and STDs case management, universal precaution, counseling, testing and surveillance. B. Supporting Activities: 1. Coordination, cooperation and community participation through: * The National, Provincial and District AIDs Commissions. * Bilateral and Multilateral Cooperation. * Cooperation with intemational agencies and NGOs. 2. Improvement of program management through improvement of functions and elements in the management of WHV/AIDS and STDs prevention and care. 3. Development of laws and regulations: * Development of laws and regulations to protect WHV serologic status confidentiality of people with - 18 - HIV and AIDS * Development of laws and regulation to prevent discrimination of people with HIV and AIDS. (b) Cofinanciers: None (c) Other partners (NGOs/private sector): None 10. Additional Information - 19- Annex 1. Key Performance Indicators/Log Frame Matrix Outcome I Impact Indicators: a000* ` to d M atr i Pjte in0R last P A it4aIcts Estimate 1. Percent of CSWs who: Data not available Data not available. - can describe correct use of a condom, Interview with a few CSWs during the ICR - can explain where and how to obtain mission found good knowledge on where and condoms, how to obtain condoms, and positive attitude - have a positive attitude toward consistent, toward consistent condom use but actual use correct condom use. of condoms was highly dependent on the preference of clients. 2. Percent of CSWs in "lokalisasi" Data not available An assessment of post syndromic approach reported ever use of condom and use in last training (1998) in district Kepri by NGOs at sexual act. three "lokalisasi", one massage parlour and MCH clinic reported 50% ever use condom. From a Behavioral Surveillance Survey conducted in two lokalisasis in Riau, on average 22% CSWs always use condoms, and 67% sometimes use condoms, 54% used condom in the last sexual act. 3. Percent of person by target group who: - Data not available Means of AIDS prevention (results from the can cite at least 2 means of STD same BSS): use sterile needle (65%), be preventions; faithful (62%), always use condom (77%). - state that HIV is a special type of STD; - state that a person can remain healthy for several years but still transmit the virus; - express the importance of positive family attitudes toward a member with AIDS. 4 Percent of STD patients reportedly Data not available. Data not available. seeking care within 24 hours of noting symptoms. 5. Percent of heaith workers trained in Data not available. Data not available. syndromic approach who can: - demonstrate appropriate use of flow chart, - demonstrate how to use a condom, - explain the importance of treating STDs as a way preventing AIDS. 6. Number of people aged 15-49 who Data not available. Data not available. reported condom use in most recent act of sex intercourse with a non-regular partner, Number of people aged 15-49 reporting sexual intercourse with a non regular sex partner in the past month 7. Reported STD Incidence (men) Data not available. Data not available. 8. Reported STD Incidence (women) Data not available. Data not available. 9. Performance on bi-annual proficiency Data not available 30 out of 33 health and hospital laboratories panel (competency reports) covered by this project conducted evaluation of proficiency panel regulariy twice a year. 75% achieved maximum score (4). - 20 - Output Indicators: IndlcatorlMatrix Prjected in last PSR ActuaULatest Estimate 1. Number of group education session Data not available. Data not available. conducted by each province. 2. Number of peer group education Data not available. 18 batches in OKI and 223 batches in Riau. session conducted by each province. Monthly data on sessions were not recorded. Various media for general community, 3. Number of educational materials Data not available. specific/closed community, adolescents/ printed and distributed by province by students, high risk groups, health workers, target religious group etc., consisting of 54,300 posters, 183,300 leaflets, 2,500 T-shirts, 5,000 modules, 75,000 handouts, 35,000 manuals, 100 fillers and 22 films, 38,000 stickers, 11,500 calendars, 12,500 booklets, 2,500 flipcharts. There were occasions reported in Batam and 4. Percent of factories/companies who Data not available. Kepri districts, but not well recorded. have participated in educational programs who develop formal policies related to HIV/AIDS in the workplace. Data not available. 5. Percent of registered CSWs who have Data not available. participated in at least one peer group education session within the past 3 months. 64 NGOs 6. Number of NGOs involved in Data not available. implementing IEC activities. Activities were reported, but data was not 7. Percent of schools reached by project Data not available. collected annually. No data recorded. that have organized peer group programs. Distribution of condoms was reported but 8. Number of condoms distributed by Data not available. actual records were non existent. each project management unit to NGOs providing IEC service Number of agencies was around 23 from 9. Number of implementing agencies that Data not available. both provinces. Annual reports were not have conducted focus group discussions done. or other activities to solicit feedback from those served by the NGOs. No report is available. 10. Percent of implementing agencies Data not available. who report at least 1 supervisory visit by PMU staff during past 6 months 11. Number of syndromic approach Data not available In Riau: 11 batches involving 330 health training workshops conducted by workers. In DKI: 30 batches involving 2,254 provinces health providers. 12. Number of condoms distributed by Data not available. Survey was not done. health center/clinics as a numerator of number received. 13. Number of health workers trained in Data not available. 2,520 persons in eight provinces new surveillance methods 14. Number of supervisory visits made Data not available No specific supervisory visits were executed. by central and provincial level staff surveillance -21 - 15. Completion of Materials Checklist Data not available. 22 provincial health laboratories and hospital by each Laboratory. laboratories completed the checklist. 22 provincial and hospital laboratories 16. Completion of Readiness Report by Data not available. provided reports, but only in the second each laboratory. year. 17. Completion of Knowledge Data not available. Some laboratories completed the Assessment (written) assessment in the second year. 18. Performance on Proficiency Panel of Data not available. - Carried out by laboratory of RSCM hospital, sera: once a year. - ability to determine CD3/418 levels by - Canied out by laboratory of RSCM flowcytometry hospital. - ability to determine p24 levels - Carried out by 27 provincial health laboratories. - ability to perform Chlamydia Data not available. Done every year. 19. Visitation and completion of the Inspection Report 20. Second year visitation and Data not available. Done. on the second year. Inspection report End of project -22 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraal ActualLatist Potenteg of Estimate Estimat Appraisal Prpect Cost 0y Componont :V$ million US$ million IEC & Behavioral Intervention 10.00 1.59 15.9 Biomedical Services 18.20 2.89 15.87 Management & Support 3.00 0.44 14.66 Total Baseline Cost 31.20 4.92 Physical Contingencies 1.60 0.25 15.62 Price Contingencies 2.40 0.37 15.41 Total Project Costs 35.20 5.54 Total Financing Required 35.20 5.54 Project Costs by Procuremet Arrangements (Appraisal Estimate) (US$ million equivalent) Ex;ienditu - as o -etho& NBro.F.tt Toat = = N = . _ . . _ 1. Works 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 5.00 4.60 2.00 0.10 11.70 l______________________ (5.00) (3.70) (1.30) (0.00) (10.00) 3. Services 0.00 0.00 3.40 0.00 3.40 (0.00) (0.00) (3.30) (0.00) (3.30) 4. Miscellaneous 0.00 0.20 14.70 5.20 20.10 (0.00) (0.20) (11.30) (0.00) (11.50) 5. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 6. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 5.00 4.80 20.10 5.30 35.20 (5.00) (3.90) (15.90) (0.00) (24.80) Project Costs by Procurement Arrangements (ActuaULatest Estimate) (US$ million equivalent) ExediuwaaW NZF loa Cot Other2 _ _ _ _ 1. Works 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 2. Goods 0.94 0.75 0.00 0.00 1.69 (0.94) (0.56) (0.00) (0.00) (1.50) 3. Services 0.00 0.00 0.72 0.00 0.72 (0.00) (0.00) (0.72) (0.00) (0.72) 4. Miscellaneous 0.00 0.00 2.87 0.00 2.87 (0.00) (0.00) (2.30) (0.00) (2.30) - 23 - 5. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) 6. Miscellaneous 0.00 0.00 0.00 0.00 0.00 (0.00) (0.00) (0.00) (0.00) (0.00) Total 0.94 0.75 3.59 0.00 5.28 (0.94) (0.56) (3.02) (0.00) (4.52) " Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2'Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) |(5204. 81 ! t!0l'021 i -i 2*w4 IEC & Behavioral 7.02 3.00 0.00 0.72 0.37 0.00 10.3 12.3 0.0 Intervention Biomedical Service 12.84 5.50 0.00 3.46 0.86 0.00 26.9 15.6 0.0 Management & Support 1.96 0.84 0.00 0.33 0.03 0.00 16.8 3.6 0.0 -24 - Annex 3: Economic Costs and Benefits An economic cost and benefit was not done due to valid supporting data were not well available. - 25 - Annex 4. Bank Inputs (a) Missions: Ste of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, I EMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation October 1994 4 1 Population & Health S S Specialist, 3 consultants, HIV/AIDS and STDs February 1995 4 1 Economist, I Population & S S Health Specialist, 2 consultants, HIV/AIDS and STD Appraisal/Negotiation July 1995 2 1 Economist, I Population & S S Health Specialist October 1995 4 1 Economist, I Population & S S Health Specialist, I Public Health Specialist, I FMS January 1996 5 1 Economist, I Population & S S Health Specialist, I Operations Analyst, 1 Senior Counsel, Legal Departnent, I Disbursement Officer. Supervision September 1996 2 1 Economist, I Public Health S S Specialist January 1997 2 1 Economist, I Public Health S S Specialist September 1997 2 1 Economist, I Public Health U S specialist June and July 1998 3 1 Economist, 2 Public Health U U (no site visit) Specialists ICR April 2000 3 2 Public Health Specialists, U U I Project Management Consultant (b) Staff Stage of Project Cycle Actual/Latest Estimate No. Staff weeks USS (,000) Identification/Preparation 24.2 113.5 Appraisal/Negotiation 17 57.3 Supervision 54 150.3 ICR 8.4 13.5 Total 103.6 334.6 -26 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating I Macro policies O H OSUOM * N O NA O Sector Policies O H OSUOM * N O NA I Physical O H OSUOM O N * NA Z Financial O H OSUOM O N * NA F Institutional Development 0 H O SU O M * N 0 NA ?Environmental O H OSUOM O N * NA Social ER Poverty Reduction O H OSUOM O N * NA F Gender O H OSUOM * N O NA LII Other (Please specify) Z Private sector development 0 H O SU O M 0 N 0 NA F Public sector management 0 H O SU O M * N 0 NA O Other (Please specify) -27 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactoiy) 6.1 Bankperformance Rating 0 Lending OHS*S OU OHU O Supervision OHS OS *U OHU O Overall OHS OS * U O HU 6.2 Borrowerperformance Rating L Preparation OHS OS O U O HU El Government implementation performance O HS O S 0 U 0 HU O Implementation agency performance O HS O S 0 U 0 HU El Overall OHS OS *U O HU - 28 - Annex 7. List of Supporting Documents 1. Staff Appraisal Report, HIV/AIDS and STDs Prevention and Management Project No. 15118-IND, January 1996. 2. Loan Agreement for HlV/AIDS and STDs Prevention and Management Project between the Republic of Indonesia and International Bank for Reconstruction and Development, Ln. 3981-IND, March 1996. 3. Aide-Memoires and BTOs for preparation and appraisal missions. 4. Aide-Memoires and BTOs for supervision missions. 5. Project Supervision Reports (PSRs). 6. External Evaluation Report of HIV/AIDS Project, April -May 1999, by National Board of Health Research and Development.. 7. Report of Independent Auditor (BPKP) on the Consolidated Financial Statement FY 1999 and 1998 of WIV/AIDS Project, January 2000. 8. Financial Consolidation Report for FY 1999/2000, of HIV/AIDS Project by Project Management Unit, March 2000. 9. Detailed Cost table for HWV/AIDS and STDs Prevention and Management Project, January 1996. 10. External HWV/AIDS Assessment, November 1999 by John Kaldor, Tonny Sadjimin. Suharyo Hadisaputro and Jeanine Bardon. 11. HIV/AIDS, STDs and related risk behavior in Indonesia, Report of a Consensus Workshop. 12. Final Report, NGO Capacity Building Project, LAHSI. 13. Dampak Bantuan Bank Dunia Terhadap Peran Lembaga Swadaya Masyarakat Dalam Penanggulangan WV/AIDS di DKI Jakarta, Yayasan Wira Bhakti Husada 14. Final Report HIV/AIDS and STDs Prevention and Management Project, IBRD Loan 3981-IND, 1996/1997-1999/2000, DG of CDC-EH. 15. Implementation Completion Report: Project Management Aspect, Widyastuti Soerojo. - 29 -