Document of The World Bank Report No: ICR00001281 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-46870) ON A LOAN IN THE AMOUNT OF US$ 150 MILLION TO THE RUSSIAN FEDERATION FOR A TB AIDS CONTROL PROJECT December 23, 2009 Human Development Sector Unit Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 23, 2009) Currency Unit = Russian Ruble 1.00 = US$ 0.033 US$ 1.00 = 30.67 FISCAL YEAR ABBREVIATIONS AND ACRONYMS AIDS Acquired Immunodeficiency Syndrome MSF Medecins sans Frontieres (Doctors without Borders) ART Anti Retroviral OED Operations Evaluation Department BCG Bacille Calmette-Guerin Immunization PDO Project Development Objective CAS Country Assistance Strategy PHC Primary Health Care CIDA Canadian Agency for International Dev PHRI Public Health Research Institute CSW Commercial Sex Workers PIP Project Implementation Plan DALY Disability-adjusted life year PPP Project Procurement Plan DFID Department for International Development PIU Project Implementation Unit (United Kingdom) DOTS Directly observed treatment, short course PLWHA People Living with HIV/AIDS GOR Government of Russia PMTCT Prevention of Mother-to-Child Transmission HAART Highly Active Anti-Retroviral Therapy RHCF Russian Health Care Foundation HIV Human Immunodeficiency Virus RSP Regional Strategic Plan HLWG High Level Working Group SSM Sputum Smear Microscopy ICB International Competitive Bidding STD Sexually-transmitted disease IDU Intravenous drug user(s) STI Sexually-transmitted infection MDR.TB Multidrug-resistant tuberculosis TB Tuberculosis MOF Ministry of Finance UNAIDS Joint United Nations Program on HIV/AIDS MOH Ministry of Health UNICEF United Nations Children's Fund MOHSD Ministry of Health and Social Development USAID United States Agency for International Development MOI Ministry of Interior USSR Union of Soviet Socialist Republics MOJ Ministry of Justice WHO World Health Organization MSM` Men having sex with men WG Working Group Vice President: Philipe H. Le Houerou Klaus Country Director: tor: Rohland Sector Manager: Abdo S. Yazbeck Project Team Leader: Patricio Marquez ICR Team Leader: :Asel Sargaldakova Primary Authors: Betty Hanan and Asel Sargaldakova RUSSIAN FEDERATION TB AIDS Control Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring (if any) I. Disbursement Profile 1. Project Context, Development Objectives and Design ........................................ 4 2. Key Factors Affecting Implementation and Outcomes ........................................ 5 3. Assessment of Outcomes.................................................................................... 14 4. Assessment of Risk to Development Outcome: Moderate................................ 23 5. Assessment of Bank and Borrower Performance ............................................... 24 6. Lessons Learned ................................................................................................. 27 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .... 28 Annex 1. Project Costs and Financing ........................................................................ 30 Annex 3. Economic and Financial Analysis ................................................................ 45 Annex 4. Bank Lending and Implementation Support/Supervision Processes............ 46 Annex 5. Beneficiary Survey Results .......................................................................... 48 Annex 5. Beneficiary Survey Results .......................................................................... 48 Annex 6. Stakeholder Workshop Report and Results.................................................. 49 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ................... 50 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ...................... 69 Annex 9. List of Supporting Documents....................................................................... 70 MAP IBRD 27188R List of Figures: Figure 1: Registered TB incidence rate in the RF, 1992-2007, all health facilities.........18 Figure 2: Overall TB incidence rate including proportion of registered TB incidence rate among penitentiary system population, 1999-2007.............................................18 Figure 3: Distribution of TB incidence rate by Federal Okrugs (counties), 1991-2007 (MOH)................................................................................................19 Figure 4: Number of HIV cases per 100,000 inhabitants.....................................20 A. Basic Information Tuberculosis & AIDS Country: Russian Federation Project Name: Control Project Project ID: P064237 L/C/TF Number(s): IBRD-46870 ICR Date: 12/28/2009 ICR Type: Core ICR RUSSIAN Lending Instrument: SIL Borrower: FEDERATION Original Total USD 150.0M Disbursed Amount: USD 110.3M Commitment: Revised Amount: USD 110.3M Environmental Category: C Implementing Agencies: Ministry of Health and Social Development Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 08/09/1999 Effectiveness: 12/11/2003 12/11/2003 Appraisal: 06/12/2000 Restructuring(s): Approval: 04/03/2003 Mid-term Review: 10/16/2006 10/14/2006 Closing: 12/31/2008 04/30/2009 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No Satisfactory at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 100 100 Theme Code (as % of total Bank financing) HIV/AIDS 29 29 Health system performance 14 14 Law reform 14 14 Population and reproductive health 14 14 Tuberculosis 29 29 E. Bank Staff Positions At ICR At Approval Vice President: Philippe H. Le Houerou Johannes F. Linn Country Director: Klaus Rohland Julian F. Schweitzer Sector Manager: Abdo S. Yazbeck Armin H. Fidler Project Team Leader: Patricio V. Marquez Olusoji O. Adeyi ICR Team Leader: Asel Sargaldakova ICR Primary Author: Betty Hanan Asel Sargaldakova F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objectives of the project are to (i) contain the growth of the epidemics of tuberculosis (TB) and HIV/AIDS in the short term and (ii) to halt and reverse the courses of these epidemics in the medium term. The achievement of these objectives will protect the Russian population and economy from uncontrolled epidemics of TB, HIV/AIDS and other epidemiologically important sexually transmitted infections (STIs). ii The project provides support to the Russian Government Federal Program on "Prevention and Control of Social Diseases, 2002-2006". Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Approval by MOH of protocols for TB diagnosis, treatment and surveillance, Indicator 1 : consistent with evidence-based approaches agreed with the World Health Organization. 1. Ministerial Decree 109 issued in 2003 for New protocols on Ministerial Decree Improving TB Control TB diagnosis, #50 for recording Ministerial Value Measures in the Russian treatment and and reporting TB Decree for quantitative or Federation. surveillance data was issued and recording/repo Qualitative) 2. No Ministerial Decree produced and adopted in all 86 rting TB data. available for disseminated regions including recording/reporting TB countrywide. MOJ facilities data. Date achieved 04/01/2003 12/31/2008 12/31/2008 04/30/2009 Comments (incl. % ACHIEVED. Ministerial Decree No. 50 was adopted in all 86 regions. achievement) Indicator 2 : Improvements in effectiveness and efficiency of TB diagnosis and treatment. Indicator 1 1. leveling off or was revised; decrease by 5% in indicators 2-4 new TB cases 1. 5 % decrease remained as New TB cases current level. approved; 2. 25 per 100,000 notification: 83.2 per 2. Increase up to indicator 5 100,000 (2003). 25 per 100,000. added. 3. 85% TB cases 3. 85% TB cases under standardized Smear positive case under the Value REVISED treatment regimen notification: 20.2 per standardized quantitative or INDICATOR 100,000 population treatment Qualitative) 1. 0-5% 4. 57% for 2004 (2003). regimen. decrease cohort; 58.3% for 4. 5% increase current level. 2006 cohort Leveling off or decrease treatment success in TB mortality: rate among new ADDITIONA 5. 10% TB 22/100,000. S+ pulmonary L mortality reduction cases. INDICATOR among new cases 5. 0-5% (18.4/100,000 in decrease in TB 2007) iii mortality (according to existing recording and reporting systems). Date achieved 12/31/2003 12/31/2008 12/31/2008 04/30/2009 Comments (incl. % ACHIEVED. All targets were achieved/exceeded. achievement) Indicator 3 : % of HIV-positive infants born to HIV-infected women Value quantitative or 13.2% 10% 10.6% Qualitative) Date achieved 12/31/2003 12/31/2008 04/30/2009 Comments PROGRESS ACHIEVED: Percentage of HIV positive infants born to HIV- (incl. % infected mothers has significantly decreased from 13.2% in 2003 to 10.6% in achievement) 2007. The target of 10% was almost met. Reduction in incidence rate of other specified STIs among general population Indicator 4 : measured against baseline values. Incidence of: Incidence of: Incidence of: syphilis: syphilis: syphilis: Incidence of: syphilis: 94.6/100,000; 65/100,000; 63/100,000; 94.6/100,000; gonorrhea: gonorrhea: gonorrhea: gonorrhea: Value 82.0/100,000; 82.0/100,000; 68.0/100,000; 82.0/100,000; quantitative or trichomoniasis: trichomoniasis: trichomoniasis trichomoniasis: Qualitative) 259.1/100,000; 259.1/100,000; : 200/100,000; 186/100,000; chlamidiosis: chlamidiosis: chlamidiosis: chlamidiosis: 100.3/100,000. 100.3/100,000 90/100,000 91.1/100,000 population. population. population Date achieved 12/31/2003 12/31/2008 12/31/2008 04/30/2009 Comments PARTIALLY ACHIEVED. Targets were achieved with the exception of the (incl. % revised gonorrhea and chlamidiosis targets, which were nearly met. achievement) Reduction in the rate of increase of HIV prevalence (or reduction in the Indicator 5 : prevalence rate) among general population. Registered HIV cases: 263,898 (36, 396 newly Reduction in the Value registered patients). rate of increase of 5.1% reduction vs. quantitative or HIV prevalence 2003 Qualitative) Rate of increase of HIV rate up to 1.5%. prevalence in 2004: 2.1% compared with 2003. Date achieved 12/31/2003 12/31/2008 04/30/2009 Comments (incl. % ACHIEVED. Targets were achieved. achievement) iv (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years % of HIV-positive pregnant women who receive ARV prophylaxis to reduce Indicator 1 : MTCT in accordance with nationally approved treatment protocol Value (quantitative 70% 85% 85% 99.9% or Qualitative) Date achieved 12/31/2003 12/31/2008 12/31/2008 04/30/2009 Comments (incl. % HIGHLY ACHIEVED. Target was significantly exceeded. achievement) % HIV-exposed neonates who received prophylactic antiretroviral drugs in Indicator 2 : accordance with nationally approved treatment protocol Value (quantitative 70% 75% 85% 97.9% or Qualitative) Date achieved 12/31/2003 12/31/2008 12/31/2008 04/30/2009 Comments (incl. % HIGHLY ACHIEVED. Targets were significantly exceeded. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 07/09/2003 Satisfactory Satisfactory 0.00 2 07/18/2003 Satisfactory Satisfactory 0.00 3 01/12/2004 Satisfactory Satisfactory 0.00 4 04/12/2004 Satisfactory Satisfactory 1.00 5 10/19/2004 Satisfactory Satisfactory 1.30 6 01/07/2005 Satisfactory Satisfactory 1.51 7 05/03/2005 Moderately Satisfactory Moderately Satisfactory 1.78 8 02/24/2006 Moderately Satisfactory Moderately Satisfactory 9.66 9 11/29/2006 Satisfactory Satisfactory 31.40 10 07/23/2007 Satisfactory Satisfactory 51.69 11 12/20/2007 Satisfactory Satisfactory 65.12 12 02/28/2008 Satisfactory Satisfactory 72.07 13 10/13/2008 Satisfactory Satisfactory 99.07 14 04/30/2009 Satisfactory Satisfactory 111.37 v H. Restructuring (if any) Not Applicable I. Disbursement Profile vi 1. Project Context, Development Objectives and Design The TB and AIDS Control Project was approved on April 3, 2003. The Loan Agreement was signed on September 12, 2003 and the Loan became effective on December 11, 2003. The Project was the fourth World Bank-financed operation supporting the health sector of the Russian 1 Federation . The TB control component of the Project was designed to complement a national program, i.e. the Federal Targeted Social Disease Prevention and Control Program (2002-2006). Implementation of the component later complemented the Government's Federal Socially Significant Disease Prevention and Control Program (2007-2011) and the National Priority Health Project. 1.1 Context at Appraisal The main sector issue was the gap between needs and the practices regarding the control of TB and HIV/AIDS/sexually transmitted infections (STIs). The factors contributing to this gap included: (i) inadequate response, both in scale and the technical quality to tackle the large burden of TB and HIV/AIDS/STIs; (ii) need for stronger management and institutional capacity for rapid implementation of large-scale programs, based on scientific evidence, across the vast territory of the Russian Federation, and (iii) financial constraints. According to the World Health Organization (WHO), Russia was one of the 22 high- burden countries for TB in the world (WHO, Global Tuberculosis control: Surveillance, Planning, Financing, Geneva, 2002). The incidence of TB increased throughout the 1990s. This was due to a combination of factors, including: (i) increased poverty, (ii) under-funding of TB services and health services in general, (iii) diagnostic and therapeutic approaches that were designed for a centralized command-and-control TB system, but were unable to cope with the social mobility and relative freedom of the post-Soviet era, and (iv) technical inadequacies and outdated equipment. Migration of populations from ex-Soviet republics with high TB burdens also increased the problem. Prevalence rates were many times higher in the prison system than in the general population. Treatment included lengthy hospitalizations, variations among clinicians and patients in the therapeutic regimen, and frequent recourse to surgery. A shrinking health budget resulted in an erratic supply of anti-TB drugs and laboratory supplies, reduced quality control in TB dispensaries and laboratories, and inadequate treatment. The social conditions favoring the spread of TB, combined with inadequate systems for diagnosis, treatment, and surveillance, as well as increased drug resistance, produced a serious public health problem. TB control in the former Union of Soviet Socialist Republics (USSR) and in most of Russia in the 1990s was heavily centralized, with separate hospitals (TB dispensaries), TB sanatoriums, TB research institutes and TB specialists. The system was designed in the 1920s to address the challenges of the TB epidemic. Case detection relied strongly on active mass screening by X-ray (phluorography). Specificity, sensitivity, and cost-effectiveness considerations were not features of this approach. Bacille Calmette-Guerin (BCG) immunization was a key feature of the TB 1 The Health Reform Implementation project was approved on March 18, 2003. control system and widely practiced, because it offered effective protection under certain conditions. It was given at birth, with repeat doses in childhood, adolescence and, in some instances, adulthood. However, the protective efficacy of the second and third BCG doses was doubtful even according to Russian scientists. By the 1990s, the directly observed treatment, short course (DOTS) strategy for TB control recommended by WHO, was either not implemented or implemented partially. DOTS is internationally acknowledged as a highly effective strategy with 148 out of 210 countries implementing DOTS in 2000. The Russian Federation was experiencing one of the fastest-growing epidemics of HIV/AIDS in the world. The cumulative total of HIV infections reported was more than 200,000 at the end of 2000, up from the 10,993 reported at the end of 1998. The estimated number of persons living with HIV/AIDS in March 2003 was thought to be around four times higher than these reported figures and, according to the Russian AIDS Center, it could have been even higher. Efforts to address HIV/AIDS by the Russian Government had been patchy for several reasons. First, despite the steep increase, HIV/AIDS prevalence rates remained low, below one percent of the adult population, compared to the worst affected regions in sub Saharan Africa, Asia, and the Caribbean. Second, the epidemic was driven mainly by transmission among the high-risk core group of injecting drug users (IDUs) in prisons and in the civilian population sharing contaminated needles. Hence there was a perception that HIV/AIDS was a problem of "social deviants." Third, political advocacy had not been linked to evidence base, with limited results from exhortations and rights-based advocacy. As such, it was politically difficult to mobilize effective programs on a large scale for a problem that was perceived to be confined to a group with limited political clout. In consequence, there was almost no regulatory framework for HIV/AIDS control prior to the Project. The predominant view of HIV/AIDS was not of an epidemic that was spreading fast among "normal" youth who experimented with drugs and sex. Yet, HIV/AIDS affected and continues to affect disproportionately the younger cohorts who, without the disease, would continue to stay in the labor force for a long time, and/or would have continued to build up human capital and expertise. By December 2001, 62 percent of the male HIV+ and 57 percent of female HIV+ individuals in Russia were between 20 and 30 years of age. Most of the diagnosed cases were males who represented 78 percent of all registered cases although the female to male ratio was steadily growing. While Russia initially demonstrated the features of an epidemic that was concentrated among the high-risk core transmitted (IDUs and commercial sex workers - CSWs), by early 2003 it was spreading into the bridge population (the sex partners of the high-risk core transmitters). Although precise predictions were impossible, it was reasonable to expect that, without effective efforts to control the trend, the epidemic was likely to spread from this bridge population into the general population. Interrupting HIV transmission among the high-risk core transmitter and the bridge populations was central to effective prevention of a generalized epidemic. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The development objectives of the Project were to: (i) contain the growth of the epidemics of TB and HIV/AIDS in the short-term, and (ii) halt and reverse the courses of these epidemics in the 2 medium-term. The achievement of these objectives was to protect the Russian population and economy from uncontrolled epidemics of TB, HIV/AIDS and other epidemiologically important sexually transmitted infections (STIs). The Project provided support to the Russian Government Federal Program on "Prevention and Control of Social Diseases, 2002-2006." For TB control: (a) Approval by MOH of protocols for TB diagnosis, treatment and surveillance, consistent with evidence-based approaches agreed with the Bank and WHO. (b) Improvement in effectiveness and efficiency of TB diagnosis and treatment over baseline values in eligible regions and reference centers, as measured by the following criteria: · Leveling off or decrease by 5 percent or more in new TB cases; · Leveling off or decrease in new multidrug-resistant TB (MDRTB) cases; and · Decrease in the TB case fatality rate among new cases by at least 10 percent. For HIV/AIDS and STIs control: (a) Approval by MOH of protocols for diagnostics, treatment, epidemiological and behavioral surveillance in line with Russian legislation, and consistent with evidence-based approaches endorsed by WHO and the Joint United Nations Program on HIV/AIDS (UNAIDS); (b) Reduction in the rate of increase of HIV prevalence (or reduction in the prevalence rate) among vulnerable population sub-groups, including among IDUs, CSWs, men having sex with men (MSM), prisoners, high risk youth, members of the armed forces, and newborns of HIV-infected mothers, as measured against baseline values; and (c) Reduction in the prevalence rate of other specified STIs among vulnerable population sub- groups, measured against baseline values. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO was not revised. The Performance Indicators were amended on June 29, 2007 following agreements during the Mid-term Review of October 2006, through an amendment to the Loan Agreement. It should be noted that the wording in the Loan Agreement for the PDO differed slightly from the text in the PAD. However, the differences were not of substance; the development objectives were the same in both documents. 1.4 Main Beneficiaries The Project was to benefit the Russian population as a whole in several ways. First, the Project aimed to reduce premature mortality, avoidable morbidity, and disability from TB, HIV/AIDS/STIs. According to the results of epidemiological and economic projection models, the Project was potentially able to save over 150,000 lives by strengthening the Government's control of TB and HIV/AIDS/STIs epidemics. Second, the Project aimed to help reduce lost productivity and avert a potential economic disaster, plausible in the event of sharp increases in HIV prevalence. Third, through the TB component, the Project aimed to benefit the poor since TB was linked to poor 3 living conditions and was a factor contributing to low labor productivity. Thus, the Project aimed to help save lives, reduce morbidity, disability, and absenteeism from work. It also sought to improve the cost-effectiveness of the TB system, which was to free resources to improve the access of services to the population. Health workers, managers and institutions directly involved in the control of TB and HIV/AIDS/STIs were to benefit from improved and stable availability of diagnostic equipment and essential supplies and consumables, and upgraded skills and morale. The institutional benefits of the Project were deemed to be important. By supporting the implementation of a new paradigm for TB control, the Project sought to strengthen the institutional capacity of health facilities to properly diagnose and effectively treat TB. This intervention aimed to help reverse the trend of several years of sub-optimal results, which was a factor behind the high incidence of TB and the increasing drug resistance. In addition, the health care system was to benefit from cost savings resulting from the reduced number of unnecessary hospitalizations and mass X-ray testing. For HIV/AIDS, the Project's harm reduction and outreach programs were to increase the collaboration between the public sector and non-governmental organizations (NGOs) in working with high-risk groups, such as the IDUs, MSM, CSWs, and vulnerable youth. The Project was to support new protocols for prevention of HIV and vertical transmission, developed in Russia and based on up-to-date scientific evidence. 1.5 Original Components (as approved) The Project was designed to support the development and large-scale implementation of interventions that were based on up-to-date scientific evidence. The major inputs were to be directed to: (i) updating and/or disseminating strategies, guidelines and protocols; (ii) assessing needs as a basis for effective planning, procurement, implementation, and monitoring and evaluation; (iii) training and on-the-job learning to improve the local capacity for effective implementation; (iv) suppling of equipment and consumables required to improve the technical quality of diagnostic and therapeutic activities; (v) procuring drugs; and (vi) monitoring and evaluation. The Project had three components: Component I - Control of Tuberculosis (US$217.30 million or 76% of total project costs) was to: (i) improve policies, strategies and protocols, (ii) strengthen surveillance, monitoring, quality control and quality assurance, (iii) improve the detection of TB cases, and (iv) improve TB treatment. Component II - Control of HIV/AIDS (US$65.32 million or 23% of total project costs) was to: (i) improve policies, strategies and public information for HIV/AIDS control, (ii) strengthen surveillance and monitoring, (iii) improve laboratory service and blood safety, (iv) prevent and control STIs, (v) deliver preventive services against HIV/AIDS, with emphasis on high-risk groups, and (vi) prevent the transmission of HIV from mothers to children. Component III - Project Management, Monitoring and Evaluation (US$3.56 million or 1% of total project costs) was to: (i) support the operation of the Project Implementation Unit (PIU) in the Russian Health Care Foundation (RHCF), (ii) provide training and study tours for RHCF, (iii) finance project audits, and (iv) ensure that monitoring and evaluation was carried out as appropriate. 4 1.6 Revised Components Components were not revised. 1.7 Other significant changes Amendment to the Loan Agreement. Following the Federal Administrative Reform of March 2004, the Loan Agreement was amended on December 29, 2004, to acknowledge the Ministry of Health and Social Development (MOHSD), which replaced the former Ministry of Health (MOH) and to identify the Federal Penitentiary Service, subordinated to the Ministry of Justice (MOJ), both as implementing agencies for the Project. Revised Performance Monitoring Indicators. As per agreement reached during the Mid- term Review (October 2006), the Performance Monitoring Indicators were revised on June 29, 2007 through an amendment to the Loan Agreement. The amendment was done because the performance indicators originally proposed were determined not to be fully relevant, especially in the case of TB as they were based on former recording and reporting systems. Closing Date Extensions. The closing date was extended twice for a total of four months: (i) from December 31, 2008 to January 30, 2009 and (ii) from January 30, 2009 to April 30, 2009. Cancellation of Loan Proceeds. At the request of MOF, on January 30, 2009, US$30 million was cancelled. In addition, the Bank cancelled US$9.73 million upon loan closing. Reallocation of Funds. Only one minor reallocation was made from the Training Category (5) to the Operating Costs category (6) to finance the costs of the PIU and the final audit of the Project up to the revised closing date of April 30, 2009. 2. Key Factors Affecting Implementation and Outcomes The Loan Agreement became effective on December 31, 2003. Delays with effectiveness of loans are common in the Russian Federation. The Project was approved in April 2003 and the Loan became effective at the end of December 2003. Shortly after effectiveness there was a major administrative reform in the Government of Russia (GOR), necessitating processing of amendments to all the Project's legal documents due to abolishment of the Federal MOH, which in turn led to the suspension of the counterpart funding provision from the federal budget during the period April 2004 to May 2005. Because of delays in execution of the required changes in the legal documents and lack of counterpart funding, Project progress lagged behind schedule. Signed contracts were implemented with delays, and new contracts failed to be signed by the established dates. Some regions had poor coordination between the regional administrations and the health facilities' management, delaying the signing of the regional participation agreements. These issues resulted in delays with shipments under the signed contracts and unduly long storage of the equipment in warehouses. 5 2.1 Project Preparation, Design and Quality at Entry Following a Government request in early 1999, the Bank initiated development of a TB project with MOH and the Ministry of Interior (MOI) 2, to which HIV was later added. The Bank identification mission took place in February 1999. The Bank initially worked with WHO, the Department for International Development of the UK (DFID), the Canadian International Development Agency (CIDA), the Soros/Open Society Institute, the United States Agency for International Development (USAID), Medecins Sans Frontieres (MSF), the United Nations Children's Fund (UNICEF), and local NGOs, who were already involved in HIV efforts. Quality of Project Design. The Bank's initial efforts were directed to helping design a project that would scale and raise GOR's HIV/AIDS program to the level of international best practice and would address TB control through the WHO's recommended DOTS approach. Several missions took place during 1999 and 2000 to engage MOH, MOI, and other counterparts in Project design. Missions included Bank staff and top notch technical assistance on Multidrug- resistant tuberculosis (MDR-TB) and harm reduction. The Bank's project team devoted considerable efforts to establish a working relationship with counterparts, NGOs, and international partners. The team introduced the idea of modeling of TB and AIDS as a valuable instrument for an effective high-level policy dialogue with GOR. DFID provided an untied grant to finance key technical assistance to carry out the modeling. Modeling of TB, and of the potential impact of a moderate to severe AIDS epidemic on TB, was carried out initially with WHO's technical assistance and funding from DFID. The TB modeling was discussed with high-level government officials, including at the State Duma. Modeling of HIV/AIDS was initially carried out as part of Project preparation in 2000, with DFID funding and technical assistance from Oxford University. In 2000-01, project development ground to a halt due to a combination of disabling factors on all sides -- Government's, Bank's, and international partners'. Perhaps the most important factor was the Government's concerns about the DOTS approach to TB control, advocated by WHO and the Bank, and the potential effects of international competitive bidding (ICB) requirements on the domestic manufacturers of TB drugs. There was the added complication of the sensitivity of the Russian authorities and the population in general to the question of acknowledging the challenges posed by TB HIV/AIDS. There was a nine-month pause during which the Bank's team approach, supported by country and sector management, was to strengthen its relationship with the Government by reducing the perception of pressure to lend, while supporting the organization of public health seminars. In parallel, the Bank team sought to maintain focus and raise commitment on HIV/AIDS by producing jointly with a Russian scientist a model of the economic impact of HIV and by supporting the planning of a high-level meeting on production of vaccines, an area where the Russian Government wanted Bank's support. The Bank team also took a broader approach to TB, which acknowledged and aimed at building on Russia's own efforts and institutions. 2 Initially the Ministry of Interior was in charge of prisons. Later the prison system was transferred to the Ministry of Justice. 6 The Sixth Quality at Entry (QAE) Assessment of August 2003 rated the overall QAE as Satisfactory, recognizing that the technical and epidemiological dimensions of Project design in relation to the TB and HIV/AIDS epidemics were based on solid analysis and testing, both in Russia and elsewhere. The ICR mission concurs with the QAG assessment and rates Quality at Entry as Satisfactory, acknowledging that the Bank's analytical approach paid off. The PAD was comprehensive and clear in defining the respective roles of WHO and the Bank in guiding implementation of the TB program. Lessons from international experience as well as from pilot programs financed by DIFD and USAID, together with Russia-specific analytic work on HIV/AIDS, were integrated into Project design. The ICR team finds that the Bank preparation team was effective in several ways. First, it raised the Government's commitment to fighting TB/HIV/AIDS. The team used analytical work as the vehicle to: (i) generate general commitment, (ii) demonstrate impact, and (iii) reduce the pressure to lend. Second, it promoted the efficiency and technical quality of the response through an emphasis on evidence-based programmatic decisions. It focused on prevention strategies that embraced behavior change in high-risk groups while raising awareness of the population, and on more efficient approaches to TB and HIV surveillance. Third, it encouraged the Government's commitment to systematic expansion of program coverage for the implementation on a national scale of several of the highly-effective smaller scale activities piloted by bilateral donors and NGOs. Fourth, it promoted links in policy dialogue and design between the epidemics of TB/HIV and STIs in both the civilian and prison populations. According to the OED3 Evaluation of the Bank's Assistance in responding to the AIDS Epidemic ­ Russia Case Study (2005), during the policy dialogue, the Bank maintained an important focus on the efficiency of public expenditure on health in general, including HIV/AIDS, aiming to help the GOR improve the effectiveness and efficiency of its HIV efforts and at helping it move its AIDS program to international best practice. The OED report credits the Bank for having a positive impact on the design of the GOR's approach to dealing with HIV/AIDS and on the commitment of Government and civil society to addressing this problem. According to the OED's report, if the Bank had not been involved, the GOR's approach to HIV/AIDS would have been less targeted to the main drivers of the epidemic and less in tune with international best practice in key areas, such as harm reduction and sentinel surveillance. It would also have paid less attention to capacity building, laboratory strengthening, and making the blood supply safe. According to the report, without the Bank's involvement, it was unlikely that the Government would have planned to scale in a timely way HIV/AIDS prevention efforts that emphasize behavior change and are consistent with international best practice. Rather, such efforts would have remained small and local, and not in step with the imperative to move ahead forcefully against the epidemic. Risk Assessment. The PAD identified nine risks, eight of which were realistic and appropriately rated. The ninth was underestimated -- political and public reaction to harm reduction activities for high-risk groups such as IDUs and CSW's may be controversial. As noted in section 1.1, the epidemic was driven mainly by transmission among the high-risk core group of IDUs and, as such, there was a perception that HIV/AIDS was a problem of "social 3 The Operations Evaluation Department is now IEG ­ Independent Evaluation Group 7 deviants" resulting in patchy efforts by GOR to address the epidemic. In fact, to some extent, this perception continues today. 2.2 Implementation The ICR mission rates implementation as Satisfactory. The Project was successful in delivering most of the outputs under the three components and achieving most of its expected outcomes. The Project was not restructured, nor was it considered at risk at any time during its implementation period. The Mid-term Review rated implementation as Satisfactory to reflect improvements in implementation during 2006, which helped overcome start-up delays in 2004 and the first half of 2005 because of the administrative reforms. Overall, the Project performed well, especially the TB Component, albeit with delays in the implementation of activities. Implementation was complex; it included participation of 85 regions for the TB component and 82 regions for the HIV/AIDS Component. In general, the Project supported the development and large-scale implementation of interventions that were based on up-to-date scientific evidence. Project inputs were directed to updating and dissemination of strategies, guidelines and protocols, assessments of needs as a basis for effective planning, design, implementation, monitoring and evaluation, training, and on- the-job learning to improve the local capacity for effective implementation. It supported the supply of equipment and consumables required to improve the technical quality of diagnostic and therapeutic activities, the procurement of drugs, and close attention to project monitoring and impact evaluation. Implementation was successful in most ways and less successful in others. Positive factors and events that influenced project achievements Stakeholder involvement and participatory processes. To assess the regional needs and develop regional project implementation plans, agreements were signed with 12 Russian core research institutes (5 research institutes for the TB component and 7 for the HIV/AIDS component). The involvement of the federal institutes, both in the needs assessments and supervision visits to the regions, played an important role in raising the importance of both TB and HIV/AIDS at the level of the regional governments. Regional needs assessments for the prison sector were undertaken by the Medical Directorate of the Federal Prison Service. The results of these assessments provided the basis for drafting Project Participation Agreements with the regions, which were signed by the Russian Health Care Foundation (RHCF) in August 2005. In addition, a Project Participation Agreement was signed with the Federal Prison Service providing participation of over 600 territorial prison health facilities in 72 regions. An agreement was also signed with WHO, with financing from USAID, to support the work of needs assessments as well as systematic monitoring and impact evaluation of the TB component. The participation agreements with the regions served as tools for greater involvement of various stakeholders, contributing to better coordination between regional and health administrations. During discussions with beneficiaries in the health institutions, it became apparent to the ICR mission that the work of the needs assessment had been a most comprehensive exercise, conducted at the federal, regional, and municipal health facility levels. To optimize resources, lists of equipment and supplies were prepared taking into account available equipment rather than preparing generic lists of equipment for the different types of facilities. Evidence from the 8 ICR field visits showed that the equipment is being utilized fully and beneficiaries noted that training conducted for the use of the equipment had been thorough. The Project was successful in establishing a truly participatory process by involving federal TB and AIDS institutes as well as regional TB facilities and AIDS centers into the needs assessments, development of the comprehensive regulatory framework, and monitoring visits to regions and municipalities. This participatory process included also a well coordinated mechanism facilitating complementarity of efforts among national and international organizations in TB and HIV control, including Ministry of Health and Social Development (MOHSD), Ministry of Justice (MOJ), federal TB institutes, federal AIDS centers, regional administrations, WHO, UNAIDS, USAID, Global Fund, and the World Bank. This is an area, which was broadly acknowledged by all during the ICR discussions. In itself, this recognition is an accomplishment, given a country of this size and the complexity and sensitivity of the issues addressed. The Project has contributed significantly to strengthening national TB/HIV/AIDS control activities. The Project promoted successfully the increase of federal budgetary allocations for these programs since 2004 (e.g. the AIDS budget increased from less than US$50 million in 2004 to more than US$400 million in 2008). It has supported improvements in case detection diagnosis, treatment, and patient follow-up, both in the civilian and prison sectors. Strengthening laboratory capacity. The Project was successful in strengthening laboratory capacity (rapid tests, ELISA, PCR, CD4, CD8 and Viral Load), in health facilities of both the civilian sector and of the prison system. This was particularly critical for the prison system which was transferred from the MOI to the MOJ, resulting in the absence of medical infrastructure, particularly laboratory capacity as this infrastructure was retained by the MOI. As such, the Project has supported the improvement of diagnosis, case management, and patient follow-up. These investments have been critical to promoting the scaling-up of anti retroviral (ARV) treatment supported under the national HIV/AIDS Program. Together with the support from the Global Fund and with technical support from UNAIDS and WHO, the Project has contributed to expanding the access of the population, particularly low income groups and prisoners, to TB and HIV detection, diagnosis, treatment, and monitoring. Strengthening human resource capacity. Staff interviewed in several institutions during the ICR mission enthusiastically acknowledged the positive role the Project has played in strengthening TB and HIV/AIDS laboratory surveillance and services, which has improved the quality, safety, and effectiveness of their work. It was particularly encouraging for the ICR mission to learn from beneficiaries how much they appreciated the training, and on-the-job learning provided under the Project. The training presented opportunities for staff of the many institutions, regions, and municipalities to exchange experiences and learn about new techniques to enable them to increase productivity and the quality of the work. The opportunity to be exposed to international best practice was also widely acknowledged as a strong element of the Project. Strong project coordination. Project implementation was coordinated well by a small PIU team, who provided strong coordination, procurement, financial management, and technical support to multiple institutions at the federal, regional, and municipal levels. Procurement and 9 financial management were particularly strong under the Project (see relevant sections). Significant problems could have been encountered in procurement management given the complex and extensive purchases, which had to be delivered to 85 regions and installed in facilities of all levels at the health system. It required sophisticated and systematic planning to ensure timely procurement, delivery, installation, and training of staff in the use of the equipment. Less effective factors and events which influenced project's achievements Administrative reforms and changes in MOHSD. Significant administrative changes in any government, including line ministries often lead to a disfunctionality until new internal procedures are established. The 2004 administrative reforms resulted in the merger of the Ministries of Health and Social Development, delaying implementation because of the need to make legal amendments. However the mid-level team on the health side of the new ministry stayed almost intact and played a strong leadership role in setting up a participatory process from the outset of Project implementation. Towards the end of project implementation, another wave of changes affected both high and middle management levels in the ministry, affecting its ability to fulfill its coordination and leadership mandates as related to Project oversight. Non-balanced approach skewed towards medical aspects such as prevention, diagnosis, treatment, and care versus targeted prevention focusing on high risk groups. While the increasingly prominent ART agenda is a positive development, it might have some implications given that HIV/AIDS is seen more as a medical problem heavily emphasized on treatment and care and prevention among the general population at the expense of targeted preventive interventions among high risk groups, the driving forces of the HIV/AIDS epidemic in the Russian Federation. Limited targeting of high-risk groups. Although targeted preventive interventions were envisaged by the Project design, their implementation was limited to needs assessment of readiness of regional institutions to launch HIV prevention interventions in high risk groups (harm reduction programs) and development and approval of methodological recommendations on: (a) behavioral surveillance, and (b) prevention of HIV/AIDS among IDUs, MSM, CSWs, mobile groups, and prisoners. This was due to: (i) changed landscape of development aid when new grants focusing on high-risk groups became available, and (ii) longer time than anticipated required for needs assessments, and development and review by WHO and UNAIDS of the regulatory framework for HIV/AIDS, and its approval by MOHSD. There was no comprehensive regulatory framework for HIV/AIDS control prior to the Project. A regulatory framework is necessary for implementation of nation-wide projects in Russia, where regions had to adhere to a framework developed and approved at the federal level. Development assistance for HIV/AIDS. While there has been a significant increase in external funding, financing modalities are not integrated into the existing systems. All external funding is off-budget and no proper mechanisms for financing of civil society organizations have been developed and tested for further scale-up to budget funding and nation-wide coverage. 10 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization Design, implementation, and utilization of M&E is rated Satisfactory. The technical criteria for M&E of the TB and HIV/AIDS components were based on WHO and UNAIDS guidelines. The PIU was responsible for producing an annual report integrating the results of the M&E activities, providing information on the: (i) progress of each of the components and sub- components, (ii) broader impact of the Project, and (iii) effect on the overall control of TB and HIV/AIDS epidemics in the Russian Federation. The technical monitoring of the TB component was guided by WHO. This was critical because of the need to monitor the performance of participating regions and to ensure that the proper diagnosis and treatment regiment were used for the basic and enhanced TB packages, as well as the monitoring of drug use to reduce the risk of drug resistance. The HIV/AIDS component envisaged the following M&E elements: (i) existing health information systems, (ii) virological surveillance, (iii) sentinel surveillance, and (iv) behavioral surveillance surveys (BSS). Following agreements at the Mid-term Review, the results framework in Annex 1 of the PAD was amended in close consultation with WHO, UNAIDS, and the Global Fund. The amendments were carried out because the performance indicators, especially for TB were determined not to be fully relevant and were based on former recording and reporting systems. The key performance indicators were revised and modified taking into account the new work plans for the individual Project components. Data collection and reporting were conducted as part of the regular surveillance system that was strengthened with Project support, using data and information and assessments made at the regional and federal levels. The Project strengthened existing systems rather than attempting to establish a project-specific system. Assistance was provided by WHO and UNAIDS to strengthen national M&E efforts. M&E was used under the Project not only as a management tool to evaluate the status of implementation of activities, but to inform policy makers for decision-making purposes, particularly for assessing the evolution of the epidemic by region and the design of targeted measures to support low performing regions. M&E has become an important tool providing the authorities at the federal and regional levels with timely and relevant data. New forms for TB recording and reporting were produced under the Project and disseminated among all regions, both in the civilian and the penitentiary sectors. Staff received training for the use of the cohort method analysis in TB detection and treatment control. Specialists from federal TB research institutes and AIDS centers make regular monitoring visits to all participating regions. Fifty two standards, norms, and protocols for HIV/AIDS prevention, treatment and care were prepared (in 5 volumes) by specialists working under the direction of an MOHSD Working Group and in collaboration with federal AIDSs centers. These documents were reviewed by WHO and UNAIDS and approved by MOHSD. Volume II addresses issues related to HIV/AIDS epidemiological surveillance, which includes collection of HIV AIDS data, behavioral surveillance, analysis of epidemiological data, and M&E of HIV prevention and treatment. The promulgation and approval of these new regulatory instruments represent a major achievement under the Project as it has laid the normative framework for HIV/AIDS activities across the Federation for the 11 implementation of the national HIV/AIDS program supported by the Government and all international partners. Training materials have been developed based on approved guidelines and protocols and the training is taking place with support from the Global Fund. 2.4 Safeguard and Fiduciary Compliance The environmental impact of the Project was rated C, therefore no environmental assessment was required. The Project did not finance rehabilitation or civil works and did not deal with disposal of medical waste. Financial Management (FM). Overall, financial management is rated Satisfactory. FM was systematically rated Satisfactory during Project implementation. Overall, accounting, FM reporting, information systems, and internal controls were effective and reliable. They provided appropriate financial information to manage and monitor Project implementation. Despite the project size and large number of accounting transactions, FM records and reports were submitted regularly, and all of the external audits were prepared on time and with unqualified opinions. As discussed before, an early stumbling block due to the Government restructuring caused delays in overall project implementation and severe issues with the availability of counterpart funding. This issue was resolved with amendments to the Loan and regional agreements. The regions reported systematically on equipment and drugs received and their distribution. Project units made final 10% payments to suppliers after receiving original acceptance reports from end users at the regional and municipal levels. Withdrawal applications and reconciliation of the special account were provided regularly and the quality of the documentation presented was good. Disbursements were initially slow for reasons outside the control of the Project (administrative reform). These picked up markedly after the MTR. The Loan was not fully disbursed; US$39.73 million was cancelled (See Section 1.7). Total disbursements were less than projected in September 2008 because the majority of the final contracts signed in the summer of 2008 were denominated in rubles or Euros, and both currencies depreciated significantly against the US dollar starting from October 2008. Procurement is rated Satisfactory. The PIU managed procurement successfully, despite the challenges of dealing with 85 regions. The ISR of May 2005 rated procurement as Moderately Satisfactory, consistent with overall project performance, due to overall delays in project implementation caused by the lingering effects of the extensive Government restructuring. Project performance in general, including procurement, accelerated after responsibility for Project implementation was transferred from the old MOH to the new MOHSD. Procurement performance, as well as overall project performance was upgraded to Satisfactory at the Mid-term review and continued as such until the end of Project implementation. The PIU procurement and financial management controls proved effective to monitor procurement, delivery and installation of equipment, training of staff in the use of equipment, and the delivery of drugs among 85 participating regions. Procurement plans were updated regularly and implemented without significant delays. The RHCF had a good system to monitor contracts after the goods were delivered and accepted by the end-users. All participating regions signed contracts with RHCF for monitoring the delivery of goods. Each such contract contained an inventory of equipment. Each end-user prepared 12 annual reports for goods received. The inventory lists included description of goods and dates of receipt and acceptance. In each of the seven okrugs, a related authorized Federal institute monitored all participating organizations. By April 24, 2009, all procurement contracts were completed with goods delivered and accepted, consulting assignments implemented and approved, training provided, and all payments made against these contracts. All in all, it is commendable that procurement was implemented successfully despite the complexities of procurement under the Project encompassing procurement and delivery of sophisticated medical equipment and supplies, drugs, and office equipment throughout the vast territory of the Russian Federation, as well as a great number of consulting contracts. 2.5 Post-completion Operation/Next Phase Support for the implementation of TB HIV/AIDS strategies is to continue with financing from federal and regional budgets. During discussions, officials reiterated their strong commitment to continuing implementing TB and HIV/AIDS programs and assured the ICR team that the GOR is committed to sustaining the achieved gains. Project inputs directed to development, updating and dissemination of strategies, and guidelines and protocols have paved the way for effective implementation of large-scale interventions, which are based on up-to-date scientific evidence. The extensive training and support to strengthen surveillance systems in both TB and HIV/AIDS have improved the local capacity for effective implementation and have improved the technical quality of diagnostic and therapeutic activities. Discussions with local implementers provide a sense of cautious optimism that the gains will be sustained. Equally, discussions with federal and regional managers provided assurances that the budgets for the implementation of TB and HIV/AIDS programs will be maintained despite financial constraints, including full support for recurrent and other operation and maintenance costs. However, after the end of 2010, it is not clear what international assistance will continue as the GOR has declared its intention not to accept financial international assistance on the grounds that the Russian Federation has now sufficient resources for development of its own health sector and provision of assistance, as a donor, to other countries. At the same time, the availability of budget funding for certain critical areas might still be inadequate, particularly for continuation of supervision/monitoring visits to regions by the federal TB institutes and AIDS centers. Their role in providing systematic guidance and monitoring will be critical to sustaining project achievements and the TB and HIV/AIDS programs in general. On the job training by TB research institutes, and their monitoring and supervision of secondary level (specialized care) and PHC facilities are also important. Budgets must be made available to ensure that these activities continue. 13 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation Relevance of PDOs. Containing the growth of TB and HIV/AIDS epidemics in the short- term and halting and reversing the courses of these epidemics in the medium-term were important development priorities of both the Government and the Bank at the time of Project approval. As noted in Section 1, the Project provided support to the GOR's federal program on prevention and control of social diseases, 2002-2006. At the time of the ICR mission, these PDOs continue to be highly relevant because: (i) TB and HIV/AIDS remain significant threats to health and development in the Federation, (ii) addressing TB and HIV/AIDS is an integral part of the Government's Federal Program on Prevention and Control of Social Diseases and the National Projects in Social Sphere, and (iii) TB and HIV/AIDS remain priorities of the current Country Partnership Strategy, recently extended to FY2011. Relevance of Design For reasons elaborated below, the relevance of objectives, design, and implementation arrangements were Substantial. Component 1 - Control of Tuberculosis. The relevance of the design of this component is Substantial. Project design included a well-defined WHO-recommended strategy for TB control, including its key five elements: (i) political commitment to effective TB control; (ii) case detection among symptomatic patients, seeking care using sputum smear microscopy; (iii) standardized treatment regimen of chemotherapy with first-line anti-TB drugs (including DOTS); (iv) uninterrupted supply of all essential anti-TB drugs of appropriate quality; and (v) establishment and maintenance of a standardized recording and reporting system to allow assessment of treatment results and efficiency for TB control measures. Focus on the technical aspects and adaptation of the key five elements to the Russian context, both to the existing health system and to the current socio- economic conditions,4 have been critical to the successful implementation of TB control in the Russian Federation. Component 2 - Control of HIV/AIDS. The relevance of this component design is rated Modest. Given that HIV AIDS control was new to the Russian Federation, the ICR mission found that the HIV component was too ambitious and perhaps too "prescriptive." It aimed to: (i) improve policies, strategies, and public information for HIV/AIDS control; (ii) strengthen surveillance and monitoring; (iii) improve laboratory service and blood safety; (iv) prevent and control STIs; (v) deliver preventive services against HIV/AIDS, with emphasis on high risk groups; and (vi) prevention of HIV/AIDS transmission from mothers to children. 4 Significant impoverishment, unemployment, migration, the sharp declines in resources for health services in the early-to-mid 1990s resulted in a sharp increase of TB incidence and TB mortality. According to published data 30- 40% of new TB cases were unemployed. High rates of poverty and alcoholism create difficulties for TB treatment compliance. Therefore, the current TB treatment practices continue to be heavily hospital-based. 14 In hindsight, while it was important to focus on goals (i), (ii), and (iii) above, perhaps the rest of the component could have allowed for more flexibility to adapt to evolving circumstances. In practice, start-up of different projects focusing on high-risk groups, as well as the significant injection of both external and internal HIV/AIDS funding from different sources, called for improved planning and coordination of the national response. In this context, the support to develop detailed Regional Strategic Plans for HIV/AIDS (RSP), including their costing and identification of specific sources of funding, could have been a good tool to help GOR to use resources more efficiently and effectively, regardless of the source. Availability of such RSPs, if not at project design but during implementation, could have helped to prioritize interventions and identify key priority areas on which the Project could focus after the MTR. The RSPs could have: (i) been a tool for better planning of future HIV/AIDS budgets in regions beyond 2010 when the country is likely to rely only on the government funding of HIV/AIDS control programs, and (ii) strengthened further links to other health issues such as blood safety modernization, laboratory services reforms, prevention and control of STIs and opportunistic infections, and MCH (particularly PMTCT), thus creating the ground work for the next generation of health reforms. 3.2 Achievement of Project Development Objectives Overall, the achievement of both PDOs is rated Satisfactory for reasons elaborated below. Interventions from domestic and international resources, including the Project, contributed to the overall objective of containing the growth and halting and reversing the course of TB and HIV/AIDS epidemics. This makes an explicit attribution to the Project almost impossible. Instead of attempting to attribute the extent to which the Project contributed to the achievement of the PDOs, the ICR examines the contribution of the Project to achieving the PDOs, as measured by the Project's outcome indicators. The ICR uses the revised indicators and targets agreed at the Mid-term review (see Section 1.7) as the basis of assessment and makes clarifications against the original targets, indicators (when necessary). The Table below shows main achievements and challenges by areas. 15 Control of TB Control of HIV/AIDS Main Achievements Outcomes · Overall TB incidence has been stabilized. · 5.1 % reduction of prevalence growth rate in 2008 · TB incidence rates have been stabilized in all seven vs. 2003. Federal Okrugs since 2004-20055. · Mother-to-child transmission decreased from 13% · TB incidence rate reduced in the penitentiary system in 2003 to 10.6% in 2008; slightly less than the from 4,347 to 1,372/100,000 over the last 5 years. agreed target of 10%. · TB prevalence reduced to 194.6 in 2007 vs. · STIs targets (syphilis, tricomoniasis, chlamidiosis) 190/100,000 in 2008 with significant variations by were achieved except the gonorrhea target. Federal Okrugs (126 in Central vs. 291.9 in Siberian · 11.9% HIV-positive among officially registered Okrug). IDUs in 2007 vs. 12% at MTR. · Improved TB smear positive case notification from 20.2 in 2003 to 23.8/100,000 in 2007. · Reduced TB mortality from 22 in the country as a whole; 130/100,000 in prisons, 18.4 and 82.2/100,000 (16.4% and 37% decrease) in 2007 as compared to 2003. · 75% of new TB cases were under standardized treatment regimen in 2007 vs. 50% in 2006 at MTR (and 44% in 2003), but not yet reaching WHO target of at least 85%. Improved policies, strategies, protocols for TB & HIV/AIDS and public information for HIV/AIDS · Comprehensive TB control regulatory framework for · Comprehensive HIV/AIDS control regulatory TB detection, diagnosis, surveillance and treatment, framework developed in consultations with consistent with evidence-based approaches and agreed UNAIDS and WHO (52 documents in 5 volumes). with WHO approved and implemented in all regions (MOHSD's Decrees #109 and #50)6. Strengthened surveillance, monitoring, quality control and quality assurance · In 2007, 1,210 MDR-TB patients were registered in · Single information system for HIV monitoring was 21 regions participating in the GLC program for the developed accompanied with capacity building of first time; no reliable data was available before. staff. · 1,000 laboratories with regular external quality control of microscopy tests vs. 670 at MTR and 0 at baseline. · Forms for recording and reporting were developed; however the post-evaluation of the system for surveillance and monitoring did not take place. Diagnosis, treatment, care and prevention · Availability of TB drugs susceptibility test results has · Strengthened laboratory capacity both in health been significantly reduced from 4-6 to 2 weeks facilities of the civilian sector and prison system, resulting in early adjustment of TB treatment. has supported improvement of diagnosis, case · Increased laboratory confirmation of new TB cases by management and patient follow up. This resulted in sputum microscopy from 25.3 in 2003 to 33.6% in scaling up of ARV treatment (1,874 items of lab 2007(32.8% increase, but not yet achieving the target equipment were supplied to regional AIDS centers, of 45%). prison facilities and research institutes). · Increased laboratory confirmation of new TB cases by · Continuous and increasing coverage with ARV inoculation from 41.3% in 2003 to 45.8% in 2007 according to national treatment protocol (35,000 or (11.7% increase, but not yet achieving the target of at 65% of HIV infected persons receiving ART in least 55%). 2008 vs. 25% in 2006). · 100% of TB patients receive controlled chemotherapy · ART is provided by the national health services free at the inpatient stage and 38%, at the outpatient stages of charge due to substantial increased funding for (survey in 67 regions in 2007-2008) vs. 50% at stage pharmaceuticals under the federal budget. of needs assessment. · 99.9% of pregnant women were tested for · 75% TB cases are under the standardized treatment in HIV/AIDS vs. baseline of 85% in 2003. 2007 compared to 44% in 2004 (70% increase from 92.3% and 97.9% of identified HIV infected 2004). pregnant women and HIV-exposed neonates were · 58.3% treatment success rate in 2006 cohort (no data provided with ARV prophylaxis for PMTCT in available previous years). 2008 vs. 70% in 2003. · 4 Federal TB institutes and 21 regions have the capacity to treat MDR-TB patients in accordance with WHO requirements vs. 4 regions at the MTR. Main challenges · Timely procurement and supply of TB drugs to ensure · A shift to heterosexual transmission of HIV/AIDS uninterrupted TB treatment and prevent MDR-TB. 16 increased HIV prevalence among CSW, 6-15% in · Increase of MDR-TB cases will result in increased 2007 in big cities vs. 5.0% at the MTR. health expenditures as the cost of treatment of MDR- · High rates of HIV/AIDS among high-risk groups ­ TB case is 10 times higher. (i) 11.9% HIV-positive among officially registered · Increased health expenditures due to an increased need IDUs in 2007 vs. 12% at the MTR; (ii) increased PDO 1: To contain the growth of TB and HIV/AIDS epidemics in the short-term Project activities, along with interventions by other financiers have contributed to containing the growth of TB and HIV/AIDS epidemics in the short-term. Significant progress has been made in increasing the effectiveness and efficiency of the national TB response, based on WHO guidelines focusing on five key elements (See Section 3.1). The Russian Federation has been able to contain the TB epidemic with TB incidence at the level of 82.7 ­ 83.2 per 100,000 in 2003-2007 (Figure 1) and to reduce TB mortality since 2005. The TB incidence among prisoners, which is still high, has fallen from 13.9% in 2003 to 11.7% in 2007 (Figure 2), despite of high HIV incidence especially among prisoners. The ICR team recognizes that national incidence rates had already started stabilizing after a peak in 2000, thus raising questions as to the Project's contribution. Thus, the team also reviewed regional trends, as described below, to seek further evidence of the Project's contribution. 17 Figure 1: Registered TB incidence rate in the RF, 1992-2007, all health facilities 100 90.7 88.5 90 Per 100 thousand population 85.2 86.3 82.7 83.3 84.0 82.6 83.2 80 76.0 73.9 70 67.4 57.8 60 48.0 50 42.9 40 35.8 30 20 10 0 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Source: RHCF. Form 8; Population: Forms 1 and 4 Figure 2: Overall TB incidence rate including proportion of registered TB incidence rate among penitentiary system population, 1999-2007 100,0 90.7 88.5 90,0 85.2 86.3 82.7 83.3 84.0 82.6 83.2 È È È 80,0 ! % ! % È È ( È $ È È & ! ( È Ã Ã Ã ! " ! $ Ã ( Ã 70,0 ! 3HU SRSXODWLRQ Ã 60,0 50,0 40,0 30,0 Civil permanent residents FSIN 20,0 10,0 0,0 1999 2000 2001 2002 2003 2004 2005 2006 2007 There is a strong regional variation in TB incidence and TB mortality, increasing from the West to the East (62.7/12.6 and 62.2/15.7 per 100,000 in Central and North-West Federal 18 Okrugs vs. 128.0/28.1 and 132.3/29.1 in the Far East and Siberian Federal Okrugs). The inter- regional differences are better proxies of the efficiency of the TB control measures, as higher levels of aggregation such as national data make it harder to separate the influence of external socio-economic factors. As Figure 3 shows, TB incidence had started stabilizing around year 2000 in Central and North-West Federal Okrugs, while it continued growing in the Far East and Siberian Federal Okrugs. The ICR team speculates that those okrugs where TB incidence had stabilized previous to the project were driving the TB incidence national average. However, since Loan effectiveness the TB incidence rates have stabilized in all okrugs. Thus, the ICR team concluded that a stable decreasing trend in the different okrugs of Russia is a reflection of efficiency TB control measures supported by the Project. Figure 3: Distribution of TB incidence rate by Federal Okrugs (counties), 1991-2007 (MOH) 120 108,6 108,9 NWFR, CFR 97,3 100 SFR, PFR 103,4 85,3 UFR 59,5 77,5 58,5 91,4 94,2 SbFR, FEFR 74,5 73,8 84,6 v 80 RF 84,7 52,1 75,0 65,1 64,8 h y 65,0 66,263,3 55,3 78,7 62,7 60,7 66,7 56,3 62,7 Ã 60 41,040,9 61,5 49,7 62,3 62,7 48,1 47,5 Ã 40,4 r 40 Q 34,3 30,5 56,6 49,5 50,0 48,249,1 42,6 44,4 44,1 54,5 47,2 25,6 40,1 48,151,7 45,6 49,1 20 36,3 30,3 47,9 40,8 35,3 43,8 33,7 0 ! " # $ % & ' ( ! " # $ % & ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ( ! ! ! ! ! ! ! ! The Project contributed to improved TB detection, diagnosis, and treatment through: (i) strengthening of 2,371 clinical laboratories in the PHC and penitentiary network of regions; (ii) equipping of 6 reference laboratories in the TB Federal institutes and the penitentiary system; and (iii) strengthening of 149 bacteriological laboratories. In addition, primary health care and TB facilities in both civilian and penitentiary systems were supplied with 3,000 X- ray/pluorography machines and provided with first-line TB drugs; equipment was accompanied by training of staff. As a result, TB detection and diagnosis rates have significantly increased in all regions by 1.5-5 times resulting in the increased coverage, with SSM reaching 98.2% of all new TB cases, from which 87.2% also had inoculations tests to confirm TB diagnosis. Compliance with the standard treatment regiment among newly detected TB cases has increased from 44% in 2004 to 76% on average; 29 of 63 regions (46%) fully achieved the target of 85% by the end of the Project. Assessment of treatment course effectiveness was done based on data of cohort analysis. Strengthening of the treatment monitoring in all regions contributed to increasing the treatment effectiveness up to 58.3% across the country, which is slightly lower 19 than the target recommended by the WHO STOP TB Partnership. Treatment effectiveness and data reliability are expected to improve in the years ahead. In summary, progress towards the achievement of PDO 1, especially containing the TB epidemic growth, was Satisfactory: (i) stabilization of TB incidence started before the Project, but continued despite the HIV/AIDS increase and the financial crisis; (ii) the small increase in TB incidence in 2007 was due to improved TB detection; (iii) TB incidence stabilized in all 86 regions during 2004-2007; (iv) reduced TB-related mortality from 22 in 2003 to 18.4 in 2007; and (v) progress is being made towards achieving WHO case detection and cure rates. In regard to HIV/AIDS, the target of reducing HIV prevalence among the general population by up to 1.5% was met. However, the HIV prevalence rate in the general population is not a good indicator of progress in reducing the spread of HIV infection. HIV incidence is a better indicator for this purpose, but it is difficult to monitor. The Project contributed to the reduction of HIV vertical transmission, as reflected in the decrease of HIV-positive infants born to HIV-infected mothers from 13.2% in 2003 to 10.6% in 2008, almost reaching the target of 10%. Figure 4: Number of HIV cases per 100,000 inhabitants 300,0 250,0 200,0 150,0 100,0 50,0 0,0 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 Number of persons living with HIV per 100 000 population, Number of new HIV-positive cases per 100 000 population 20 The Project's intermediate outcome indicators are better indicators of the HIV/AIDS prevention progress. Targets of increased percentage of HIV-positive pregnant women and HIV- exposed neonates, who received prophylactic ARV drugs, were met (92.3 % and 97.9% in 2008 vs. 70% in 2003), significantly exceeding the target of 85%. These achieved targets reflect only one of the sub-components. Achievement of PDO 1 for HIV/AIDS is modest. The Project contributed to this HIV/AIDS related PDO through supporting: (i) development and dissemination of the comprehensive regulatory framework; (ii) procurement of equipment, test kits and supplies to improve laboratory services and blood safety; and (iii) procurement of ARV drugs. Project activities were closely coordinated with other health programs and projects supported by the Global Fund, USAID, WHO, Government federal programs, thus minimizing potential duplications. PDO 2: To halt and reverse the courses of TB and HIV/AIDS epidemics in the medium- term Progress toward this PDO, particularly on halting and reversing the course of TB epidemic, is rated Satisfactory. The Project contributed to this PDO through supporting development and capacity building for implementation of Decrees No. 109 and No. 50, which provide the comprehensive regulatory framework for TB control. These decrees are mandatory for execution in all 86 regions of the Federation. Achievement of PDO 2 related to halting and reversing HIV/AIDS epidemics is rated Moderately Satisfactory. The Project was able to establish a participatory process for development of the comprehensive regulatory framework and complete all planned procurements of goods despite significant delays with start-up of project implementation along with changes in the Project environment. The Project contributed to this PDO through: (i) development of a comprehensive regulatory framework; (ii) provision of CD4, CD8 count machines, other laboratory equipment, test kits, and supplies to improve HIV/AIDS diagnostics, scaling up of ARV treatment, and prevention of HIV/AIDS transmission from mothers to children; and (iii) strengthening of blood safety. However, the Project did not deliver preventive interventions against HIV/AIDS with emphasis on high risk groups, mainly because of the delays in start-up activities by almost a year (see Section 2.2). The training in the use of the regulatory framework is being supported by the Global Fund. 3.3 Efficiency Available data is insufficient to carry out a detailed economic analysis of the Project at closing. On the basis of the considerations below, the Project's efficiency is rated as Substantial. The PAD included a cost-benefit analysis, which identified both direct and indirect benefits. The direct benefits were to be the expected tangible benefits accruable to the Project from the reduced hospital stays, reduced diagnostic expenses, elimination of unnecessary discharges, and reduction of consultations. Indirect benefits were related to the potential life years to be saved by the Project and the economic and financial value of increasing productivity. The Project was to yield a present value of net benefits, after investments and recurrent costs of US$380 million over five years and US$1.4 billion over ten years. The internal rate of return was estimated at 143 percent. While it is difficult to calculate specifically direct or indirect benefits of the Project, it is clear that both TB and AIDS outcomes have improved as the result of the Project's investments. 21 Specifically, the Project, together with other federal, regional, and international assistance interventions has: (i) played an important role in reducing hospitals stays; (ii) reduced diagnostic expenses because of the ability of staff to conduct tests in a more efficient manner thanks to the medical equipment procured under the Project; and (iii) reduced the number of consultations required to obtain results of tests (see also Section 3.5 (b). It is not possible, however, to quantify life- years saved by the Project and/or other interventions and therefore to calculate the economic and financial value of increasing productivity. What is clear is that the levels of hospital and health institution outputs have increased because of project's investments, including medical equipment and supplies, and training of staff. 3.4 Justification of Overall Outcome Rating The Project's overall outcome is considered Satisfactory based on the Project's significant achievements, its continued relevance, its contribution towards strengthening the Federation national programs, and making them more accountable and efficient. As the Russian Federation confronts its next generation of challenges regarding TB and HIV/AIDS, the Project's contributions will continue to be highly relevant. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development While the Project was not explicitly classified as a poverty-targeted operation, it has had a positive impact on the poor. It aimed to contain TB, typically acknowledged as a disease of the poor with high at risk populations living in overcrowded and poor environments contributing to generally weaker health and nutritional status. The Project expanded the access of the population, particularly of low income groups and prisoners, to TB and HIV detection, diagnosis, treatment and monitoring. Progress towards decreasing mother-to-child transmission has been good. Further, the Project has supported regions with difficult socio-economic conditions, where the incidence of AIDS is increasing significantly. (b) Institutional Change/Strengthening While it is difficult to establish direct links between capacity improvements and improved TB and HIV/AIDS outcomes, particularly without solid baseline data, process and output indicators provide useful pointers to possible impact. It is also difficult to attribute success in capacity building to a single project as other intervention factors are always at play. However, it is evident that Project investments resulted in substantial institutional development impacts at the federal, regional, and municipal levels. It strengthened institutions and supported development of the comprehensive regulatory framework, which is mandatory for implementation in all regions. Project investments also played an important role as catalyst for change. The project catalyst role was acknowledged by international partners interviewed during the ICR mission. While the financial contribution of the Project was rather small in the scheme of all the financial resources available for TB and HIV/AIDS, in the view of the ICR team, the Project's impact was significantly greater. The importance of the Project was invariably acknowledged during ICR discussions. Staff highlighted the indirect benefits of the Project on improving the overall implementation of programs. Without the Project, some authorities and development partners acknowledged that it was likely that implementation of TB 22 HIV/AIDS strategies would have moved more slowly and would have been subject to more political influences. The Project financed significant investments in training and was successful in improving overall surveillance systems in both TB and HIV/AIDS programs, thus contributing to capacity building. At the federal level, the Project contributed to strengthening the TB institutes and AIDS centers coordination and leadership roles, programs planning, and implementation. The Project also strengthened the laboratory capacity at the federal, regional and municipal levels, as well as M&E capacities. In addition, the Project supported partnerships with various government entities, including close collaborations between MOHSD and MOJ, at both federal and regional levels, which are likely to have been absent without the Project. Further, collaboration with international partners, including the technical agencies ­ WHO and UNAIDS, as well as the Global Fund, helped promote a coordinated effort for the implementation of TB and HIV/AIDS activities. (c) Other Unintended Outcomes and Impacts (positive or negative) The Project has contributed to health system strengthening by putting in place a rigorous M&E system, with technical assistance from WHO, which can serve as a model for other health areas. It has also supported coordination and integrated provision of diagnostic services at the PHC level to respond to the changing epidemics. The Project's support in the development and approval of the comprehensive regulatory frameworks for both TB and HIV/AIDS control represents a major achievement, which has laid the normative framework for TB and HIV/AIDS activities across the Russian Federation. The regulatory framework should be of interest and use (with modifications) in other countries of the region with similar organization of TB and HIV/AIDS services. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops A Beneficiary Survey was not conducted. 4. Assessment of Risk to Development Outcome is rated Moderate Institutional Risks. The GOR's commitment to continuing its response to TB control and HIV/AIDS is clear. However, there are institutional risks. The main one is posed by management and staff turnovers in MOHSD and departure of the PIU staff, which has resulted in the loss of institutional knowledge and memory. The PIU capacity was not absorbed within MOHSD. Availability of budget funding to continue supervision/monitoring visits to regions by the TB federal institutes and AIDS centers is also potentially a risk. Their role in providing systematic guidance and monitoring is critical to sustaining the achievements of TB and HIV/AIDS programs in general. On-the-job training by TB research institutes, and their monitoring and supervision of secondary- level and PHC facilities are also important. Budgets must be made available to ensure that these activities continue. Institutional risks at the regional and municipal levels might be considered relatively moderate due to the strong involvement and commitment of staff at the health facilities for systematic implementation of TB and HIV control activities. There is also relatively low turnover of staff in these facilities. Long-term sustainability will depend on timely availability of Government's funding. USAID is presently considering an extension of its financial support to WHO for the provision of technical assistance to continue the independent monitoring and impact evaluation of the TB program. This support would also extend to the Federal TB institutes. 23 Financial Sustainability. TB and HIV/AIDS programs' funding has been decentralized to the regions. According to GOR, financial needs for first-line and second-line TB and ART drugs will be fully covered without interruptions from public funds (see also Section 2.5). However, timely availability of drugs might be affected with an increase of number of MDR-TB and HIV cases, requiring an increase in health expenditures to purchase second-line TB drugs. Lack of timely supply of drugs could play a negative role in sustaining gains achieved, particularly of the affected population, which so far has been supported by the Global Fund. As noted earlier, the Fund's contribution is not likely to be available beyond the end of 2010. The ICR mission was informed by federal authorities that most regions have a stock of drugs for a maximum of six months. Treatment Sustainability. The PAD identified a substantial potential risk of treatment failure resulting from resistance to TB drugs. While treatment is a long-term agenda, the rate of default at the end of the project was 9.6 %, which is below the 10 percent threshold for withdrawing second-line drugs. Treatment sustainability will also depend on timely availability of drugs and compliance with WHO-recommended standard regiments for TB treatment. Prevention. The budget allocation for the HIV/AIDS program in the 2010 draft budget is at the level of 2009 despite the economic constraints. This funding is earmarked for ART drugs and primary prevention activities among the general population. One of the challenges with funding of HIV/AIDS prevention activities for high risk groups is linked to budget restrictions that currently do not allow funding for NGOs. This is a critical issue that should be resolved in time for the next year budget execution. This budget restriction can lead to an inability to target the drivers of the epidemic with effective preventive interventions. The problem becomes more critical when considering that funding from international sources for these interventions might not be available beyond 2010. 5. Assessment of Bank and Borrower Performance The Bank's team's leadership (TTL) changed three times during project preparation and twice during implementation. The Moscow-based staff changed only once during implementation. 5.1 Bank Performance: Satisfactory (a) Bank Performance in Ensuring Quality at Entry: Satisfactory Over four years (1999-2002), the Bank engaged in high-level policy dialogue, including provision of technical assistance and analytical work. Several missions took place in 1999 and 2000, but between 2000 and 2001 project development ground to a halt. While development of the HIV/AIDS component had moved forward, difficult issues arose in preparation of the TB component, which stalled discussions. The most important factor related to concerns in the medical establishment about the DOTS approach to TB control and concerns in the domestic TB pharmaceutical industry about possible loss of markets (see also Section 2.1). The Bank team tried to build consensus among partners and other actors, including NGOs, to set out a clear policy context for further Project development. The team used analytical work as the vehicle to generate commitment, demonstrate likely impact, and reduce the perception that the Bank was pressured to lend. It promoted links in policy dialogue and design for both the civilian and prison populations. 24 Project design made best use of the most up-to-date information and evidence available. It promoted the efficiency and technical quality of the response through an emphasis on evidence-based programmatic decisions, focusing on prevention strategies that embraced behavior change in high- risk groups, while raising awareness of the population, and more efficient approaches to TB and HIV surveillance. Project design encouraged the Government's commitment to systematic expansion of program coverage for the implementation on a national scale of several of the highly effective smaller scale activities piloted with support from bilateral donors and NGOs. (b) Quality of Supervision: Satisfactory Generally there were two supervision missions annually conducted in conjunction with supervision of another Bank-financed health project, the Health Reform Implementation Project. The skills mix was adequate, comprising HQ-based and Moscow-base staff, as well as international partners (WHO, UNAIDS, Global Fund, USAID). Project supervision promoted strong coordination arrangements among a great number of government and international partners, with the Bank taking the lead in providing effective and strong leadership to coordinate Project implementation among many actors. Aide Memoires (when written, see below) and Implementation Status Reports (ISRs) identified key implementation issues and ways to resolve them. The respective roles by the Bank and WHO, spelled out clearly in the PAD, worked well. The Bank's close relationship with WHO and UNAIDS ensured their technical contribution to the review of the 52 documents (standards, norms and protocols) produced under the Project, which have provided a comprehensive legal framework for the implementation of HIV/AIDS strategies and programs. Positive points of supervision can be summarized as follow: · Missions were effective in reviewing all aspects of project-specific progress. · Team composition was good, comprising HQ, Moscow-based staff, and consultants. · The team promoted a culture of regular contacts with counterparts to help resolve issues and/or address client requests and needs during and between missions, for which the role of the Moscow-based staff was critical. · Missions coordinated well with partners to ensure complementarity rather than duplication of efforts. In fact, most of the missions included the relevant development partners. · The Bank team developed strong relations with Federal TB institutes and AIDS centers staff, which extended to regions. · Supervision promoted strong relations between the civilian and prison sectors. · Aide Memoires systematically reported on indicators. · The Bank team revised the performance monitoring indicators during the MTR jointly with partners and counterparts. · Financial management guidance was systematic and timely. · Procurement management guidance was timely and helped identify and resolve issues. In terms of the less positive aspects: · Aide Memoires were not prepared during 2008. Although, management letters sent after mission identified the main issues and provided recommendations, in the view of the ICR team these letters did not have the same impact as Aide Memoires, which are prepared and 25 discussed with the authorities prior to missions' departures. Distribution of the detailed Aide Memoires among federal and regional institutions could have further contributed to a stronger ownership of the Project and further capacity building within the health sector, especially at the regional level. · While recognizing that the Project encompassed 85 regions in the vast territory of the Russian Federation and not all regions could be visited during Project implementation, the ICR mission holds the view that based on discussions in the field, supervision missions could have attempted more systematically to visit regions. (c) Justification of Rating for Overall Bank Performance The Bank provided strong technical and financial support to help GOR to deal with the epidemics. Although the financial contribution by the Bank was small compared to other resources available domestically and internationally, the Project played an important catalyst role in helping ensure a comprehensive approach to the implementation of TB and HIV/AIDS strategies. The Bank's overall performance is rated as Satisfactory because of its: (i) focus on evidence-based strategies and results-base activities at the federal, regional, and municipal levels in the civilian and prison sectors; (ii) efforts to ensure full involvement of federal, regional, municipal, and penitentiary authorities, promoting dialogue among them; (iii) successful efforts to engage the Federal TB institutes and the AIDS centers from the outset of project implementation by signing contracts with these institutions for carrying out needs assessments; (iv) successful efforts to help coordinate international assistance interventions; and (v) the successful role in leading and facilitating policy dialogue. 5.2 Borrower Performance: Satisfactory (a) Government Performance: Satisfactory In the past several years, GOR has increased substantially its resources to combat TB and HIV/AIDS, demonstrating political will and commitment for confronting the diseases and the myriad of issues that surround them. Although project regional agreements provided for 50% contribution of project resources from regional budgets, regional governments contributed significantly more through provision of resources for civil works to prepare premises for new equipment, and for the purchase of medical supplies and drugs. GOR complied with all legal covenants and agreements. MOF systematically increased budget allocations for TB HIV/AIDS strategies and, despite financial constraints, it has not reduced budgets for the twin epidemics or for the health sector more broadly. (b) Implementing Agency or Agencies Performance: Satisfactory The performance overall of MOHSD in establishing a policy framework and guiding the implementation of strategies that include the civil and prison populations has been mixed. As noted earlier, following the federal administrative reorganization, responsibility for Project implementation was transferred from the former MOH to the new MOHSD and middle management staff in the new ministry were assigned responsibility for overseeing project implementation. These staff took their role seriously and provided oversight and guidance during the early stages of project implementation. However, later changes in the ministry affected both high and middle-management 26 levels leading to a loss of institutional memory, which in turn reduced the ability of the MOHSD to ensure systematic oversight of Project implementation. The performance of the federal TB institutes and federal AIDS centers has been strong. They have played a crucial role in guiding implementation at the regional and municipal levels. Their systematic efforts are acknowledged by the regions. Equally, the performance of regional and municipal health facilities have in general been strong, judging by their commitment, enthusiasm and energy to implement project activities. The performance of the MOJ and, more specifically of the Correction System, has been exemplary. From the outset, MOJ's involvement in project design was strong and its commitment during implementation has been effective. The general good performance of the Project reflects the strong coordination among all related institutions assisted by strong efforts from the PIU. The effective role played by the PIU was crucial to ensure effective implementation of a complex project dealing with 85 regions. Overall coordination, as well as procurement and financial management, worked well. The PIU had to deal with challenges from the outset to coordinate the amendment of the Loan Agreement and Project regional agreements, as well as changes in leadership in MOHSD and the difficulties inherent in translating result-based management practices into effective interventions in the field. The PIU was successful in building and maintaining productive relationships at all levels of the health system ­ federal, regional and municipal. It worked effectively to help achieve the PDOs and in facilitating the full participation of both the public and prison sectors in the TB HIV/AIDS responses. (c) Justification of Rating for Overall Borrower Performance: Satisfactory The key reasons include: (i) overall commitment to the goals of the Project at the federal/regional and municipal levels, (ii) adequate allocation of budgets, and (iii) compliance with fiduciary requirements, despite the complicated federal/regional/municipal environment. 6. Lessons Learned Lessons learned can be categorized into the following major headings: Government ownership and capacity development · Efforts require evidence-based advocacy. Efforts under the Project from the design to the implementation stages were crucial to changing practices regarding the control of TB and HIV/AIDS. Active involvement of different stakeholders and the mobilization of support from "national champions" to implement Project activities and ensure their long-term sustainability were crucial. · Comprehensive coverage plays a critical role in TB and HIV programs. Working not only in the civilian sector but also in the penitentiary sector has been critical to reducing the epidemics. It was recognized by the authorities during the ICR mission that it was the Bank that encouraged the two sectors to work together from the design stage. · Comprehensive needs assessments from the outset of Project implementation played an important role to ensure effective utilization of resources. The assessments conducted under the project should be considered best practice and served well to guide procurement actions. 27 Coordination · Understanding the roles and reaching agreements with technical agencies and development partners can optimize benefits of programs. WHO's role was part of the institutional arrangement of the Project from the outset for the provision of technical assistance for needs assessments, monitoring, and impact evaluation. UNAIDS role has been critical to raise the importance of both TB and HIV/AIDS to the highest levels of Government. Also the investments funded by the Global Fund have complemented those under the Project, thereby reducing duplication of efforts and investments. Implementation Guidance by the Bank · The efforts displayed by the Bank to influence the Russian TB HIV/AIDS programs in ways that would make it more effective, efficient and in line with emerging international experience paid off. Through policy dialogue, analytical work, and Project preparation activities, the Bank preparation work was effective to improve the efficiency and technical quality of the response by raising high-level government commitment for TB and HIV/AIDS. · The catalytic role played by the Bank was crucial and widely acknowledged by Government and international partners. The Bank's greatest contribution was not only in helping to reduce the number of cases of TB or HIV/AIDs; the Bank played a crucial role in putting these issues on the Government's agenda, moving the country toward internationally accepted standards to address the epidemics, and playing an important coordinator role with other international partners. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies All the federal and regional institutions visited by the ICR mission commented positively about the role the Project has played in the implementation of strategies to reduce the twin epidemics. They commented particularly on the strong role the Project played in: (i) getting national and international experts talking to each other in the "same language;" (ii) providing opportunities for Russian scientists and health staff to become familiar with international best practice, through provision of in-country technical assistance and external study tours; and (iii) raising awareness at high levels of the federal and regional governments of the severity of the epidemics and the need for concerted efforts to improve the situation, including the provision of increased budgets. A draft ICR was circulated broadly among government and non-government stakeholders. Their inputs were introduced in the final version of this ICR. (b) Cofinanciers The Project was cofinanced with WHO with funding from USAID. Both WHO and USAID highly complemented the efforts of the Bank team to bring all government and international partners together to support the implementation of common strategies. In the view of the ICR team, the role 28 that WHO played in the systematic monitoring and impact evaluation of the TB component has been crucial to building at the federal and regional levels a culture of understanding of the merits of a strong surveillance system. It was particularly effective also that WHO technical support was independent from Project funding and, as such, was more effective in the eyes of both the federal and regional authorities. The ICR acknowledges particularly the efforts by the Moscow-based WHO staff, who reviewed the draft ICR thoroughly and provided clarifications and updated data to strengthen the ICR. (c) Other partners and stakeholders The Global Fund and UNAIDS praised the Bank's efforts to promote a comprehensive approach and utilization of technical expertise of all partners. In the view of the ICR team, the financial and technical efforts by all international partners were executed in an effective and coordinated manner. WHO and UNAIDS, for example, participated in the review of technical guidelines prepared by local scientists for TB and HIV/AIDS to ensure international best practice in these fields. International partners participated regularly on supervision missions. 29 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million equivalent) Appraisal Estimate Actual/Latest Percentage of Components (USD million) Estimate (USD Appraisal million)i Control of Tuberculosis 205.88 195.17 95 Control of HIV/IDS 65.32 46.26 71 Project Management, Monitoring and 3.56 3.41 96 Evaluation Total Baseline Cost 274.76 244.84 89 Physical 6.06 Contingencies Price Contingencies 5.36 Total Project Cost 286.18 244.84 86 Amounts financed by the regions have been estimated on the basis of discussions during field visits of the ICR team as well as Project Regional Agreements, which specified contributions by the regions of at least 50% of the loan for civil works to prepare premises for equipment installation, supply of consumables and reagents and support/remuneration for regional project coordinators. Amounts for taxes and duties for equipment under the project were considerably less than planned as the project benefitted from tax exemptions under several packages. US$28.29 million had been allowed for taxes and duties in the PAD. (b) Financing Appraisal Estimate Actual Estimate Percentage of Source of Funds (USD million) (USD million) Appraisal Borrower 134.11 131.54 99 International Bank for 150.00 111.40 74 Reconstruction and Development World Health Organization 2.07 2.07 100 TOTAL 286.18 244.84 86 (c) Project Cost by Category (in USD Million equivalent) Appraisal Estimate Actual Percentage of Categories (USD millions) Disbursement Appraisal (USD millions)ii (1) Civil Works 6.13 10.00 163 (2) Goods 219.63 200.50 96 (3) Consulting Services 19.34 12.44 64 (4) Training and Study Tours 15.47 8.60 56 (5) Operating Costs/ O&M 25.61 13.30 52 Total Project Costs 286.18 244.84 86 See footnote above 30 Annex 2. Outputs by Component Component Planned outputs at Appraisal Actual outputs/outcomes at ICR (a) Development/revisions of legislation and (a) Development/revisions of necessary Component 1 ­ Control of regulations legislation and regulations were Tuberculosis (US$217.30) undertaken, including Order 109 and 50. (b) Next phase of work of Technical (b) Evidence-based protocols and 1.1. Policies, Strategies and Working Group (TWG) and High Level guidelines were prepared and Protocols for TB Control Working Group (HLTWG) to develop, disseminated. Training manuals and publish and disseminate evidence-based materials were developed, approved and protocols and guidelines published. Extensive training was conducted at the Federal and Regional levels. (c) Plan for restructuring and development (c) Needs assessments and development of of strategy of TB services, including needs project implementation plans were assessment of needs undertaken in 86 participating regions. (d) Public Information Campaign (d) An international tender was held to select a PR campaign. However, because similar activities were being implemented under federal programs, these activities did not proceed with project financing. 1.2 Strengthening Surveillance, (a) Needs assessment for design of TB (a) and (b) Needs assessments and work Monitoring, Quality Control and surveillance, monitoring, quality control and plans for strengthening the system of TB Quality Assurance quality assurance detection and treatment in the civilian and (b) Preparation/implementation of a work prison sector were conducted in the 86 plan at each level of the system participating regions, 5 TB and lung disease research institutes and the Medical Directorate of the Federal Prison Service. 31 (c) Costs associated with implementation of Directorate of the Federal Prison Service. the plan (training, implementation support (c) The sub-component supplied office (local and foreign TA), logistics costs for furniture and equipment, communication supervision and monitoring, equipment, equipment (546 units) and 93 vehicles for software and vehicles, publication of forms regional monitoring centers and federal TB for recording/reporting research institutes. With TA from WHO, extensive training took place on TB treatment and monitoring in federal research institutes for 788 participants and 40 seminars for 1,800 participants from regions. In 2005-06, recording and reporting forms were designed and disseminated to the regions for TB monitoring based on cohort analysis. In 2007, a development of series of recording and reporting forms for MDR-TB case management monitoring was initiated. (d) Pre and post-evaluation of the system (d) Specialists from federal TB research for surveillance, monitoring, quality control institutes made monitoring visits to regions and quality assurance financed by: (i) MOH in 2003, (ii) Federal Health Social Development Agency in 2005, and (iii) the project 2006-08. 1.3 Improvement of TB Case (a) Assessment of needs/specification of (a) Needs assessments were conducted. Detection and Diagnosis selection criteria for facilities, staff and (b) and (c) TA was provided through the equipment Federal TB research institutes to train 358 (b) Implementation support (local and specialists at 19 seminars in microscopy foreign TA) for TB detection and 288 specialists at 14 (c) Training of staff in bacteriology and seminars in culture testing for TB radiology diagnosis. At the regional level, 1,322 32 radiology diagnosis. At the regional level, 1,322 specialists were trained at 47 seminars in microscopy for TB detection. (d) Equipment and supplies for bacteriology (d) and (e) Over 38,000 units of equipment and diagnosis (MOH and MOJ facilities and were procured and supplied. Retrofitting external quality control was completed in 6 reference labs of (e) About 200 stationary X-ray equipment federal research institutes and the federal for eligible MOH and MOJ facilities prison service. 159 bacteriological labs and 2,371 clinical and diagnostic labs of PHC in the civilian and prison sectors in participating regions. (f) In 2005-06, WHO in cooperation with (f) Pre and post-evaluation of the system for the Swedish Institute of Communicable TB case detection and diagnosis Diseases conducted 3 cycles of external quality control for micro-bacteria TB first- line drug susceptibility testing. 1.4 Improvement of TB Treatment (a) Assessment of needs (a) Needs assessments were conducted. (b) Training in clinical management of TB (b) and (c) The federal TB research patients under new guidelines and protocols institutes hosted 26 seminars for 788 (c) Implementation support (local and participants. At the regional level, TB care foreign TA specialists were retrained in TB case at the municipal level during 5-day methodological seminars. PHC staff received regional workshops. A total of 80 workshops and seminars covered about 8000 specialists involved in TB control in PHC facilities. 33 (d) logistical expenses for monitoring and (d) The project supported logistical supervision expenses as required. (e) US$10 million for second-line anti-TB (e) and (f) The project supplied to the drugs for the MOH facilities regions for civilian and prison facilities (f) US$8 million reserve fund for first-line first-line drugs worth US$19.3 million. and/or second-line anti-TB drugs for MOH and MOH service delivery systems Component 2 ­ Control of a) Further updates of policies and (a) A great number of policies, guidelines HIV/AIDS (US$65.32 million) development of strategies for HIV/AIDS and strategies were developed. control (b) The AIDS Centers served as 2.1 Policies, Strategies and public (b) Establishment of a Coordination Center coordination centers Information for HIV/AIDS Control (c) Preparation of technical proposals to (c) and (d) The project supported support legislation preparation of regulations. All developed (d) Formulation of regulations to ensure guidelines and regulations were published policy environment in 5 volumes and disseminated to federal (e) Public information campaign, including and regional institutions. Training material formative research for the design and on the basis of the regulations was production of appropriate materials and developed and training is taking place with messages support from the Global Fund. (f) Implementation of the Campaign (e) (f) and (g) An international tender was (g) Pre and post-evaluation of the Campaign prepared to select a company to undertake (h) Assessment of availability of a Public Information campaign. However, commodities for prevention MOHSD decided not to proceed with this (i) Procurement of commodities as per the tender as information activities were needs assessment implemented under the federal programs (j) Early phase of technical work on a (h) Assessment was conducted and concept and baseline studies for implementation plans prepared for federal development of vaccines against HIV in the research institutes RF (working with research (i) Over 4,000 items of equipment were 34 institutions/consulting multilateral agencies procured for civilian and prison health working on vaccine development facilities. In addition, the project supplied (k) Scientific consultation for 1,684 items of office furniture and Russian/international experts on HIV equipment for regional and federal centers. vaccines. (j) Technical/scientific consultations were promoted under the project. 2.2 Surveillance and Monitoring (a) Establishment of a Coordination Center (a) The AIDS Centers served as for Treatment, diagnosis and psychosocial coordination centers support (b) Assessment of needs and development ( b) The assessment was done and a work of a work plan focusing on gaps in plan prepared. epidemiological, behavioral, and virological surveillance (c) Surveillance at each level of the system (c) A single information system for HIV (d) Training, implementation support (local monitoring was developed. Training was and foreign TA) and logistics expenses for conducted with support from the AIDS supervision/monitoring centers. (e) Equipment and software (e) The project procured 194 units of (f) Publication of forms for recording and communication equipment and 22 vehicles reporting for regional AIDS centers. (g) Pre and post-evaluation of the system (f) Forms for recording and reporting were for surveillance and monitoring prepared, however the post-evaluation of the system for surveillance and monitoring did not take place. 2.3 Laboratory Service and Blood (a) Needs assessment for laboratories in (a) and (b) Needs assessments were Safety Improvement immunology, hematology and virology conducted for laboratories and for blood (including facilities and equipment) banks. (b) Needs assessment, system design and inputs for safe blood banks 35 inputs for safe blood banks (c) Guidelines/regulation were developed (c) Development and publication of and issued under 5 volumes distributed to guidelines all regional and municipal facilities (d) Provision of blood banking equipment (d) 1,874 items of laboratory equipment and supplies were supplied to regional AIDS centers, (e) Training prison facilities and federal research institutes. In addition, equipment for complying with blood quarantine regulation was procured and supplied to 18 facilities of the blood service. (e) Training was provided to relevant staff in the use of the equipment. 2.4 Prevention and Control of STIs (a) Establishment of a Coordination Center (a) The AIDS Centers served as (b) Needs assessment in the dermato- coordination centers. venereological service (b) Needs assessment was conducted. (c) Development of evidence-based (c) Guidelines and protocols were guidelines and protocols for STI control developed and issued to federal, regional (d) Training of staff on the dermato- and municipal facilities. venereologicla services, the AIDS Centers, (d) Training materials were developed and Family Planning and Reproduction Centers, training is taking place with support from Narcologists and Infectionists the Global Fund. (e) Establishment/upgrading of 20 (e) (h) (j) and (l) 174 items of laboratory diagnostic and treatment centers equipment was supplied to regional STI (f) Assessment of the incidence and centers and 3 federal STI treatment and prevalence of STIs and HIV among IDUs diagnosis centers. (g) Implementation support (local and (i) and (m) Training materials were foreign TA) developed and training is taking place with (h) Provision of equipment and furniture assistance from the Global Fund. (i) Training on the basis of needs assessment 36 assessment (o) Post evaluation of the system for the (j) Establishment and/or upgrading of 3 control of STIs did not take place as specialized youth-friendly centers for project implementation was delayed by one children and teenagers with STIs. year because of the Administrative Reform (k) Implementation support (local and and the project implementation period was foreign TA) shorter than expected. (l) Equipment, furniture (m) Training (o) Pre and post-evaluation of the system for the control of STIs 2.5 Targeted Prevention of (a) Development of a strategy and policies (a) Strategies and policies were developed HIV/AIDS and STIs in High Risk for HIV prevention in high risk groups Groups (b) Needs assessment and development of (b) (c) and (d) Needs assessments were implementation plans conducted to evaluate the readiness of (c) Training of specialists in the regional institutions to launch HIV implementation of preventive programs prevention program for high risk groups. (d) Development and implementation of However, the delays in drafting regulations targeted HIV/AIDS and STI prevention and guidelines prevented from launching programs programs at the beginning of the project. In 2006-08, funding for these activities significantly increased funded by the federal and regional budges and international projects. Therefore, implementation of these activities under (e) Establishment and equipping of a the project became less urgent. Federal Coordination Center for training in (e) Laboratory and office equipment as HIV/AIDS and STI prevention well as furniture were provided to regional and federal institutions. 37 2.6 Prevention of Mother-to-Child (a) Needs assessment for prevention MTCT (a) Needs assessment was conducted. Transmission (MTCT) (b) Development and publication of (b) and (c) Protocols and guidelines were protocols developed with local and foreign TA. (c) Implementation support (local and (d) Training materials were development foreign TA) and training is being conducted with (d) Training support from the Global Fund. (e) Procurement of drugs from PMTCT (e) In view of the substantial increased (f) Strengthening of 2 treatment centers for funding for pharmaceutical under the HIF-infected children, including training, federal budget, the allocated funding under equipment and supply of drugs. the project was reallocated for procurement of lab equipment. (f) Laboratory equipment, including reagents and furniture were procured and supplied to the Republican Clinical Isolation Hospital in St. Petersburg. The PIU was established within the RHCF and the Director and other staff appointed. Component 3 ­ Project (a) Establishment of the PIU and Management, Monitoring and appointment of staff (Director, 2 TB Evaluation (US$3.56 million) Component Coordinators, 4 HIV/AIDS Component, 2 Procurement Officers, 3.1 Project Implementation Disbursement Officer, FM Specialist Project Accountant, Translator, and an Administrative Assistant 3.2 Training and Study Tours for (a) Training on procurement and (a) Training on procurement was RHCF Staff disbursement conducted following Bank guidelines 38 (b) Training of Component Coordinators to (b) and (c) Component Coordinators and improve knowledge and skills Director received training to improve (c) Project management training knowledge and skills (d) Coordinator of training under the project (d) The PIU coordinated well training under the project. 3.3 Project Audits (a) Annual audits by independent auditors (a) External audits were conducted by independent auditors and submitted to the Bank within six months of the end of the FY. 3.4 Monitoring and Evaluation (a) Baseline assessments and M&E (a) and (b) Base line assessment was (b) Implementation of MIS system to conducted and M&E system strengthened. support M&E (c) The PIU prepared comprehensive (c) Annual report integrating results of the annual reports, integrating results of M&E M&E activities activities. 39 Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Not applicable. 45 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Joana Godinho Team Leader EACHD J-J de St. Antoine Team Leader EACHD Olusoji Adeyi Team Leader EACHD Supervision/ICR Anne Margaret Bakilana Economist ECSH1 Alexander Balakov Procurement Specialist ECSC2 Maria E. Gracheva Senior Operations Officer SASHN Olesya Klimenko Team Assistant ECCU1 Alyona Korneva Consultant ECCU1 Galina S. Kuznetsova Sr Financial Management Spec. ECSC3 Tatyana Loginova Operations Officer ECSHD Senior Program Assistant Jennifer Manghinang ECSHD Patricio Marquez ECSHD Team Leader Flore Martinant de Preneuf Team Leader OPCRX Anahit Poghosyan Office Manager ECSHD Irina Reshetnikova Program Assistant ECCU1 Sevil Kamalovna Consultant ECSHD Salakhutdinova Nikolai Soubbotin Sr. Counsel LEGEM Marina Vasilieva Sr. Communications Officer ECCU1 Maria D. Zhorova E T Temporary ECCU1 Asel Sargaldakova Health Specialist ECSH1 Implementation Specialist Betty Hanan ECSHD (Consultant) (b) Staff Time and Cost 46 Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY99 110.44 FY00 118 462.54 FY01 62 235.21 FY02 16 72.17 FY03 27 113.34 FY04 0.14 Total: 223 993.84 Supervision/ICR FY02 0.01 FY03 4 13.87 FY04 24 95.27 FY05 27 100.08 FY06 41 85.13 FY07 18 86.63 FY08 33 88.54 FY09 28 94.15 Total: 175 563.68 47 Annex 5. Beneficiary Survey Results Not applicable. 48 Annex 6. Stakeholder Workshop Report and Results Not applicable. 49 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR A. Introduction In the early 1990s, the TB and HIV/AIDS epidemiological situation began worsening due to social and economic changes, lowered living standards, conflicts within the Russian Federation and in some CIS countries, large-scale migration, lowered manageability of the health and social sectors, inadequate financing from the federal and regional budgets, and inadequate efforts to prevent and detect these diseases through PHC facilities. By 2000, there was more than a two-fold increase in TB incidence, and mortality from TB increased 3 times, compared with 1990. The lowered treatment effectiveness of the recent years resulted into an increase in the number of TB chronic patients, creating a permanent `breeding ground' for the infection. At that moment, the share of pulmonary TB cases confirmed by bacterioscopy did not exceed 25%, and the share of such cases confirmed by culture testing was no more than 41% due to suboptimal effectiveness of laboratory diagnosis, which led to poor detection of smear-positive TB cases. Being a social disease, TB affected the most socially and economically marginalized populations in Russia. With a view to stabilizing the TB situation, Federal Law # 77-FZ -- On Preventing the TB Spread in the Russian Federation was adopted in June 2001, and in December 2001, the Russian Government issued its Resolution to implement this law. In November 2001, the Government approved a Federal Targeted Program entitled Social Disease Prevention and Control (2002-2006), which included such sub-programs as: Urgent TB Control Measures in Russia and Urgent Measures to Prevent the Spread of the Disease Caused by the Human Immunodeficiency Virus in the Russian Federation. Later, a new Federal Targeted Socially Significant Disease Prevention and Control Program was adopted for 2007 2011. MOH developed and issued Order No. 109 of March 21, 2003, on Improving TB Control in the Russian Federation, and Order No. 50 of February 13, 2004, on Introducing Recording and Reporting Documentation for TB Monitoring to define treatment standards, methods of TB case detection and arrangements for collection and reporting of TB information. On October 2, 2006, the MOHSD signed its Order 690 on Approving the Recording Documentation on TB Detection by Bacterioscopy. In accordance with statistical reports, from 1999 to 2005, the number of officially registered HIV cases in Russia increased roughly 16 times. In 2005, 35,526 new HIV cases were recorded, which is by 1,200 cases (4%) more than in 2004 (34,306). Among all people living with HIV/AIDS, the share of those with HIV infection detected at the age of 15-30 was 79%, while the share of people with HIV infection detected at the age of 18-25 was 51%. In 1989, to improve AIDS control, MOH issued an order on Establishing an AIDS Prevention Service in the USSR. In 1995 a law was adopted -- Federal Law on Preventing the Spread of the Disease Caused by the Human Immunodeficiency Virus (HIV Infection. It incorporated key international principles of combating the epidemic and provided for protection of HIV-positive people. Those and later adopted laws and regulations of the 50 Russian Federation provided a framework for the establishment of over 100 territorial HIV/AIDS prevention and control centers and over 100 HIV diagnostic laboratories. In addition, during that time, more than 100 non-governmental and international organizations engaged in the HIV epidemic response were established and continue to operate. An analysis showed that the federal targeted program failed to provide sufficient funding to meet the regional needs in modern diagnostic equipment. The program earmarked extremely scarce funds for activities to raise public awareness of TB and AIDS prevention methods, to train health workers in up-to-date highly effective methods of prevention, diagnosis and treatment of these diseases, and to establish a system of monitoring the volumes and quality of TB and AIDS control activities. To make the response to the TB and HIV epidemics more effective, the Bank- financed TB and AIDS Control Project (project) was developed with due regard to the design/content of the federal targeted program and included activities which did not receive sufficient funding from the federal budget. The National Priority Health Project (NPHP) was launched during the implementation of the project in the context of the federal targeted social disease prevention and control program (2002-2006), the National Priority Health Project and the new federal targeted socially significant disease prevention and control program (2007- 2011). B. Financial implementation A major part of the goods and services procured under the project was funded out of the IBRD loan. The Russian Federation provided counterpart funding from the federal budget in an amount of about US$11.5 million, meant primarily to pay taxes and duties. Some activities related to the project implementation were financed under the Federal Targeted Program and from other sources of the federal budget. Such activities included mandatory preventive epidemic control measures in infection outbreak areas, provision of health facilities of the MOHSD and the Federal Prison Service with modern pharmaceuticals, consumables and computers. The participating regions caused local current funding to be made available to: prepare premises for equipment installation, supply consumables and reagents for their health facilities, and support/remunerate the regional project coordinators. WHO implemented some project activities related to review of new protocols and guidelines, monitoring the introduction of new TB control practices, developing training and learning materials and training, with these activities funded from other development partners funding. The regions were provided with equipment and services free of charge, as the Loan is to be repaid from the federal budget. C. Regional and federal project participants In accordance with the Loan Agreement, participating regions were selected by the MOH and MOJ as the executive authorities responsible for project implementation; the Russian Health Care Foundation provided operational and technical support in selecting the regions. MOH disseminated to the Governors of all the Russian regions its letter # 51 2510/11672-03-24 of October 20, 2003 referring to selection of regions to participate in the TB and AIDS Control Project. An attachment to this letter contained a sample letter of application from the administration/government of a Russian region and the form for submission of the information to be prepared by regional health authorities. These forms listed the key conditions for regional participation in the project and the requirements to submission of the project participation applications. A mandatory condition for acceptance of a regional application was provision of a written guarantee of the regional administration/government to the effect that it would agree to meet the following key conditions for participation in the project: 1) To introduce full-time positions of the Regional Project Coordinators and appoint Regional Project Coordinators for: (a) the TB Component; (b) the AIDS Component. 2) To sign a Project Implementation Agreement with the Russian Health Care Foundation. 3) To ensure coordination among regional health, prison, education and law enforcement authorities. 4) To cause the equipment and other goods procured under the project to: (a) be accepted, unloaded and stored; (b) have prepared (renovated) premises for installation; (c) be recorded in the balance sheet, undergo annual inventories with reporting of their results to be submitted; (d) be supplied with reagents and consumables in an interrupted manner; (e) be covered with operational maintenance services. 5) To provide logistic support and premises for personnel training. 6) To provide access to information and logistic support for Russian specialists and international experts working under the project. 7) To provide support in implementing public awareness and preventive activities under the project, including placement of `social marketing' (free of charge or at lower rates) in regional mass media. Regional administrations/governments were also to assure that project implementation in their regions would not lead to cuts of the funding for TB and AIDS control from the regional and municipal budgets. Submitted applications were reviewed and as the result, 86 regions were selected to participate in the TB Component and 83 in the AIDS Component7. MOJ prepared the list of participating prison facilities in the regions. On March 25, 2004, the Interagency Working Group for the Project Implementation approved the results of selecting the regions. The following regions did not submit their applications for participating in: Both components: 1. Komi-Perm Autonomous Okrug 2. Khanty-Mansiysk Autonomous Okrug 3. Chukotka Komi-Perm Autonomous In the HIV/AIDS Component: 1. Kostroma Oblast 2. Moscow City 52 3. Evenk Autonomous Okrug To assess the regional needs and develop regional project implementation plans, agreements were signed with 12 Federal research institutes (with 5 research institutes for the TB Component and with 7 ones for the HIV/AIDS Component). Regional needs assessments for the prison sector were undertaken by the Medical Directorate of the Federal Prison Service. Their results were agreed with representatives from the participating regions, endorsed by the IBRD and approved by the MOHSD and the Federal Prison Service in July 2005. The results of the needs assessments provided a basis for drafting Project Participation Agreements with the regions, which were approved by the MOHSD. In August 2005, the Russian Health Care Foundation signed the Agreements and sent them to all the 86 participating regions. By mid-2006, 85 regions countersigned their Agreements, and one region (the City of St-Petersburg) sent a written notification of its refusal to countersign the respective Project Participation Agreement. Therefore, as of the project start date, 85 out of the 88 Russian regions participated in the TB Component and 82 regions participated in the HIV/AIDS Component. The Project Participation Agreement with the Federal Prison Service provided for participation of over 600 territorial prison health facilities from 72 Russian regions. The following federal institutions were included in the project to act as centers of monitoring, quality control, resource support and training: Under the TB Component: 1. Central TB Research Institute under the Russian Academy of Medical Sciences 2. TB and Lung Disease Research Institute named after Sechenov 3. St-Petersburg TB and Lung Disease Research Institute 4. Ural TB and Lung Disease Research Institute 5. Novosibirsk TB Research Institute Under the HIV/AIDS Component: 1. Central Research Institute of Epidemiology - Russian Research and Resource Centre for AIDS Prevention and Control (further on referred to as `the Federal AIDS Centre) 2. St-Petersburg Research Institute of Epidemiology and Microbiology named after Paster 3. Rostov Research Institute of Microbiology and Parasitology 4. Nizhniy Novgorod Research Institute of Epidemiology and Microbiology 5. Yekaterinburg Research Institute if Viral Infections 6. Omsk Research Institute of Feral Nidal Infections 7. Khabarovsk Research Institute of Epidemiology and Microbiology 8. Centre of Consultation and Diagnostic Services for HIV-Positive Children Treatment under the Regional Clinical Hospital, Ust-Izhora 9. Central Skin and Venereal Disease Research Institute 10. Nizhniy Novgorod Skin and Venereal Disease Research Institute 11. Ural Research Institute of Skin and Venereal Diseases and Immunopathology 12. Moscow State Medical and Dentistry University 13. Research Institute of Virology named after D.I. Ivanovsky 53 14. National Research Centre of Immunology under the Federal Medical and Biological Agency 15. Institute of Haematology under the Russian Academy of Medical Sciences Project Participation Agreements were signed with all the federal project participants. Under the TB Component, needs of federal participants were assessed under a gratuitous agreement with WHO, and under the HIV/AIDS Component, their needs were assessed under contracts for consultant services between the Russian Health Care Foundation and international and Russian experts. E. Project Evaluation: Achievement of project objectives Control of Tuberculosis MOH adopted Order # 109 of March 21, 2003, Order # 50 of February 13, 2004. These regulations provided a basis for implementing the TB control strategy revised in accordance with WHO's recommendations. The project complemented the national projects designed for country-wide implementation of the revised strategy. The project implemented a program of personnel retraining, built capacity to increase the coverage with MDR-TB treatment, strengthened the laboratory service, including laboratories performing TB drug susceptibility tests, and included external quality assessment of drug susceptibility tests (testing of control samples) and a study to assess the prevalence of drug resistance. Project activities laid the grounds for the Global Fund's 4th Round Project. Both independent monitoring of the project in participating regions and national and health sector statistics revealed improvements of the project indicators. In 2005, five WHO coordinating offices were opened under federal TB research institutes and helped the institutes to intensify their supervision and monitoring activities in their assigned zones. In 2006-2008, the five federal research institutes delivered 59 training seminars in core research institutes; as a result, 1,434 specialists were trained in provision of TB care at the municipal level; TB detection by microscopy; and TB detection by culture testing. In 2008, regional- level training of specialists covered such aspects as: the role of PHC physicians in TB control; TB care provision at the municipal level; and TB detection by microscopy. This output contributed to capacity building in the health sector; the total number of people trained was 9,315. About 43,000 units of laboratory equipment were supplied to microbiological laboratories of the federal TB research institutes, regional bacteriological laboratories of TB facilities and clinical and diagnostic laboratories of PHC facilities in the civilian and prison sector in participating regions and other departments. Such activities resulted into substantial strengthening of the laboratory network providing TB diagnosis services, including drug susceptibility tests. The project-supplied bio-safety cabinets helped to enhance infection control in laboratories of different levels. The project supplied civilian and prison health services with anti-TB first-line drugs worth of US$ 19.3 million, which ensured uninterrupted treatment. In addition, the project procured 191 fluorography machines. Independent monitoring of the project rested with WHO TB Control Program in the Russian Federation assisted by Russian experts and included independent supervision over 54 the quantities and eligible use of equipment and other goods supplied under the project and the achievement of the targets for the project indicators. In the period of May 01, 2007 - October 06, 2008, 67 monitoring visits were arranged and made. A review of the reports on the visits yielded the following conclusions: targets were achieved for 16 out the 23 project indicators; 2 out of the 8 indicators of the epidemiological situation and TB detection, diagnosis and treatment at the national level were achieved, and for 1 of these indicators, data are not as yet available. Forty six out of the 63 regions covered with the WHO independent monitoring (73%) demonstrated good progress towards the targets of the project indicators, and 4 regions achieved the targets for all the indicators. In spite of good progress towards the targets of key project indicators, there are still some hindrances for improving the effectiveness of TB care, including: low compliance with the TB diagnosis and treatment standards, a slow implementation pace of Order # 690, insufficient administrative measures of infection control, sub-optimal skills of PHC personnel for TB detection, inadequate coverage of TB patients with observed outpatient treatment, lack of physicians' knowledge of and experience in preparing the TB documentation as required in accordance with Order # 50. To a great degree, it is accounted for by the delay in the implementation of key project activities, including monitoring and supervision, training and supply of equipment for longer than 1.5 years and their completion by 2-2.5 years later than set forth in the original schedule. A.1 Improved policies, strategies and protocols for TB prevention, diagnosis and treatment. MOHSD, the Federal Prison Service and other stakeholder ministries and agencies established cooperation with international partners under the High-Level Working Group for TB Control (HLWG) and the International Coordinating Committee set up to work out proposals on policy and strategic frameworks for effective prevention of the TB spread in the Russian Federation under a constructive dialogue between Russian and international TB control experts. Under the HLWG, thematic working groups were put together (TWGs) consisting of Russian and international experts in such areas of TB control in Russia as: (i) epidemiological surveillance; (ii) laboratory and diagnostic support and quality control; (iii) diagnosis, treatment and drug resistance; (iv) health policy and regulatory framework in the Russian Federation ; (v) TB in prisons; (vi) drug supply (procurement and supplies); (vii) TB in children; (viii) TB in marginalized populations; (viii) training and public health education, etc. Such arrangements improved the cooperation at the high level as well as planning, coordination and supervision of TB control activities under the TB Component. As of the project start, the Russian Federation had substantial capacity to implement TB control interventions in line with key principles of the Stop TB Global Plan for 2001- 2005. The adopted course of action was aimed at making undertaken interventions more targeted, effective and comprehensive. By the beginning of the project implementation, Russian and international experts led by the HLWG managed to reach agreements on many principal positions through developing and agreeing strategic recommendations on: - TB epidemiology and monitoring; - TB prevention, diagnosis and treatment in HIV-infected cases; and 55 - laboratory service. Since the end of 2004 till May 2005, activities were implemented to assess needs and to develop regional project implementation plans for the civilian sector in 86 regions selected to participate in the project. The work was done by specialists from the following five federal TB research institutes: the TB and Lung Disease Research Institute under the Sechenov Medical Academy, Central TB Research Institute of the Russian Academy of Medical Sciences, St-Petersburg TB and Lung Disease Research Institute, Ural TB and Lung Disease Research Institute, and Novosibirsk TB Research Institute. Activities to assess needs and develop project implementation plans for penitentiary facilities were implemented by the Medical Directorate of the Federal Prison Service. This work was supported by the WHO TB Control Program in the Russian Federation; the support included: training seminars for Russian experts, assistance in preparing regional and consolidated reports. The results were approved by the MOHSD, the Federal Prison Service, and endorsed by the IBRD. Reports from all 86 regions were placed in the website of the World Bank. In April-September 2005, to help attain project objectives identified for the federal TB research institutes, needs assessment missions worked in all the TB research institutes; the missions consisted of WHO experts who provided support in developing the toolkit for needs assessment at the federal level (questionnaires for experts and officials from the TB control institutes and the Federal Prison Service, a list of standard equipment for federal laboratories, etc.). The results of the assessment missions underlay the development of a number of key documents assessing the available resources at the federal and regional levels and setting forth guidelines for further improvement of the regulatory and organizational framework of TB care. In particular, a draft regulation on supervision was prepared to guide operation of federal TB research institutes in their capacity of resource centers for TB control in their assigned zones; that regulation was submitted to the MOHSD for review and approval. In 2005, five federal TB research institutes opened and equipped coordinating offices to improve cooperation and coordination among partners and project implementation. Orders of the MOH ## 109 and 50 helped to mainstream the WHO guidelines for TB Control into the national system of TB care. In 2005, the revised TB control strategy covered 60% of the Russian population; and by the end of 2007, its coverage reached 100%. Thus, the target for a key indicator of the Stop TB Global Plan was achieved. Good arrangements for epidemiological surveillance of drug resistance are of great importance and require a network of high quality bacteriological laboratories and a national system of quality assurance. High priority was attached to MDR-TB control at the HLWG meeting on July 13, 2006. The Thematic Working Group for Diagnosis, Treatment and Drug Resistant Tuberculosis under the HLWG continues to prepare national guidelines on multi-drug resistant tuberculosis control in line with the recently published WHO Guidelines (May 2006). A.2 Strengthened Surveillance, Monitoring, Quality Control and Quality Assurance. This sub-component strengthened and improve surveillance, monitoring, quality control and quality assurance relying on the exiting health institutions (federal research institutes) and regional centers (regional TB Dispensaries) in all participating regions, with 56 vesting them with the functions of the federal system of monitoring and quality control of TB response. A.3 Improved TB Detection and Diagnosis. The sub-component upgraded infrastructure of reference laboratories in federal research institutes, bacteriological laboratories and clinical and diagnostic laboratories of the civilian primary health care network and prison health facilities in participating regions. All in all, about 3,000 sets of equipment were supplied with reagents and consumables to make the equipment operational at the initial stage. These facilities were also supplied with 191 small-dose digital stationary fluorography units (50% for the civilian health sector and, 50% for the prison health sector). This subcomponent also included training activities for laboratory physicians. A.4 Improved TB treatment. This sub-component included training of specialists in up-to-date methods of TB treatment (in accordance with the protocols approved by the MOH), and procurement of second-line TB drugs for civilian and prison health facilities. WHO contribution. WHO's involvement in the project to render technical assistance and conduct monitoring was provided for in the Loan Agreement. The WHO is to provide assistance at its own expense under the following project activities: · endorsement of the protocols developed by Russian specialists for TB diagnosis, treatment, epidemiologic surveillance, monitoring and laboratory quality control; · development of agreed recording and reporting forms in collaboration with Russian experts; · development of training guidelines and materials and regulations for training of specialists in implementing the agreed protocols for TB detection, diagnosis and treatment; · development of methods for needs assessments and project implementation planning as well as methods for project monitoring in Russian regions; · provision of technical assistance to federal-level institutions in assessing the needs and developing project implementation plans at the regional and federal levels; · training and certification of trainers at the federal and federal district levels to train specialists in implementing the new protocols; · independent monitoring of the scope and quality of TB control activities in the participating regions. Below are presented the findings from the WHO's independent monitoring, review of the reports from the federal TB research institutes and recommendations. Findings from WHO's independent monitoring and the review of the project reports of the federal TB research institutes used for project evaluation and recommendations. The technical assistance for the project was provided under Agreement # 4687/ .1.1.5 between the WHO TB Control Program in Russia and the Russian Health Care Foundation. Since May 1, 2007, 67 independent monitoring visits were made with their results recorded in respective reports: as of the date of this report preparation, 67 monitoring reports were submitted and reviewed. 57 In 2007, for purposes of independent monitoring, specialists from the Russia Office of WHO designed a methodology to monitor and evaluate the project based on approved indicators and including a standard plan of 5-day monitoring, an expert's work notebook, and, a standard report template. The methodology corresponded to the guidelines for visits by the TB research institutes; it was tested in some regions and then used for independent monitoring. Russian independent experts from federal TB research institutes and regional TB control services were involved in the independent monitoring, with experts from some federal research institutes visiting zones assigned to other research institutes to ensure independent nature of such visits. Two training seminars were held: on March 27-29, 2007 in Suzdal (Vladimir Oblast) and on December 2-7, 2007 in Oryol, which permitted to train 48 specialists from 4 TB research institutes, 2 TB Chairs, the Federal Prison Service and 9 lead TB facilities. In March 2008, 10 specialists completed a repeated one-day training in the WHO's Russia Office. Overall for the Russian Federation, targets were achieved for 16 out the 23 project indicators, 2 out of the 8 indicators of the TB epidemiologic situation and effectiveness of measures to detect, diagnose and treat TB, and for one indicator, data will become available in 2009. Forty eight out of the 63 regions (72%) covered with the evaluation demonstrated good progress towards the targets of the project indicators, including: · 4 regions which achieved the targets for all the indicators (the Oryol, Vladimir, and Pskov Oblasts, and the Chuvash Republic). · 5 regions were close to achieving the targets for all the indicators: (the Arkhangelsk, Kaluga, Novgorod Oblasts, and the Republics of Mariy El and Khakasia). · 37 regions failed to achieve 3-4 indicators or demonstrated good progress towards the targets (the Republic of Karelia, Kaliningrad, Murmansk, Penza, and Tambov Oblasts, Krasnodar Kray, Republic of Dagestan, Kemerovo Oblast, etc.). · Seventeen regions (28%) had poor performance in terms of achieving the targets for project performance indicators and/or no progress towards them and require additional interventions. Some regions which were not among 63 regions covered with the WHO evaluation in 2007-2008 also achieved the targets for all the indicators, including Ivanovo and Belgorod Oblasts. In most regions, key hurdles in achieving the targets were: · Delay with the start-up of key project activities, including monitoring and supervision, training and supply of equipment by more than 1.5 years and their completion by 2-2.5 years later than scheduled; · Poor capacity of PHC and TB care medical workers at the rayon level; · Non-compliance with standards for TB detection, diagnosis and treatment; · Lack of control over TB drug administration, especially at the outpatient stage of treatment; · Lack of administrative tools for infection control; · Sub-optimal compliance with Orders of the MOHSD ## 109, 50 and 690. B. Control of HIV/AIDS 58 Russia has been fighting HIV for more than 20 years, but as of the project start, the regulatory and methodological framework, established in the late 1980s and early 1990s, failed to reflect the specifics of the HIV epidemic and its related important problems which emerged in the recent decade. The regulatory and methodological framework was confined to Federal Law # 38 of March 30, 1994 On Preventing the Spread of the Disease Caused by the Human Immunodeficiency Virus in the Russian Federation (with amendments set forth in Federal Laws # 112 of 18.07.96, and # 8 of 07.01.97), statistical reporting forms for HIV infection cases and several other documents on HIV prevention and treatment in adults and children which were also developed in the early 1990-ies. The TB and AIDS Control Project's aim was to develop regulations and guidelines based on international practices and to have them approved by the MOHSD. To prepare information for the development of regulations under the TB and AIDS Control Project in an efficient manner, the MOHSD issued Order # 251 of 01.03.05 to establish a Working Group to draft regulations and guidelines on diagnosis, prevention, treatment, epidemiological and behavioral surveillance of HIV/AIDS and its associated diseases. The Working Group consisted of officials from the MOHSD, federal health agencies and services, specialists from core research institutes, international and civil society organizations and the Russian Health Care Foundation. Efforts of the Working Group resulted into a list of 52 documents to be prepared and to cover all aspects of HIV control (epidemiology, prevention, diagnosis, and treatment). The Russian Health Care Foundation signed contracts with Russian experts to develop the documents. Management arrangements were put in place to administer outputs of the work, have them agreed with international organizations and submitted to the MOHSD for approval. By the end of 2007, all 52 documents were drafted, agreed with the WHO and/or UNAIDS and approved by the MOHSD. Some of the developed and MOHSD-approved documents embrace quite new aspects of response to the HIV epidemic, e.g., Order of the MOHSD # 619 of 17.09.2007 On Measures to Provide Palliative Care to HIV Patients, Order of the MOHSD # 107 of 12.02.2007 On Approving the Procedures for Provision of Counseling Services to HIV Patients under Regular Monitoring, Order of the MOHSD # 474 of 09.07.2007 474 On Approving the Standards of Care Provision to HIV Patients, Order of the MOHSD # 475 of 09.07.2007 475 On Approving the Standards of Care Provision to HIV Patients under Specialist Health Care, Guidelines of the MOHSD # 5952-RX of 06.06.07 On Pre- and Post-Testing Counseling for HIV-Tested Individuals, Guidelines of the MOHSD # 5960-RX of 06.08.07 On Antiretroviral Treatment Adherence Stimulation, Guidelines of the MOHSD # 6834-RX of 20.12.2006 On HIV Prevention among Various Populations. Other developed documents reflect experience gained in Russia and practices related to HIV epidemiologic studies, quality control and HIV treatment. No less important are documents to govern HIV interagency cooperation, prevention programs and a number of other documents for health mangers. Therefore, the Russian Health Care Foundation signed contracts for publishing (in 5 volumes) and disseminating these documents to Russian regions to be used as national guidelines and recommendations for Russian health facilities at 59 all levels. Volume # 1 was published and disseminated by the UNAIDS under joint initiatives to implement the AIDS prevention and control component in the Russian Federation. Below is the list of the published documents: Volume # 1: Operation arrangements for the AIDS Prevention and Control System in the Russian Federation: 1) Health Care for HIV Patients (Glossary), recommended by the MOHSD in its letter # 4173- of 04.08.2006; 2) Guidelines of the MOHSD # 3212- of 20.03.2007: Arrangements for Interagency Cooperation under HIV and Associated Disease (TB, STI, Hepatitis, Drug-Use) Prevention and Treatment Programs; 3) Guidelines of the MOHSD # 5957- of 06.08.07: Procedures for Operating the AIDS and Communicable Disease Prevention and Control Centers; 4) Guidelines of the MOHSD # 6503- of 07.12.2006: On Developing Regional HIV and Associated Disease (TB, STI, Hepatitis, Drug-Use) Prevention and Treatment Programs; 5) Guidelines of the MOHSD # 5953- of 06.08.07: Needs Assessment for Pharmaceuticals to Treat HIV and their Distribution Planning; 6) Guidelines of the MOHSD # 5959- of 06.08.07: On Recording, Storing and Using Pharmaceuticals for HIV Treatment in Health Facilities. Volume # 2: HIV Epidemiologic Surveillance in the Russian Federation: 1) Guidelines of the MOHSD # 5950- of 06.08.07: Procedures for HIV Testing in the Russian Federation; 2) Guidelines of the MOHSD # 5962- of 06.08.2007: On Collecting Data on HIV and AIDS Cases; 3) Guidelines of the MOHSD # 6966- of 20.09.2007: On Behavioral Surveillance (including risk groups); 4) Guidelines of the MOHSD # 6964- of 20.09.2007: Methods for Epidemiologic Analysis of HIV and Associated Diseases (TB, STIs, and Hepatitis) 5) Guidelines of the MOHSD # 5958- of 06.08.07: Procedures for Surveillance of the Circulation of HIV Genetic Variants, including the Circulation of ARV Drug Resistant Strains; 6) Guidelines of the MOHSD # 6965- of 20.09.07: On Monitoring and Evaluation of HIV Prevention and Treatment. Volume # 3: Response to the HIV Epidemic in the Russian Federation: 1) Guidelines of the MOHSD # 5952- of 06.08.07: On HIV Pre- and Post-Testing Counseling; 2) Guidelines of the MOHSD # 5954- of 06.08.07: Procedures for Warning Partners of HIV-Infected Individuals; 3) Guidelines of the MOHSD # 6963- of 20.09.07: Support for a Response to the HIV Epidemic in Areas of HIV Outbreaks; 4) Guidelines of the MOHSD # 5955- of 06.08.07: Prevention of HIV Transmission from Mother to Child; 60 5) Guidelines of the MOHSD # 5961- of 06.08.07: HIV Prevention in the Workplace, Including Prevention among Health Workers; 6) Guidelines of the MOHSD # 7067- of 27.09.07: HIV Prevention during Blood Transfusion. Volume # 4: Provision of Health Care to HIV Patients in the Russian Federation: 1) Guidance Letter of the MOHSD # 5922- of 10.11.2006: HIV Diagnosis Rules 2) Guidance Letter of the MOHSD # 4174- of 04.08.2006: HIV Laboratory Testing (including immunity testing and viral load testing in HIV cases); 3) Guidelines of the MOHSD # 7125- of 29.12.2006: Indications for Prescribing Treatment to HIV Patients; 4) Order of the MOHSD # 612 of 17.08.2007: Standard of Health Care for HIV Patients; 5) Order of the MOHSD # 785 of 21.11.2007: Standard of Health Care for HIV Patients (under Specialist Health Care); 6) Order of the MOHSD # 107 of 12.02.2007: On Approving the Indicative Procedures for Providing Counseling to Registered and Monitored HIV Patients; 7) Guidelines of the MOHSD # 7124- of 29.12.2006: Medical Monitoring of Registered HIV Patients; 8) Guidelines of the MOHSD # 7128- of 29.12.2006: Prevention and Treatment of Secondary Diseases in Adults and Adolescents with HIV; 9) Guidelines of the MOHSD # 7126- of 20.12.2006: Prevention and Treatment of HIV-Associated Diseases (TB, STIs, hepatitis) in Adults and Adolescents with HIV; 10) Order of the MOHSD # 610 of 17.09.2007: Provision of Palliative Care to HIV Patients; 11) Guidelines of the MOHSD # 5951- of 06.08.07 On Monitoring of Drug Resistance in HIV Cases; 12) Guidelines of the MOHSD # 5956- of 06.08.07: Guidance for HIV Drug Susceptibility Testing; 13) Guidelines of the MOHSD # 5960- of 06.08.07: Arrangements for Improving Antiretroviral Therapy Adherence. Volume # 5 Arrangements for HIV Prevention among Various Populations in the Russian Federation: Guidelines of the MOHSD # 6834- of 20.12.2006): 1) On HIV Prevention among General Population. 2) HIV Prevention among Students of General Schools. 3) HIV Prevention among Injecting Drug Users. 4) HIV Prevention among Commercial Sex Workers. 5) HIV Prevention among MSMs. 6) HIV Prevention among Mobile Populations, Including Refugees, Long-Distance Drivers, Seasonal and Temporary Workers; Foreign Workers, Tourists, People in Business Trips (Secondments). 7) HIV Prevention among Patients of Health Facilities. 8) HIV Prevention among Prisoners. 61 Pursuant to MOH's Order # 312 of August 7, 2000 On Improving the Structure and Operation of AIDS Prevention and Control Institutions, the functions of HIV/AIDS prevention and control monitoring at the federal level were assumed by 7 federal core research institutes which host the Federal-Okrug AIDS Centers. However, no funding was made available for such monitoring due to the economic situation in the country at that time. Infrastructure and human resource capacity (in terms of the number of personnel) of the institutes were rather poor. In July 2004, upon agreement with MOH, the Federal Oversight Service for Consumer Rights Protection (Rospotrebnadzor) and the World Bank, the RHCF signed agreements with the above referred institutes for assessing the regional needs in HIV/AIDS diagnosis equipment and treatment quality control, and by the end of 2006, agreements with these institutes were signed for monitoring of project activities in their assigned zones of supervision until October 2008. Under those agreements, specialists from the institutes visited each region once or twice a year to monitor the situation in the health facilities participating in the project, prison facilities, Regional AIDS Centers, skin and venereal disease centers, and centers of psychological and educational support for adolescents. Data collected during the visits (including description of identified flaws in the project implementation and use of equipment procured under the project) and recommendations on mending them were forwarded to regional and local health administrations, lead regional AIDS institutions, Directorates of the Federal Prison Service for the regions and the Federal Prison Service of Russia as well as to the Russian Health Care Foundation. The institutes' reports on project performance monitoring in the regions of their assigned zone were accepted upon endorsement of such reports by Rospotrebnadzor, review and approval by the MOHSD and the World Bank. The initiated supervision of the regions by the federal institutes enabled to restore the vertical of organizational and resource support for regional and federal services, to enhance HIV control activities of regional and municipal health administrations and specialists, to increase local specialists' responsibility for volumes and quality of the work done. Due to unavailability of funding from the federal budget, upon completion of the project, supervision of the regions will cease, which will make the project achievements unsustainable. In accordance with the needs assessments undertaken by the federal institutes, the project supplied equipment for HIV/AIDS diagnosis and treatment quality control, office furniture and communications to 82 Regional AIDS Centers. Laboratory equipment was supplied to 16 regional STI centers, information centers and centers of psychological and educational support, and medical and social care, and 35 regional prison laboratories. All in all, the project supplied to the regions and federal institutions 4,198 units of equipment, including 398 units for regional prison facilities. So, a continuously functioning laboratory infrastructure was established at the regional level for HIV and STI diagnosis and their treatment quality control. It is of particular importance that this infrastructure was developed in the period when needs of HIV patients in highly active antiretroviral therapy sharply rose. Currently, increasingly more patients are at advanced stages of HIV development, which 62 calls for putting in place adequate arrangements for detection and treatment of secondary diseases and control over antiretroviral therapy. In view of the growing HIV prevalence both in risk groups and among general population in Russia, the HIV detection workload of laboratories increased. Overall, at the regional level, the project installed 38 automated systems for immune-enzyme analysis to diagnose HIV, 42 systems for immune-enzyme analysis to diagnose HIV in a semi-automated mode in Regional AIDS Centers and STI centers and 35 such devices for regional prison laboratories, 66 cytometers (to measure levels of CD 4 lymphocytes) and 37 systems for viral load measurements and quality control of antiretroviral treatment. In addition, regional centers were provided with office equipment and furniture (689 items), and communication equipment for HIV monitoring (195 items). Moreover, the project supported substantial strengthening of physical and technical infrastructure of the 7 federal research institutes, under which AIDS Centers are operated, 3 STI research institutes and 4 other federal institutions participating in the project. All in all, federal research institutes were provided with equipment of 1,756 types. Equipment supplies were based on assessment of needs of these institutions for establishing centers of HIV drug resistance monitoring, and HIV diagnosis and treatment quality control in their assigned zones. Under contracts with suppliers, the federal research institutes were supplied with 7 systems for immune-enzyme analysis to diagnose HIV in a semi-automated mode, 10 laser cytometers to assess immunity of HIV patients, 35 systems for PCR and viral load measurement. Apart from these, the project procured equipment for electrophoresis, gel documentation systems, as well as ophthalmologic, ultrasound, and physiotherapy equipment. It also procured 9 DNA sequenators to measure drug resistance of circulating HIV strains and work out a system of antiretroviral therapy adjustment. These institutions were supplied with over 600 items of office, laboratory and communication equipment. The project assessed the readiness of regional institutions to launch HIV prevention programs for risk groups (harm reduction programs) and general population. However, there were factors which prevented from launching such programs under the project. B.1 Improved Policies, Strategies and Public Information for HIV/AIDS Control. This sub-component includes needs assessment for the AIDS prevention and control system in medical and laboratory equipment, pharmaceutical products, prevention products; development of teaching and learning materials on public information with due regard to the effective legislation; preparation and implementation of social marketing and prevention campaigns and their evaluation; development and publishing of social marketing and public awareness products for HIV and STI prevention among various populations, and expanded operation of hotlines. This subcomponent was also designed to supply equipment to regional information centers and centers of educational support and medical and social care, and to train HIV/AIDS and STI prevention specialists; provide assistance in establishing a federal coordination centre for HIV monitoring, treatment, diagnosis and psychological support to HIV-infected people. Two on-going projects, financed by the World Bank and the Global Fund (with both of them implemented by the Russian Health Care Foundation) were to support the development of regulations and guidelines on HIV/AIDS/STI prevention, diagnosis, 63 treatment and epidemiologic and behavioral surveillance in line with international practices. To avoid duplications under the two projects, it was decided to finance the development of regulations and guidelines out of proceeds of the TB and AIDS Control Project. The amounts earmarked for similar services under the Global Fund Project will be spent to support the development of training programs to train trainers, teaching/learning and information materials and provision of HIV/AIDS/STI training. To have information for the development of regulations prepared under the TB and AIDS Control Project in an efficient manner, the MOHSD issued Order # 251 of 01.03.05 to establish a Working Group for drafting regulations and guidelines on diagnosis, prevention, treatment, epidemiological and behavioral surveillance of HIV/AIDS and its associated diseases. The Working Group consisted of officials from the MOHSD, federal health agencies and services, specialists from core research institutes, international and civil society organizations and the Russian Health Care Foundation. By the end of 2007, all 52 documents were drafted, agreed with the WHO and/or UNAIDS and approved by the MOHSD. All the developed documents were published (in 5 volumes) and disseminated to federal and regional institutions designated to address HIV/AIDS/STIs. The full list of the documents is presented above. Training materials were developed on the basis of the above referred 5 volume collection of regulations and guidelines and submitted to the MOHSD for approval. Due to the failure to obtain such an approval, the project failed to implement broad-based training for specialists. Pursuant to Order of the MOH # 312 of August 7, 2000 On Improving the Structure and Operation of AIDS Prevention and Control Institutions, the functions of HIV/AIDS prevention and control monitoring at the federal level were assumed by 7 federal core research institutes which host the Federal-Okrug AIDS Centers. However, no funding was made available for such monitoring due to the economic situation in the country at that time. Infrastructure and human resource capacity (in terms of the number of personnel) of the institutes were rather poor. In July 2004, upon agreement with the MOH (MOHSD), Rospotrebnadzor and the World Bank, the RHCF signed agreements with the above referred institutes for assessing the needs of regions in equipment for HIV/AIDS diagnosis and treatment quality control, and by the end of 2006, agreements with these institutes were signed for monitoring of project activities in their assigned zones of supervision until October 2008. Under those agreements, specialists from the institutes visited each region once or twice a year to monitor the situation in health facilities participating in the project, prison facilities, Regional AIDS Centers, skin and venereal disease centers, and centers of psychological and educational support for adolescents. Data collected during the visits (including description of identified flaws in the project implementation and use of equipment procured under the project) and recommendations on mending such flaws were forwarded to regional and local health administrations, lead regional AIDS institutions, Directorates of the Federal Prison Service for the regions and the Federal Prison Service of Russia as well as to the Russian Health Care Foundation. 64 The institutes' reports on project performance monitoring in the regions of their assigned zone were accepted upon endorsement of such reports by Rospotrebnadzor, review and approval by the MOHSD and the World Bank. B.2 Strengthened Surveillance and Monitoring. This sub-component includes development of a plan to improve the existing epidemiological surveillance system and support in its implementation, including drafting of protocols for monitoring and recording and reporting forms as well as evaluation of the epidemiological surveillance system; publishing of national standards of recording and reporting documentation on HIV monitoring agreed with the WHO, and establishment of a single information system for HIV monitoring and procurement of specialized equipment and software for such a system; and training of specialists in implementing the new standards. In addition, the project was to provide equipment for the Federal Coordination Centre for HIV Monitoring, Treatment, Diagnosis and Psychological Support to HIV-infected patients and regional monitoring centers. B.3 Improved Laboratory Quality and Blood Safety. This sub-component includes development and publishing of national standards and guidelines on quality control of HIV laboratory diagnosis and quarantine requirements to donor blood agreed with the WHO; project implementation support; procurement of specialized equipment and software for the information network of donor blood safety; procurement of equipment for blood transfusion stations of the federal okrug level, laboratory equipment for diagnostic centers of the regional and federal-okrug level, including prison facilities; training of laboratory specialists in operating the equipment procured under the project and implementing the new standards. B.4 Prevention and Control of STIs. This sub-component includes evaluation of the STI epidemiological surveillance and prevention system during the entire project period; technical assistance in implementation; development and publishing of national standards on epidemiological surveillance, monitoring, treatment and diagnosis of STIs; establishment and equipment of a federal STI coordination centre; equipment provision for 13 regional and 3 federal centers for prevention, treatment and diagnosis of STIs which work with high-risk groups and 3 regional specialized centers for children and adolescents with STIs; development of training materials and delivery of training for specialists from the federal STI coordination centre, skin and venereal disease dispensaries of different levels, AIDS Centers, Family Planning Centers and other health facilities. B.5 Targeted Prevention of HIV/AIDS and STIs in High Risk Groups. This sub- component includes development of a strategy and policies for HIV prevention in high risk groups in accordance with the effective legislation; needs assessment and development of implementation plans; training of specialists in implementing preventive programs in high risk groups, development and implementation of targeted HIV/AIDS and STI prevention programs for high risk groups, including around 30 harm reduction programs for drug users, establishment and equipment of a Federal Coordination Centre for Training in HIV/AIDS and STI Prevention in High Risk Groups. The project assessed the readiness of regional institutions to launch HIV prevention programs for high risk groups (harm reduction 65 programs) and general population. However, the delays in drafting regulations and guidelines prevented from launching such programs in the beginning of the project. In 2006- 2008, funding for such activities significantly increased owning to allocations from the federal and regional budgets and under other international projects. That made this problem much less relevant for this project. B.6 Prevention of Vertical (Mother-to-Child) Transmission. This sub-component includes development and publishing of national protocols and guidelines on of mother-to- child transmission prevention agreed with the WHO; drug supply for preventive treatment of 2,500 HIV-infected pregnant women, including postnatal prevention among newborns, and for HAART of 150 HIV-infected children; provision of medical equipment, consultant services, consumables and training, including those to strengthen two (2) Federal Consultation and Diagnosis Centers for HIV-infected Children. In view of the substantially increased funding to procure pharmaceuticals under the federal HIV/AIDS control programs, it was decided to reallocate funds meant for procurement of pharmaceuticals under this subcomponent and use them to procure laboratory equipment; this decision was agreed with Rospotrebnadzor, MOHSD and IBRD. F. LESSONS LEARNED TB Control Component: · Comprehensive approach to the development of activities (protocols, training, strengthening of the laboratory diagnostic services, monitoring, and treatment) was key to the effective implementation of the project. · Efforts for 100% unification and standardization of approach in the development and implementation of activities throughout the country was essential. · Realistic targets were established during project implementation which helped relevant institutions to keep an eye on results. · Cooperation with WHO was essential to ensure independent monitoring and evaluation of TB-related efforts. WHO's role in guiding and training the TB Federal institutes continues to be important to ensure sustainability of outcomes. HIV/AIDS Control Component: · Dissipation of efforts over too many areas, beneficiaries and target groups should be avoided at all cost. · Participants diversity (health facilities, non-commercial organizations, various centers) ensured comprehensiveness of approach. · Substantially improved quality of laboratory diagnostics (data from the Federal Centre of Quality Control) resulted into improved quality of treatment · The project substantially benefited from involvement and enhanced the managerial and advisory role of core research institutes · The project played a catalyzing role to ensure proactive involvement of national authorities and international organizations into TB and HIV/AIDS control 66 · The current rates of TB and HIV/AIDS call for a reinforced response both at the national and international level · National and international programs must complement one another, with their synergies to be safeguarded at the planning stage · There is a need to enhance the active role of core research institutes at the follow-up stage, and ensure their further cooperation with WHO G. Evaluation of the Bank and Borrower Performance8 Ministry of Health and Social Development MOH performance during project design was Satisfactory. Following agreement in the technical content and scope of the project, MOH moved forward in the completion of necessary documentation to ensure project approval by the Bank. During project implementation, there were early delays caused by the Government Administrative Reform in 2004, which necessitated amendment of the Loan Agreement to ensure that the new Ministry of Health and Social Development could legally become the implementing agency. During project implementation, MOHSD worked with the Russian Health Care Foundation to review and approve protocols, regulations, guidelines, needs assessments. MOH's performance is considered Satisfactory. Ministry of Justice MOJ performance during project design and implementation is considered Satisfactory. During implementation of the project, MOJ demonstrated high commitment to the objectives of the project and cooperated well with MOH and the RHCF in all related activities. Activities to assess needs and develop project implementation plans for penitentiary facilities were implemented by the Medical Directorate of the Federal Prison Service in close collaboration with Activities to assess needs and develop project implementation plans for penitentiary facilities were implemented by the Medical Directorate of the Federal Prison Service. This work was supported by the WHO TB Control Program in the Russian Federation; the support included: training seminars for Russian experts, assistance in preparing regional and consolidated reports. The results were approved by the MOHSD, the Federal Prison Service, and endorsed by the IBRD. Reports from all 86 regions were placed in the website of the World Bank. Russian Health Care Foundation The PIU in the RHCF played an important role in the design and implementation of the project. The staff of the PIU demonstrated high commitment to the objectives of the project and worked hard to coordinate activities among the many government and non government institutions working in areas of TB and HIV/AIDS control. Implementation of the project was complex because of the many institutions that participated in the 67 implementation of the project throughout the Russian Federation. Implementation encountered enormous challenges, starting with delays during the initial period after effectiveness following the Government's Administrative Reform which necessitated amendments to the Loan Agreement. The PIU coordinated well the work overall the entire implementation period and its performance both during preparation and implementation is considered Satisfactory. World Bank After initial difficulties during the preparation phase where there was disagreement regarding the TB component, the Bank moved forward to work with local counterparts and conduct analytical work and seminars for better understanding of the situation in the Russian Federation. Project preparation and negotiations took place when agreements were reached on overall project scope and detailed project description. The performance of the Bank during the initial phase can be considered Satisfactory. During implementation, the Bank team worked closely with the various agencies to identify issues and help resolve bottlenecks. There was good collaboration between the Bank team and the various levels of the TB and HIV/AIDS systems which resulted in overall good performance of the project. The Bank team promoted good collaboration also among international organizations working in the areas covered by the project. Overall, the Bank's performance is rated Satisfactory, including by regions and municipal institutions. 68 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable. 69 Annex 9. List of Supporting Documents Ministry of Finance, Ministry of Health and Social Development, Federal Correction Service, "Russian Health Care" Foundation; Russian Federation; Project Regional Agreements between the "Russian Health Care" Foundation and Regions of the Russian Federation, 2005. Ministry of Health and Social Development, Federal Correction Service, "Russian Health Care" Foundation; Russian Federation; Progress Reports, 2003 ­ 2008. Twigg, Judith and Richard Skolnik. Evaluation of the World Bank's Assistance in Fighting the AIDS Epidemic: Russia Case Study. Case Study for the OED Evaluation of the Bank's HIV/AIDS Assistance. Operations Evaluation Department, World Bank, Washington, D.C., 2004. World Bank. Committing to Results: Improving the Effectiveness of HIV/AIDS Assistance. An OED Evaluation of the World Bank's Assistance for HIV/AIDS Control, Washington, D.C., 2005. World Bank. Country Assistance Strategy for the Russian Federation - Fiscal Year 2003 ­ 2005 (extended to FY 2006), 2002. World Bank. Country Partnership Strategy for the Russian Federation ­ Fiscal Year 2007 ­ 2009, 2006. World Bank. Project Concept Note for Tuberculosis and AIDS Control Project, Russian Federation, Internal Documents. World Bank. Minutes of Project Concept Note Review Meeting for Tuberculosis and AIDS Control Project, Russian Federation, Internal Documents. World Bank. Minutes of Appraisal Review Meeting for Tuberculosis and AIDS Control Project, Russian Federation, Internal Documents. World Bank., Loan Agreement for Tuberculosis and AIDS Control Project, Russian Federation (Loan Number 4687 RU), 2003. World Bank. Amendments to Loan Agreement for Tuberculosis and AIDS Control Project, Russian Federation, 2004. World Bank. Aide Memoires and Back-to-Office Reports, Internal Documents. World Bank. Implementation Supervision Reports (ISRs), Internal Documents. 2003- 2008. 70 World Bank. Management Letters, Internal Documents. 2003-2008. World Bank. Project Appraisal Document for the Tuberculosis and AIDS Control Project, 2003. WHO Reports. Reports on WHO Provision of Technical Assistance to Implementation of the Project, Tuberculosis Component2003-2008, 71 ii 73 IBRD 27188R UNITED STAT ES OF AMER ICA The boundaries, colors, denominations Bering and any other information shown on Strait this map do not imply, on the part of The World Bank Group, any judgment RUSSIAN FED ERATION Chukchi on the legal status of any territory, or any endorsement or acceptance of REPUBLICS, KRAIS, OBLASTS, OKRUGS Sea such boundaries. NATIONAL CAPITAL B e r i n g RIVERS S e a ECONOMIC REGION BOUNDARIES OBLAST, KRAI, OR REPUBLIC BOUNDARIES UNITED 86 KINGDOM Norwegian Sea AUTONOMOUS OBLAST, OKRUG, OR REPUBLIC BOUNDARIES* INTERNATIONAL BOUNDARIES East Siberian Sea * Including republics of Adygeya, Altai, Karachaevo-Cherkess, and Khakasiya N o r t h S e a N O R WAY B a r e n t s L a p t e v Koly S e a NETHERLANDS SWEDEN 82 DENMARK S e a ma In d ig ir R. of Bothnia 85 ulf ka G GERMANY 5 R. Kara Sea FINLAND 81 Baltic Sea RUSSIAN 3 FEDERATION 78 CZECH REP. ESTONIA 11 L AT V I A 10 Le POLAND LITHUANIA na 9 2 Sea of 1 89 R. 8 SL 7 Okhotsk O VA K HU RE BELARUS 6 NG P. 21 R. Ob AR 20 4 Y Circle 17 24 Moscow 15 R 69 Arctic N 12 14 13 U O T I 16 S S 76 83 23 88 O VA ROMANIA F E D E R A 22 UKRAINE 18 52 49 I A N A m ur R. 19 LD 26 58 O M 27 50 51 48 57 68 Ob R 25 28 59 . 29 42 60 45 47 80 En JAPAN ise R. R. iR 46 44 43 An gara R. B l a c k 53 66 . a 67 73 84 lg ol 75 Vo b 54 To S e a 34 37 55 56 71 87 65 70 R. 35 Lake Sea ur 41 Am 40 64 Baikal of 38 39 Japan 63 74 72 33 TURKEY 36 77 C H I N A 32 30 61 GEORGIA K A Z A K H S T A N 79 31 AR M EN IA Caspian Aral Sea 0 100 200 300 400 500 MILES 62 SYRIAN Sea D.P.R. 0 200 400 600 800 KILOMETERS OF ARAB REP. AZERBAIJAN IRAQ ISLAMIC REP. OF IRAN UZBEKISTAN M O N G O L I A KOREA I. NORTH II. NORTHWEST III. CENTRAL IV. CENTRAL V. NORTH CAUCASUS VI. VOLGA VII. VOLGO-VYATKA VIII. URAL IX. WESTERN SIBERIA X. EASTERN SIBERIA XI. FAR EAST 1. Arkhangel 7. Novgorod 12. Bryansk 22. Tula CHERNOZYOM 30. Chechen 40. Astrakhan 48. Chuvash 53. Bashkortostan 61. Altai 70. Buryat 80. Amur 2. Nenetz 8. Pskov 13. Ivanovo 23. Vladimir 25. Belgorod 31. Daghestan 41. Kalmykia- 49. Kirov 54. Chelyabinsk 62. Gorny Altai 71. Chita 81. Kamchatka 3. Karelia 9. Leningrad 14. Kaluga 24. Yaroslavl 26. Kursk 32. Ingush Khalmg Tangch 50. Mariy El 55. Kurgan 63. Kemerovo 72. Agin Buryat 82. Koryak 4. Komi 10. St.Petersburg City 15. Kostroma 27. Lipetsk 33. Kabardino-Balkar 42. Penza 51. Mordov 56. Orenburg 64. Novosibirsk 73. Irkutsk 83. Khabarovsk 5. Murmansk 11. Kaliningrad 16. Moscow 28. Tambov 34. Krasnodar 43. Samara 52. Nizhniy Novgorod 57. Perm 65. Omsk 74. Ust-Ordyn Buryat 84. Jewish AO 6. Vologda 17. Moscow City 29. Voronezh 35. Adygeya 44. Saratov 58. Komi-Permiak 66. Tomsk 75. Krasnoyarsk 85. Magadan 18. Orel 36. North Ossetia 45. Tatarstan 59. Udmurt 67. Tymen 76. Evenk 86. Chukot 19. Ryazan 37. Rostov 46. Volgograd 60. Sverdiovsk 68. Khanty-Mansi 77. Khakas 87. Primorski 20. Smolensk 38. Stravropol 47. Ulyanovsk 69. Yamalo-Nenets 78. Taimir 88. Sakhalin 21. Tver 39. Karachaevo-Cherkess 79. Tuva 89. Sacha (Yakut) DECEMBER 1995