ICRR 14023 Report Number : ICRR14023 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/24/2014 Country : Kyrgyz Republic Project ID : P099453 Appraisal Actual Project Name : Avian Influenza US$M ): Project Costs (US$M): US$6.4M US$5.54 Control & Human Pandemic Preparedness & Response Project L/C Number : CH203 Loan /Credit (US$M): Loan/ US$M ): US$4.0 US$3.76 Sector Board : Agriculture and Rural US$M): Cofinancing (US$M ): US$2.4 US$1.78 Development Cofinanciers : PHRD, AHIF Board Approval Date : 02/09/2006 Closing Date : 12/30/2010 12/31/2011 Sector (s): General public administration sector (38%); Health (23%); Agricultural extension and research (18%); Animal production (13%); Other social services (8%) Theme (s): Rural services and infrastructure (29% - P); Other communicable diseases (29% - P); Health system performance (14% - S); Participation and civic engagement (14% - S); Social safety nets (14% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Sarina Abdysheva Stephen Hutton Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: Original objective : The original project objectives were “to minimize the threat posed to humans and to the poultry industry by Highly Pathogenic Avian Influenza (HPAI) infection and to prepare for, control, and respond to an influenza pandemic �. (Financing Agreement, Schedule 1) The objectives in the appraisal document (PAD, p. 6) were substantially identical. Revised objective : After restructuring, the revised objectives were : "to minimize the threat in the Kyrgyz Republic posed by the Highly Pathogenic Avian Influenza (HPAI) infection as well as other poultry and livestock diseases and to prepare for the control and response to animal influenza pandemic and other zoonotic or infectious disease emergencies in humans �. (Restructuring Paper, November 23, 2010) The revised objectives were largely the similar to the original objectives, except that they expanded the scope to cover zoonoses other than avian influenza, and pandemics and infectious disease emergencies other than those from avian influenza. b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 05/09/2010 c. Components: The project contained five components . During restructuring, all components, except for Component IV, were slightly revised to include other human and animal concerns : 1. Animal Health (US$2.88M at appraisal, US$2.41M actual). This included support for prevention and preparedness capacity, disease control capacity, surveillance and diagnostic capacity, and outbreak containment and control, and improving bio-security in poultry production and trade . It would include policy reform, technical assistance for veterinarians, epidemiological studies, laboratory upgrades, equipment and training for culling, and a Compensation Fund to compensate owners of poultry and other domestic animals for the loss of assets caused by mandatory culling. After the restructuring the money for the compensation fund was removed, additional civil works and equipment for the animal health laboratory were added to ensure that the laboratory would be operational, and additional technical assistance on field epidemiology and laboratory diagnosis for other zoonoses were added . 2. Human Health (US$ 2.3M at appraisal, US$ 2.61M actual). This component targeted the reduction of the impact of the virus of a pandemic influenza and other zoonoses through support for coordination and planning, improvements to the national public health surveillance system, and improving health care response capacity . It included creation of plans, technical assistance and training to set up year round surveillance system, laboratory upgrades, treatment facility upgrades, and supply of vaccine and antivirus . 3. Public Awareness (US$ 0.44M at appraisal, US$0.24M actual) was to develop and implement a communication plan to inform the public about the threat of avian Influenza and other zoonoses and steps they could take to reduce transmission and spread . It involved information and communication activities to increase the attention and commitment of government, the private sector and civil society organizations and raise awareness, knowledge and understanding among the general population about the risk and potential impact of a pandemic . It also included development and dissemination of communication materials, communication training, and awareness raising with community groups. The restructuring has broadened scope of the communication activities to address other priority zoonoses. 4. Implementation Support and Monitoring and Evaluation (US$0.38 at appraisal, US$0.34 actual) would support costs associated with project management and coordination and M&E . 5. Emergency Imports would, in case of pandemic, finance emergency imports under preparedness and response program, including pharmaceuticals / vaccines, medical / veterinary supplies / equipment, communication equipment, protective gear and clothing . Allocations for this component of the Project were incorporated into costs of other four components. The project restructuring shifted funds from this component to allow for purchase of emergency supplies in the case of outbreaks of other zoonoses . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: During restructuring in 2010, a reallocation of the funds was carried out to accommodate the expansion of scope of the project, in order to address other veterinary emergency issues while keeping the original focus on avian influenza preparedness. Funds were redirected (from categories "Eligible Imported goods and commodities ", "Compensation Fund", "Unallocated", and "Operating Costs") to increase civil works and goods to upgrade the existing vaccine storage facilities and veterinary objects under the State Veterinary Department . In addition, during the 2010 restructuring, the project added support for a compensation scheme to the brucellosis control program . The project cost at appraisal was estimated at US$ 6.4 financed by commitment from IDA (US$ 4M), a grant from the Japanese Policy and Human Resources Development trust fund (US$ 1M), a grant from the Avian and Human Influenza Facility US$ 0.10M) and from a Borrower contribution (US$0.3M). An additional US$ 1M was contributed by the ongoing IDA funded Agricultural Services Support Project . Actual disbursements were US$ 3.74M (IDA), US$ 0.72M (PHRD), US$ 1.06M (AHIF) and US$ 0.08M (Government). The Closing Date was extended from December 30, 2010 to December 31, 2011 in order to allow time for completion of activities. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Relevance of original objectives : Substantial The project was responding to a global emergency and the HPAI posed a serious risk to the country . In 2005, the AI virus had spread to several countries in Europe and Central Asia, including Russia, Kazakhstan, and Ukraine . These countries and neighboring China had registered outbreaks in their regions that were geographically close to Kyrgyzstan. In addition, there was an outbreak in Turkey (a close trade partner), where thousands of poultry have been culled to prevent infection in humans and about 20 people were hospitalized with avian influenza like symptoms. In addition, a number of poultry died within the country, but lack of diagnostic capacity did not allow confirmation of outbreak. The virus could have spread to Kyrgyzstan through migration of wild birds or through illegal imports of poultry. During the project, HPAI was still present endemically in some countries, in particular in South and South East Asia and the pandemic threat remained . Avian influenza posed a direct threat to the poultry industry, but also a threat to humans through direct infection, and through the potential to trigger a pandemic in humans . Controlling avian influenza in poultry would have global public good benefits, as it would reduce the risks posed to other countries. Veterinary and public health services had a number of weaknesses in capacity, public awareness and ability to fund compensation, which contributed to the risk . The project would provide some contribution to two overarching objectives identified in the National Poverty Reduction Strategy (NPRS) for the period of 2004 - 2007 and the Bank's Country Assistance Strategy (CAS) for the period of 2003 - 2006. The NPRS has emphasized a greater attention on public health and combating communicable diseases and on public awareness campaigns on key health issues . One of the 16 specific poverty indicators for monitoring progress towards the NPRS goals was a reduction in the mortality from infectious diseases . The CAS also addressed critical need for effective provision of health care services and for continued efforts to promote agricultural development as a key driver of growth and poverty alleviation . Relevance of revised objectives : Substantial The objectives were revised to address the risk of other zoonoses (diseases that originate in livestock or wildlife and can affect humans). The revision was motivated by a government request to integrate a number of other zoonoses into the project (such as brucellosis, anthrax, rabies, TB, echinococcosis ), as no outbreaks of avian influenza had occurred but other zoonotic diseases posed risks . The revision of the PDO to include other zoonotic diseases was substantially relevant considering the economic importance of the livestock sector . b. Relevance of Design: Substantial . The original project design followed the broad outline of the Global Program on Avian Influenza . It aimed support the development of institutional and technical capacity, and through improved communication . The awareness and information component was designed to communicate risks to the public and to reduce behaviors that might lead to disease transmission from poultry to humans or between humans . The animal health component was designed to improve the ability to identify any avian influenza outbreaks that occurred among poultry and to contain them by culling infected and potentially infected birds, thus limiting the spread of disease . The human health component was designed to increase the capacity to identify and treat humans infected by HPAI, and to build capacity to mitigate a pandemic. The activities supported under the original project design were still in the revised design, and remained relevant to the revised objectives. While maintaining the original project design, minor changed introduced during the restructuring were related to training activities, technical assistance in epidemiology, support to surveillance system and public awareness on animal disease other than AI . 4. Achievement of Objectives (Efficacy): Achievement of original objectives : 1. Minimizing the threat posed to humans and to the poultry industry by HPAI : Substantial Outputs : The Republican Central Veterinary Laboratory was renovated, equipped with diagnostic devices and upgraded to biosecurity level 2; Laboratory veterinarians were trained to detect HPAI including through field simulation exercises; Active surveillance on wild and domestic birds in strategic areas were implemented annually and two in -depth studies on wild birds migration were prepared; Contingency plans for HPAI were developed and adapted to regions; Veterinary Services were evaluated through the OIE's Performance of Veterinary Services tool and received support to develop a modern legal framework; Regulation on specimen transport was developed and allowed accurate delivery of samples to Kyrgyz laboratories in accordance with the WHO standards; National Veterinary Strategy was adopted in accordance with the OIE standards; Poultry farms introduced stringent biosecurity measures through the communications plan and training of veterinarians and staff, and construction of 27 sites to dispose of dead birds and animals . Outcomes : There were no documented cases of HPAI among poultry or humans during the project, though it cannot be determined whether any cases would have occurred in the absence of the project . The Veterinary Services were enabled to conduct reliable HPAI diagnosis, through field detection, sampling, shipping and laboratory diagnosis; The National Central Veterinary Laboratory achieved 100% success in diagnosing HPAI samples through a blind test organized by an OIE/FAO Reference laboratory; The ICR reported that the performance of the poultry and wild bird surveillance systems was improved through regular surveys and sampling for laboratory diagnosis . 2. Prepare for, control, and respond to avian influenza pandemics : Substantial Outputs : Hospitals for infectious diseases established three Intensive Care Treatment Units; Hospital staff were trained in treatment guidelines (total number is not provided in the ICR ); A virology laboratory received equipment and training and received accreditation by the WHO for virus detection; Priority target groups were vaccinated annually for seasonal influenza with varied coverage levels; An unspecified quantity of antiviral drugs were obtained and stored at central level for case of possible future outbreak and pandemic; A communication strategy was developed and implemented . The strategy aimed to improve understanding of disease transmission and prevention measures among specific audiences, including public health workers, veterinarians, poultry farmers and traders, and school children; A modular training program for state agencies / departments, mass media, and other involved stakeholders developed and implemented through workshops on HPAI surveillance, control and monitoring; Workshops on crisis communication and relations with mass media conducted with total of 750 participants; Materials on external communication were disseminated in the workshops, including : methodical guidelines, booklets, leaflets on avian influenza prevention (more than 80,000 copies distributed); 423 information broadcasts on national TV were carried out . Outcomes : 88% of respondents were aware of the AI and other zoonotic disease risks (vs low awareness at the project start). No evidence was available on changes in behavior; The ICR reports that public health program planning and coordination was enhanced by setting up the National Epidemic Committee, with clear lines or responsibilities and well equipped to manage outbreaks; The ICR reports that coordination and command mechanism at a central level were strengthened; The ICR reports that coordination between the animal and human health sectors was improved . Achievement of revised objectives : The original objectives were revised to include other animal specific diseases such as brucellosis, anthrax, rabies, TB, echinococcosis, and others . Outputs and outcomes of the original objectives were also relevant to the revised objectives. 1. Minimize the threat posed by HPAI infection as well as other poultry and livestock diseases : Substantial The main outputs were : Strategies for the control of six major diseases (brucellosis, anthrax, rabies, FMD, echinococcosis and PPR ); Additional training on diagnosing animal diseases other than avian influenza was provided to Veterinary Department laboratory and other; The National Animal Information System Database was further developed to include brucellosis control; The Veterinary Vaccine Warehouse was renovated with additional cold storage capacity; Compensation mechanisms were tested in pilot regions for brucellosis control by compensating owners of livestock infected with brucellosis in 8 districts; The Central Virology Laboratory was refurbished and equipped to increase capacity to prevent zoonozes other than avian influenza; A brucellosis control program in sheep was piloted; Nine million sheep were vaccinated . Outcomes : Initial steps of brucellosis control implemented successfully in the pilot regions, with the abortions in sheep from brucellosis reduced by 75% in the pilot areas; The ICR reports that laboratory diagnostic capacity was improved for rabies and brucellosis diagnosis; The National Central Veterinary Laboratory continues to act as a national reference laboratory for animal health and coordinates the work for avian influenza and other infectious zoonotic diseases . 2. Prepare for the control and response to an influenza pandemics and other zoonotic or infectious disease emergencies in humans : Modest The main outputs were : The outputs of the original avian influenza objective remained relevant to the revised objectives; The communication program was expanded to include the six other priority diseases; An inter-agency working group was established to cover zoonotic diseases other than avian influenza . Outcomes : The outcomes of the original avian influenza objective remained relevant to the revised objective; No information was provided in the ICR on the incidence of the other priority diseases . 5. Efficiency: Due to inherent difficulties in quantifying the underlying risk and the likely efficacy of activities, it is extremely difficult to conduct a meaningful economic analysis for avian influenza projects . Nonetheless, it is plausible that the project had significant economic benefits . An economic analysis conducted at appraisal attempted to estimate the benefits of the project from reducing the likelihood of outbreaks among poultry and associated losses in poultry . The analysis also considered the economic benefits from preventing secondary economic costs of outbreaks (change of poultry prices), human mortality and morbidity. For the purpose of economic analysis, protection rates (i.e. the assumed reduction rate in the number of cases due to the project ) between 33 and 80 % for both poultry and humans have been assumed and the analysis considered a range of scenarios . For instance, at the lowest protection rate of 33%, the number of saved lives would be 7,577 and reduced number of hospitalization would be 50,379. The likely impact of an HPAI pandemic on unprotected humans was derived using WHO estimates of human -to-human transmission, which calculates possible infection rates, hospitalization rates, and mortality rates . The analysis calculated an economic rate of return of 26%. An ex post economic analysis would not have been meaningful, given that no outbreaks of avian influenza occurred . Instead, the ICR presents a forward -looking analysis conducted in a similar matter to the appraisal analysis . However, in this case it calculates an ERR of 34%, because of slightly different methodologies and an increase relative to the appraisal analysis in the assumed proportion of the human population affected by HPAI outbreaks . The ICR notes that the analysis relies heavily on assumptions with a weak evidentiary base, as the number of human and poultry affected in an outbreak of HPAI and death rate estimates were difficult to estimate reliably . The ICR did not contain an economic analysis of the impact of the project on diseases other than influenza . Project efficiency is rated Substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 26% 100% ICR estimate Yes 34% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project was largely carried out according to its design . When no outbreak of HPAI had occurred, the project broadened its scope to address other zoonotic infectious diseases . The relevance of objectives of the original and revised objectives and design were rated substantial . The project made substantial progress in boosting the capacity of the animal and human health sectors to detect and respond to HPAI and other zoonozes, including development of control strategies of six major diseases and launch of a brucellosis control program . Though traditional economic analysis would not be meaningful for the project, the project may have had significant economic benefits . There were no obvious sources of inefficient use of resources in project supported activities, but the project had undisbursed funds which were returned to the Bank . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The Government remains committed to the project ’s objectives and concept . It is likely to support these areas in the future, given that it has requested the World Bank and other donors (IFAD, USAID, SDC, EU) to provide support in developing follow-up projects. Overall, the risk is assessed as moderate mostly due to the consideration of political volatility in the country, which may lead to instability in the institutional arrangements, attrition of skills, as well as wavering government and budgetary support to the project outcomes . In particular, the ICR refers to instability of the State Veterinary Institutions and uncertainty about reform decisions, which may undermine the longer term impact of the project. The ICR does not provide information on financial commitments or budgetary support to the project outcomes. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The Bank team prepared the project in emergency context relatively quickly, over six months, drawing heavily on the Global Program on Avian Influenza design template (and including animal health, human health, and communication activities) while aligning it with the national strategies and CAS . Preparation was coordinated with other donors involved, including WHO, CDC, IFAD, FAO, USAID, and SDC, that were involved in addressing issues of animal and human health . The project benefitted from having two joint TTLs, from agriculture and health sectors. The Project had ambitious goals considering that existing institutions and infrastructure were weak . The project addressed this through support for institutional capacity building and training . The original Results Framework was comprehensive but difficult to implement, with more than 30 intermediate indicators (which were reduced to eight through consolidation). Proper assessment of financial and procurement capacity was carried out during the project preparation to ensure that there were financial management arrangements and legislative framework acceptable to IDA. The project took into account possible critical risks to achievement of outcomes and possible controversial aspects (PAD, p.19). For instance, it has considered the risks of social distancing measures (i.e. quarantines, bans on mass gatherings, travel restrictions, etc ), which may cause political and social controversy . The project provided a detailed design of procedures for establishing the culling and compensation system, including institutional and legal framework . at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: Implementation support missions took place twice a year and included both animal health and human health specialists. The Bank responded effectively to changing Government priorities by supporting restructuring of the project to address additional infectious livestock diseases . The project was managed in close cooperation with other development partners, including WHO, OIE, EU, USAID and FAO, in areas of institutional reforms, policy assistance and support to veterinary services . During implementation the Bank supported the project in revising the M&E indicators. However, the ICR reports delays in production of Bank supervision reports and other documentation, which caused delays in updating the changes . These delays affected the M&E implementation, as the indicators introduced during the May 2010 were not reflected in the project ’s internal reporting system, and therefore not tracked immediately. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The government's engagement during the project and its further commitment were strong, despite frequent turnover of the staff at decision making level . During the project preparation, the government initiated and coordinated high-level, multi-agency meetings and workshops to set the project ’s scope and shape its concept (PAD, pp. 8,16). The Government had demonstrated a great deal of interest and commitment to the project and conducted regular meetings throughout the project life to discuss the progress and address the outstanding issues in a timely manner. It also provided the necessary facilities for project management and coordination, including field facilities. The Government put substantial efforts to coordinate the project with other resource partners, including OIE and EU. The Government has also proposed to expand the PDO scope to address other zoonozes and continued its strong support to the One Health approach . There were some delays in implementation caused by frequent rotation of responsible staff in the government agencies. In addition, low capacity of the local partners at the start of the project was another major contributing factor to slow pace of implementation at the beginning . Government Performance Rating Satisfactory b. Implementing Agency Performance: The Ministry of Agriculture and the Ministry of Health were the implementing agencies . The Ministries appointed Component Coordinators to oversee the implementation of project activities within their respective agencies and internal coordination . The ICR cites effective coordination between the animal and human health national counterparts, as well as coordination with external resource partners (WHO, CDC, OIE, USAID, FAO, EU, SDC). The Agricultural PIU (APIU) was entrusted with the coordination of project activities, technical assistance and training, as well as with fiduciary tasks of procurement and financial management . The ICR reports that the APIU had demonstrated effectiveness and responsiveness during the entire project implementation with an effective coordination with the Component Coordinators and other counterparts . Project progress reports were produced regularly with a good level of monitoring of activities of all components . Initially, monitoring was slow, and so the APIU recruited an M&E specialist in order to expedite the process . Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: While the M&E system was able to track the completion of major project activities and allowed assessment of the impact of the project, it contained several weaknesses in design . The system was designed to be effective in case of HPAI outbreaks and most of the indicators were not framed in a way where they would be meaningful if no outbreak occurred. For example, it required "culling, disposal and disinfection activities completed as needed " and "% of national and local agencies submitting regular weekly and monthly reports on the influenza pandemic ". During project restructuring, three indicators were added to cover other zoonoses . The M&E system and the list of key indicators (original and revised) covered the PDO adequately, however the tracking system was not clarified and adjusted during restructuring . Most indicators and targets were framed in qualitative terms . For example, the ICR does not report on total number of people trained . b. M&E Implementation: Monitoring of the indicators and data collection were coordinated by both implementing agencies, but was not done correctly and systematically at the beginning . For instance, the three indicators introduced during the May 2010 restructuring were not tracked at all . The project management unit was not clear what and how to monitor and measure. This caused delays in the process and later it recruited an M&E specialist . c. M&E Utilization: M&E data contributed to adaptive management in the Project and it used to share project concepts, results and lessons learned with government, donors and civil society (ICR, pp. 8, 12). M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project was not expected to generate significant adverse environmental impacts, and therefore received Environment Category B. The Environmental Management Plan addressed issues related to zoonotic disease containment, waste management, emissions, culling of poultry and disposal of carcasses . The Plan included mitigation plans and monitoring plans to ensure appropriate attention to environmental issues, and tracking progress or problems in their management. No other safeguards were triggered . The ICR reported that safeguard compliance was satisfactory throughout the project's life . b. Fiduciary Compliance: The project had regular supervision by financial management and procurement specialists . Audit reports were delivered on time and the final report was expected by June 30, 2012. The ICR reports that both FM and procurement were fully compliant with the Bank rules and procedures . The APIU hired a qualified local procurement specialists with experience in international and national procurement . No major procurement issues were identified . c. Unintended Impacts (positive or negative): Collaboration between this project and other donors' funded projects on AI control significantly strengthened technical inter-agency cooperation and liaison between animal and human health institutions . The ICR reports, that the project increased prestige of Veterinary Service and indirectly contributed to increase in enrollment to Veterinary department of the Agrarian Academy . The project improved cooperation and coordination between health agencies in Central Asia, including on sample sharing and use of laboratory facilities . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR reports a number of lessons, including : A global template can enable rapid, good project design . The GPAI template was in place and was used as the basis for the preparation of this project; Crisis response can be an opportunity for supporting long term capacity improvements . This longer-term vision opened the opportunities for progressive improvements and establishment of sustainable systems; Expanding objectives to beyond a single disease can help to support a more comprehensive solution . In this case, the project maintained relevance by expanding objectives to address other needed diseases, and this help to support a longer term One Health approach through a follow up regional project 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is thorough and well written, including a clear exposition of the intended results chain, and a good description of how the project was changed under restructuring . The ICR drew on several sources of evidence, including an endline assessment, beneficiary survey, and workshop evaluations . It also included comments from the Implementing Agency, participating technical agencies, and external partners . However, there were gaps in the descriptions of some outputs, and little information on outcomes . There was little information provided on the scope and scale of many training and capacity building activities . The ICR relied on the output-oriented results framework indicators and provided little additional evidence on which to assess outcomes . In addition, the ICR did little to describe the impact of the project on zoonotic diseases other than avian influenza or other infectious diseases though these had been added as a project development objective at restructuring . a.Quality of ICR Rating : Satisfactory