ICRR 14577 Report Number : ICRR14577 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/13/2015 Country : Uganda Project ID : P110207 Appraisal Actual Project Name : Preparedness And US$M ): Project Costs (US$M): 12.00 10.63 Control Of Avian Influenza L/C Number : C4482 Loan/ US$M): Loan /Credit (US$M): 12.00 10.63 Sector Board : Agriculture and Rural Cofinancing (US$M): US$M ): 0.0 0.0 Development Cofinanciers : Board Approval Date : 06/19/2008 Closing Date : 06/30/2012 12/31/2013 Sector (s): Agricultural extension and research (39%); Central government administration (36%); Animal production (13%); Health (11%); Agro-industry (1%) Theme (s): Rural services and infrastructure (34% - P); Other communicable diseases (33% - P); Natural disaster management (33% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : Hassan Wally Stephen Hutton Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: According to the Emergency Project Paper (p.4) the Project Development Objective was to : "substantially reduce the threat posed to the poultry industry and humans in Uganda by HPAI infection and other zoonoses and to prepare for, control, and respond effectively to future AHI pandemics and other infectious disease emergencies in livestock and humans ." According to the Financial Agreement (p. 5) the Project Development Objectives was to : "support the Recipient’s efforts to substantially reduce the threat posed to the poultry industry and humans by HPAI infection and other zoonoses, and to prepare for, control, and respond effectively to future AHI pandemics and other infectious disease emergencies in livestock and humans ." The PDO stated in the Emergency Project Paper is almost identical to the one in the Financial Agreement . This Review will evaluate the project against two objectives as stated in the Financial Agreement : (i) support the Recipient’s efforts to substantially reduce the threat posed to the poultry industry and humans by HPAI infection and other zoonoses . (ii) respond effectively to future AHI pandemics and other infectious disease emergencies in livestock and humans . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: US$ 5.5 million ). Includes the following six 1. Animal Health (Appraisal cost US$ 5.66 million, Actual cost : US$5 subcomponents: (i) Strengthening Capacity for Animal Health Surveillance, Monitoring and Emergency Disease Control . The project would use the existing institutional framework for surveillance, monitoring and emergency disease control. The project would support: (a) Ministry of Agriculture, Animal Industry and Fisheries (MAAIF) to strengthen its epidemiological unit : (b) MAAIF to put in place a system in all the districts to detect the appearance of Highly Pathogenic Avian Influenza (HPAI) in migratory birds and domestic animals; (c) strengthening the surveillance capacity of MAAIF, HPAI Rapid Response Teams and Uganda Wildlife Authority through training, provision of Personal Protective Equipment and procurement of necessary equipment and supplies for control of HPAI and other emerging diseases . ii) Strengthening of National Veterinary Services (NVS). (ii) NVS ). This subcomponent would support activities to strengthen NVS to enhance animal disease surveillance, diagnosis and control . The support would cover the following: (a) upgrading knowledge and skills through training, equipping laboratories, provision of transport; (b) development and distribution of Standard Operating Procedures for reporting between MAAIF and other line ministries; (c) dissemination of the World Organization for Animal Health (OIE) developed Performance Vision Strategy document; (d) Sero-prevalence surveys to establish baseline data on the epidemiology of re -emerging diseases; (e) review of curricula of the Faculty of Veterinary Medicine, Makerere University and Para -Veterinary schools to cater for HPAI and other Transboundary Animal Disease; and (f) strengthening of the regulatory framework. iii ) Prevention, early detection and rapid response Improved Veterinary Technical Competence for emerging (iii) disease . The project would build capacity in risk analysis, through training and research in various areas of specialization (risk analysis, infectious disease epidemiology, wildlife surveillance ). Simulation exercises for HPAI and other infectious diseases would also be undertaken . iv ) A Balanced Veterinary Epidemio -surveillance programme capable of early detection of emerging (iv) infectious diseases and other TADs . The project would support: (a) passive, active and targeted surveillance, (b) improving animal health information flow among relevant agencies; (c)early detection and timely reporting and follow-up of suspected or positive cases; (d) public and Community-Based Surveillance System by establishing a community-based early warning system; (e) routine serological surveys and epidemio-surveillance; and (f) upgrading of the laboratories earmarked to carry out diagnostics for HPAI including upgrading of the MAAIF National Diagnostics and Epidemiology laboratory to Bio -Safety Level 3 (BSL-3). Animal disease surveillance would be strengthened through provision of the necessary infrastructure, equipment, supplies and transport capacity . (v) Infectious Diseases Preparedness, Detection and Control Evidence -Based Veterinary Rapid Response Capacity for Emerging . To enhance the capacity for effective rapid response, the project would : (a) establish an Incident Command System for animal HPAI outbreak response; (b) develop national policies and Standard Operating Procedures for HPAI rapid response and control; (c) procure rapid response supplies and equipment for all levels of NVS, including protective personal equipment, disinfection units and culling equipment; (d) improve commercial farm and backyard poultry production bio -security; (e) strengthen National Veterinary Service quarantine capacity; and (f) support preparation of compensation policy and guidelines . vi ) Coordination, monitoring and evaluation . MAAIF would be responsible for coordination, monitoring and (vi) evaluation of animal health activities planned under the project . The Office of the Prime Minister would be responsible for the overall M&E activities . The project would provide funds to all the implementing ministries responsible for their own coordination, monitoring and evaluation . 2. Human Health (Appraisal cost : US$2US$ 2.06 million, actual cost; US$ 3.42 million ). It includes the following seven subcomponents: (i) Strengthening of Surveillance of humans for influenza . Surveillance activities for human influenza would be strengthened in all aspects and the number of sentinel sites expanded from the present three sites to at least eight sites. The project would provide funds to strengthen Ministry of Health’s existing field disease investigation system and expand sentinel surveillance to cover at least the regional hospitals and community based surveillance in selected high risk districts ii) Strengthening the Capacity of the Uganda National Health Research Organization (ii) UNHRO ). This would (UNHRO). include: (a) strengthening of the capacity of UNHRO for developing policies to ensure effective coordination and to harmonize research in new and emerging infectious diseases including avian influenza (b) upgrading the National Influenza Center laboratory at the Uganda Virus Research Institute to BSL3, including provision of laboratory diagnostic equipment and reagents; and (c) training of staff for quality control in AI diagnostics, field specimen collection, and testing for influenza viruses . iii ) Consolidation of preparedness for Avian and Human Influenza Prevention, Containment and Control . This (iii) sub-component would support: (a) development of infection control procedures and educational programs for the most vulnerable and at risk communities and health workers and (b) establishment of isolation facilities and procurement of infection control materials and personal protective equipment for at least six regional hospitals. iv) Improving Food safety and Environmental Hygiene . The project would support activities and assessments (iv) aimed at improvement of environmental, occupational and food safety of poultry and poultry products in poultry markets and along major highway food vending points . (v) Health Services and Systems Prepared to handle suspected cases . Based on the lessons learnt from the Ebola outbreak, the implementation of this subcomponent is aimed at increase the surge capacity and flexible deployment of both human and infrastructure resources . This would be achieved mainly by training of health workers in case management and rapid response, development and conducting of simulation exercises and the development of a contingency plan of human resource management in case of an outbreak . vi ) Preparation for Effective Case Management . The project would: (a) provide limited stock of (vi) pharmaceuticals for emergencies (TamifluB and other antibiotics) while adequate budgetary provision should be made for additional supplies when the situation warrants; (b) support preparation of guidelines and standard Operating Procedures; and (c) procurement of necessary accessories and medical supplies (including vaccines) vii ) Response to Outbreak of pandemic influenza of Avian origin . The Project would help create and sustain a (vii) system that can rapidly contain the infection as this is one of the key elements which would drastically reduce human to human transmission, suffering and death . Support would be provided for activating Rapid Response Teams to undertake active case investigations, intensify collaboration with the veterinary services in the areas of disease surveillance, provision of personal protective equipment, prophylaxis to investigating teams and treatment of human cases to control the spread of the epidemic . In the pandemic phase support would be provided for exceptional measures like quarantine and restricting movement of people, closure of schools and markets. 3. Communication (Appraisal cost : US$1 US$ 1.25 million ). To support preparedness and US$ 1.83 million, actual cost : US$1 prevention, the Project would provide funds to : (a) brief policy makers on AI through development and presentation of policy papers; (b) enhance the communication skills of key policy makers and spokespersons through training; (c) establish a communications center within the coordination operation centers at central and district levels, designate Communication Focal Points (CFPs) within them, and equip the CFPs with dedicated communication facilities, ready supply of posters, and standard protocols for reporting any outbreaks; (d) mount media campaigns using Information, Education and Communication (IEC)materials specially developed, and establish a national AH 1 website; (e) train farmers and households with back yard poultry, local government officials, and religious and cultural leaders on the risks of AI; (f) build strategic partnerships with the media through briefings and simulation exercises; and (g) develop an M&E system. To support response and Recovery the project would support : (a) establishment of Standard Operating Procedures for the outbreak communication strategy; (b) dissemination of IEC materials for outbreaks; (c) activation of media plan for outbreaks; (d) Strengthening communication to the public at the outbreak foci or ‘ground zero’; (e) plan for officially communicating the end of the emergency phase; and (f) the evaluation of the impact of the communication effort. 4. Coordination, Monitoring & Evaluation (Appraisal cost : US$0 US$ 0.46 million ). This US$ 0.45 million, actual cost : US$0 component would support implementation costs associated with project planning, coordination, management, as well as overall monitoring and evaluation (M&E) at the national and district levels . The project would support the development of, and integration of an M&E system for project activities into the National Implementation, Monitoring and Evaluation System. The project would also support the M&E aspects of the project activities at MAAIF and Ministry of Health. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost . The total project cost at appraisal was US$ 12.00 million (Emergency Project Paper, Annex 3). Actual project cost was US$10.63 million (ICR Annex 1). Financing . At appraisal the project was expected to be financed through a US$10.00 million IDA credit under the Global Program for Avian Influenza Control and Human Pandemic Adaptable Program Loan window . The Avian and Human Influenza Trust Fund (AHITF) was expected to contribute a grant of US$ 2.0 million to finance technical assistance and training under the Integrated National Action Plan. The project was also expected to receive parallel financing from USAID and EC/EU worth US$0.46 and US$0.80 million, respectively. According to the ICR (Annex 1) the actual disbursed amount at project completion was a total of US$ 10.63 million including an IDA credit of US$8.65 million and an AHITF grant of US$1.99 million. Borrower Contribution . The project was 100% financed by IDA and AHITF. The Emergency Project Paper (p. 16) notes that "the GOU may claim a small amount of expenditures (estimated at US$100,000) under the IDA credit as retroactive financing." Dates . The project closing date was extended twice for a total of 18 months "to allow completion of project activities, especially those related to civil works (ICR, p. 4)." Such extension was necessary to compensate for a seventeen months delay from Board approval (June, 19, 2008) to effectiveness (December, 3, 2009). The delay was due to the lengthy process of securing Parliamentary approval for the IDA Credit (ICR, p. 4). Mid-Term review was on August 11, 2011. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial . At project completion, objectives remain highly relevant . Urbanization activities associated with population growth often lead to expansion into forests and other natural habitats and is usually accompanied by intensification of livestock production . The growing interaction between humans and animals increases the risk of viral mutations and disease transmission from animals to humans . In addition, there is a persistent threat of Avian Influenza worldwide, most notably in Egypt and Cambodia . Uganda is also vulnerable to other zoonosis including Ebola in West Africa and the Middle-East Respiratory Syndrome (MERS). At appraisal, the project was responding to a global emergency, rather than to existing national or Bank development strategies. The objectives were not formally a part of the 2004-2008 World Bank Country Assistance Strategy (CAS) for Uganda. In addition, Uganda was among the sub -Saharan African countries at risk of HPAI outbreak given an outbreak in Juba (South Sudan) in 2007, prevalence of backyard poultry rearing combined with unregulated poultry markets in most rural and urban areas of the country . Finally, the geographic location of the country not only coincides with the major routes for the wild migratory birds, but also is combined with the presence of large masses of water which would facilitate close interaction between wild birds and domestic poultry, hence increasing the chances of disease transmission (Project paper, p. 1). b. Relevance of Design: Substantial . Design includes a simple and realistic statement of objectives . To achieve the stated objectives, design features four components: the first supports activities related to animal health and would be implemented by the Ministry of Agriculture, Animal Industry and Fisheries . It would support the achievement of the objective through capacity building activities of relevant veterinary institutions, improving prevention and early detection measures as well as rapid response capacity to emerging diseases . Surveillance and detection on the animal health side are important because they lead to the ability to contain and control outbreaks through culling . In addition, the animal health activities benefit humans, as controlling the disease in animals is the most effective way of reducing the likelihood of human cases. The second component supports activities geared towards human health and would be implemented by the Ministry of Health. It would support the capacity of relevant health institutions to detect and develop rapid response plans to emerging diseases . Design also featured a dedicated component for communication which would support the achievement of the objectives through improving the communication capacity of relevant institutions, key policy makers, community leaders and spokes persons, hence increase community awareness of the risks of Avian Influenza and other zoonotic diseases . Communicating safer behaviors to people can reduce the risk of disease spread. The fourth component supports the project management activities . Overall, the Results Framework includes clear and logical links between the project inputs, outputs, and expected outcomes. However, it could have benefitted from better articulation to adequately reflect human health outcomes as reflected in the development objective . One notable design shortcoming is the absence of an effective mechanism to coordinate and consolidate all Highly Pathogenic Avian Influenza activities in the country . Design also suffered from poor arrangements to coordinate with other donors . Also, it is not clear why BSL-3 labs were needed. 4. Achievement of Objectives (Efficacy): (i) support the Recipient’s efforts to substantially reduce the threat posed to the poultry industry and humans by HPAI infection and other zoonoses . Substantial . Outputs By project completion the following outputs were achieved : All districts in the country have a functional computer set for data capturing . Also, 94 districts were supplied with GPS equipment (Target 112) and ten high risk districts have been supplied with 4 wheel Double Cabin pickups, and 48 districts that lacked transport have been provided with a motorcycle each . The curricula of three institutions (target: 3) which are offering certificates, diplomas and degrees in veterinary medicine were reviewed by a consulting firm and final report presented to MAAIF . It is not clear whether curricula were revised/updated. Eight sero-prevalence surveys covering HPAI and other zoonotic diseases and non -zoonotic priority diseases including Pestes des Petits Ruminants, Foot and Mouth Disease, Anthrax, Contagious Caprine Pleuro-Pneumonia, Rift Valley Fever, and Brucellosis, were undertaken compared to the four surveys that were envisaged at appraisal. Community based surveillance systems have been established in two high risk districts (Busia and Tororo) compared to an appraisal target of five . 100% of commercial poultry farms and breeders are registered (target:100%, baseline: none). 100% of districts became linked to an incident command system established within the National Veterinary System for the management of logistics at all levels (target: 100%, baseline: none). Three updated HPAI protocols, Standard Operating Procedures, and case definitions were printed and distributed (target: 3, baseline: none). Five quarantine centers and 40 checkpoints were renovated and operational (target: 4, baseline: none). Outcome : The number of districts reporting promptly on veterinary service reached 83 compared to a baseline of none and a target of 40. Electronic reporting is operational in all districts (112 districts have been supplied with modems and 30 districts have been supplied with computers and their accessories ). The diagnostic competence at MAAIF veterinary laboratory for HPAI differential diagnosis was partially achieved (target: full). The ICR (p. vii) notes that full diagnostic competence will be achieved after the recently completed BSL -3 laboratory becomes fully operational. In a further communication the project team indicated that the facility has since become operational. The project efforts led to improved practicing of biosecurity standards along the poultry supply chain . The ICR (p. vi) reports that the percentage of commercial poultry, poultry slaughter facilities, and live bird markets practicing acceptable biosecurity standards at project completion reached 63% compared to an end target of 60% and a baseline of none. However, it is not clear how the project efforts led to such massive change in biosecurity practices. A compensation policy for HPAI for induced death or culling of poultry was drafted and now awaits cabinet approval. However, the fact that the project closed with no approval casts doubt on the approval of such compensation policy . Compensation systems are extremely important for disease reporting and effective surveillance and control and without approval, effective surveillance and control is in doubt . The project improved the capacity of relevant veterinary institutions to detect and fully investigate HPAI cases in poultry in ten high risk districts . The capacity to diagnose HPAI has been increased in three out of four veterinary facilities in high risk districts. There was also improvement on reporting and investigating other zoonotic diseases including anthrax, poultry diseases, swine fever, ruminant plague, and foot and mouth disease, but the extent of such improvement and the contribution of the project are both unclear . That said, it is worth noting that no outbreaks of avian influenza ever occurred, so it is very hard to gauge how much capacity was really improved at all levels. (ii) respond effectively to future AHI pandemics and other infectious disease emergencies in livestock and humans . Modest . Outputs 92% of districts have preparedness and response plans for HPAI (target: 50%, baseline: 30%)which demonstrates the improved state of preparedness and response, in addition 100% percent were reporting data on influenza-like illness using the weekly Integrated District Reporting System compared to a target of 50%. 14 epidemiological mapping exercises (target:10, baseline: none) of avian and human influenza risk factors were undertaken in high risk districts and 100% of districts had bulletins/Integrated District Reporting System feedback reports. 250 health personnel (target: 250, baseline: none) received training and the National Influenza Center Laboratory in Uganda Virus Research Institute was upgraded to BSL 3 and handed over to MOH in December 2013. However, only one isolation facility linked to Entebbe Grade B Hospital was constructed instead of the six originally planned due to limited funds . Laboratories in 15 at risk districts were supplied with rapid detection kits that screen influenza type A, however screening results had low reliability . Outcome The project supported stocking all districts in the country with sufficient stocks of Personal Protective equipment (PPE), disinfection units, and burdizzos ready for use in case of an outbreak . In addition, asphyxiation units were procured and were ready for dispatch in case of a HPAI outbreak . However, asphyxiation systems were only completed in June 2011, 1.5 years after effectiveness despite being extremely important in the case of an avian influenza outbreak. The ICR (p. 13) highlights that through the efforts of the project and other donors the response capacity of the Ministry of Health improved as demonstrated by the case of Ebola . Despite the increase in Ebola outbreak incidence, the duration, geographical spread, and fatality rates from the outbreaks has considerably declined which reflected better preparedness and containment . However, the project had two notable shortcomings . First, only one out of six isolation units was constructed; second, the completion of two BSL -3 laboratories was delayed. The construction of the single isolation unit was towards the project's closing which made it difficult to confirm its functionality with respect to staffing, equipment, and overall management. Similarly, the delay in the BSL-3 labs made it difficult to ensure their functionality during the life of the project. It is also unclear the extent of their future functionality . The availability of functional isolation centers and operational labs is critical to achieve accurate diagnosis and effective containment in case of an outbreak. These shortcomings cast doubt on the impact of the project on improving the ability to respond to any future influenza pandemic . In a further communication, the Bank team confirmed that the equipment installation work at the diagnostic laboratories at the Uganda Virus Institute and the National Animal Disease Diagnostics and Epidemiology Center had been finalized and both labs had been officially handed to the Government and are currently functional. 5. Efficiency: Economic and Financial Efficiency At Appraisal The Emergency Project Paper (Annex 9) includes an economic analysis that aimed to examine the economic impacts of Avian Influenza in terms of costs incurred without the project and economic impacts of Avian Influenza in terms of benefits and costs accruing with the project. The Project Paper (p. 72) highlights two major difficulties in estimating the economic impact of AI in Uganda:" first, scenarios for “gross attack rates� (GARS) of AI in the case of a pandemic, based mostly on past events, are only approximate; second, there is no data to assist in predicting the probability of any of the scenarios for which a possible impact of AI can be estimated ." Therefore, it was not possible to provide an estimate for the economic impact of the AI without the project . Impact with the project was analyzed against three possible GARs scenarios for poultry population infection, at 0.7%, 1.0% and 2.5% which represent poultry lost from the disease or culling . For an outbreak in year 2, Internal Rates of return (IRRs) ranged between 11% to 70% while for an outbreak in year 4, IRRs ranged between 7% and 34%. These results show that the project impact would be considerable under reasonable assumptions of GARs on poultry alone . Only if the GARs are extremely low and are delayed in the life span of the project are the returns likely to be lower . At Closure It is plausible that the project had significant economic benefits by reducing the risks of disease outbreaks in poultry and in humans, but due to inherent difficulties it is extremely difficult to rigorously quantify these benefits . The ICR (Annex 3) follows the same approach as the Emergency Project Paper . The ICR analysis is actually not ex post; it is a repeated forward looking analysis which is appropriate since no outbreaks actually occurred yet . The analysis provides estimates for Internal Rates of Return (IRRs) in 2014 under the same GAR scenarios envisaged at appraisal. Internal Rates of Return were between 12% to 40%. The ICR (Annex 3) highlights that the lack of reliable data made it not possible to include other impacts of the project in the analysis. This includes the decline in the demand for poultry /meat products, the impact of the project ’s surveillance and disease prevention and containment activities for other livestock (cattle, small ruminants, pigs) and the value of reduced human morbidity from avian influenza related illness; all of which would have added to the project’s rate of return. The methodology also addresses only the impact on poultry and not on humans (as the assumption was that the attack rate for humans of 0 in all cases) and so the rate of return is a lower bound estimate . Finally, the analysis does not explain its key assumption regarding the impact of the project on the attack rates, only the IRRs are presented. Institutional and Administrative Efficiency The project suffered from a delay of about seventeen months from Board approval (June, 19, 2008) to effectiveness (December, 3, 2009). The delay was due to the lengthy process of securing Parliamentary approval for the IDA Credit (ICR, p. 4). To compensate for such delay the closing date was extended by eighteen months . This delay was a particular problem for what was designed as an emergency response project, aimed at rapidly building capacity to be able to respond to a global health threat should it strike Uganda . The cost of component 2 exceeded the appraisal estimate by 66% (US$2.06 million compared to actual cost of US$ 3.46 million). In a further communication the project team explained that part of the increase was due to reallocation of funds from activities that would have taken place in case of an outbreak . A significant amount of project resources were spent on building facilities (BSL-3 labs) that were not fully operational as of project closure . The project also suffered from weak coordination between the two implementing agencies. Civil works suffered from delayed remittance of funds . This was partially mitigated through reallocation of funds in -between project components. Procurement activities were slow and suffered from weak capacity in both implementing agencies which undermined project implementation and progress . The weak capacity combined with the decision not to adopt emergency procurement procedures as allowed under the provisions of OP/BP 8.50, contributed to implementation delays and according to the ICR (p. 5) "the absence of expeditious procurement features, at least for emergency items, created a disconnect between the project ’s emergency response nature and its cumbersome procurement arrangements ." That said, procuring complicated technical laboratory equipment under emergency procedures might not be the best option given that there was no particular need for labs to be built under emergency conditions; only some of the activities supported by the project needed to be done on an emergency basis, for example, equipment like personal protective gear, and rapid test kits . On balance efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 13% 100% ICR estimate Yes 19% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objectives and relevance of design were both rated substantial . Efficacy of the first objective was rated substantial where the project activities helped to improve poultry biosafety standards and strengthened the capacity and readiness of veterinary institutions . Efficacy of the second objective was rated modest due to shortcomings on reaching the target number of isolation units and delays in completing two BSL -3 laboratories. Efficiency was rated modest due to institutional and administrative weaknesses and the lack of data on the full impact of the project. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The project enhanced the capacity of relevant institutions to respond to zoonotic and non-zoonotic threats. The sustainability of the project development objective is dependent on the availability of adequate funding to maintain monitoring activities and preparedness . Also, close collaboration between the animal, health and wild life institutions needs to be maintained. There is also a concern on the ability of MAAIF to maintain regular collection of field data post project completion; where the data covers a wide array of animals across the country would require continuous effort and focus by the Government . Finally, the laboratory diagnostic capacity established under the project needs to be functional. There are concerns regarding the availability of funding to maintain such capacity post project completion. In a further communication, the Bank team explained that the diagnostic capacity established under the project is currently functional after finalizing the work at the diagnostic laboratories at the Uganda Virus Institute and the National Animal Disease Diagnostics and Epidemiology Center . The team also highlighted that the "Government has shown commitment to reducing the ‘risk to development outcome� by establishing strong collaboration between the animal, health and wildlife institutions in disease surveillance and monitoring, and this has been formalized through a specially dedicated National Task Force that is in charge of the One Health agenda . In addition, regular collection of field level surveillance and monitoring data continues through Government funding as this is one of the core functions of Ministry of Agriculture Animal Industries and Fisheries (MAAIF) and the Ministry of Health (MoH)." a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: At the request of Government of Uganda the Bank agreed to process the project as an Emergency Recovery Credit (ERC) under the Bank’s policy for emergency operations (OP/BP 8.00). At appraisal, the project could have benefitted from better analysis of other projects in the same field e .g, Makerere University-Walter Reed Project in which the U.S. Department of Defense Global Emerging Infections Surveillance and Response System funded a BSL-2 lab as well as with activities funded by other donors. e.g. USAID supported the distribution of personal protective equipment in some areas . Such analysis would have facilitated better coordination and resulted in better synerg y. The project design benefited from lessons emerging form the Global Program for Avian Influenza most notable was that strengthening national veterinary infrastructure and capability, adequate surveillance systems, proper diagnosis and linkage to reference laboratories, multi -sector collaboration specifically with human health services, and effective public communication were pivotal to prevention and response to an AI outbreak. However, the project design did not reflect an important lesson from other emergency operations regarding the use of expedited procurement process rather than the traditional one --which proved to be lengthy and contributed to project delays . Project design included a realistic and simple objective . Featuring a design that necessarily included both animal health and human health agencies, and aimed to increase their cooperation . Design called for strengthening both veterinary and health institutions to address a common objective . Design also included a dedicated communication component. Design suffered from weak coordination arrangements between the two implementing institutions . Also, the M&E design lacked key performance indicators that would adequately capture the impact of the project on the human health dimension. At appraisal several risks were correctly identified and three of these materialized . The most considerable risks to materialize were poor project coordination and weak procurement management while delays in availing financial resources also materialized, but to a lesser extent . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: The project benefitted from the country -based Bank team who provided support on work programming, financial management, procurement, and safeguards, among others . Project support missions were composed of a broad skill mix. Aide memoires were detailed and countersigned by the Government and included clear actions that need to be carried out. Supervision also included smaller missions that were supported by the country office which were carried out in-between the principal missions. Bank supervision could have benefitted from better coordination with other donors funding parallel operations . More attention should have been given to weaknesses in procurement activities and M&E at an earlier stage of the project . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government requested the support of the Bank to address zoonotic diseases . It showed mixed commitment towards the project with stronger support during preparation and the second half of project implementation. However, the period following effectiveness, Government actions were less supportive . It took almost one year after Board approval to seek parliamentary approval . This delay indicates that the Government had not adequately engaged parliament for an expedited review of the project. Furthermore, an initial parliamentary decision was to refuse the project on the basis that Avian Influenza was not a threat . Such refusal demonstrated that the Government failed to sensitize the parliament on the project's broad scope beyond Avian Influenza. On the heels of the first case of swine flu in Uganda, the parliament approved the project on October 10, 2009. This late approval came after the expiration of the effectiveness deadline which required processing a retroactive effectiveness extension . The project was finally effective on December 3, 2010, almost eighteen months after Board approval. By project’s completion, the draft compensation policy was still not approved by the Cabinet. According to the terms of the Financing Agreement, the Government committed to preparing the Environment and Social Management Plan (ESMP) within three months of project effectiveness . However, ESMP was prepared 2.5 years after effectiveness . Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: The project was implemented under the Ministry of Agriculture, Animal Industry and Fisheries for the animal health component and Ministry of Health for the human health component . Coordination between the two implementing agencies was ensured through the Office of the Prime Minister (OPM). Overall, coordination arrangements under OPM were inadequate . Project coordination started weak and improved after personnel changes and greater involvement by the Permanent Secretaries of the implementing entities . According to the ICR (p. 7) such coordination weakness adversely affected priorities managed by the OPM including the production of the project Environment and Social Management Plan (ESMP), the development of the overarching M&E framework for the project, and designing modalities for public /private partnerships, all of which were only completed in early 2012, over two years after project effectiveness . At a later stage of implementation, personnel changes and greater commitment from the Permanent Secretaries improved coordination . In addition, the National project Steering Committee met infrequently . Both implementing agencies suffered from weak procurement capacity. Civil works suffered from procurement delays and poor contract management . According to the ICR (p. 7) there were also "delays in hiring architectural design consultants; delays in payment to contractors (this was the case for the National Animal Disease Diagnostics and Epidemiology Center ) of up to 4 months which led to the works stalling various times as the contractors run out of funds to sustain their cash flow requirements; and delays attributed to inadequate site management skills ." Implementation activity improved within the 18 months extension period. Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The Results Framework (PAD, Annex 2) includes a set of outcome/output indicators that covers different project activities. However, the key performance indicators are exclusively focused on the animal sector outcomes . Design lacks adequate indicators to capture the impact of project activities on the human health dimension . For example, the outcome indicators only relate to the threat to the poultry industry, and none to the threat to humans . It was envisaged at appraisal that two independent impact evaluations of the project would be carried out, one at mid -term and another at end of the project (PAD, p. 10). b. M&E Implementation: M&E implementation was handled by the two relevant ministries, MAAIF and MOH . The OPM was supposed to consolidate project information and feed it into the overall national M&E system . Ministerial level M&E systems collected data generated from project activities and inputs were included into the M&E system . The ministerial level M&E system was simple and matched the existing capacity level within the ministries . However, establishing the overarching M&E framework at the OPM was problematic and required an M&E consultant . The hiring process of the consultant was slow and the system was only put in place in early 2012. No impact evaluation was carried out . c. M&E Utilization: It is not clear how the M&E information was utilized to inform project management and decisions . The ICR (p. 9) highlights that 83% of the districts were reporting useful information on animal disease compared to 15% in 2009. Such data is expected to play a critical role in filling a major analytical gap in understanding the epidemiology of the various diseases. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: According to the Emergency Project Paper (Annex 8) the project was a Category B. It triggered Environmental Assessment (OP/BP/GP 4.01). Environment . According to the terms of the Financing Agreement, the Government committed to preparing the Environment and Social Management Plan (ESMP) within three months of project effectiveness . However, ESMP was prepared 2.5 years after effectiveness . This, however, did not impact project implementation since there was no outbreak of Avian Influenza . In addition, civil works were carried out towards the end of the project when the ESMP was already in place. The ESMP was approved by Uganda’s National Environmental Management Authority, and was found satisfactory by the World Bank . However, the protocols on handling highly infectious patients and medical waste generated at the Isolation laboratory and the BSL 3 facility at the National Influenza Center laboratory in the Uganda Virus Research Institute had issues that need to be addressed . It is not clear whether such issues were addressed. In a further communication, the project team explained that an incinerator was in place to deal with hazardous waste materials. Social . At appraisal the project did not trigger any Social Safeguards as there were no plans for land acquisition, voluntary or involuntary, nor any displacement of people . In the event of future HPAI outbreaks, the main social concern was the maintenance of livelihoods for poultry producers, especially backyard producers . This was to be mitigated through establishing a transparent compensation program with broad stakeholder involvement . Also, in the case of outbreak a public awareness campaign would aim to deter widespread consumer rejection of poultry products from outbreak free areas . Finally, a compensation policy was developed, but has yet to be approved by the Cabinet. b. Fiduciary Compliance: Financial Management . Financial reports were submitted on time except for the audited financial statements for June 30, 2013 which were submitted after the submission deadline of December 31, 2013. The audit report was unqualified opinion on the financial statements . While financial management was good across the three implementing agencies, the three audit reports reported some weaknesses . The FY2010/2011 Auditor General report issued a clean opinion, but raised concern on poor bookkeeping at MAAIF, poor advance management at MOH, and inadequate fixed asset recording at OPM . The FY 2011/12 audit issued a qualified opinion on the basis that project payment vouchers and supporting documents for US $ 323,338 at the OPM were not availed for audit. These were later availed in the subsequent audit and cleared . According to the ICR (p. 10) any weaknesses identified during in Financial Management during supervision missions were satisfactorily addressed . Procurement . Procurement activities were generally compliant with the World Bank Procurement guidelines . However, procurement was undermined by weakness in the capacity of the staff at the implementing agencies . This contributed to delays in project implementation especially with regards to civil works . Procurement delays were exacerbated by the decision not to adopt emergency procurement procedures, as allowed under the provisions of OP/BP 8.50. According to the ICR (p. 10) "there were no reports of procurement violations although there were some instances of poor record keeping ." c. Unintended Impacts (positive or negative): None d. Other: None. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are emphasized from the ICR with some adaptation of language : Flexible implementation arrangements are more suitable in an emergency operation . There is need for greater pragmatism and flexibility during emergency projects especially if there is a threat of a disease outbreak. Options might range from using the procurement staff of existing /ongoing projects with experienced personnel for the more urgent items, while non -urgent items are procured by the relevant line ministry staff whose capacity might still need some strengthening, to hiring experienced consultants for the duration of the project, to pairing line ministry staff with more experienced short -term consultants for on the- job training. In addition, it is important to constructively use the time before project effectiveness to make progress on procurement (especially where procurement accredited staff in the ministries already exist ), instead of waiting for project effectiveness before starting the procurement process . Engagement with parliament during project preparation can facilitate timely approval of projects by parliament . Adequately engaging key parliamentarians about the project ’s broad scope and the varied and persistent nature of the threats addressed under the project could have potentially facilitated an expedited and favorable consideration of the project in Parliament . It is particularly important that relevant parliamentary sectoral committees be closely associated with the project preparation process to ensure timely approval . Working across sectors can be an effective way of addressing zoonotic diseases . With the growing and persistent threat from zoonotic diseases, a culture and effective modi operandi of collaboration between the ministries in charge of health and livestock are critical in ensuring adequate monitoring of, and rapid response to major zoonotic disease outbreaks . Implementation arrangements through Government structures should take into account not only staff ’s technical competency but also their actual availability . The Office of the Prime Minister (OPM) tends to be a favored choice for coordinating projects of a multisectoral nature, as was the case in this project . With the growing tendency for multi-sectoral approaches, hence the risk of overwhelming OPM ’s capacity, the choice of OPM for project coordination should be weighed against the available capacity to fulfill the function . Alternative implementation arrangements (such as conferring coordination responsibilities to the ministry with the preponderance of project activities ) should also be considered . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides through coverage of project activities and a candid account on most shortcomings . It includes six lessons that reflect the project experience . Two lessons are project specific and four could be generalized to other operations. The ICR's assessment of outcomes was based on evidence, however, assessment of the human related health activities was limited by the weaknesses of the project ’s M&E frameworkt. It could also have explained its actual assumptions about the incremental impact of the project on the likelihood of poultry losses. a.Quality of ICR Rating : Satisfactory