ICRR 14268 Report Number : ICRR14268 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/25/2014 Country : Bangladesh Project ID : P102305 Appraisal Actual Project Name : Avian Influenza US$M ): Project Costs (US$M): 20.00 12.59 Preparedness And Response Project L/C Number : C4340 Loan/ US$M): Loan /Credit (US$M): 16.00 10.16 Sector Board : Agriculture and Rural Cofinancing (US$M): US$M ): 2.00 1.17 Development Cofinanciers : Avian and Human Board Approval Date : 06/28/2007 Influenza Trust Closing Date : 12/31/2012 12/31/2012 Fund-TF90662 Sector (s): General public administration sector (48%); Health (31%); General agriculture fishing and forestry sector (17%); Other social services (3%); Solid waste management (1%) Theme (s): Other communicable diseases (25% - P); Natural disaster management (25% - P); Rural services and infrastructure (24% - P); Pollution management and environmental health (13% - S); Health system performance (13% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Ebru Karamete Stephen Hutton Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The project development objectives stated in the Financing Agreement is (p. 5): "to minimize the global threat posed by the highly pathogenic avian influenza to humans by controlling such infections in domestic poultry, and preparing for, controlling, and responding to possible human infections, and to contribute to the control of zoonotic and other infectious diseases, in terms of building overall response capacity ". The Project Appraisal Document statement of objectives is (p. 9 ): "to minimize the threat posed by HPAI to humans in Bangladesh by controlling such infections in domestic poultry, and preparing for, controlling, and responding to possible human infections, especially an influenza epidemic and related emergencies ". As per IEG’s evaluation methodology, this Review ’s assessment is based upon the formulation of the project objective in the Financing Agreement . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: I - Animal Health (Appraisal Estimate US$ 15.5 million, Revised Estimate $ 9.8 million, Actual US$ 8.6 million). (Please note that revised estimates and disbursements by component do not include contingencies, and the project team was unable to provide accurate data ). I.A. Enhancing Prevention and Preparedness Capability by (a) adapting and updating a National Policy Framework supporting a review of national policies with private sector and NGOs involvement, (b) evaluating national veterinary services; (c) reviewing national compensation policy; and (d) fostering better understanding of HPAI by organizing conferences and workshops . B . Strengthening Disease Surveillance and Diagnostic Capacity by (a) legislation to strengthen veterinary services; I B. (b) reorganizing Department of Livestock Services; (c) improving animal health information flow; (d) improving detection, reporting and follow-up of reported outbreaks in all of the high -risk districts; (e) establishing community-based surveillance networks; (f) conducting epidemiological surveys; (g) improving laboratory diagnostic capacity; and (h) supporting applied research on HPAI through competitive grants . I.C. Strengthening Control Programs and Containment Plans by (a) controlling disease outbreaks in poultry; (b) setting up effective movement controls on poultry; (c) supporting poultry vaccination through a contingency fund; and (d) providing laboratory, field staff, and avian flu workers with training and personal protective equipment . I.D. Improving Bio-security in Poultry Production and Trade to improve the bio -security of commercial and semi-commercial poultry producers by supporting Department of Livestock Services staff to train producers to improve their bio-security measures. I.E. Support for Surveillance of Migratory Birds funds for improving surveillance and reporting unusual wild bird deaths from HPAI, including the provision of limited amount of personal protective equipment and bio -secure containers to transport samples laboratory for analysis . I.F. Establishing a Contingency Fund (a) establishment of a compensation program to reimburse affected poultry farmers; and (b) the procurement of personal protective equipment and avian flu vaccine . The amount for this sub-component was reduced at Mid-term review from US$ 3.265 million to US$ 1.746 million by cancelling funds allocated for personal protective equipment, in view of the substantial amounts of personal protective equipment were being provided by donors . A sub-component entitled Bio-secure Poultry Market Chains was added to the Animal Health component in 2011, comprising the establishment of nine piloted bio -security poultry market chains in nine of the Districts at highest risk to HPAI. II - Public Awareness and Information Component (Appraisal Estimate US$ 1.8 million, Revised Estimate $ 1.5 II- million, Actual US$ 0.3 million). II. II .A. Advocacy and Collaboration with Stakeholders support to disseminate information from the national level to public and private sectors; improve coordination and collaboration among all major stakeholders . II.B. Production and Dissemination of Communication Materials development and testing of messages and materials II. to disseminate communication from national to local level public and private sectors . II .C. Developing Community Based Communication Pilots , the development and implementation of training courses II. in risk communications methodology for extension and veterinary staff, health workers, and NGOs at national and local levels. III: III : Implementation Support and Monitoring and Evaluation Component (Appraisal Estimate US$ 2.3 million, Revised Estimate $ 1.4 million, Disbursement US$ 1.3 million) The sub-component supported establishment and operational costs of Project Coordination Unit within Department of Livestock Services for project implementation, coordination and management, communications, procurement, financial management, safeguards, and implementation management as well as M&E with technical assistance from FAO. (Please note that ICR's report on project costs and disbursements was different in different sections of the report and also it did not match the figures presented in the operations portal . As the operations portal does not report figures by components, this review used PAD amounts for appraisal estimates and ICR figures (p. 24) for actual costs). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : At appraisal, Project costs were expected to be US$ 20 million. The project cost was revised to US$ 14.7 million in January 2010. The actual project cost was US$ 12.6 million. Financing : The project was to be financed by an IDA credit of US$ 16 million and a US$ 3 million grant from the Avian and Human Influenza Trust Fund, and a borrower contribution of $ 2 million. In January 2010 the project was restructured, leaving expected financing of $ 12.2 million from the IDA credit and $1.2 million from the AHIF grant. Actual disbursements were $10.2 million from the IDA credit and $1.2 million from the AHIF grant. The Borrower contribution at appraisal was expected to be US$ 2 million. In January 2010 this was revised to US$ 1.25 million, and the actual contribution was US$ 1.25 million. Dates : The project was closed on as expected on December 31, 2012. On 25 January 2010, US$ 4.67 million was cancelled as some planned project activities were to be carried out by other donors and so did not need to be financed under the project. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Substantial . Highly pathogenic avian influenza posed a significant threat to humans in Bangladesh . The project was directly responding to outbreaks in poultry that had been detected in 2006, and there were significant risks of further outbreaks as biosecurity was weak on both backyard and commercial poultry farms, and as Bangladesh lies on major migratory bird pathways which could serve to spread the disease within Bangladesh and to neighboring countries . Outbreaks in poultry could pose a serious threat to the poultry industry and could affect humans through direct infection from birds. This risk of direct infection was high, because of Bangladesh ’s population density of 1,072 people per km2 and poultry density of 1,194 per km2, and because 75% of people living in rural areas were heavily dependent on poultry production and many lived in conditions where the chance of disease transmission from animal to human was high. The weak capacity of veterinary services and low awareness of general public increased the potential impact of outbreaks. If not contained, avian influenza could also develop a strain that was transmissible between humans, triggering a pandemic . However, the project objectives of minimizing the global threat posed by avian influenza were very ambitious, as any activities within Bangladesh could play only a moderate role in mitigating the threat to non -neighboring countries. Although the objectives were not relevant to the latest Country Assistance Strategy (FY 11-14), the project objectives were relevant to those of the Global Program for Avian Influenza and Human Pandemic Preparedness and Response and the Bank’s policy of assisting countries in responding to emergencies . Furthermore, the objectives were also directly linked to the National Avian Influenza and Human Pandemic Influenza Preparedness and Response Plan that was approved in April 2006. b. Relevance of Design: Modest . The activities supported under the project would all contribute to the project objectives, as controlling avian influenza among poultry would help to reduce the threat of avian influenza among humans . Under animal health component, activities such as strengthening surveillance and diagnostic systems would increase the ability to detect outbreaks of avian influenza, and strengthening outbreak response capacity would then help to allow those detected outbreaks to be controlled. Under public awareness component, increasing awareness and information on avian influenza through communication would again help to control the outbreaks and thereby minimize the threat posed by the virus . However, there was a serious disconnect between the project development objectives and the project components . The project did indirectly contribute to controlling human infections through controlling the disease in poultry; however the project did not have any activities that would help to respond to possible to human infections of avian influenza, or to contribute to building capacity for the control of other infectious diseases, though these were included in the project development objective . Human infections were to be addressed under a different program (the Human Health sub-component under the avian influenza component of Health Nutrition and Population Sector Program ) that was carried out by Ministry of Health and Family Welfare, but if they were not to be carried out under this project then the project should have had more focused objectives . 4. Achievement of Objectives (Efficacy): It is important to note that there was difficulty in attributing the outcomes to the project . As also pointed out by the ICR (p. 6 and p. 16), the project joined forces with the already ongoing activities of other development partners with whom the Bank shared credit for achievement of objectives : US$ 22.5 million was provided by other donors under separate projects, and so this project contribute roughly 35% of total financing. (i) Minimizing the global threat posed by the highly pathogenic avian influenza to humans by controlling such infections in domestic poultry; rated Substantial . Outputs : • National Avian Influenza Policy was adopted and updated to cover 2009-2012. • Commercial poultry farms across the country with their poultry inventories were registered (however, it is not clear if this included all commercial poultry farms ). • A community based surveillance system was established for detecting outbreaks and communicating with field response teams, utilizing a Short Message Service (SMS) text message system. Under the system 1,117 workers monitored all 306 high risk districts. 86% of avian influenza outbreaks were detected using this system . • Training was provided to surveillance workers (22,057 person days), veterinary staff (22,057 person days), poultry farmers, containment personnel, and animal health workers (28,898 person-days across 1,888 people). • 7 existing field veterinary laboratories were renovated and provided with supplies and equipment, and two new ones were constructed . Staffing at these laboratories increased from 30 % of required level to 63 %. • The project conducted pilots to improve bio -security at live bird markets and on farms in 9 high risk districts. Under the pilots, 1,854 farmers, transporters, bird market workers, and other stakeholders were trained in bio-security. • The Project enforced poultry movement controls and quarantine in infected areas . • A contingency fund was established for compensating farmers for culled birds and for purchase of emergency supplies. At Mid Term Review, the emergency supplies aspect was cancelled . The fund provided compensation of US$ 3.88 million to 822 farmers within 4 weeks. • Wild bird surveillance was not expanded . • A public awareness campaign was carried out, with the project supporting training in workshops in risk-based communication, production and distribution of materials and implementation of the national communication strategy for pandemic influenza . Outcomes : The ICR reports a reduction in response time between outbreak detection to culling from 4.8 days to 1.4 days. (The project team reported that this data came from an independent survey carried out by UNICEF in partnership with the Ministry of Fisheries and Livestock and other development partners .) Veterinary laboratories could diagnose HPAI with 98 % accuracy based on an international assessment . (source: The National Reference Laboratory for Avian Influenza in 2012 sent 110 split samples to the World Animal Health Organization Avian Influenza Reference Laboratory in Padova, Italy, which fed back a diagnostic accuracy of 98.2%, which is higher than the 90% targeted). The baseline laboratory accuracy was not known . Culling and disinfection of farms were carried out for all identified outbreaks of avian influenza, with a total of 3.46 million poultry culled and 2.97 million eggs destroyed. Although the rate of avian influenza outbreaks declined by 70% during the project, with 21 outbreaks recorded in 2012, avian influenza has not been eradicated in Bangladesh : it still has the highest incidence in South Asia and the country is one of six countries globally where the virus remains endemic. The ICR also provided a survey result on public awareness (p. 16): 96 % of surveyed knew how to protect themselves against avian flu . Also, 90% of surveyed commercial poultry farmers and 70% of backyard poultry farmers and 65% of targeted live bird handlers were aware of good biosecurity practices . ii) Preparing for, controlling, and responding to possible human infections; rated Negligible (ii) . Outputs : • The project helped to adopt and update the National Avian Influenza Plan for the period of 2009-2012. The Government designated Ministry of Fisheries and Livestock and Ministry of Health and Family Welfare to implement its National Avian Influenza Plan, with appropriate support from other agencies, but this was carried out under a separate project. Outcomes : During the project period, no human fatalities occurred, although 6 human cases have been identified . iii ) Contribute to the control of zoonotic and other infectious diseases, in terms of building overall response (iii) capacity, rated Modest . Outputs : The ICR did not provide any specific evidence on this . The task team subsequently reported that Seven Field Disease Investigation Laboratories have been upgraded through rehabilitation construction works and advanced lab equipment and materials. The laboratories are fully operational with varying levels of utilization of the new equipment: (i) In addition to carrying out simple diagnostic tests, including parasitological, bacteriology and rapid tests to detect avian influenza, all laboratories perform post -mortem procedures on various animals, such as poultry and ruminants. (ii) Two of the labs also perform relatively advanced serological tests . (iii) More advanced tests, such as PCR, egg inoculation and virus isolation are conducted by the two central labs. Outcomes : Although there was not much evidence on the project's contribution to control of other zoonoses, the veterinary laboratories have also improved their capacity to diagnose other zoonotic diseases in animals . The Project team subsequently stated that the investments in a national strategic response, critical infrastructure and technology for surveillance and detection, and increased public awareness will impact the ability of Bangladesh to respond to other emerging zoonotic threats. However, there was no evidence on project ’s specific contribution on other infectious diseases. 5. Efficiency: Substantial . The project may have had significant economic benefits, though it is very difficult to estimate these benefits due to uncertainties in key parameters . The economic analysis at appraisal estimated the number of infected humans across 6 possible scenarios and calculating the economic benefit from prevented infections for each scenario . The PAD rightfully argued that the ex -ante analysis was difficult due to a lack of scientific knowledge on factors affecting spread of the virus in humans and probability of its occurrence, and in the efficacy of project activities on the number of cases. The analysis estimated that 0.7% to 20% of poultry would be killed by disease or culling, and 0% to 30% of humans would be infected. The likely economic impact ranged from 0.3 % of GDP to 2.2 % of GDP. During the project period 1.7 % of the poultry population was actually affected (culled), which was higher than the average of estimated at appraisal, but the actual economic costs were much lower . An ex post economic analysis conducted at project closure estimated project benefits based solely on a reduction in the number of poultry lost to disease or culling . As mentioned by the ICR (p. 17), it is impossible to know what would have occurred without the project . Nonetheless, the analysis attempts to tackle this by using an epidemiological model to simulate possible disease scenarios . The analysis estimates that the project together with the projects of donors/development partners contributed to a reduction in the average cost of the disease from US$ 347 million to US$ 17 million, a saving of US$ 330 million. The ICR provided a Net Present Value figure by subtracting total intervention costs of US$ 34.5 million (including numerous agencies and development partners ) from the total benefits. The ICR attributed 35 % of the Net Present Value amount to this project based on the proportion of total expenditure on avian influenza in Bangladesh that was under this project as opposed to those under other projects . [While methodologically invalid, it is difficult to imagine an alternative method for separating the benefits from this project from those of other donors .] The key assumptions are buried within the simulation modeling that projects a presumed counterfactual within its simulation modeling, and so are difficult to assess . In terms of operational efficiency there were some delays in implementation . The project closed on time without any extensions, but a number of activities were not completed due to delays, leaving $ 2 million undisbursed at closing. The delays were a result of the need to revise the main project document to accommodate hiring of FAO for technical assistance, the Government's delays in approving the revisions, and weaknesses in project management and procurement in the project coordination unit . After the FAO team was hired in late 2010, implementation progress improved. The efficiency of the project is rated substantial but marginally so, noting the difficulties in calculating economic efficiency for such projects and difficulty in attributing the economic benefits to this single project, as well as considering operational inefficiencies . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of objectives was substantial given the emergence of avian influenza as a global threat and massive outbreaks in the country and limited capacity to respond . The relevance of design was modest due to the lack of components dealing with : preparing for, controlling, and responding to possible human infections, even though this was part of the PDO. Achievement of minimizing the global threat posed by the highly pathogenic avian influenza to humans was rated substantial; achievement of preparing for, controlling, and responding to possible human infections is rated negligible due to lack of project activities contributing to this objective, and achievement of contributing to the control of zoonotic and other infectious diseases was rated modest due to a lack of evidence . Efficiency was rated substantial due to potentially high economic benefits but operational delays . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The ICR did not provide any information on the institutional risks to the capacity improvements that have been achieved. The risks from avian influenza remains high as outbreaks continue and the disease has become endemic . a. Risk to Development Outcome Rating : High 8. Assessment of Bank Performance: a. Quality at entry: The project was developed as an emergency operation to respond to serious avian influenza outbreaks in Bangladesh, and to join to the already ongoing avian influenza activities financed by other donors . The project design followed much of the Global Program for Avian Influenza and Human Pandemic Preparedness and Response template, though it did not include a human health component . There a number of weaknesses in quality at entry: • There was a disconnect between the project development objectives and project activities, as activities to build capacity to respond to cases in humans were not included although this was part of the project development objective. • Poultry chain bio-security was not included in original project design although this was important to reduce spreading of disease, and would contribute to achieving the development objective . This was only identified and a component was added for establishment of nine piloted bio -security poultry market chains following the Mid-term Review; • The long period between the concept note and appraisal and approval (1.5 years from concept review to approval) did not fit the scope of an emergency operation . • The project was not well coordinated with interventions from other donors, which meant that 25% of the loan was cancelled in order to avoid duplication • The institutional arrangements failed to involve Department of Forestry, which resulted in lack of surveillance of wild birds; • Monitoring and evaluation framework was weak, as the results framework did not adequately measure outcomes (see section 10). at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: The ICR stated that (p. 20), the bank team was committed and made significant efforts in providing regular support to Department of Livestock Services and Project Coordination Unit with timely supervision missions and a useful Mid Term Review which led to significant restructuring . The Bank team strongly reminded the Government of Bangladesh to complete the approval of revisions to the project design and to meet its other project commitments. The Bank provided significant support for procurement to the implementing agency, which improved overall procurement management performance . However, the disconnect between the project objectives and design was not repaired during restructuring, even after the midterm review. The ICR also made the following critique : (i) supervision failed to ensure full restructuring at Mid Term Review, particularly the project could have included a wider scope for biosecurity of poultry chains activity instead of carrying out 9 pilots given the importance of these activities in reducing spread of disease among poultry; and (ii) the Bank team should have worked out an alternative option to ensure that the wild bird surveillance activity was effectively implemented . In addition another issue was that the team did not revise the PDO to reflect the actual activities of the project . All of these shortcomings in supervision yield a moderately satisfactory rating . Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: According to the ICR, the Government engaged early with a high commitment to the Project in terms of establishing the Project Coordination Unit and ensuring the availability of adequate staff . However, in various cases the emergency nature of the project was disregarded and the lengthy bureaucratic process in terms of obtaining multiple-level government clearances for procurement delayed implementation . For example the revision and approval of changes to the project design and hiring of an FAO team were significantly delayed . Also the agreement with Department of Mass Communication was delayed which further postponed communication activities. These delays in supporting an emergency operation mean that government performance is rated moderately unsatisfactory. Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: The Department of Livestock Services (DLS) under the Ministry of Fisheries and Livestock implemented the Animal Health component. The Department of Mass Communication under Ministry of Information collaborated with DLS to implement parts of the public awareness and information component . Since the Ministry of Health and Family Welfare implemented human health activities under a different project, DLS assigned a liaison officer to maintain linkages with that component ’s implementation progress. The Government contracted an FAO Technical Assistance Team team to assist in the technical implementation aspects (producing communication materials and guidelines, and curricula for different stakeholders, such as biosecurity intermediaries, laboratory technicians, and community surveillance agents ). The Project Coordination Unit (PCU) as well as Department of Livestock Services initially had weaknesses in project management and procurement due to low human resources capacity . Consistent delays in processing procurement activities, particularly in case of civil works and equipment for laboratories created difficulties . Mis-procurement for several contracts led to the removal of the project director shortly before project closure . Approximately US$ 2 million funds remained undisbursed at project closing due to delays in procurement . According to ICR (p. 13), delays in setting up an adequate M&E system delayed identifying and addressing bottlenecks in project activities Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The results framework did a reasonable job of assessing the project ’s impact on avian influenza with 3 outcome and 23 “intermediate outcome� indicators (though many were output indicators ), all with measurable, reasonable targets (though the framework would be useful if it reported the total number of outbreaks occurring over time rather than just in the initial year and the final year ). However, the ICR reported that (p. 22), these many indicators overburdened the local implementation capacity . And the indicators did nothing to assess the impact of the project on zoonotic diseases other than avian influenza or other infectious diseases . Data collection responsibilities for indicators was clearly assigned, mostly to the Department of Livestock Services . According to the PAD (p. 16), a team of consultants would be hired (probably through FAO) to carry out a baseline survey for the key performance indicators, collect data at a specified frequency, analyze those, and prepare baseline, mid-term and end-of-the-project impact evaluation reports . b. M&E Implementation: It is unclear whether any M&E was carried out prior to 2010. The ICR reported that (p. 13) a comprehensive Management Information System and Monitoring and Evaluation capacity was only set up after the arrival of the FAO Technical Assistance Team in October 2010, 3 years after effectiveness . After this system was in place, reports were produced on financial management, program reporting, and quarterly progress reports on project indicators . The external M&E firm collected data from 17 districts in six divisions to produce a Knowledge Attitude and Practice survey on training and public awareness, and completed baseline and impact assessment reports published in August 2012. Some of the outcome data including baselines came from an independent survey carried out by UNICEF in partnership with the Ministry of Fisheries and Livestock and other development partners . c. M&E Utilization: As an example demonstrating utilization of M&E system, the ICR reported that (p. 13), M&E system helped to identify the weak farm biosecurity situation in the country, which the original project design did not address, and which was one of the causes of continuing outbreaks in the country; as a result a new sub -component was included after mid-term review. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The Project was classified as environmental category B . An Environmental Management Plan was prepared to address potentially adverse environmental impacts and guide project implementation, particularly on safe culling, safe disposal of carcasses, biosafety at Field Disease Investigation Laboratories, biosecurity at Live Bird Markets and the pilot Biosecure Poultry Market Chain facilities . A Bio-safety consultant hired by FAO worked closely with the project team. A review of existing environmental safeguard and biosafety of the facilities between July - December 2012 indicated that structural and operational biosecurity improved substantially after project intervention . However, improving the following was still necessary : dead bird disposal practices at the Live Bird Markets, and biosafety practices in the Field Disease Investigation Laboratories scientific disposal of postmortem and laboratory wastes, personal and laboratory hygiene practices . The ICR does not report on whether the procedures under the environmental management plan were followed . b. Fiduciary Compliance: A control framework was established at the start of the project, outlining major expenditure eligibility and indicators of fraud, corruption, collusion, and coercion . Operational guidelines for accessing to the compensation fund were developed. Financial management capacity within the project coordination unit was initially weak, but improved through the course of the project . Project Audit reports were timely without major issues and the team followed the timeline for submission of financial reports . During one of the regular Procurement Post Reviews (PPR) of the project, in FY12 , three contracts amounting to $320,902 were found with significant deviations, in terms of accepting inferior quality goods by Department of Livestock Services compared to contract specifications, and not conducting proper due diligence by the Tender Evaluation Committee. These were being considered for mis -procurement at the time the ICR was written and the team subsequently reported that those contracts were finally declared as mis -procurement to be refunded by the government. Based on that a set of mitigation / improvement measures were agreed and the implementing agency took proactive actions on some of these measures including debarment of concerned suppliers, and removal of Project Director. In FY13 two contracts were found with major deviations and required further investigation, though these contracts were signed before these mitigation measures were introduced . c. Unintended Impacts (positive or negative): d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Relevance of design was modest due Satisfactory Unsatisfactory to lack of components dealing with : preparing for, controlling, and responding to possible human infections, even though this was part of the PDO. Also, under efficacy, sub-objective of preparing for, controlling, and responding to possible human infections was negligible due to lack of project outcomes. Risk to Development High High Outcome : Bank Performance : Moderately Moderately Ratings for the Quality at Entry and Satisfactory Unsatisfactory Quality of Supervision are the same as the ICR, but under the harmonized evaluation criteria these result in a moderately unsatisfactory overall Bank Performance rating, as the outcome rating is moderately unsatisfactory . Borrower Performance : Moderately Moderately Both the government and implementing Satisfactory Unsatisfactory agency performance rating was moderately unsatisfactory due to delays at various levels in implementation conflicting with the emergency nature of the project . Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR reports the following lessons : • Complex results framework may not be possible to fully implement when they overwhelm the limited local implementation capacity . In this project, there were too many complex indicators which overburdened M&E institutional capacity. • When donor coordination is weak, interventions may not be well harmonized and project designs may be unnecessarily duplicative . It is important to streamline and complement donor financing when many donors are involved. The project had to cut down significant amount of funds since other agencies also financed similar activities. • Important interventions may not be undertaken if not all relevant agencies are included as implementing partners . In this project, the Department of Forestry was not included, which meant that wild bird monitoring was not carried out, and so the extent and distribution of disease infection in wild birds remains unknown. • Failing to improve biosecurity can contribute to an inability to eradicate avian influenza . The project did not pay enough attention to this particular issue both during design and implementation . IEG also finds that: • Project objectives should be chosen so that they match the sectors of what is actually planned to be achieved under the specific project . In this case there was a disconnect between the stated objectives and the project design, because the objectives included human health response goals that were not to be achieved under this project. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was in many aspects well written, with a good narration of implementation progress and challenges . However, the following points needed attention : (i) The ICR failed to consider the disconnect between project development objectives and project design . (ii) Figures reported on project costs and financing were not clear, with conflicting amounts reported in different sections (for example in Annex 1 the total disbursements exceeded the total financing required); (iii) the section on risk to development outcome considered only the continuing risks from avian influenza, not the risks to achievements made under the project; (iv) The ICR could have provided more evidence to support some lessons drawn . a.Quality of ICR Rating : Satisfactory