ICRR 14581 Report Number : ICRR14581 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 02/02/2015 Country : Cambodia Project ID : P100084 Appraisal Actual Project Name : Avian And Human US$M ): Project Costs (US$M): 11.00 10.83 Influenza Control And Preparedness Emergency L/C Number : Loan /Credit (US$M): Loan/ US$M ): 6.00 5.83 Sector Board : Agriculture and Rural US$M): Cofinancing (US$M ): 5.00 5.00 Development Cofinanciers : Japan PHRD Grant Board Approval Date : 03/24/2008 (TF056832), Avian Closing Date : 12/31/2011 04/30/2014 Influenza Trust Funs (TF058146) Sector (s): Central government administration (49%); Agricultural extension and research (26%); Sub-national government administration (13%); Health (8%); Animal production (4%) Theme (s): Other communicable diseases (34%); Natural disaster management (33%); Rural services and infrastructure (33%) Prepared by : Reviewed by : ICR Review Group : Coordinator : Ebru Karamete Stephen Hutton Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The project development objectives stated in the Financing Agreement is (p. 5): "to assist the Recipient in implementing its Comprehensive Avian and Human Influenza (AHI) National Plan, specifically, to minimize the threat posed to humans and to the poultry industry by AHI infection, and to prepare for, control, and respond if necessary to human influenza pandemics ." The appraisal document (Technical Annex) statement of objectives is very similar (p. 8 ): "to support the implementation of the Cambodia Comprehensive AH 1 National Plan, which is designed to minimize the threat posed to humans and the poultry sector by AH 1 infection in Cambodia, and to prepare for, control, and respond if necessary to a human influenza pandemic." As per IEG’s evaluation methodology, this Review ’s assessment is based upon the formulation of the project objective in the Financing Agreement . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: A. Animal Health (Appraisal Estimate US$ 5.80 million, Actual US$ 5.90 million ). The component aimed to strengthen veterinary services to enhance the capacity to prevent, recognize and respond to avian influenza and other emergency animal diseases . This was to be achieved through (i) the strengthening and consolidation of training resources, reviews of training needs and curricula for Village Animal Health Workers (VAHW), provision of resources to support disease responses for district and provincial level veterinary services, and development of appropriate animal health laws; (ii) training in emergency vaccination and necessary resources to implement emergency responses; (iii) epidemiological studies on wild birds; (iv) the review of high-risk practices in production and marketing that facilitated the persistence and transmission of infection and appropriate measures to reduce these risks; (v) strengthening existing animal health Information, Education and Communication (IEC) programs, as well as to enhance pandemic preparedness response for animal health staff . During implementation, additional activities were added to improve biosecurity and poultry -handling behavior in pilot villages (15 villages increased to 30 villages later on). B. Human Health (Appraisal Estimate US$ 3.50 million, Actual US$ 2.64 million ). The component aimed to prevent the emergence of a human pandemic in Cambodia through the rapid identification and effective management of human cases . The component planned to include : (i) training of health care workers and the development of materials for use by health care providers and Village Health Volunteers (VHV); (ii) support operating costs associated with surveillance, investigation and response; and the transportation for those undertaking both activities in the field, as well as the procurement and distribution of medicine, medical equipment and Personal Protection Equipment (PPE); (iii) clinical training on case management and infection control as well as the identification of and support to activities necessary to ensure appropriate health care waste management and infection control; (iv) laboratory activities through the development of a national laboratory strategy and continuous education and refresher training for laboratory staff on -site (using international trainers) or at the local level using existing trainers in the country; (v) provision of equipment, specimen collection kits, laboratory consumables and reagents, and transport for specimen transport; (vi) training activities for provincial level staff to pandemic preparedness plans; (vii) activities on rapid containment of an outbreak with a novel influenza virus that has the potential to create pandemic influenza such as case management, infection control, isolation, quarantine, and equipment and supplies. Project funds planned to support pandemic preparedness plans and training for province, district and health staff, border quarantine staff, and the Rapid Response Team, as well as for military, police and local authorities. Under an October 2012 restructuring, nearly all component B activities that had not been completed were dropped from this operation and transferred to a separate Bank -financed project. The ICR did not clearly identify which activities remained in the scope of this operation . C. Inter -Ministerial Coordination for Pandemic Preparedness, Project Coordination and Management (Appraisal Estimate US$ 1.70 million, Actual US$ 2.21 million ). The component aimed to strengthen the capacity to coordinate a multisectoral response for an influenza pandemic . It included: (i) strengthening National Committee for Disaster Management (NCDM’s) capacity to oversee this cooperation through training and staffing; (ii) NCDM preparing and testing a Pandemic Preparedness, Response, and Recovery Program (PPRRP); (iii) designing and setting up NCDM emergency management information system (MIS) and an M&E system (for the overall national PPRRP); (iv) integrating pandemic preparedness, response and recovery activities into the existing Community Based Disaster Risk Reduction Programs (CBDRM); and (v) as the responsible body for coordinating and overseeing institutional and implementation arrangements for the project as a whole, supporting NCDM on these functions . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : At appraisal, Project costs were expected to be US$ 11 million. The actual project cost was US$ 10.83 million. However, it should be noted that the actual expenditures did not cover the full set of planned activities, as under the 2012 restructuring human health activities were moved to a separate project but the overall financing of the avian influenza operation was not reduced . Financing : The project was to be financed by an IDA grant of US$ 6 million and a US$ 2 million grant from the Avian and Human Influenza Trust Fund, and a Japanese PHRD Grant of US$ 3 million. Actual disbursements were $ 5.83 million from the IDA grant, US$ 3 million from PHRD Grant and US$ 2 million from the AHIF grant. Several reallocations were made, the last one (October 18, 2012) was due to transfer of Component B activities to another project and therefore, reallocate the related funds to Component A and B . This is unusual in that the overall funding of this operation was not reduced, and the other operation was expected to fund the transferred activities without additional financing . There was no borrower contribution . Dates : A project paper in October 2012 restructured the project, noting that over January -July nearly all activities from the Human Health component had been transferred to the Second Health Sector Support Program Pooled Fund project, which included activities on communicable diseases including H 5N1 and other emerging diseases. The stated rationale was to reduce fragmentation of World Bank financing to the human health sector . The project closing date of December 31, 2011 was extended twice (on December 28, 2011 and June 26, 2013) for a 28 months to April 30, 2014. due to delays stemming from lack of familiarity by implementing agencies to Bank procedures, a 15 month delay in signing contracts with FAO and WHO due to technical and legal issues, and expansion of the program to additional villages. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High Highly pathogenic avian influenza posed a significant threat to humans and the poultry sector in Cambodia . Cambodia is situated between two large poultry producing countries (Thailand and Vietnam) where outbreaks occurred and so the spread of the disease to Cambodia was likely . The project was directly responding to outbreaks in poultry that had been detected in Cambodia in 2003, and outbreaks repeated in 2005 and 2006 that had severe impact on smallholder farms that raised majority of the country's poultry in subsistence conditions . The poultry sector remained vulnerable, with roughly 17 million chickens and 7 million ducks constituting an important source of protein and livelihoods, especially in rural areas where the majority of households kept poultry . Outbreaks in poultry posed a serious threat to the humans through direct infection from birds; four fatal human cases were reported in 2005, two fatal cases in 2006 and another fatal case in 2007. The repeated outbreaks and loss of human life and livelihoods were due to an inadequate disease surveillance system and limited capacity to control the disease . In addition, Cambodia was vulnerable to an influenza pandemic in humans because of weak capacity for disease surveillance, disease control and prevention, outbreak investigation and emergency response . The project objectives were not specifically relevant to the Country Assistance strategy at the time of approval, but rather were responding to a specific emergency and was consistent with the objectives of the Global Program for Avian Influenza and Human Pandemic Preparedness and Response and the Bank ’s policy of assisting countries in responding to emergencies. Furthermore, the objectives were also directly linked to the National Avian Influenza and Human Pandemic Influenza Preparedness and Response Plan that was approved in April 2006. There was no active Country Assistance Strategy at the time of project closing, but the objectives remained relevant as the threat continued with ongoing outbreaks . The project objectives were relevant to the Cambodia Comprehensive National Plan on Avian and Human Influenza that was prepared in 2005 to address the outbreaks of avian influenza H 5N1 in poultry and sporadic human infections in all neighboring countries . The program included a multi-sectoral approach including animal health, human health, communications and pandemic preparedness to be supported by various international agencies including the World Bank (ICR p. 6). b. Relevance of Design: Substantial Overall the results framework presented a logical causal chain between project activities and expected attainment of the objective. Under the animal health component, strengthening veterinary services, particularly village animal health workers training on AI monitoring, reporting and control were important and would help raise awareness not only among animal health workers as well as small farmers served by these health workers and would help improved detection and control of the disease . The human health component included activities for surveillance, diagnosis and response to outbreaks by providing medical, personal protection and laboratory equipment as well as trying to enhance pandemic preparedness response for the health sector . The third component tried to improve inter-ministerial cooperation for pandemic influenza preparedness . As mentioned in Section 3a, the project was part of an overall program that was funded by the Government and other donors. The project team subsequently informed that activities such as public awareness and communications were carried out by other donor projects . The project arguably could have done more to improve biosecurity on smallholder farms, since most poultry are held on smallholders who are very poor and in need of awareness, training, and funds, and are on farms with weak biosecurity . During implementation some activities were added to address this gap, but only on a pilot basis . Also, the project did not include compensation for the smallholders to cover the losses from culling of infected poultry, although this is a critical factor in encouraging disease reporting by farmers and thus the ability to identify and contain outbreaks . The ICR reported that the government was reluctant to include that aspect. The pilot programs aimed at using other means to encourage community disease reporting, but these came late in the project and were implemented only in 30 pilot villages, as the project funds would not be sufficient to make improvements for the 1.9 million smallholder and village producers . 4. Achievement of Objectives (Efficacy): The project had two objectives : (i) minimize the threat posed to humans and to the poultry industry by AHI infection; (ii) prepare for, control, and respond if necessary to human influenza pandemics . i) Minimize the threat posed to humans and to the poultry industry by AHI infection, Substantial . Outputs : • 2122 Village Animal Health Workers (VAHW) were trained on Avian and Human Influenza surveillance (AHI) using the revised curriculum. Refresher courses were provided to existing VAHWs on AHI . New training materials and kits with basic equipment for VAHWs were provided to all those trained and certified . About 83% of trainees are providing disease reports regularly . • 125 district vets were trained on emergency vaccination (achieving the target). An additional 90 vets were trained under the Healthy Livestock Healthy Village, Better Life Program . • 648 farmers and 211 traders trained in basic biosecurity measures . Biosecurity booklets printed and disseminated. The ICR reported that (p. vi), 85% of farms provided with recommendations had adopted the measures, however it was not reported how this adoption was measured . • 100 percent of provincial and district Rapid Response (exceeding the target of 80 %) Teams were trained on applied epidemiology. • The project supported District Veterinary Staff via resources to travel to villages including the supply of 210 motor cycles and fuel. Provincial and Municipal Offices were also provided with vehicles to allow these staff to conduct field visits including disease investigations . • Equipment was provided to National Veterinary Research Institute laboratories (together with other donor projects). • The project supported studies on risks and risk reduction in market chains that is likely to serve as input for future efforts on restructuring of the marketing chains . • Strengthening of the national surveillance systems and response activities to outbreaks included technical assistance, information technology support for surveillance, funding of Ministry of Health ’s Rapid Response Teams (RRT) for emergency outbreak responses, training and feedback sessions by RRT for improving outbreak responses, and RRT workshop in four provinces . Outcomes : The ICR did not provide evidence that the threat to humans and poultry industry was minimized, but it is not clear how this could have been done . However, there was sufficient evidence that the systems and institutions were strengthened to detect, diagnose and respond to the AHI disease, which would contribute to the minimizing of the threat; therefore the achievement of this objective is rated as substantial . • There was a fourfold increase in the number of suspected cases of avian influenza in poultry reported compared to 2009 (180 in 2013, 141 in 2012, 54 in 2011, 48 in 2010 and 42 in 2009) (the achievement was less than the target of 200 reports). This is likely evidence of improved disease reporting, but could also be evidence of increased disease prevalence . The ICR also stated that (p. 18): "....poultry disease reporting is still imperfect...Possible human H5N1 cases are not always being identified at the local level quickly enough to save lives, largely because these cases are being treated first by private sector health care workers and the early signs of this disease in humans mimic other common diseases ". • There were 8 human cases identified in 2011, all of which were fatal (which suggests that there were additional non-fatal cases that were not identified ). The number of new identified cases in humans increased from 21 in 2005 to 35 in 2012. (ICR p. 18). This showed that the disease was continuing to occur in poultry without being reported and communicated messages were not very effective . (ICR p. 15). Also, Cambodia accounted for two thirds of the world’s human H5N1 cases in 2013 and overall 30% of reported global cases from 2011 to June 2014. (ICR p.18). The lack of a compensation system for poultry culling may have contributed to weaknesses in disease reporting. However, the ICR argued that (p. 18) although the number of cases increased between 2005 and 2012, the proportion of fatalities declined (from 91% to 51%, which is lower than the global fatality average of 59 %), this is partially due to improved detection ability as a result of better testing, as well as earlier treatment of the disease . • 439 (96%) out of 465 suspected HPAI cases were investigated within 24 hours (exceeded the target of 90 %). • 142 (82%) cases typical of AI had samples collected out of 174 investigated (exceeded the target of 80 %). • All samples collected were transported and tested in National Veterinary Research Institute . • 100 percent of identified clusters of patients with clinical symptoms of influenza, were closely related in time and place followed by field investigation within 24 hours. The ICR reported that (p. vi) Rapid Response Teams respond to all within 24 hour. People in same household & village with H 5N1 case were followed up. Timely sharing of information was carried out through International Health Regulation mechanism . Most outbreak detection were made from clinical cases and some from sentinel surveillance . • The ICR claims that (p. 17) on the human health side, rapid response teams from the human health sector ensured that there was no onward human to human transmission of avian influenza virus . So far, no evidence of human to human transmission has been detected in Cambodia . However, it is not possible to determine whether human to human transmission would have occurred in the absence of the project . • No cases of avian influenza were detected in periurban commercial poultry .30 villages adopting two or more disease preventive measures recommended under the "Healthy Livestock, Healthy Village, Better Life " scheme (the indicator was included with the project restructuring in October 2012). ii) Prepare for, control, and respond if necessary to human influenza pandemics, Modest ii) (ii . Outputs : • 465 hospital staff/clinicians were trained in case management, infection control, and waste management . 170 staff also got refreshment training . Antiviral drugs were purchased and are now available at Sub /National hospitals . • The Ministry of Health finalized a Pandemic Preparedness Plan, Rapid Containment Plan, and Risk Communication Plan in June 2013, with support from the Project. • A Pandemic Preparedness Response and Recovery Program was developed and tested in targeted in nine high risk provinces (although with some delay: six in Year 4 instead of Year 2 of the project). • Emergency MIS and M&E systems were designed, developed and established (although with some delay: in Year 3 instead of Year 1 of the project). Outcomes : The ICR did not provide sufficient evidence that this objective was achieved, therefore the rating is modest . • The ICR reported that as a result of the project, testing was extended to a larger number of laboratories and about 70 tests per week are conducted for H 5N1 as part of active surveillance programs, and this was not available before the project. • The number sites reporting on influenza -like illness (ILI) was increased from 6 to10; the percentage of sites submitting their reports on time increased from 30% to 100%. Severe Acute Respiratory Infection detection 4 sites operated to get accurate data for seasonal influenza& new Pandemic Influenza strains . • The 2009-10 influenza pandemic occurred before most project activities had taken place and so was not affected by the project. All H5N1 cases were reported officially to WHO and OIE as soon as they are identified, in line with international obligations. These reports are available to the general public 5. Efficiency: Modest It is plausible that the project may have had significant economic benefits, but it is extremely difficult to conduct a meaningful economic analysis. The appraisal document presented (p. 16) an economic rate of return calculation, noting that analysis was difficult due to the inherent uncertainties about the numbers of humans and poultry that might be affected by the disease and the efficacy of the control measures employed . The analysis considers the impact of the government’s full avian influenza program (of which the Bank financed roughly 1/3) on an influenza pandemic in humans, and on outbreaks of avian influenza in poultry . The likely impact of an influenza pandemic for without project situation was derived using World Health Organization estimates : (a) 30 percent of unprotected humans were likely to be affected, 10 percent would require hospitalization and 1.5 percent of human cases would be fatal. It does not identify the probability with which a pandemic is assumed to occur (and seemingly implies that this is 100%). For the with project scenario, major economic outcomes were modeled based on protection rates of 30, 80 and 100 percent However, it is important to note that it is overly optimistic to assume that the outputs of this project could have reduced the attack rate of a pandemic by 30-100%The scenario of 80 percent protection resulted in an estimated 50,400 human lives saved and 420,000 hospitalizations averted. The scenario of protection rate of 30 percent estimated to save 18,900 lives. The net economic effect of benefits ranged from US$ 70.6 million to US$ 165.1 million using very conservative estimates of the value of a human life . The ICR did not estimate an economic rate of return and mainly used the appraisal document ’s estimations in order to present cost benefit results for the government ’s program. The ICR argued that (p.20), it is not possible to quantify the extent to which the measures implemented reduced the likelihood of emergence of a pandemic strain of virus . The appraisal estimate for the direct costs of a severe human influenza pandemic to Cambodia were estimated to be $195 million and the ICR argued that (p. 40) an early response to a such a pandemic could reduce this cost significantly and the total budget for activities under pandemic preparedness represented only 1.1% of the estimated cost of such a severe pandemic . Similarly, the investments in the health sector would be recovered if early identification and treatment of cases reduced hospitalization rates in a severe pandemic by only 2% and if survival of patients infected with the virus increased by 1.3%, as a result of investments in enhanced treatment protocols and isolation facilities in hospitals then the investments in to the Health Sector . In terms of the overall effects of avian influenza for the poultry sector, the appraisal calculations assumed that 17.5% death rate, an average value per head of $1.5 for 5 million poultry, and a total cost estimated at $ 7,612,500; the appraisal document assumed that these costs would not occur due to the project . In actuality, during the project implementation, most of the avian influenza outbreaks have been localized and overall resulted in the death and destruction of fewer than 30,000 poultry in the past 5 years and assuming that only 20% of cases are reported the total cost to the Cambodian economy is still relatively low (150,000 poultry times $4.50, the value of meat poultry at market, at approximately $ 675,000 for the life of the project). It is again important to note that, it was not possible to determine the extent to which the measures implemented reduced the number of cases in poultry . The ICR also noted that (p. 20) it is not possible to estimate the extent to which actions by animal health services reduced the likelihood of the emergence of a pandemic strain of virus . In terms of operational efficiency there were serious delays in approval and implementation . The emergency project took 23 months to approve and a further 5 months to reach effectiveness . The project closing date was extended for a total of 28 months. According to the Bank team the implementation delays were a result of lack of familiarity by implementing agencies to Bank procedures and severe delays (15 months) in signing agreements with FAO and WHO, whose technical expertise was required to implement the program . The contract signing delays constituted a significant weakness in efficiency as they delayed progress on the urgently needed emergency operation . The project team stated that one source of delays was the expansion of the program to additional villages and that implementation of these activities required extra time . But due to the initial operational inefficiencies, which are particularly problematic for an emergency project, the efficiency is rated as modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: There are moderate shortcomings in the achievement of objectives and efficiency . The relevance of objectives is high given the emergence of avian influenza as a global threat and massive outbreaks in the country and limited capacity to respond. The relevance of design is substantial due to close links between project activities and the achievement of the objective despite of limited activities to control the disease in backyard poultry . The achievement of the objective, minimizing the threat posed to humans and to the poultry industry by AHI infection is substantial, due to the evidence that the systems and capacity for surveillance, diagnosis and response for the disease has improved, however t the achievement of the objective of preparing for, controlling, and responding if necessary to human influenza pandemics is modest due to lack of evidence . Efficiency is rated modest due to substantial operational delays that pose efficiency problems particularly for an emergency operation . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risks from avian influenza remains high as outbreaks continue, the virus is entrenched in the region and continues to evolve. The project helped to reduce some of the institutional risk through capacity building activities particularly for the veterinary services .. However, surveillance of avian influenza and other emerging diseases require continuous sources of funds, otherwise there is a risk that the required surveillance testing especially in the veterinary area would diminish, but the ICR notes that in the short term there may be EU funding (para 114).. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The project was developed as an emergency operation to respond to serious avian influenza outbreaks in Cambodia, and to support the Government's Comprehensive National Plan on Avian and Human Influenza in a collaborative fashion with FAO, WHO and UNDP . The Bank-financed operation was to finance a subset of activities to be delivered under the Comprehensive Plan, rather than being a fully standalone project . The project design followed in general the Global Program for Avian Influenza and Human Pandemic Preparedness and Response template. There are a number of weaknesses in quality at entry : • Despite being an emergency operation, there was excessive delay in preparation and approval of the operation-a total lapse from Project Concept Note to Board approval of about 23 months and until effectiveness of about 28 months. This is particularly serious as Cambodia had already suffered outbreaks of avian influenza in poultry and needed rapid assistance . (The delays were due to long discussions regarding the implementing agencies. The original design relied on the Borrower ’s agencies to coordinate and implement the project but due to concerns about local capacity to manage the program efficiently and transparently, the Bank advised the project should be implemented through UNDP, FAO and WHO, as executing agencies acting on behalf of the government. Although the project design was revised accordingly, the UN agencies felt that their own policies would conflict with Bank policies for procurement and financial management and intensive discussions resulted in more than a year delay in project negotiation and approval . Consequently, it was agreed that the Cambodian government would assume full responsibility for the implementation of the project with the technical support of the specialized agencies of the UN system . However, another lengthy discussion over the TOR for the UN agencies led to more delays in processing, allowing the Board approval to take place only in March 2008 and effectiveness in August 2008. Under these circumstances, UNDP decided not to participate as a provider of technical assistance). • As noted above, the original project design did little to improve biosecurity for smallholder farms . The project did not include compensation for the smallholders to cover the losses from culling of infected poultry, although this is quite effective to control the disease . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: The ICR reported that (p.26), eight supervision missions were conducted during implementation supported by relevant experts and actions plans after each mission provided guidance to the project . The team mobilized technical expertise from FAO experts on animal health sector for the poultry program when needed . The ICR argued that the task team demonstrated flexibility as reflected in the four project restructurings . It also argued that the Bank's relationship with the partners and other donors was strong, resulting in coordinated financing of activities. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: According to the ICR (p. 26), the Government was engaged throughout the project . The Ministry of Economy and Finance tried to resolve implementation issues such as payment arrangements for petrol for district staff for motorbikes supplied by the project . Also, the Government showed commitment by introducing “the Healthy Livestock, Healthy Village, Better Life � program to address biosecurity issues for small farmers and committed to expand this pilot with their own resources . Counterpart funds in the form of salaries were provided for staff in Implementing Agency staff working on the project without any issues . However although, a commitment from government to introduction of compensation was obtained at the end of the project, compensation was not implemented during implementation, despite its importance to control the disease; There were delays in the legislative package ratification . The ICR did not provide information on the consequences of this delay and the project team subsequently stated that these did not affect project implementation as they were related to pandemic preparedness and since no pandemic occurred in since 2010. These shortcomings lead to a moderately satisfactory rating . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: According to the ICR (p. 26), the three implementing agencies (IAs) (Ministry of Agriculture, Forestry and Fisheries-MAFF, Ministry of Health-MOF, National Committee for Disaster Management-NCDM) completed all of the activities in the project, followed the work plans set after supervision missions . The implementing agencies demonstrated their initiative when there were delays in signing contracts with FAO and WHO by continuing work on project activities related to work to be conducted by the two technical agencies . There were some coordination issues and delays due to lack of familiarity with the Bank procedures during the initial phases of the project but it improved throughout the project . It is also important to note that a big factor of the pre -approval delays was the concerns over whether the Cambodian agencies would be capable of implementing effectively and transparently, and that it turned out that they were . Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: As noted by the ICR (p. 10) measuring Avian Influenza projects ’ achievements is challenging due to complex results chain, the difficulty in observing outcomes, the lack of a credible counterfactual, and challenges in determining attribution. Given these difficulties, the approach of the results framework was to use intermediate outcome indicators to assess capacity improvements and the functioning of various systems and institutions on surveillance, diagnosis, and response . However, there are some indicators that could be improved . For example, the indicator, improved effectiveness of the animal and human response teams in responding to avian influenza risk, was not specific and clear enough . The ICR noted that (p. 7), this PDO indicator was qualitative and was not supported by any intermediate outcome indicators . Also, an indicator added to the project on progress in pilot villages did little to capture the impact of the project at the national level . The ICR reported that (p. 11), total case numbers in poultry or humans were not used as outcome indicators because there was no prospect of disease elimination during the course of the project and there were issues with attribution of successes and apparent “failures�. Despite not being used as outcome indicators, total case numbers in both humans and poultry were still measured and used to help guide the project. National Committee for Disaster Management (CDM) was responsible for coordinating and managing the M&E process and worked with the other IAs to ensure that there was appropriate understanding of the indicators and systems in place to record progress . Staff from the three IAs were trained to operate the system online . The project not only built the capacity of national project staff but also provincial staff on project M&E and the Emergency Management Information system. b. M&E Implementation: Appointment of M&E consultants was delayed and the framework and systems for collecting relevant data were not introduced until 2010. NCDM organized four training sessions at four provinces which covered almost all the relevant provincial line department staff IAs of 23 provinces. c. M&E Utilization: The ICR reported that (p. 10) the M&E system was used by NCDM, MOH and MAFF . Staff from the three IAs were trained to operate the system online . The project not only built the capacity of national project staff but also provincial staff on project M&E and the Emergency Management Information system (EMIS). M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The safeguard classification was not included in the PAD or in the ICR . The project team subsequently stated it was Category B. According to the PAD (p. 20), the triggered safeguards policies were the Environmental Assessment (OP4.0 1) and Indigenous Peoples (OP4.10). The PAD also stated that a d draft EMP had been prepared and would have to be completed after effectiveness as well as an Ethnic Minorities Planning Framework would need to be prepared. In terms of social safeguards, the ICR reported that (p. 11), an Indigenous People’s Planning Framework (IPPF) was developed in May, 2008. The framework was used to ensure that the communities of indigenous people could benefit from the project implementation by receiving information in local languages and in culturally appropriate and accessible ways. Based on the policies and procedures set out in the IPPF, both MAFF and MoH prepared Indigenous People’s Activities Plans (IPAPs) on an annual basis. The project tried to make sure that a wide range of the Cambodian population, including poor and indigenous people, benefit from the project . However, in terms of environmental safeguards, the ICR did not provide information on whether the EMP was completed during implementation. The project team subsequently stated that EMP was completed and implemented . The ICR reported that (p. 12), culling and disposal of poultry was undertaken based on standards developed by FAO and implemented by MAFF with the small number of carcasses of culled poultry burned and /or buried. Samples were transported to laboratories in purpose built bio -safe containers. Hospital waste management was handled in accordance with standard MoH procedures for infectious waste . b. Fiduciary Compliance: The ICR reported that (p. 13) the financial management performance rating of the Project was moderately satisfactory throughout the project implementation . Audit opinion on the audited annual financial statements was unqualified; however, the submission was late for the last two audit reports . The final audit reports for FY13 and FY14 were expected to be submitted after preparation of the ICR, so no information on those could be provided . The quarterly Interim Financial Reports submissions from MOH was generally on time whereas there were delays with the reports from MAFF and NCDM of less than a month . In terms of procurement the ICR reported that (p. 13) despite significant procurement delay, particularly regarding hiring of UN Agencies (FAO and WHO) as consultants, at the project start up, all procurement activities were successfully completed during the project extension without non -compliance to the Bank’s procurement guidelines. c. Unintended Impacts (positive or negative): d. Other: The project's support on veterinary services particularly training of the village animal health workers generated additional benefits in controlling other animal diseases such as foot -and-mouth disease, blackleg, hemorrhagic septicemia, hog cholera, fowl cholera, duck virus enteritis and Newcastle disease (ICR p. 41). Assuming that the training would result in vaccination of various numbers of animals, the savings would amount to 6.07 million per year. However, it was not clear if the training would lead to that many vaccinations, if there were existing plans for vaccination and also how the necessary would be provided . 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately Government performance is rated Satisfactory moderately satisfactory due to reluctance by the government to establish compensation payments for culled birds and delays in ratifying legislation on pandemic preparedness . Quality of ICR : Satisfactory NOTES NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR reports the following lessons : In the absence of a mechanism to compensate farmers for birds culled by government veterinary workers, disease reporting by farmers is likely to remain weak . The problem of resisting to cull the poultry will be alleviated once the smallholders can be paid appropriate compensation . prevention. Poverty is identified as a driver of certain behaviors such as Poverty hinders disease control and prevention. preparation of sick and dead poultry for food, and non-reporting of disease. Any measures that help to reduce poverty will assist in preventing zoonotic influenza (H5N1). Farmers will also need constant training and awareness raising for the necessary behavioral changes . When working with UN agencies, if procurement and financial management arrangements are not clearly implementation. Most of the established in advance at the corporate level then there may be serious delays in implementation. delay related to legal procurement issues that should be anticipated in any new projects in which these agencies are involved. IEG also finds that: Failing to improve biosecurity in small holder farms can contribute to an inability to eradicate avian influenza . However, this is also very a difficult objective to achieve, as it requires many years and significant amount of funds . The project tried to demonstrate the importance biosecurity by implementing the Healthy Livestock, Healthy Village, Better Life program as a pilot. Lengthy delays in emergency operations are particularly problematic as the situation can get worse due to inaction . Although there was no pandemic in Cambodia during the project implementation, if it occurred the project delays would pause major issues for pandemic response efforts . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was in many aspects well written, with a good narration of implementation progress, challenges and delays . Also, the M&E section was very thorough and showed some critical thinking about the M&E challenges . However, the following points needed attention : (i) The weakness of PDO indicators to measure the achievement of objective was not adequately reflected; (iii) Environmental safeguard category, EMP completion and implementation was not reported. a.Quality of ICR Rating : Satisfactory