ICRR 12218 Report Number : ICRR12218 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/24/2005 PROJ ID :P003589 Appraisal Actual Project Name :Disease Prevention (health Project Costs 162.64 152.18 7) US$M ) (US$M) Country :China Loan/ US$M ) Loan /Credit (US$M) 100.00 100.00 Sector (s):General ): public US$M ) Cofinancing (US$M) 0 0 administration sector; General education sector; Health L/C Number :C2794 FY ) Board Approval (FY) 96 Partners involved : Australian Aid Closing Date 12/31/2001 06/30/2004 Evaluator : Panel Reviewer : Group Manager : Group : Elaine Wee-Ling Ooi Ridley Nelson Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives The project had two objectives : a) prevent and control vaccine -preventable diseases to reduce morbidity, disability and mortality, especially in the poorest provinces of China, by improving the immunization services and by introducing policy measures to improve vaccine quality and ensure access of all population groups, and add new vaccines to the Expanded Program on Immunization (EPI); and b) improve the capacity of the health sector to design and implement health promotion programs to prevent and control the rising prevalence of non -communicable diseases (NCDs), sexually transmitted diseases (STDs), HIV and injury through policy and institutional improvements, and by implementing pilot programs, including surveillance, staff training, health education, and other health promotion interventions at selected locations . b. Components (or Key Conditions in the case of Adjustment Loans ): i)Immunization Component (targeted at ten of the poorest provinces )- cold chain improvement, inservice health workers training in technical and managerial aspects of immunization, improve management and surveillance (especially collection and use of immunization coverage and disease data ), promoting public awareness and generating demand for immunization services and vaccine preventable diseases (planned $113.67m; actual 137.7m); ii) Health Promotion (targeted at seven cities, HIV /AIDS control also included Yunnan province ) - instituting policy measures supporting prevention work and healthy behavior change, developing strategic health promotion plan, mobilizing intersectoral and community participation in health promotion, capacity building of local institutions to provide skills training on long term basis, improve project management, supervision and evaluation skills, establish surveillance system on behavioral risk factors for NCDs, STDs, HIV and injury, and improve use /analysis of data to plan and evaluate activities (planned $17.71m; actual $22.43m); and iii) Central (National) Component - provision of national policy guidance, coordination of project activities across provinces, cities and several national institutions, provision /coordination of technical assistance, and dissemination of project experience (planned $2.03m; actual $2.49m). c. Comments on Project Cost, Financing, Borrower Contribution, and Dates At appraisal, total project costs were estimated at $ 162.62m with IDA providing $100m. After project effectiveness, an additional $1.2m was provided by AusAID. According to the ICR, final government expenditure was at 98% or $1.25m less, while the credit was fully disbursed (actual expenditures were at 91% due to fluctuations between the SDR and the US$). It is unclear (although unlikely) in the ICR whether the AusAID Trust Fund spending was included in the final $152.18m estimate. The original closing date was extended twice : i) by 2 years due to delays in cold chain equipment procurement and inadequate counterpart financing, and ii ) by another year due to the SARS epidemic . The project closed on September 30, 2004. 3. Relevance of Objectives & Design : The project objectives were in line with the government's "8-7 Plan" aimed at reducing poverty by the year 2000 and at narrowing the widening health gap between the poorer rural and richer urban areas . The objectives were consistent with the Bank's CAS 1995 which was underpinned by 1990 analytical work, "China: Long Term Issues and Options in the Health Transition". The CAS 1995 stipulated that the health sector would i ) meet the basic health needs of the rural poor and ii) retain gains made against infectious diseases and implement innovative programs against non communicable diseases (NCDs). The objectives remained relevant throughout the project lifespan, and were also consistent with the CAS 2003. Project design appeared to be sound and was supported by stakeholder (all levels of government and external technical experts) consultations. Risks identified were appropriate with respect to adequate counterpart financing at subprovincial level, and agreement was reached on a number of policies with the participating provinces / cities to facilitate implementation. 4. Achievement of Objectives (Efficacy) : Objective a) achieved. The project strengthened implementation of - and access by vulnerable children to - the EPI program in 10 poverty provinces, contributing to lowering of infant and child mortality . With the exception of Henan and Shaanxi (where there were slight increases ), IMR and U5MR in project provinces dropped by an average of 20-45% although there are some questions about attribution . An external evaluation (by Chinese tertiary institutions ) reported an overall decline in incidence rates of EPI diseases (eg. measles, pertussis, neonatal tetanus and hepatitis B) in project areas. By project close, the incidence from measles and neonatal tetanus in seven project provinces were lower than the national average . A policy was put in place expanding the EPI program to include free Hepatitis B vaccination of newborns; and there has been widespread implementation, facilitated with funds from the Global Alliance for Vaccines and Immunization (GAVI). Project support to the cold chain and immunization procedures improved immunization coverage and safety . The ICR also reported 53% fewer Disability Adjustable Life Years (DALYs) lost due to vaccine preventable diseases than in 1995. Objective b) achieved. Modern theories and concepts in health promotion were introduced /adopted in the project cities and Yunnan, with NGOs collaborating with municipal governments to design / implement/evaluate promotional programs for healthy lifestyles . In project areas, there were gains in positive behavior change with respect to smoking, blood pressure testing, and knowledge of AIDS prevention . Smoking cessation recorded a decrease in adult smoking from 34.6% to 28.3%. Little or no gains were recorded for obesity and physical exercise . New policies are in place regulating tobacco advertising, smoking in public areas, free and anonymous HIV testing, and protocols for blood use and donation. A behavioral risk factor surveillance system has been developed and implemented . Management and professional staff benefited from 100,000 person days of training in international health promotion and disease prevention techniques . As a result of the project, central funding for NCDs has increased, a central unit has been established dedicated to NCD issues, and 31 new national level NCD demonstration projects have been implemented by the government - notable achievements given the importance of NCDs in the financial and functional sustainability of the Chinese health care system . 5. Efficiency : Neither the SAR nor ICR calculated an economic rate of return, but they provided an estimate of project benefits by calculating the Disability Adjusted Life Years (DALYs) potentially avoided by implementing the immunization (largest) component. The SAR estimated a return on project investment of $ 622 million from the immunization component. According to the ICR, while there are methodological limitations on the calculation of the DALY, it estimated project savings of 53% DALYs against the SAR's 57% DALYs. 6. M&E Design, Implementation, & Utilization: Project M&E was comprehensive, with input, output, process and outcome indicators identified at appraisal and mechanisms established to routinely measure them . There was special targeting and monitoring of the most vulnerable populations. These were supplemented with special surveys, including baseline -, midterm- and end of the project surveys, all of which were implemented . Additionally there were provisions for evaluations conducted by "external" public health research and academic groups . The project also established standardized, country wide surveillance systems for vaccine preventable diseases and for risk behavior for NCDs . While variations may exist in the data quality, utilization of the M&E data was systematic and fed into program improvement (one city provided results of the health promotion component to the media for discussion ). However to facilitate comparability of the data/results (collected from different sources ) within and across provinces, agreement needs to be reached on which data sources should be utilized . The ICR also noted that while arrangements for domestic supervision were impressive, the Bank project status reports did not systematically record progress against many of the key indicators as required. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): The project contributed to sensitizing and engendering greater commitment by senior policy makers to NCDs . However there remains the issue of the long term financial sustainability (an issue not unique to the project but relevant to HD projects in poor Chinese provinces /counties) of the Immunization component. After project closure it is not known to what extent the poorer provinces will be able to secure long term funding for the required renewal and upkeep of the cold chain. 8. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely There is still concern about the financial resilience in some provinces, and the impact that would have on upkeep of the cold chain, and access to immunization by the poorest. Bank Performance : Satisfactory Satisfactory However, as noted in the ICR, during supervision there was poor reporting of performance against indicators . Further, there is a period of two and half years (98-01) with only two supervision missions, which seems inadequate for such a large and complex project . This may be indicative of insufficient supervision resources. Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Immunization is a public good for which adequate central and provincial funding is required . Thus a solution needs to be found to deal with the generic problem of counterpart financing at the poorer subprovincial levels . To achieve success in health promotion, regulation and administrative orders are not likely to be enforceable unless the target population understands the value of compliance . Thus social mobilization is critical . Multiple interventions against a single risk factor, such as the approach towards tobacco control, are more likely to have an impact than single interventions . 10. Assessment Recommended? Yes No Why? There may be lessons to be gleaned from the project experience especially in health promotion that can be applied to guide future such projects. Some of these activities where NGOs and government have worked collaboratively, have been very innovative and successful. 11. Comments on Quality of ICR: The quality of the ICR is satisfactory overall . It provided a sound and realistic assessment of the project experience and outcome. The inclusion of the internal and external evaluations of the health promotion component was helpful and informative. This is a project which has a multitude of data collected from different sources . It would have been easier for the reader if the ICR had stipulated which sets of data /results it finds more reliable and why, instead of reporting them side by side. The ICR did not report on the number of staffweeks used in the project . Also there is some confusing language related to the rating of Bank supervision in para 7.2 and 7.3 which may lead the reader to believe supervision was poor overall .