Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00003974 IMPLEMENTATION COMPLETION AND RESULTS REPORT Loan Number 7899-CN ON A LOAN IN THE AMOUNT OF US$ 100 MILLION TO THE PEOPLE'S REPUBLIC OF CHINA FOR THE JILIN AGRICULTURAL PRODUCT SAFETY AND QUALITY ( P101716 ) December 18, 2017 Agriculture Global Practice East Asia And Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective December 6, 2017) Currency Unit = Renminbi (RMB) RMB 6.61 = US$1 FISCAL YEAR January 1 – December 31 Regional Vice President: Victoria Kwakwa, EAPVP Country Director: Bert Hofman, EACCF Senior Global Practice Director: Juergen Voegle, GFADR Practice Manager: Nathan M. Belete, GFA02 Task Team Leader(s): Carolina V. Figueroa-Geron, GFA02 ICR Main Contributor: Xueming Liu, FAO/CP ABBREVIATIONS AND ACRONYMS CNAS China National Accreditation Service for Conformity Assessment CPS Country Partnership Strategy EMP Environmental Management Plan FAO Food & Agriculture Organization GAP Good Agricultural Practices IPM Integrated Pest Management ISO International Organization for Standardization M&E Monitoring and Evaluation MIS Management Information System PDO/PDOs Project Development Objective/s POCAD Provincial Office for Comprehensive Agricultural Development PIU Project Implementing Unit PPMO Provincial Project Management Office PRC Peoples’ Republic of China TABLE OF CONTENTS DATA SHEET ........................................................................................................................... I I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 1 A. CONTEXT AT APPRAISAL .........................................................................................................1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................5 II. OUTCOME ...................................................................................................................... 5 A. RELEVANCE OF PDOs ...........................................................................................................5 B. ACHIEVEMENT OF PDOs (EFFICACY) .....................................................................................6 C. EFFICIENCY ........................................................................................................................ 10 D. JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................. 11 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ........................................................................ 11 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 13 A. KEY FACTORS DURING PREPARATION ................................................................................ 13 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 13 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 15 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 15 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 15 C. BANK PERFORMANCE ........................................................................................................... 18 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 19 V. LESSONS AND RECOMMENDATIONS ............................................................................. 19 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 21 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 32 ANNEX 3. PROJECT COSTBY COMPONENT ............................................................................ 34 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 35 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 38 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 41 ANNEX 7: REPORT ON THE ESTABLISHMENT OF RISK BASED MONITORING SYSTEM IN JILIN PROVINCE (PREPARED BY THE JILIN PROVINCIAL AGRICULTURAL COMMISSION). ................. 42 ANNEX 8. ASSESSMENT BY SUB-COMPONENT TO THE ACHIEVEMENT OF PDOS .................... 46 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name JILIN AGRICULTURAL PRODUCT SAFETY AND QUALITY ( P101716 P101716 ) Country Financing Instrument China Specific Investment Loan Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Jilin Provincial Office of Comprehensive Agriculture PEOPLE'S REPUBLIC OF CHINA Development Project Development Objective (PDO) Original PDO The objectives of the Project are to assist Jilin Province in improving its agricultural product quality and reducing agricultural product safety risks through: (i) introducing good agricultural practices, (ii) improving the implementation of agricultural product safety related regulations, and (iii) strengthening the agricultural product safety monitoring system. i The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 100,000,000 81,109,301 81,109,301 IBRD-78990 Total 100,000,000 81,109,301 81,109,301 Non-World Bank Financing Borrower 42,560,000 44,570,000 0 Total 42,560,000 44,570,000 0 Total Project Cost 142,560,000 125,679,301 81,109,301 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 13-May-2010 02-Sep-2010 18-Jun-2013 30-Jun-2016 30-Jun-2017 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 17-Feb-2015 37.67 Reallocation between Disbursement Categories 25-Nov-2015 49.10 Change in Disbursements Arrangements 09-Jun-2016 56.96 Change in Loan Closing Date(s) Reallocation between Disbursement Categories KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Substantial ii The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 16-Apr-2011 Satisfactory Satisfactory 8.25 02 04-Dec-2011 Satisfactory Satisfactory 8.25 03 20-Dec-2012 Satisfactory Moderately Satisfactory 8.29 04 23-Jun-2013 Moderately Satisfactory Moderately Satisfactory 16.61 05 15-Oct-2013 Moderately Satisfactory Moderately Satisfactory 16.61 06 05-May-2014 Moderately Satisfactory Moderately Satisfactory 19.13 07 12-Nov-2014 Moderately Satisfactory Moderately Satisfactory 28.67 08 11-Jan-2015 Moderately Satisfactory Moderately Satisfactory 37.92 09 09-Nov-2015 Moderately Satisfactory Moderately Satisfactory 49.35 10 27-May-2016 Moderately Satisfactory Moderately Satisfactory 57.21 11 22-Dec-2016 Moderately Satisfactory Moderately Satisfactory 65.10 12 30-Jun-2017 Moderately Satisfactory Moderately Satisfactory 76.20 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 100 Agricultural Extension, Research, and Other Support 27 Activities Public Administration - Agriculture, Fishing & Forestry 59 Education 100 Tertiary Education 1 iii The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Industry, Trade and Services 100 Agricultural markets, commercialization and agri- 13 business Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 33 Finance for Development 33 Agriculture Finance 33 Urban and Rural Development 66 Rural Development 66 Rural Markets 33 Rural Infrastructure and service delivery 33 Environment and Natural Resource Management 1 Environmental policies and institutions 1 ADM STAFF Role At Approval At ICR Regional Vice President: James W. Adams Victoria Kwakwa Country Director: Klaus Rohland Bert Hofman Senior Global Practice Director: John A. Roome Juergen Voegele Practice Manager: Ede Jorge Ijjasz-Vasquez Nathan M. Belete Task Team Leader(s): Iain G. Shuker Carolina V. Figueroa-Geron ICR Contributing Author: Xueming Liu iv The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Food Safety has economic and political consequences for the People’s Republic of China (PRC). This project was initiated at a time when the Chinese economy was changing rapidly from a centrally planned to a market economy, and was becoming more integrated into the global economy with a rapid increase in exports of Chinese food products. To adapt to this new environment, food producers and the government were required to move beyond the old system of detailed production norms and heavy administrative control to a more market oriented approach. Food safety and quality were of major concern in China at that time as illustrated by the widely-publicized European Union import prohibition of Chinese honey and aquaculture products containing prohibited antibiotics and the adulteration of milk products with melamine. For the agribusiness sector, this meant more self-regulation and the use of certification systems, adoption of new technologies and improved knowledge of the cost and benefits of food safety compliance and the establishment of food quality standards and brand name recognition. The Government also had to adjust its procedures to focus on more risk based monitoring and enforcement of standards, education and training on food safety and food quality, and facilitate better private sector production practices. 2. Food quality, as opposed to the basic level of food safety, was also becoming an increasingly important issue in China with increasing urbanization and rising incomes and consumers demanding higher quality food and reliable certification systems to determine quality of food. This project explored ways to improve product certification, labeling and traceability and applied them as a way of adding value to food products. 3. The Government of the PRC made food safety a top priority and was taking clear actions to upgrade their food safety system. The Law of the People's Republic of China on Agricultural Product Quality and Safety was passed in 2006 and the Law of the People's Republic of China on Food Safety in 2009. The process of upgrading China’s food safety infrastructure, procedures and enforcement capacity to implement the Law on Agriculture Product Quality and Safety had commenced. Financial investment into agricultural product safety had historically lagged other areas of the food supply chain such as industrial food processing, retail food outlets and food exports. For this reason, this project was focused on improving the primary agricultural production segment of the food supply chain. 4. Several provinces in China expressed interest to undertake the project. However, Jilin Province was selected by the Central Government for its reputation as a “granary” for China, with the intension to upscale nationwide the good practices piloted within the Project. Jilin Province has a population of 27.34 million. It has 75.76 million mu (5 million ha) under agricultural production and is a major supplier of agricultural products to other parts of China. At the start of the project, Jilin Province did not have a system for systematically monitoring and collecting information on pathogens, organic and inorganic contaminants including pesticides and veterinary drug residues in agricultural and livestock products. Theory of Change (Results Chain) 5. The PDO formulation and results chain are clear and logical. The activities built around the 5 components aligned closely with the PDO and there was a balance between “hardware” and “software” investment and between private and public investments. The project interventions, outputs, intermediate results and the desired outcomes are directly contributing to the achievement of the desired project development objectives. The entire project and most of the individual components provided the platform by which behavioral, 1 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) attitudinal and operational practice changed among regulators, farmers, enterprises, and consumers within the value chain using a risk-based approach, when it comes to agricultural product quality and food safety risk issues. These hard and soft investments form the core of positive advances in food safety management systems which are expected to be initially demonstrated by the project and to be achieved fully over the longer term. 6. The results chain and theory of change are graphically represented in the next page: 2 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Legend: Blue - low level – activities; middle level - outputs and intermediate outcomes Green – intermediate results indicators Red/Orange – PDO indicators Total # product % ag products certifications certified? received in Jilin %of participating farmers #of product certifications Improved capacity of gov. agencies (Ag adopting GAP received by farmer groups or Committees, Livestock Bureaus and Viable business models Farmers/groups apply enterprises for the province Water Resources Bureaus) in Jilin in for high value supply good ag practices in monitoring and managing ag product chains developed and agricultural safety and quality demonstrated production #farmers visiting demo # of demo sites achieving System sites certification established % of successful sub-projects New standardized good % of farmers #Ag product quality Risk-based system of ag that demonstrate integration ag practices adopting GAP within and safety site product safety of small farmers into high one year after inspections/year implemented in 200 monitoring established value chains visiting demo sites demonstration sites Supervision capacity of responsible agencies Small farmers and Increased % of processors integrated in # technical improved targeted high value food supply standards # of farmers audiences who chains adopted successfully trained Initial sampling results and annual update have minimum based on exit exams for risk-based ag products safety understanding #of food safety labs monitoring system established of food safety Standardized good Established control and operational 1.2 TA, training, agricultural practice 1.2 Sub-grant appropriate use of Agencies use Improved capacity of equipment, notes developed and financing to agricultural chemicals, management tools to Study results gov. staff, farmers and package Labs are upgraded and Public at large is better Value addition distributed to farmer groups and product tracing manage/monitor ag Baseline improving ag processors in testing, materials perform ag product aware about ag product infrastructure farmers/groups system product safety and inventory safety and monitoring and safety tests and provide safety and quality constructed quality completed quality managing safety and certification Increased % of quality farmers who # of government staff have minimum Collection rate for sub- Computer system for safety successfully trained understanding loans is satisfactory monitoring operational based on exit exams of GAP 1.3 Pilot programs for ag 2.4 Establishing a 3.2 Training gov. staff, 4.1 Sub-loans to 1.1 Improving/ 2.1 Establishing and 2.3 Improving labs for 2.2 Developing ag product baseline inventory in farmers and processors 3.3 Public awareness processing enterprises developing local products certification for equipping agencies testing ag product 3.1 Sub-grants quality and safety areas unsuitable for ag in testing, monitoring campaigns on ag and farmer groups for agricultural product sales outlets: equipment, supervising and safety and providing for research in information systems at production due to and managing af product safety and incorporating small safety regulations and training, sub-grant managing ag certification of ag universities relevant agencies in Jilin environmental products safety and quality farmers in high value standards financing products product quality contamination quality supply chains 3 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 7. Furthermore, it should be noted that the project was designed as part of a broader strategy by the PRC in its efforts to ensure food safety. The PDO clearly states that the project will assist Jilin Province in improving its agricultural product quality and reducing agricultural product safety risks. As such, these will generate longer- term outcomes beyond the project’s closing. It addresses the project’s contribution to the borrower’s higher- level objectives for the sector, as well as the relevant PRC-World Bank Country Partnership Framework priority on food safety. In this regard, the project outcome indicators: (a) the adoption rate of “non-polluting” agricultural production standards, as defined by the Ministry of Agriculture, in the province; and (b) the establishment of a risk-based agricultural product safety monitoring system, measure the efforts by both Government and the Project. The interventions supported under the Project were designed as pilot initiatives and demonstration undertakings and were part of a much larger investment and development efforts of the Government for food safety in Jilin. Project Development Objectives (PDOs) 8. The objectives of the Project are to assist Jilin Province in improving its agricultural product quality and reducing agricultural product safety risks through: (i) the introduction good agricultural practices, (ii) improving the implementation of agricultural product safety related regulations, and (iii) strengthening the agricultural product safety monitoring system. Key Expected Outcomes and Outcome Indicators 9. The PDO outcomes are two-fold: (a) improved agricultural product quality; and (b) reduced agricultural product safety risks in Jilin province. They are intertwined and mutually complement each other. 10. The Outcome Indicators are: (a) the adoption rate of “non-polluting” agricultural production standards, as defined by the Ministry of Agriculture, in the province; and (b) the establishment of a risk based agricultural product safety monitoring system. Components 11. Component 1: Promotion of Good Agriculture Practices for Agriculture Product Quality and Safety (USD 17.91 million at appraisal; USD 14.76 million actual costs). Major activities included (1) Developing or improving local regulations and standards for safe agricultural products and promoting standardized good agricultural practices. (2) Establishing and managing about 200-300 demonstration sites to promote the adoption of standardized good agricultural practices; and (3) Strengthening agricultural product certification systems in Jilin Province., 12. Component 2: Public Monitoring of Agricultural Product Quality and Safety (USD 68.54 million at appraisal; USD 60.81 million actual costs). Carrying out a program to strengthen the capacity of the Agricultural Committees, the Livestock Bureaus and the Water Resources Bureaus at the provincial and local levels in Jilin Province in managing and monitoring agricultural product safety and quality, including, inter alia: (1) Establishing and equipping government agencies supervising and managing agricultural product quality and safety at the provincial, city and county levels. (2) Developing agricultural product quality and safety information systems as agricultural product safety management tools at the relevant agricultural, livestock and water resources agencies in Jilin Province. (3) Constructing or renovating provincial and municipal level laboratories for testing of agricultural product safety and for certification of agricultural product quality. (4) Establishing a baseline inventory for areas in Jilin Province that are unsuitable for agricultural production because of environmental contamination. 4 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 13. Component 3: Agricultural Product Quality and Safety Applied Research, Training, and Awareness Raising (USD 13.90 million at appraisal; USD 11.34 million actual costs). Carrying out a program to increase the level of knowledge of agricultural product safety and quality in Jilin Province, including, inter alia: (1) Providing Sub- grants to selected research institutes or universities for conducting research that can be directly applied to improving agricultural product safety. (2) Providing training to government staff, farmers and agricultural processors engaged in testing, inspection, supervision, and monitoring of agricultural product safety and quality; and upgrading curricula at university departments that train students in the areas of food safety and food quality testing. (3) Conducting public awareness campaigns utilizing local television, radio, and newspaper media outlets to promote agricultural product safety amongst farmers and the public. 14. Component 4: Demonstration Models for Safe Agriculture Supply Chains (USD 28.57 million at appraisal; USD 29.10 million actual costs). Carrying out a program to develop and demonstrate viable business models for incorporating farmers into high value food supply chains, including, inter alia, provision of Sub-loans to selected private enterprises or farmer associations to develop and demonstrate models for integration of small-scale farmers into high quality, high value, and safe agricultural product supply chains. 15. Component 5: Project Management and Monitoring (USD 7.39 million at appraisal; USD 9.67 million actual costs). Strengthening the capacity of the Project implementing agencies at the provincial, municipal and county levels in Project management, financial management, procurement, social and environmental safeguards, monitoring and evaluation. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) 16. PDOs, outcome targets, PDO Indicators, and components remained unchanged throughout project implementation. 17. Reallocations between Disbursement Categories and increased disbursement ratio were effected during project implementation. 18. The project was extended for one year to ensure completion of all physical works (laboratory construction and rehabilitation), as well as to allow for at least a full year of operationalization of the laboratories, as originally envisioned. By the time the project became effective, several improvements on the national building standards were introduced. Thus, the laboratory designs were required to take these changes into consideration. These revisions improved the quality of the structural integrity of the project laboratories but at the same time, had delayed project implementation substantively, especially with the lack of in-house capacity for procurement and engineering design. The extension was also crucial to ensure that the Agriculture Quality and Safety Information System was put into place and utilized by the concerned technical agencies prior to project closing. Rationale for Changes and Their Implication on the Original Theory of Change II. OUTCOME A. RELEVANCE OF PDOs (Rating: High) Assessment of Relevance of PDOs and Rating 19. At Design: This project was designed as part of a broader strategy by the PRC to integrate into the world economy and to improve the well-being of its population. Food safety is a necessary condition for integrating into export markets and improving domestic welfare. At appraisal, the Bank supported this strategy under Pillar 1 of the then Country Partnership Strategy, “Integrating China into the World Economy”. This project was 5 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) designed to contribute to this higher-level objective by establishing in Jilin Province a comprehensive and internationally accepted risk-based approach to reducing agricultural product safety risks. It also supported Pillar 2 of the Country Partnership Strategy, “Reducing Poverty, Inequality, Social Exclusion,” by expanding the economic opportunities for small-scale farmers through the development of new technologies and institutions that would allow them to enter higher value agricultural product chains. Higher food safety and quality standards can be a barrier to market access for small-scale farmers. This project aimed to overcome this potential problem. 20. At Completion: The project remains highly relevant in the context of the latest CPS (2013 - 2016), in particular related to its Outcome 1.4: Promoting Sustainable Agriculture Practices, in which it states “concerns about food safety further hamper potential for expanding production of high value crops for both domestic and international markets”; and for that it proposes: “Testing new approaches to ensuring agricultural product quality and safety by piloting a certification process at the provincial level, with the potential to transfer lessons to other regions and segments of the food supply chain”. B. ACHIEVEMENT OF PDOs (EFFICACY) RATING: Substantial Assessment of Achievement of Each Objective/Outcome 21. The objectives of the project are to assist Jilin Province in: (a) improving its agricultural product quality and (b) reducing agricultural product safety risks through the introduction of good agricultural practices; improving the implementation of agricultural product safety related regulations; and strengthening the agricultural product safety monitoring system. Hence, the desired PDO outcomes are two-fold: (a) improved agricultural product quality; and (b) reduced agricultural product safety risks in Jilin province. They are intertwined and mutually complement one another. 22. The indicators to measure these outcomes are: (a) Percent of agricultural production adopting certified production standards (pollution free) in the province; and (b) Establishment of a risk-based agricultural product safety monitoring system in the province. Both outcome indicators measure the achievements of the much larger investments and overall efforts of the Government for food safety in Jilin, of which the project interventions were designed as pilots and demonstrations. 23. While the outcome indicators were well-chosen to reflect the achievement of the two parts of the PDO, it should be noted that on Indicator 2, the chosen indicator, is at best, an initial measure of the desired objective of reducing agricultural product safety risks. As earlier explained, this reflects the fact that this project is but a pilot yet critical part of the overall food safety risk system of Jilin province. 24. The project objectives have been fully achieved as measured by the outcome indicators (see results framework in Annex 1). a) 16.47%1agricultural production adopting certified production standards (pollution free) in the province, exceeding PAD target of 15%; and 1The data on the total output of agricultural, livestock and aquaculture production in 2016 are from the “Statistical Bulletin of National Economic and Social Development in Jilin Province in 2016”; the total agricultural production (pollution free) in 20 16 are based on statistics from the line agencies (agriculture, livestock and aqua-culture). The methodology and data collection are based on the requirements as outlined in the PAD Para. 10 (pollution free as defined by Ministry of Agriculture), and PAD Annex 3(for PDO Indicator 1, data collection instrument is national/provincial statistics). 6 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) b) Establishment of a risk-based agricultural product safety monitoring system in the province has been accomplished2 .As stated in the PAD (page 23, Annex 3 Results Framework, Component B), the agreed indicators for measuring the establishment of a risk-based system are the following: (a) Quality and Safety Information system in place; (b) Baseline risks identified; and (c) Laboratories completed and operational. Although the fulfillment of these three measures have taken place a few months after project closing (in August 2017) and with the operationalization of all laboratories firmly set by no later than June 2018 as per the action plans committed to by Jilin province, it is quite clear that the Province is well on its way to establish a risk-based safety system for its agricultural products. Details of these would be expounded below. 25. Equally important, Jilin Province, based on good practices from the Project, has invested heavily and set up the five-tier supervision mechanism at the levels of the province, municipality, county, township and village. Horizontally, it has developed the joint meeting mechanism among the three agencies of agriculture, livestock and aquaculture. In collaboration with seven provinces and regions in Northeast China and North China, it has established the inter-province linkage mechanism of agro-product quality and safety supervision. The Food Safety and Quality system implemented in Jilin Province meets the requirements of the “Food Safety Law of People’s Republic of China” which came into force on 1 October 2015, and specifically stresses the risk based monitoring system3. 26. As such, it would be more relevant to evaluate the impact of project interventions on the achievement of the PDOs. Food safety and quality were of major concern in China when this project was proposed, as illustrated by the widely-publicized European Union import prohibition of Chinese honey and aquaculture products containing prohibited antibiotics and the adulteration of milk products with melamine. New regulations including the Law of the People’s Republic of China on Agricultural Product Quality and Safety, 2006, had been promulgated to address these issues. Jilin Province, as a major supplier of agricultural products to other parts of China, was investing with World Bank assistance to meet the new regulatory challenges by strengthening its supervision and monitoring infrastructure. Despite the many operational obstacles and delays, the primary objective to improve agricultural product quality and reduce agricultural product safety risks in Jilin Province has been achieved. The primary objective to improve agricultural product quality and reduce agricultural product safety risks in Jilin Province has been achieved through the following project interventions4: I. The development and adoption of 119 standards in the arena of crop, horticulture, livestock and aquaculture production and processing; Jilin Provincial Government has bestowed “Innovation Contribution Awards” to 28 of these standards; one standard won the first prize, one second and two third prizes. The standards setting process in Jilin Province has been enhanced from one that involved using readily available literature information or guidelines with limited consultation with relevant stakeholders to an objective, evidence-based process that involves working with suitable stakeholders combined with appropriate experimental and field work to determine practices best suited to conditions in Jilin Province. This can be a good model for other provinces in China. Examples of standards developed and applied in the field include: a) Biological control using the entomopathogenic fungus Beauveria bassiana for control of corn 2 See Annex 7. The Establishment of Risk-Based Monitoring System, which gives details on the state of the monitoring system. 3 See Annex 7: “Report on the Establishment of Risk Based Monitoring System in Jilin Province”, prepared by Jilin Agricultural Commission. 4 See Annex 8. Assessment by Sub-component to the Achievement of PDOs, for more detailed assessment on contribution by sub-component to the achievement of PDOs. 7 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) borer, applied province wide in the areas of Gongzhuling City, Lishu County, Taonan City, Baicheng District, Jilin Province, Tongyu County, Zhenlai County and Liaoyuan with a very significant economic benefit of an average net profit of 63.05 yuan per mu. This technology has greatly reduced the use of insecticides, increased the number of natural enemies of insects and beneficial microorganisms in the field, and protected the ecological environment of farmland. In the future, the technology will be popularized and applied within the province with an annual output of 3 million mu, which will surely bring considerable economic and social benefits to farmers in Jilin Province; and b) Plant protection technical regulations for safe production of Ginseng; Guidelines to promote the safe use of environmentally friendly biological pesticides and low toxicity chemical pesticide has reduced ginseng pest damage rate, from more than 20% down to less than 10% in 2015, and enhanced the province's ginseng International market competitiveness and has resulted in significant social and ecological benefits. II. Enhanced competencies for agricultural product quality and safety supervision system; III. A Product Quality and Safety Monitoring Information System incorporating a product traceability system tied to a web-based platform accessible to all stakeholders; IV. Increased capacity for agricultural product safety testing in well-equipped laboratories operating in a quality-assured and accredited environment. By project closure, the Agriculture Committee, Changchun Municipality and Baishan Laboratories were fully equipped, staffed and operational. The Agriculture Committee laboratory has achieved CNAS/CL01:2006 (equivalent to ISO 17025). Changchun Municipality and Baishan laboratories are also operating to the required quality standards and are expected to be CNAS-accredited by end of December 2017. These laboratories cover a large proportion of the regulatory and enforcement testing for agricultural products in the province. A commitment letter, signed by the Director General of the Comprehensive Agriculture Development Office (POCAD) of Jilin Provincial Finance Bureau together with a plan (can be found in the project files) for achieving full operational status for all the other laboratories by June 2018 has been provided; V. A baseline survey for environmental contamination of agricultural production areas has been completed and a relationship between the farmland environmental conditions and the agricultural product quality in Jilin province has been established. It covered agricultural land in 99 towns and villages in 9 prefectures and cities, 42 counties and suburban vegetable fields in 6 counties (Yushu, Dehui, Jiutai, Gongzhuling, Nong'an and Shuangyang). Sampling sites included farmlands surrounding areas around industrial and mining enterprises and landfills. Over 2,300 soil samples, 88 irrigation water samples and 340 rice, corn and vegetable samples were collected along a geographic grid and analyzed for inorganic contaminants such as heavy metals, nitrogen, phosphorus, potassium and organic contaminants such as chlorinated and organophosphorus pesticides, polychlorinated biphenyls (PCBs) and polycyclic aromatic hydrocarbons (PAH). The results were collated in a database and soil-based thematic maps (1:250,000) created using ArcGIS 10 software, a computerized mapping tool with a GIS (Geographic Information System) functionality to capture, query, analyze and display geographic information. The maps show soil environmental quality in the sampled regions of Jilin Province. a) The main pollutants detected are cadmium, nickel, copper, arsenic, mercury, lead, DDT and polycyclic aromatic hydrocarbons (PAHs). The results show that per national environmental quality standard for arable land soil quality, most areas reached “grade II” and deemed suitable for “pollution free” production. The results of 826 soil sampling sites in corn fields showed that 8 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) the heavy metals and organic pollutants did not exceed the national secondary standard of soil environmental quality, indicating that the quality of farmland soil in corn planting area in Jilin Province is good and can be used as the organic corn production. Heavy metals and organic pollutant levels in samples from 396 irrigated paddies and 775 suburban vegetables growing sites do not exceed the national agricultural production safety standards. b) The databases together with the analytical tools have been transferred to the line agencies and shared among different functional departments. This will assist them to accurately understand the spatial distribution of environmental quality conditions of agricultural land and support development of agricultural policies in the province and enable formulation of management measures to reduce risks form inorganic and organic chemical contaminants. c) To ensure sustainability of the database and improve it, Jilin Province Agriculture Committee, in cooperation with the Provincial Department of Science and Technology, have planned additional environmental studies to monitor changes occurring due to demographic changes and increased urbanization. d) Much of the detail of this sub-component is deemed to be confidential under current national and provincial regulations. Nevertheless, it can be a good model for other parts of China and other countries. In this regard, discussions and technical exchanges have taken place between Guangdong Provincial Agriculture Committee, the State Environmental Protection Department of the Nanjing Institute of Environmental Sciences and the United Nations Office for Project Services (UNOPS). VI. Funding of research projects on topics related to environmental, chemical or microbiological risks associated with crop, livestock and aquaculture production or food processing techniques and approaches to risk analysis or hazard detection. The commissioning of the projects and evaluation of the outcomes have been reviewed by an expert group. Results are of a very high standard and many have been published including several in peer reviewed journals and some have had an impact in the field. An example is the development of probiotics for use in livestock breeding instead of using antibiotics medicated feed. New probiotics have been developed to promote substitution of antibiotics in livestock production. They have been used in large-scale pig farms to reduce feed costs, increase the survival and growth rate of pigs. The reduction in use of antibiotics also results in lower incidence of drug residues in animal products, providing a guarantee for food safety and human health. There is an increase in the economic benefits of 150 yuan per each finished pig. VII. Training of supervision officials, village units, farmers, laboratory scientists and development of university curricula strengthened capacities and spread knowledge for food safety; VIII. Public awareness activities on quality and safety of food products and good agricultural and processing practices has been carried out at Province, Municipality and County levels through media programs and newspaper articles. An online and telephone support system for farmers and consumers has been well received by farmers, food processors and consumers. Information technology and ecommerce has also had a positive impact and contributed to attitudinal and behavioral changes in food safety among regulators, farmers, enterprises, and consumers. The provincial authorities are committed to provide sustainable support for this component to ensure long-term continuation of this activity; and IX. Supporting demonstration sites and cooperatives to promote the adoption of standardized good agricultural practices by demonstrating their effectiveness in farmer fields and exposing other farmers to these practices. It has been recognized that these cooperatives can attain a price premium from the higher quality produce and use part of that premium to pay for renewal of such certifications and 9 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) other costs which are crucial to keep operating these on-farm quality assurance programs in a sustainable way. An example is the green rice specialty cooperatives located in the Korean township of Huinan County. Through WB support to provide space, office equipment, pest and disease control equipment and hired technical assistance to guide the standardization of rice production, the cooperative members have grown from an initial 26 households and 136 people to 315 households with more than 1,200 people. The area of collectively managed land has been expanded from 70 to 200 hectares. The cooperative has obtained organic certification, Good Agricultural Practice (GAP) certification and has applied for registered trademarks. Training has been provided in standardized production technologies and pest control to 101 farmers, of whom 21 were women. Since receiving organic certification, the price of organic rice produced was about 3.0 yuan per kilogram higher than that of ordinary rice, and the average household could receive an additional 1200 yuan per acre. The economic effect is obvious. In the production process, the base operates in strict accordance with the production standards, pest and disease control plans, reduces the amount of fertilizer used, and effectively controls various pests and diseases of rice. Justification of Overall Efficacy Rating (Substantial) 27. Based on the above assessment, the overall efficacy rating is Substantial. The project achievements have ensured that Jilin Province meets the requirements of the Food Safety Law of the People's Republic of China (adopted at 11th National People’s Congress, 2009 and amended at the 12th National People’s Congress, 2015). In summary, the various components and interventions under the project all worked together to achieve the PDOs on improving agricultural product quality and reducing agricultural product safety risks. The project provided the catalyst to influence the regulators and actors in the whole agriculture and food production chain to deviate from practicing ‘business as usual’ approaches, but rather, distinguishing hazards from risks. C. EFFICIENCY Assessment of Efficiency and Rating (Moderate) 28. Economic Analysis: The Project will predominantly generate long term positive economic benefits related to human health and environment, including (i) reduction in livestock mortality and long term environmental damage from the use of bad agricultural practices; (ii) reduction in long term human mortality and health costs that result from exposure to unsafe agricultural products or production practices, by consumers and producers respectively and (iii) a reduction in reputational risks, and consequent financial losses, from food safety failures. These benefits are difficult to quantify in a reasonably reliable manner. Further, ensuring food safety and reducing food risk is a “must do” case as required by the food safety law of China. 29. Therefore, the economic analysis adopted a cost effectiveness (least cost or cost minimization approach). Civil works and equipment for inspection offices, laboratories and the management information system was based on the minimum quantities identified during project preparation and appraisal to achieve the desired inspection and testing coverage. In the case of laboratories, the minimum equipment and civil works requirements were calculated based on required test sample loads for each proposed laboratory coverage area to achieve a required level of agricultural safety, based on well accepted statistical methods. 30. The cost effectiveness has been achieved through (i) the improved design to fit the specific requirements of the laboratories and enhanced building codes, contributing to the assets’ longer economic life and efficient gains; and (ii) Bank competitive bidding procurement procedures for all civil works and equipment, minimizing the 10 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) cost for the establishing an agricultural product safety monitoring and enforcement system to ensure the desired level of food safety. 31. Financial analysis: Sub-projects under Component D were selected through a financial analysis to ensure that the expected financial rate of return from these subprojects would be higher than the opportunity cost of capital. At ICR, the financial analyses for 10 out of the approved 14 sub-projects were repeated. The FIRRs range from 9- 22%, indicating most of the enterprises funded by the project are financially viable. 32. Subprojects under Sub-component A.2 (Demonstration Sites for Good Agricultural Practices) and sub- component A.3 (Pilot Programs for Certification of Agricultural Products) have promoted the adoption of standardized good agricultural practices by demonstrating their effectiveness in farmer fields and exposing other farmers to these practices. Among the 218 demonstration sites, 215 have received various food safety related certifications, and 41 of them have registered trademarks for their produce. Certification and branding strategy serves these producers to capture price premiums, which range from 15-20% for rice and corn, and 20-50% to specialized products such as indigenous mushrooms and edible roses5, showing the financial attractiveness to farmers. 33. Implementation efficiency: Although all project subcomponents (except B2 and B3) were completed on time as originally designed, project implementation efficiency was negatively impacted by the delay of lab construction, and consequently, the operationalization of the laboratories, resulting in the one-year extension. On hindsight, better designed and a focus on a network of reference laboratories could have achieved economies of scale, both in terms of equipment usage and in terms of achieving a critical mass of qualified laboratory personnel. Moreover, inefficiencies were also brought about by the Bank task team through frequent turnover and transitions. On the part of the project, there was also inadequate engineering and procurement capacity and management by the project. Undisbursed loan proceeds were mainly due to (i) a reduction in the original number of laboratories6; and (ii) cost savings generated from the competitive bidding process. However, the unspent loan could have been reallocated to other successful and productive components such as training and on-site research studies. D. JUSTIFICATION OF OVERALL OUTCOME RATING (Moderately Satisfactory) 34. Based on the evaluation above, the Project’s Relevance has been rated” High” for the project’s full convergence with the priorities in the PRC-Bank country partnership framework and strategy at both appraisal and ICR stages. Its Efficacy has been rated “Substantial “for its satisfactory achievement of the PDOs; and Efficiency has been rated “Moderate” for its long- term significant benefits and least costs for project interventions but with less than satisfactory implementation efficiency. As such, the overall outcome rating is Moderately Satisfactory for the Project. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 35. With rapid rural –urban migration during the project implementation, the involvement of women in farming 5See case studies in the government ICR report. 6One laboratory (in Siping Municipality with Agriculture Bureau) had continuing problems with a contractor which failed to deliver up to required standards while the rehabilitation of another Lab (Provincial Lab with Livestock Department in Changchun) had to be cancelled due to the compromised integrity of the original building. These were factors which were not foreseen during preparation. 11 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) has been evident in the project area. The project beneficiaries involved in farming and training activities are predominantly female. As such, women participation in the training and workshops was high (over 60%), and they have been active in full line of farming activities. Some of them have become village leaders and successful entrepreneurs in specialized crop production and marketing. Institutional Strengthening 36. The project directly contributed to institutional strengthening in food safety, particularly in relation to the establishment of risk based monitoring system and the food safe safety standards and regulations. The capacities of the POCAD as well as the line agencies were strengthened during project implementation. Training also provided for the government officials related to food safety oversight agencies; and capacities of universities in safety education were enhanced through curricula development. Mobilizing Private Sector Financing 37. The Project’s Component D on demonstration models for safe agriculture supply chains has demonstrated viable business models for incorporating farmers into high value food supply chains, to selected private enterprises or farmer associations to develop and demonstrate models for integration of small-scale farmers into high quality, high value, and safe agricultural product supply chains. The sub-projects supported under the project created 1,667 jobs; and 12,820 farmer households were linked into the market and dynamic value chains. Successful operations of these enterprises have successfully promoted the private sector (including farmers’ cooperatives) investment in the food supply chains. Poverty Reduction and Shared Prosperity 38. The project interventions covered 41 counties, among which 14 were designated poverty counties. In coordination with government poverty reduction program, up to 20,000 poverty households benefitted through their participation in GAP, product certification, integration into supply chains and training activities. Other Unintended Outcomes and Impacts 39. The results from Sub-component B.4 (Establishing a baseline inventory for agricultural production for environmental contamination) were officially transferred to the Jilin Provincial Agricultural Commission in July 2017 to support the establishment of farmland environmental quality platform and forecast system soon. This could be used as an excellent model for similar work not only in other parts of China but also in other countries. 40. Moreover, the rapid development in the use of information technology and ecommerce has had an unexpected positive impact. In fact, the awareness campaign took advantages of mobile texting and WeChat messaging, which turned out to be more efficient and had much wider coverage than originally planned. As indicated by surveys done by the monitoring agency (Jilin Agricultural University), retention rate of food safety knowledge was much higher than PAD estimation. This has contributed significantly to attitudinal and behavioral changes in food safety among regulators, farmers, enterprises, and consumers. The provincial authorities are committed to provide sustainable support for this component to ensure long-term continuation of this activity. It can also be a useful model for other provinces in China embarking on the development of their own improved food safety systems. 12 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 41. Realistic objectives: The project objectives were clearly set in line with the government strategy for food safety. The background analysis identified that investment into agricultural product safety - the initial part of the food supply chain prior to processing- had historically lagged other areas of the food supply chain such as industrial food processing, retail food outlets and food exports. The project therefore was focused on improving the primary agricultural production segment of the food supply chain, making the project objective realistic and almost at the right level of ambitiousness. 42. Appropriateness of design: The project design was clearly structured with 5 components to achieve the PDOs. The major project interventions, such as in good agricultural practices, monitoring of agricultural product safety, applied research, training and awareness raising, were complementary to each other and implemented by competent institutions in the specific sub-sectors (agriculture, livestock and aquaculture). Both public sector institutions (such as agricultural and livestock bureaus for public goods including laboratories standard setting) and private sector players (individual households and enterprises for GAP demonstration and safe agriculture supply chains) were engaged, depending the nature of the project interventions. Further, incorporating the Bank project into the bigger government efforts turned out to ensure the upscaling and sustainability of the project interventions and impact. 43. Results Framework and M&E. Overall, the Results Framework was sound with indicators aligned with operational objectives. M&E was properly designed with baselines and appropriate targets. Methodologies, frequencies, and responsibilities for data collection to evaluate the achievement of the PDOs were clearly defined. 44. Adequacy of risk and mitigation measures identification: Critical risks on project implementation and sustainability such as those related to laboratory construction and private sector involvement of food supply chain were adequately identified and corresponding mitigation measures were integrated in the project design. The overall project risk was rated Moderate after mitigation measures were identified for all risks rated either substantial or modest. The risk of quality of laboratory design was clearly identified, pointing to the fact that detailed laboratory design completed prior to project effectiveness could have poor quality designs which would cause delays. This risk turned out to be substantial during implementation, causing the project delay of one year since the legislated improvements on the national building parameters were not foreseen during preparation. Hence, the project spent a lot of time in upgrading the laboratory designs to comply with the new building standards. B. KEY FACTORS DURING IMPLEMENTATION (i) Factors Subject to Government Control 45. Support from Government Legislation: The government legislation remarkably supported the implementation of the project. The PRC had been continuously modernizing its regulatory framework, both in terms of legislation and institutions. China enacted major new Food Safety Laws in 2013 and in 2015. Particularly conducive and relevant to the achievement of the project PDOs are government efforts in establishing a risk-based agricultural product safety monitoring system. 46. The “Food Safety Law of the People's Republic of China” of 2015 stipulates that “the Health authorities of the provincial, autonomous region and municipality directly under the central government, together with the food 13 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) and drug administration, the quality supervision authorities at the same level, shall make and adjust the food safety risk monitoring plan based on the national food safety risk monitoring plan in combination with the administrative region’s actual situations,” and “the technical organizations undertaking the food safety risk monitoring should carry out their monitoring according to the food safety risk monitoring plan”. Jilin Province has made great efforts and investments in the monitoring and supervision of risk-based product quality and safety. The Project piloted and contributed to the establishment of a risk-based system in Jilin Province while the government efforts directly supported the achievement of the PDO7. 47. Within such favorable policy environment, the commitment and leadership demonstrated at various levels of government had secured sufficient human and financial resources for project implementation, and facilitated the coordination of line departments involved in the project. (ii) Factors Subject to Implementing Agency Control 48. Positive Impact of ICT technologies: Fast ICT technology development (internet, mobile phone apps, WeChat, product traceability scanning and e-commerce) during project implementation greatly facilitated the awareness campaign and publicity to forge partnership between and among the government, the enterprises and the consumers in ensuring and sustaining the food safety systems and practices. These ICT innovations complemented the effective implementation of the project interventions. 49. Delays in Laboratory Construction /Renovation: Many of the detailed construction drawings were prepared well before the start of the project implementation in 2010. During implementation, these had to be revised because of the rapidly changing national regulations for buildings, fire and environmental control and the time required to get clearances from various provincial authorities was lengthy. Changes in the design and variations in the civil works contracts caused the delays. A permanently assigned project supervision engineer would have helped to overcome many of these issues, instead of the Project relying on the review provided by the Bank during its regular supervision missions. 50. Procurement Management: A qualified procurement officer should also have been assigned to the Project throughout the duration of the project. Unfortunately, the frequent changes in personnel for procurement processing also contributed to the delays in laboratory construction/renovation. In the meantime, the Bank task team should have provided more technical assistance to the client in procurement procedures and management and had lesser transitions. 51. The big number of laboratories (originally 12) may have also contributed to the substantial delays. On hindsight, a better option would have been to focus on establishing the core central laboratories of the province. The expertise of these central laboratories could then be used to develop a network of smaller regional laboratories. Furthermore, a sufficient implementation period for the overall food safety system is needed for the introduction of new safety systems procedures and considering appropriate gestation time for risk-based systems to be in place and to be proven operational and effective. (iii) Factors outside the control of government and/or implementing entities 52. The severe and long winter makes the construction period short (only 6 months in one year), causing further 7Annex 7: Report on the Establishment of Risk Based Monitoring System in Jilin Province. 14 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) delays of lab construction and rehabilitation. 53. The high turnover of Bank task team leaders and members during the project life had caused some delays in responding to client’s queries and providing timely advice. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 54. The M&E design was in line with the theory of change, outlining how Project activities of each component would contribute to achieving the Project Development Objective. The first outcome indicator reflects well the overall achievement of the PDO on improving agricultural product quality through the introduction of good agricultural practices, improvement and enforcement of agricultural product safety regulations and strengthening of the agricultural product safety monitoring system. The second outcome indicator on the establishment of a risk- based agricultural product safety monitoring system is but an initial step to the overall PDO of reducing agricultural product safety risk, which would still be fully achieved much later after project closing. The project was designed as part of the much larger government efforts in securing food safety. As such, both Outcome Indicators are to be monitored using Government data and processes/systems. Implementation 55. Overall M&E implementation was the responsibility of the PPMO, which guided and liaised with the county PMOs to ensure timely and quality reporting through an independent external monitoring agency. Specialized agency - Jilin Agricultural University (JAU) - was contracted to undertake the project M&E. As a third party with rich knowledge in food safety research and development, JAU prepared quality M&E report following methodologies and frequencies as defined in the PAD, and maintained good working relations and communication with PMOs. Further, PMOs at all levels had designated M&E officers during the project implementation, and training on M&E procedures and methodologies were provided to them. Utilization 56. The M&E system as designed was established and extensively used by project management and supervision missions to gauge progress and to identify problems and follow-up actions. M&E data provided vital information to assess project outcomes and impact on beneficiaries during the preparation of the Implementation Completion and Results Report (ICRR). For example, the monitoring and evaluation of Indicator “percentage of participating farmers adopting GAP” enabled the PPMO to refine the size and scope for specific GAPs for higher adoption rates. The PPMO has indicated that methodologies and procedures used for M&E in the project will be adopted by domestic food safety projects in Jilin Province. Justification of Overall Rating of Quality of M&E (Substantial) 57. Overall M&E is rated “Substantial” based on (i) a generally satisfactory design with a clearly structured theory of change (the design provides for the causal links between project intervention and PDO-level outcomes); (ii) good implementation arrangement with the selection of a qualified external M&E agency; and (iii) effective utilization of M&E data to identify issues, propose solutions and refine interventions during implementation. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 58. Two environmental safeguard policies were triggered for the project, i.e., Environment Assessment (OP/BP 4.01) and Pest Management (OP 4.09). During project preparation, the Environmental Management Plans (EMPs) 15 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) were developed for each laboratory under Component B.3; and the Environmental Management Framework (EMF) was developed to support the environmental management during the implementation of Component A.2 (establishment of demonstration sites), Component D (demonstration models for safe agriculture supply chain) and Component C (Agricultural product quality and safety applied research). In addition, a Pest Management Plan was also prepared to guide the implementation of Component A.2. 59. As anticipated during project preparation, the project has contributed in a positive way to environmental sustainability and public health in relation to the improvements of agricultural product safety and quality in Jilin Province. The project was appropriately classified as Category B with limited adverse environmental impacts which were mostly related to the construction and operation of project laboratories (Component B.3) and other facilities. 60. During implementation, the project compliance with the Bank’s environmental safeguard policies was generally deemed satisfactory, as described below. 61. Environment Assessment (OP/BP 4.01). Appropriate mitigation measures have been taken during the laboratories' construction and operation to address the resulting environmental impacts based on the EMPs. Although only ten out of the twelve project laboratories were physically completed by project closing, all the exhaust gas/wastewater treatment facilities or services have been or will be available upon the laboratories’ operation, and a qualified service provider based in Changchun has been contracted by the project laboratories for the disposal of generated hazardous waste. In addition, all these laboratories have signed commitment letter to complete the domestic procedure for environmental inspection and acceptance at latest by Sep. 30 th, 2018. To guarantee the sustainable operation of project laboratories, relevant laboratory owners also committed to assign dedicated staff to be responsible for the daily O&M of installed emission and wastewater treatment facilities as part of the laboratories’’ operational plan. 62. During the implementation of Component A.2 (establishment of demonstration sites), Component D (demonstration models for safe agriculture supply chain) and Component C (Agricultural product quality and safety applied research), the Environmental Management Framework developed for the project during preparation has been applied by the PPMO to support the screening, preparation, review and approval, implementation and monitoring of identified subprojects in terms of environmental management. As reported by the PPMO, minor environmental impacts/risks (e.g. soil run-off) has been mitigated in an appropriate manner during project implementation. 63. Pest Management (OP4.09). The policy was triggered for Component A.2 in relation to the establishment of 215 demonstration sites on farmers’ field. As an integrated part of good agricultural practice, the Integrated Pest Management (IPM) technology has been promoted at the demonstration sites by following the guidelines of the Pest Management Plan. Local agricultural technologists were engaged to provide targeted and effective training to the farmers for safe and scientific pest management. 64. On compliance with fiduciary requirements, the required interim financial reporting and annual audit reports were submitted to the Bank in a timely manner and all project audits were unqualified. Supervision missions regularly reviewed project accounts and procedures to ensure compliance with fiduciary requirements. Financial management assessment was carried out during preparation and concluded that the organization, structure, skills and control systems at provincial and county levels were adequate. Financial management risks were rated as “modest” based on the assessment result. A Financial Management Manual was prepared to guide project 16 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) implementation. Bank supervision missions regularly reviewed project financial management procedures being followed in the project to ensure that Fiduciary requirements were being complied with at all levels. Although the Bank’s mission and annual project audit noted some financial management weaknesses, such as improper treatment of retention funds and incompletion accounting of project activities in some counties/cities, PPMO and related county/city PMOs took these issues seriously and remedial actions were taken to address the problems the soonest time possible. 65. On procurement, the Provincial PMO played the leading role to carry out the procurement activities and made remarkable efforts to make the project procurement in compliance with the Bank procedures. The project involved multiple city Project Implementing Units (PIU) with wide geographical spread, which brought difficulties for the Provincial PMO to manage and coordinate the overall project procurement. The Provincial PMO endeavored to monitor the performance by each PIU and to make the project procurement moving forward. The Provincial PMO was responsive to the Bank suggestions/comments and assigned designated staff to take care of day-to-day procurement transactions. The Provincial PMO focused on building the procurement capacity for itself and the PIUs at the city level. However, there were frequent substitutions of procurement staff for the PIUs at the city level and at the PPMO, which undermined the effect of the regular procurement training. 66. Lack of experience on Bank procurement policies and procedures was identified as one of the key risks at the time of appraisal. However, till the late stage of the project implementation, the situation was not mitigated for the PIUs at the city level. The post review found a few instances that the influences of local practice had not been properly rectified during the bid evaluation, i.e. the bids were rejected due to non-compliance which did not constitute substantial non-responsiveness. Although the Bank post-review identified the key issues of implementation and provided recommendations accordingly, those recommendations were not fully addressed and there were some cases that similar non-compliance re-occurred afterwards. Moreover, some important supporting documents were missing in the project archives. 67. There were quite several cases that the defects in the design and inadequate technical requirement in the bidding documents led to large value variations during the implementation. Due to the lack of knowledge and expertise, the PPMO and the PIUs failed to make timely decisions while requesting the Bank to make judgements on their behalf even for the post-review contracts. This has led to some serious delays in works implementation and disbursement. On handling of complaints, the PPMO and the PIUs were cooperative in the process of complaint handling and prepared responses to the complainants. However, it was noted that the Project has not established an efficient mechanism to handle such complaints since the handling process involved time- consuming efforts to coordinate the different entities. Despite all these difficulties however, the Project did not have any cases of misprocurement. 17 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) C. BANK PERFORMANCE Quality at Entry 68. The project is the first Bank-assisted project in food safety in China. The Bank Task Team incorporated international good practices and relevant lessons in the design of the project. The project PDO was highly relevant to strategic priorities in the 2006 CPF for China. Implementation arrangement was appropriate with POCAD coordinating the line agencies involved in specific sub-sectors (agriculture, aquaculture and livestock). The Bank team expertise covered all relevant project aspects, technical, social and environmental safeguards, fiduciary, economic aspects. Innovations in project design were forward-looking and maximized long term project impact. The Project Results Framework M&E design were generally sound. By incorporating the Bank project as part of the large efforts of government, the project also generated positive impact on poverty reduction and gender. Risk assessment was realistic and candid, and the project was thorough in the analysis of, and turned out to contribute to, policy and institutional aspects. 69. The forward looking and innovative project design in the areas of risk-based monitoring system and demonstration of good agricultural practices has contributed to the policy formulation and legislation for food safety, not only in Jilin Province but also for the rest of China. The incorporation of the Bank project into much bigger government efforts in food safety will help ensure project sustainability and facilitate the upscaling of the good practices and successes generated by the project. The awareness campaign and publicity to forge partnership between the government, the enterprises and the consumers can go a long way in ensuring and sustaining the food safety systems and practices. Quality of Supervision 70. Bank supervision missions, composed of multi-disciplinary team of international professionals and project management staff, were fielded regularly (almost every six months). The supervision teams were proactive in identifying the key implementation issues and followed up with intensified efforts (such as additional follow up supervisions and monthly joint tele-conferences with the PPMO during the project extension period) to address implementation delays, particularly in laboratory renovation/construction during the project extension period. 71. On the last year of the project, Bank missions focused on project impact and sustainability. The Bank team worked with the PPMO closely to come up with specific work plans for the major project interventions, particularly for the risk-based monitoring system (including the plan for achieving full operational status for all the laboratories). These are all in the project files and committed to by the responsible officials of Jilin province and the concerned technical agencies. The mission findings and evaluations were fair and candid (implementation progress and Progress towards Development Objectives had been rated MS, and sub- component for Construction/ Renovation of laboratories had been rated MU two months prior to project closing), highlighting issues and proposing specific follow-up actions. These focused follow ups have led to the completion of all works and the preparation of substantial and confirmed action plans to ensure operationalization and sustainability of the project-supported laboratories and agricultural quality and safety information system even beyond project closing. 72. Although the Bank Task Team had maintained good working relationship with the counterparts, the high turnover of task team members had caused some delays in responding to the Government requests for clarifications and advice. 73. The Bank supervision missions should have made more efforts to strengthen implementation support in procurement management, as the issues in procurement had cropped up during a long period of project 18 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) implementation (see section on fiduciary management). 74. The Bank team could have worked with the PPMO to adjust laboratory construction focusing on key reference laboratories, and to encourage the Project to reallocate Bank loan proceeds to other successful components such GAPs and certifications. Justification of Overall Rating of Bank Performance 75. Overall Bank performance is rated Moderately Satisfactory based on (i) satisfactory quality at entry for the innovative and forward looking project design and appropriate implementation arrangements; and (ii) overall good quality supervision to address implementation and sustainability issues; and (iii) supervision which could have been more effective with (a) less turnover of Bank task team leaders and members; and (b) more focused implementation support in procurement. D. RISK TO DEVELOPMENT OUTCOME 76. The risk to development outcome is low due to (i) favorable legislation and policy environment (Sections 48&49) will directly contribute to long term project impact in improving its agricultural product quality and reducing agricultural product safety risks; (ii) key project interventions in areas of adoption of GAP, strengthening food safety regulations and morning system have been incorporated into on- going government efforts; (iii) GAP and production certification have generated price premiums for improved production; (iv) sustainability plan for laboratories built under the project has been fully endorsed by the government; and ( v) enterprises financed to promote farmers’ involvement in the supply chain are financially viable. V. LESSONS AND RECOMMENDATIONS Lessons (positive) 77. Incorporation of the Bank project into a much bigger government effort in food safety further ensures the project sustainability and facilitates the upscaling of good practices and successes generated by the project. . 78. The forward looking and innovative project design in the areas of risk-based monitoring system and demonstration of good agricultural practices has contributed to the policy formulation and legislation for food safety, not only in Jilin Province, but also for the rest of China. 79. A conducive and supportive legislation and policy environment is key to the successful implementation and sustainability of project interventions and innovations. 80. Taking advantage of the strides in ICT and developing ICT tools and innovations during project implementation can greatly facilitate successful communication campaign and outreach on food safety issues among various stakeholders (producers, traders, private firms, consumers, public, etc.) in the whole value chain. Lessons (Negative) 81. Sub-optimal laboratory design in terms of physical structure and testing system/network impairs the efficiency and effectiveness of the food monitoring system. 82. Operationalization of a risk-based system takes time and testing/iteration. 19 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 83. High staff turnover on the part of the World Bank Task Team negatively impacted the communication with project authorities, and slowed down project management decision making and implementation. 84. It may be good to incorporate agro-product quality safety supervision into the performance management of government cadres; and organizing the authorities and agricultural agencies to sign “liability statement on agro- product quality safety objectives 85. Lack of understanding of Bank procurement policy impeded project implementation. Recommendations 86. To take advantage of economies of scale in terms of equipment usage and a critical mass of qualified laboratory personnel, focus should have been on the building, staffing and capacity building of a reference laboratory(ies), or the like. 87. Due to the need for introduction of new safety systems procedures and considering appropriate gestation time for risk-based systems to be in place and to be proven operational and effective, careful consideration should be given to the duration period of these types of projects . 88. Key project staff for technical and fiduciary aspects of the project should be assigned right at the beginning of the project and staff turnover should be managed and addressed more efficiently during project implementation. The adequacy and competence of such staff should form part of the implementation covenants under the project. 89. The use of the Bank’s PForR instrument for future food safety projects in China may be worthwhile considering. This would allow for more accountability for agreed results and more intensive use of government systems for better institutionalization, ownership and sustainability. . 20 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To assist Jilin Province to improve its agricultural product quality production adopting certified production standards (pollution free) in the province Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of agricultural Percentage 10.00 15.00 15.00 16.47 production adopting certified production standards 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 (pollution free) in the province Comments (achievements against targets): Objective/Outcome: To assist Jilin province to reduce agricultural product safety risks Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Establish a risk based Text N/A Yes Yes Yes agricultural product safety monitoring system (as 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 defined in Para. 36 of the PAD) 21 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Comments (achievements against targets): A.2 Intermediate Results Indicators Component: All of the components Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of technical Number 0.00 100.00 100.00 119.00 standards developed and adopted 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of demonstration Number 0.00 50.00 50.00 211.00 sites support under the project achieving 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 certification Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 22 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Number of farmers visiting Number 0.00 3000.00 3000.00 41700.00 demonstration sites 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): of which, 25000 are female farmers. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of participating Percentage 0.00 70.00 70.00 100.00 farmers adopting GAP 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percent of visiting farmers Percentage 0.00 15.00 15.00 90.54 adopting GAP within one year after visiting demo-sites 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of product Number 0.00 800.00 800.00 936.00 23 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) certifications received by 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 farmer groups or enterprises for the whole province Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Agricultural Number 0.00 3.00 3.00 4.00 Product and Quality and Safety Supervision Offices 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 operational Comments (achievements against targets): target: 3 provincial; 27 city; 126 county Actual: 4 provincial; 28 city; 161 county Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of agricultural Number 0.00 6000.00 6000.00 6500.00 product quality and safety site inspections per year 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 24 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Computer system to support Text N/A Yes yes yes supervision activites and risk based agro products safety 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 monitoring operational Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Initial sampling results and Text N/A Yes yes yes annual up-date for risk based agro products safety 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 monitoring system established Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of food safety Number 0.00 12.00 12.00 13.00 laboratories operational 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Formally Revised Completion 25 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Target Number of government staff Number 0.00 5000.00 5000.00 20000.00 successfully trained based on exit exams 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of farmers Number 0.00 40000.00 40000.00 72000.00 successfully trained based on sample exit exams 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 Comments (achievements against targets): of which, 43000 are female farmers. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased percentage of Percentage 0.00 5.00 5.00 10.24 farmers who have minimum understanding of GAP as 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 measured by a general survey of all farmers in media target area Comments (achievements against targets): 26 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased percentage of Percentage 0.00 5.00 5.00 11.35 targeted audiences who have minimum understanding of 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 food safety as measured by a general survey in the media target area, including similar percentage increases for minorities Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of successful sub- Percentage 0.00 70.00 70.00 100.00 projects that demonstrate the integration of small 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 farmers into high value food chains, and record lessons learned. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Collection rate for sub-loans Percentage 0.00 97.00 97.00 100.00 27 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) is satisfactory (greater than 02-Sep-2010 12-Apr-2010 30-Jun-2017 11-Aug-2017 97 percent) Comments (achievements against targets): 28 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) B. KEY OUTPUTS BY COMPONENT Objective/Outcome: To assist Jilin Province to improve its agricultural product quality production adopting certified production standards (pollution free) in the province Percent of agricultural production adopting certified production Outcome Indicator standards (pollution free) in the province 1. Comprehensive set of technical standards for safe agricultural practices developed and adapted to local conditions; Intermediate Results Indicators 1. Good agricultural practices demonstrated 2. Pilot programs for certification of agricultural products launched 3. Safe agricultural production chains demonstrated A. 1. Number of technical standards developed and adopted each year of implementation based on schedule. A.2. Number of demonstration sites supported under the project achieving certification Key Outputs by Component A.3. Number of farmers visiting demonstration sites (linked to the achievement of the Objective/Outcome 1) A.4. Percent of participating farmers adopting good agricultural practices A5. Percent of visiting farmers adopting good agricultural practices within one year after visiting demo sites 29 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) A.6 Number of product and process certifications received by farmer groups or enterprises D. 1. Percentage of successful subprojects that demonstrate the integration of small-scale farmers into high value food chains and record lessons learned. D.2. Collection rates for sub-loans is satisfactory Objective/Outcome 2: To assist Jilin province to reduce agricultural product safety risks Establish a risk based agricultural product safety monitoring system Outcome Indicator 1. Government quality and safety supervision system improved 2. Quality and safety information system established 3. Procedures and infrastructure for risk-based agro products safety monitoring and testing system established and operational Intermediate Results Indicators 4. Capacity of Government and private stakeholders in food quality and safety issue improved 5. Public awareness raised B.1. Number Ago-product Quality and Safety Supervision Offices operational B.2. Number of agricultural product safety inspections per year Key Outputs by Component (linked to the achievement of the Objective/Outcome 2) B.3. Computer system to support supervision activities and risk-based agro- products safety monitoring operational B.4. Initial sampling results and annual up-date for risk based agro products safety monitoring system established. 30 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) B.5. Number of food safety laboratories operational. C. 1. Number of government staff successfully trained based on exit exams. C.2. Number of farmers successfully trained based on sample exit exams C.3. Increased farmer understanding of good agricultural practices as measured by a general survey of all farmers in media target areas C.4. Food safety information retention rates of target audience 31 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Iain Shuker Lead Economist (TTL) Xiaolan Wang Senior Finance Specialist (Co-TTL) Josef Ernstberger Senior Economist/Costing Raj Patel Laboratory Specialist Klaus Ziller Laboratory Specialist Catherine Bessy Agricultural Product Safety Yi Dong Senior Financial Management Xiaowei Guo Senior Procurement Sp. Yuan Wang Procurement Analyst Robert O’Leary Senior Finance Officer Haiyan Wang Finance Officer Jun Zeng Social Development Spec. Feng Ji Environmental Specialist Li Li Senior Ext Affairs Cornelius van der Meer Food Safety Policy John Lamb Sr. Agribusiness Specialist Brian Bedard Sr. Livestock Specialist Steven Jaffee Sr. Food Safety Specialist Luz Dias Rios Food Safety Specialist Patria Consuelo Morente Sr. Program Assistant Minhnguyet Le Khorami Program Assistant Xiuzhen Zhang Program Assistant Niu Zijing Program Assistant Chunxiang Zhang Sr. Program Assistant Mei Wang Senior Counsel Marta Molares-Halberg Lead Council Supervision/ICR Carolina V. Figueroa-Geron Task Team Leader(s) Zheng Liu Procurement Specialist(s) Yi Dong Financial Management Specialist Rabih H. Karaky Former TTL Jun Zhao Team Member Feng Ji Safeguards Specialist Jun Zeng Safeguards Specialist Yunqing Tian Team Member 32 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Xiaodan Huang Safeguards Specialist Steven Jaffee Lead Agriculture Economist Zhefu Liu Social Development Specialist Lynn Wang Financial Management Consultant Ting Zhang Engineer, Consultant Raj Patel Food Safety Specialist/Chemist, Consultant B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY07 11.995 85,977.90 FY08 13.000 89,250.50 FY09 25.800 172,049.73 FY10 11.750 92,851.21 Total 62.55 440,129.34 Supervision/ICR FY11 9.807 57,727.01 FY12 20.651 113,011.03 FY13 14.663 97,677.73 FY14 20.702 127,326.86 FY15 11.570 74,317.64 FY16 11.374 50,687.91 FY17 12.192 53,294.27 FY18 5.817 38,429.10 Total 106.78 612,471.55 33 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 3. PROJECT COSTBY COMPONENT Amount at Closing Actual at Project Percentage of Approval Components (US$M) Approval (US$M) (US$M) Promotion of Good 14.76 82% Agricultural Practices for 17.91 Agricultural Product Quality and Safety Public Monitoring of 60.81 89% Agricultural Product Quality 68.54 and Safety Agriculture Product Quality 11.34 82% and Safety Applied Research, 13.90 Training and Awareness Raising Demonstration Models for 29.1 102% Safe Agriculture Supply 28.57 Chains Project Management and 9.67 131% 7.39 Monitoring Total 125.68 136.31 92% 34 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 4. EFFICIENCY ANALYSIS 1. The economic analysis at ICR follows the approach adopted at PAD. 2. The project is expected to have a significant positive economic benefits resulting from: (a) a reduction in long term human mortality and health costs that result from exposure to unsafe agricultural products or production practices, by consumers and producers respectively; (b) a reduction in livestock mortality and long term environmental damage from the use of bad agricultural practices; (c) a reduction in reputational risks, and consequent financial losses, from food safety failures; and (d) the social and financial benefits of increased income for small-scale farmers integrating into high value food supply chains. 3. A complete analysis of the benefits of an agricultural product safety system would require an evaluation of the underlying risks and the degree to which they are mitigated by the system. Risks are calculated by using the health and mortality rates associated with known hazards (such as pesticide or veterinary residues, environmental contaminants such as heavy metals, and microbial contaminants) and combining these with observed exposure levels (such as consumption levels of products with high levels of pesticide or veterinary residue, heavy metals or microbial contaminants). Many of the links between health and mortality due to exposure to agricultural chemicals, veterinary residues, environmental contaminants and microbial 4. Contaminants are not immediately observable and tend to be based on the results of long term medical studies. 5. As such, it is difficult to quantify the economic benefits of the Project on these aspects. In addition, food safety goes beyond the purely economic trade-offs to issues of socially acceptable minimum food standards. A single food safety scare can cost a country and producers billions of dollars in lost revenue. The 2008 Melamine incident in China where Melamine, a plastic compound, was added to milk to boost protein readings cost dairy producers and the country billions. It was the direct result of a failure in the food safety monitoring system. 6. Another example is the incident in 2002 where Japan discovered excess pesticide residues in frozen spinach from China. Vegetable imports were suspended for a period, but even after they resumed, Chinese frozen vegetable imports into Japan traded at a 25 - 40% discount because of a perception that they may not be safe. Likewise, poor animal health practices can result in the development of disease epidemics, such as avian flu or the recent HlNl (swine) flu that can also result in billions of dollars in losses. The magnitude of the losses that result from food safety and animal health failures are so large that they dwarf any reasonable investment, such as this project, that could significantly reduce the risk. Thus, the rate of return for these types of investments could be very high, but are difficult to estimate because of the large amount of uncertainty in the long-term outcomes. No major food scare incidents happened during project implementation, owing to enormous efforts made the government, which the project is a part of. 7. Given the social prerogative of establishing minimum acceptable agricultural product safety standards and the large downside risks of failing to meet these minimum standards, the economic evaluation method used to evaluate investments under this project is cost effectiveness within the constraint of providing a prescribed minimum acceptable level of agricultural product safety and within the context of Jilin province. 8. Laboratories and the management information system was based on the quantities identified during project preparation and appraisal to achieve the desired inspection and testing coverage. In the case of laboratories, the equipment and civil works requirements were calculated based on required test sample loads for each proposed 35 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) laboratory coverage area to achieve a required level of agricultural safety, based on well accepted statistical methods. The design was further improved with enhanced national building codes and changed requirements of specific laboratories. 9. Civil works and equipment for inspection offices, laboratories and the management information system was based on the quantities identified during project preparation and appraisal to achieve the desired inspection and testing coverage. In the case of laboratories, the minimum equipment and civil works requirements were calculated based on required test sample loads for each proposed laboratory coverage area to achieve a required level of agricultural safety, based on well accepted statistical methods. All contracts for civil works and equipment awarded following bank procurement procedures to minimize the cost for the establishing an agricultural product safety monitoring and enforcement system that meets the desired level of food safety. 10. The cost effectiveness has been achieved through (i) the improved design to fit the specific requirements of the laboratories and enhanced building codes, contributing to the assets’ longer economic life and efficient gains; and (ii) Bank competitive bidding procurement procedures for all civil works and equipment, minimizing the cost for the establishing an agricultural product safety monitoring and enforcement system to ensure the desired level of food safety. 11. The technical assistance expenditure such as training, food standard development, demonstrations of good agricultural practices were also subjected to Bank procurement procedures 12. Financial analysis for Demonstration Models for Safe Agriculture Supply Chains Sub-projects (component D). Fourteen (14) enterprises were selected through financial analysis to ensure that the expected financial rate of return is higher than the opportunity cost of capital. At ICR, the financial analyses for 10 out of the 14 sub-projects were repeated. The FIRRs range from 9-22%, indicating that most of the enterprises funded by the project are financially viable. 13. Subprojects under Sub-component A.2 (Demonstration Sites for Good Agricultural Practices) and sub-component A.3 (Pilot Programs for Certification of Agricultural Products), have promoted the adoption of standardized good agricultural practices by demonstrating their effectiveness in farmer fields and exposing other farmers to these practices. Among the 218 demonstration sites, 215 have received various food safety related certifications, and 41 of them have registered trademarks for their produce. Certification and branding strategy helped these producers to capture price premiums, which range from 15-20% for rice and corn, and 20-50% to specialized products such as indigenous mushrooms and edible roses, showing the financial attractiveness to farmers. 14. Fiscal impact: the fiscal impact after project completion is related to the funding of laboratory operating costs, which is vital to the sustainability of the laboratories constructed /rehabilitated under the project. The government has firmly committed that operating costs of the laboratories will be fully covered by local government budgets after project closing. 15. Implementation efficiency: Though all project subcomponents (except B2 and B3) were completed on time, as originally designed, project implementation efficiency was negatively impacted by the delay of lab construction, resulting in the one year extension. The project could have focused on designing a network of provincial reference laboratories since this could have achieved economies of scale both in terms of equipment usage and in terms of achieving a critical mass of qualified laboratory personnel. Furthermore, less Bank staff TTL turnover and strengthened procurement management could have enhanced implementation efficiency. 36 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 16. Undisbursed loan proceeds were mainly due to (i) a reduction in the original number of laboratories; and (ii) cost savings generated from the competitive bidding process. However, the unspent loan could have been reallocated to other successful and productive components. 37 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Dear Ms. Carolina V. Figueroa-Geron, First of all, please allow me, on behalf of all my colleagues of the PPMO, to express our gratitude and appreciation to you and your team for your contributions to the project, whose successful completion would have never been possible without your hard work and dedication to it. The ICR team led by you visited Jilin Province and carried out in-depth review of the project completion in August 2017, when we basically reached a consensus based on good communications between our both sides on the project completion, rating, etc. Since December 18, when we received the ICR from you, we have made a careful study and analysis of it. I am very happy to see the that ratings for project development objective relevance and the project development objective are highly satisfactory and satisfactory in the ICR, which is obviously a manifestation of our consensus in these two areas, but I think that the ratings, “Its efficacy has been rated Substantial and the overall outcome rating is Moderately Satisfactory for the Project”, are subject to further discussions. First, project extension, which, I think, constitutes a major reason for the moderately satisfactory rating of the project. I strongly agree to your remarks, “Similar projects in the future should have longer implementation period.” In the project implementation period of six years plus eight months, we both sides made concerted efforts to bring a satisfactory end to the World Bank project of so complicated contents, so extensive coverage and so many stakeholders. Of all the components of the World Bank project, laboratory building and renovation is the most complex, and it took a long time to complete so many tasks as equipment selection, staffing, skill training, accreditation and effective operation. Give that, the project implementation efficacy was satisfactory to me. You mentioned a number of adverse factors during the project implementation, such as frequent replacement of procurement staff, delayed reply to PIU consultation, cold climate to hinter construction, variation of lab design drawings, etc. However, all this serves as a testimony of satisfactory project implementation efficacy because the project was successfully completed despite so many adverse factors. Second, capital surplus, which constitutes another major reason for your moderately satisfactory rating of the project. In the first place, we proposed plans promptly for new project activities and reasonable reallocations of the capital. We submitted capital use plans for B2, B3, B4, C2, C3, upon the World Bank’s confirmation. Nevertheless, we failed to get the no objection from the World Bank for a long time, which left us with insufficient time for the new activities. Meanwhile, we solicited, reviewed and approved a numbers of production chain projects, but the World Bank only approved very few of them. In addition, in my opinion, the set targets and objectives of the project are attained with good project benefits, and capital surplus should be deemed as a good thing that benefits the nation and the people. Third, project management capacity insufficiency. The project was the first pilot project of the World Bank in China. Our provincial leadership attached great importance to it. We established project steering groups at the three levels of province, municipality and county, assigned offices and staff to track the project implementation, allocated ear-marked counterpart funds to ensure project management, set up technical teams and organizations in consultation, bid-based procurement guiding, consultation of policies, laws and regulations, financial auditing, monitoring & evaluation, etc. to offer in-time supervision, review and 38 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) find solutions to problems when they occur during the project implementation. So, there was no capacity insufficiency of the project management. Based on the above, I do not think that your rating of moderately satisfactory of the project efficacy, which further leads to your rating of moderately satisfactory of the entire project has impartially reflected the entire project situations, the project results and the project’s far-reaching influence on the future. We hope to further discuss with Ms. Carolina V. Figueroa-Geron and reach a consensus on the above points of mine. Finally, I would like to once again express our gratefulness to you and your team for your efforts to the project whose significance will emerge in the years to come. Regards, Li Xiaofeng PPMO of the Jilin Province Agricultural Product Quality and Safety Project Funded by the World Bank December 18, 2017 关于对吉林省世行贷款农产品质量安全项目（项目编号：101716）实施竣工报告的反馈意见 尊敬的卡罗琳娜 菲格罗亚-杰龙女士： 首先，请允许我代表吉林省世行项目办的全体同事，向您和您的团队多年来对项目所做的工 作表示感谢，项目的圆满完成与你们的辛勤付出密不可分。 2017 年 8 月，您带领世行项目竣工验收团来到吉林省，对项目的完成情况进行了深入考察， 我们双方就项目的完成情况、评级情况等进行了良好的沟通，并达成了基本共识。12 月 8 日，收 到了您的项目实施竣工报告后，我们进行了认真的研究和分析，在您的评价中，项目发展目标相 关性、项目发展目标的评级分别为高度满意和满意，我很高兴看到双方在这两方面的一致意见。 但是，对于效率和总体结果的评级我认为有待商榷： 第一，项目延期问题，我认为这是您对此项目评级为谨慎满意的重要原因之一。我非常赞同 您在建议中提到的观点，“未来，类似性质的项目实施期应该更长一些”。在项目建设的 6 年 8 个月里，是双方共同的努力促成建设内容庞杂，覆盖范围广泛，涉及部门和单位众多的世行项目 圆满竣工。在世行项目众多的子项目中，实验室建设是其中最为复杂的一个，如设备的选型确定、 人员队伍组建、实施技能培训、资质认证的获得、实验室的有效运行，都需要较长的时间才能完 成。鉴于此，我认为项目的实施效率是令人满意的。您也提及在实施过程中存在着世行采购人员 39 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 频繁更换、项目建设方咨询回复延迟、天气严寒不利施工、实验室设计图修改等诸多不利因素， 在这么多不利因素存在的情况下，项目仍顺利完成，这就是实施效率令人满意的一个证明。 第二，资金结余问题，这是您对项目评级为谨慎满意的另一个重要原因。首先，对于结余资 金，我们及时提出了新增项目建设计划，对资金进行合理分配。在得到了世行认同后，提交了 B2、 B3、B4、C2、C3 等项目资金使用方案。可是，在很长的一段时间里，未得到世行方面的不反对意 见，导致项目的建设时间不足。同时，我们还征集并评审通过了一批生产链建设项目，但世行仅 批复了其中极少的一部分。其次，我认为，在完成项目既定目标、内容，并获得良好项目效益基 础上，资金结余是一件利国利民的好事。 第三，项目管理能力不足问题。本项目作为世界银行在我国实施的第一个试点项目，我省给 予了高度重视，我方组建了省、市、县三级项目管理领导小组，专设办公室、专配人员监督跟踪 项目的建设情况，设置专项配套资金保障项目管理顺利进行；成立技术咨询、招标采购指导、政 策法律咨询、财务审计、监测评价等团队和机构，及时监督、检查、发现并解决项目实施过程中 可能出现的问题。为此，在项目管理上，并不存在不足情况。 基于以上事实，我认为由于项目延期、资金结余等原因，您对项目的效率评级为谨慎满意， 进而影响到项目总体评级为谨慎满意，并不能公正地反映项目的完全情况、所取得的成果以及对 未来产生的深远影响。 以上几点意见，期望能与卡罗琳娜 菲格罗亚-杰龙女士进行深入探讨并达成一致意见。 最后，我方再次向您和您的团队表示感谢，感谢你们数年来为这个有意义的项目所做的工作。 祝工作顺利 刘晓峰 吉林省世行贷款农产品质量安全项目领导小组办公室 2017 年 12 月 18 日 40 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) 1. JAU M&E Reports. 2. List of the Research Topics financed by the Project 3. List of Food Standards financed by the Project 4. List of Certifications financed by the Project 5. Food Safety Awareness Surveys by JAU. 6. Government Project Completion Report (In Chinese) 41 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Annex 7: Report on The Establishment of Risk Based Monitoring System in Jilin Province (Prepared by the Jilin Provincial Agricultural Commission). 1. Jilin Province follows the concept of “risk-based management” as advocated by the Ministry of Agriculture. This is quite in line with the Bank-assisted Project Development Objective. Jilin province is committed to establish a comprehensive and internationally accepted risk-based approach to reducing agricultural product safety risks. The focus is on prevention and control at its root by monitoring and controlling all sectors of the food chain and involving regulatory authorities at all levels to deal with both the symptoms and causes of food safety problems whilst ensuring adequate supplies of safe, quality agro-products. It has closely centered on the main objective of “striving to ensure that there is no major incident of agro-product quality and safety”, and with World Bank support, has strengthened the supervision capacities and co-ordination between the three line agencies responsible for safety and quality of agriculture, livestock and aquaculture products. 2. Jilin Province agro-product quality and safety system has been strengthened by concentrating on five key areas of standardization, testing, certification, supervision and traceability. The Province has made substantial progress in terms of standardized production, quality and safety monitoring, risk assessment and regulatory enforcement by increasing the agro-product quality and safety awareness of the agricultural agencies, agro-product producers, dealers and consumers at all levels. 3. Since 2010, the monitored compliance rate of major agro-products (livestock and aquaculture products) of Jilin Province has risen from 95% to over 97%. There has been no major incident on agro-product quality and safety. The overall safety and quality of production and consumption of agro-products remains good. Relevant line agencies have adopted the strategy of “grasping two key links at the same time”: On one hand, they have focused on standardized production to promote transformations of agricultural development to safeguard agro-product quality and safety from the production origin and on the other, they have intensified supervision to stringently enforce regulations and take enforcement action where violations of laws and regulations have occurred. They have focused to resolve outstanding problems and eradicate potential safety hazards. 4. The Province has achieved this by: (1) Strengthening institutional leadership to demarcate supervision responsibilities by promulgating the “Opinions of the General Office of the Jilin Province People’s Government on Overall Intensification of Agro-product Quality and Safety” in order to step up institutional leadership of agro- product quality and safety by clearly defining supervision responsibilities in the province; (2) Organizing the authorities and agricultural agencies at all levels to sign the “Liability Statement on Agro-product Quality and Safety Objectives”. (3) Completed the confirmation of the powers of the administrations of agro-product quality and safety, and incorporated agro-product quality and safety supervision into the performance management of the governments and the agencies. 5. Vertically, Jilin Province has set up the five-tier supervision mechanism at the levels of province, municipality, county, township and village. Horizontally, it has set up the joint meeting mechanism among the three agencies of agriculture, livestock and aquaculture. In collaboration with seven provinces and regions in Northeast China and North China, it has established the inter-province linkage mechanism of agro-product quality and safety supervision. 6. The Food Safety and Quality system implemented in Jilin Province meets the requirements of the “Food Safety Law of People’s Republic of China” that was passed by the Chinese Executive Committee 14th Meeting on 24 April 2015 and came into force on 1 October 2015. 42 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 7. On capacity building to improve the supervision system, the Jilin Food Safety Project has set up: • a five-tier supervision systems at the levels of province, municipality, county, township and village; • a four-tier testing systems of the levels of ministry, province, municipality and county; • and a three-tier certification system at the levels of the province, municipality and county. 8. The Province has promulgated the “Guiding Ideas on Building the Agro-product Quality and Safety System at the County Level” and the Jilin Province Agricultural Committee promulgated the “Guiding Ideas on Building the Agro- product Quality and Safety System at the Provincial Level”. 9. On strengthening regulations, Jilin Province has enacted complete local laws and regulations on agro-product quality and safety, launched the survey on the “Regulations of Jilin Province on Agro-product Quality and Safety” and amended the “Regulations of Jilin Province on Livestock Slaughtering Management”. It has est ablished a traceability management system, a production work log system, an emergency management system, a risk monitoring and spot-check supervision system, a risk response system, a production site exit system, an instant reporting system of rapid detection, a grid-based supervision system, a complaint reporting system, and an expert involvement system. 10. On enhancing risk-based management to eliminate hidden risks, Jilin Province has identified major points for supervision, engaged in risk management, emergency management, risk monitoring, risk assessment, discussion system of hidden risks, and special treatment of agro-product quality and safety. 11. On increasing standardization to improve safe production level, Jilin Province has established a standardization technology committee, engaged in standards innovation, organized formulation of local standards, and implemented standardized production. It has initially formed an agro-product quality and safety standard system composed of national, industrial, local and corporate standards. 12. On strengthening certification to create brands of quality and safety, Jilin Province has studied and promulgated incentive policies to overall promote certification, encourage and guide production entities to vigorously develop pollution-free, green and organic products to realize improved quality and effect of quality and safe agro- products. Jilin Province has intensified certification management and prior-report review, acceptance of applications for the titles of pollution-free, green and organic foods in the province, and onsite inspection per the requirements. It has intensified post-certification management, laying emphasis on post-certification spot-check supervision of quality and labels. It has intensified standardized constructions of raw materials bases for green food. 13. On enhancing food safety and quality with increased financial support, per the unified arrangement by the Ministry of Agriculture, Jilin Province has carried out the activity to create agro-product quality and safety counties. At present, three counties have been named “National-level County of Agro-product Quality and Safety”, and eleven counties have completed the creation of provincial-level counties of agro-product quality and safety. Ear-marked capital for agro-product quality and safety to support projects of agro-product quality and safety has been established. The financing emphasis is placed on supervision projects, pilot projects and public awareness projects. 14. On enhancing capacity through Training, Publicity and Research, Jilin Province has organized training workshops on supervision, testing, certification, and standardization within the province. On top of the training, it has 43 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) organized testing agency capacity certification and inter-laboratory comparisons. It has organized training on network applications and practical training to the information platform staff and corporate personnel in a region- based manner. It has also presented training workshops on “creation of agro-product quality and safety” at the county level. 15. Jilin Province has made use of such media as the Jilin Agricultural Net, the “Jilin Agriculture”, the agro-product quality and safety information platform, etc. to carry out publicity of agro-product quality and safety laws and regulations, safety knowledge and popular science, created the journal of “Jilin Agriculture” on agro-product quality and safety, opened up columns on agro-product quality and safety, and provided platforms for agro- product quality and safety supervising, testing and law-enforcing personnel to publicize and display their work contents to correctly guide public opinions. 16. The project financed research projects on topics covering crop and livestock production, food processing techniques and approaches to risk analysis or hazard detection. Completed projects were reviewed by an expert group and the results are of a very high standard and many have been published, including several in peer- reviewed international journals. 17. To sum up, the Province will continue to: a. actively promote industrial and corporate self-discipline, b. increase producers’ awareness of entity responsibility by strengthen their professional quality and ethics, c. establish transparency in production of quality and safe agro-products, d. accelerate operations of the laboratories of agro-product quality and safety, the information platform of our province, focus on the management of the laboratories and the platforms, expedite lab accreditation to meet the Chinese National requirements and endeavor to achieve CNAS accreditation (equivalent to ISO17025) for the all laboratories. In this regard, the Provincial Agriculture Commission Laboratory has already achieved CNAS accreditation. e. complete the networking of the information platform within the province to include municipalities and counties, f. fully utilize the functions of the information platform in terms of traceability management, information disclosure, capacity building, etc. and encourage townships and enterprises to use the information platform, and g. extend the information platform to encompass the five tiers of province, municipality, county, township and enterprise. 18. Over the past six years, the World Bank funded Jilin Province Agricultural Product Quality and Safety Project has played a key role in ensuring food safety and quality in the Province through guidance for good practices and demonstration promotion. The guidance on project conceptualization, project financing and project implementation has been of great significance for the Province. It has introduced advanced management concepts, helped to understand the development trend of agro-product quality and safety around the world, and to raise the overall agro-product quality and safety level in the Province. 19. In the future, the Province will continue to give full play to the functions and “seed” effect of the Bank-assisted Jilin Food Safety Project to ensure that the concepts introduced by will take root, germinate, bloom and bear fruits in the world as well as in China and Jilin Province. 44 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 20. It is hoped that the World Bank will disseminate China’s development concepts of “innovation, coordination, green, opening-up and sharing” to the world, so that China and all the other countries of the world will embark on a road of agriculture with efficient production, safe products, resource conservation and good environment and sustainable development. The dream of China and the dream of the world will march hand in hand to benefit humankind, benefit nature and benefit our planet. 45 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Annex 8. Assessment by Sub-component to the Achievement of PDOs Sub-component A1: Developing standards and standardized good agricultural practices 1. Jilin Provincial Quality and Technical Supervision Bureau (JQTSB) has prepared or revised and disseminated 119 standards to assist the promotion and adoption of good practices for products and/or processes in the arena of crop, horticulture, livestock and aquaculture sectors. 105 of these are new and 14 are revisions; 39 are for agricultural products, 10 for livestock, 21 for aquaculture and 49 for other issues including testing methods. Most these take the form of guidelines or recommendations for producers and processors to apply various methods for them to comply with national regulations related to safe and good agricultural practices. Others relate to methodologies for applying regulatory measures in relation to Jilin agriculture. 2. Standards developed and promulgated under the WB project have played a significant role to promote the development of agricultural standardization in Jilin Province. 28% of the agricultural standards issued by the China Quality and Technical Supervision Administration (2011 – 2014) are from the World Bank project. In 2017, the Jilin Provincial Government bestowed “Innovation Contribution Awards” to 28 standards; 1 World Bank project standard won the first prize, 1 second and 2 third prizes. 3. This sub-component has enhanced the standards setting process in Jilin Province. The earlier approach involved using readily available literature information or guidelines with limited consultation with relevant stakeholders. The new approach is an objective, evidence based process that involves working with suitable stakeholders combined with appropriate experimental and field work to determine practices best suited to conditions in Jilin Province. This can be a good model for other provinces in China. This sub-component was financed with counterpart funds provided by the Provincial Department of Finance. Sub-component A2: Demonstration sites for good agricultural practices 4. This component sought to promote the adoption of standardized good agricultural practices by demonstrating their effectiveness in farmer fields and exposing other farmers to these practices. Among the 218 demonstration sites, 215 have received various food safety related certifications, and 41 of them have registered trademarks for their produce. Undoubtedly, certification and branding strategy not only serves these producers to capture likely premium by exploring niche markets, differentiating their own products from others, and better linked to the high value markets. It also required them to develop standard operation procedures including on-farm quality assurance measures which changed small producers’ behaviors, which is a capacity building process. 5. The subcomponent laid a solid foundation for small farmers to take collective action and coordinate their farming. Moreover, small holders from adjacent areas benefited from the demonstration sites through exposure to hands- on good farming practices in terms of high-quality seedlings, technologies, production and postharvest management. The most recent survey conducted by the PPMO suggests that 47,101 farmers benefited directly from this sub component by receiving various trainings. Because of these subcomponent activities, about 37,756 households adopted standardized farming practices. On average, each household increased their annual income by RMB 3,000. In addition, 41,700 farmers visited the demonstration farms/cooperatives to learn good agricultural practices introduced by the project. The 112,982 hectares of farmland received various food safety related certifications, 6.81 million tons of produce were grown in these land, which generated RMB 160 million incomes for producers. The research institutes/universities delivered demand-driven training/technical assistance packages through interactions with cooperatives/associations, thus improving the cost-effectiveness 46 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) of such extension activities. Fifty-three universities and research institutes engaged in this service delivery to small farmers and their groups. 6. It has been recognized that these cooperatives can attain a price premium from the higher quality produce and use part of that premium to pay for renewal of such certifications and other costs which are crucial to keep operating these on-farm quality assurance programs in a sustainable way. Sub-component A3: Pilot Programs for certification of agricultural products 7. This sub-component aimed to strengthen capacities at provincial and county levels to issue certifications to farmers and firms meeting GAP or GMP standards through training and provision of equipment. Sub-grants were made to firms, farmer cooperatives and others to adopt measures which enable them to gain certification. 8. A total of 936 certifications have been issued. Bank financed certifications have enabled the cooperatives/firms to successfully explore niche markets for their higher quality products with a price premium. This has clearly demonstrated the value of these certifications and they will continue to invest in such activities. Sub-Component B1: Agricultural product quality and safety supervision system 9. The objective of this sub-component was to improve the supervision and management capabilities of the offices managing agricultural product quality and safety at province, municipality and county levels. In this regard, 31 provincial-level, 9 municipal-level and 41 county-level supervision units have been set up for education, supervision and enforcement of regulations. The activity covered agricultural inputs, production processes, agricultural technology promotion stations, production area environment supervision systems, rural energy and environmental protection stations, agricultural products certification processes, green food packaging logo supervision system and the agricultural products quality safety inspection and testing systems. The agricultural products quality safety database, supervision information platform, traceability and early warning system have been developed. It should be noted that the training for supervision staff was provided under Sub-component C2. 10. Sampling of fruits, vegetables, aquaculture and livestock products from producers, markets, slaughter houses and shops is carried out for testing of contaminants. The network of laboratories with qualified staff and highly sophisticated instrumentation funded by the project should provide an opportunity to review and modify this activity to further enhance implementation of risk-based food control strategies based on robust scientific principles. For example, “fast-tests” used for rapid screening of contaminants like pesticides, mycotoxins, antibiotics and prohibited compounds like beta agonists should be based on validated performance verified techniques to ensure there are no false-negative results and any positives are confirmed. For example, National and Provincial regulatory authorities require sampling and testing to monitor illegal use of the beta-agonist, clenbuterol in livestock and aquaculture farming. Urine samples are collected from every animal being processed at slaughter and tested using rapid immuno-chemical assays. 11. A more cost-effective approach would be to utilize statistically-based random sampling with appropriate target tissues. The use of urine as a target matrix for clenbuterol has limited value in a control program. About 2 weeks after treatment is stopped, clenbuterol levels in urine are well below detection limits. Such targeted analysis of clenbuterol using urine would also miss the illegal use of other beta-agonists, e.g. ractopamine. Now that Jilin Province laboratories are operational, multi-analyte assays together with random sampling of other tissues such 47 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) as liver or retina, could be introduced into the surveillance programs to monitor illegal use of a range of beta- agonists. Sub-component B2: Product Quality and Safety Monitoring Information System 12. The system is now operational and covers all 3 line agencies (Agriculture, Livestock and Aquaculture) and incorporates province's nine cities and 60 counties. The platform can integrate testing results from the laboratories for the supervision and enforcement officers to assess and plan risk based surveillance. The platform has some innovative features to help the supervision and enforcement officers; for example, it has a link to the Institute for the Control of Agrochemicals, Ministry of Agriculture (ICAMA) that provides real-time updated guidance on the use and application of pesticides. Such information helps regulators at all levels with information that enables efficient use and control of pesticides. 13. The use of a product tracing system was demonstrated to the ICR team at one of the enterprises. The vegetable produce was packaged and labelled with a 2-dimensional barcode (QR code) that can be scanned using a WeChat scanner to see traceability information on the product. The sustainability plan presented to the mission states that by end of 2018: 1) agricultural products from at least 100 enterprises will be registered in the platform by early 2018 and their products will be fully traceable by end of that year; 2) livestock products from 1 large slaughter house, comprising 175 farmers, 47 wholesalers and 125 retailers will be traceable; 3) aquaculture platform will register 179 pollution-free enterprises and a will develop a traceability system. Sub-component B.3: Construction / Renovation of laboratories 14. To meet Paragraph 9 of the Project Loan Agreement and meet the key objective of providing high quality analytical support to the food safety and quality regulatory system in Jilin Province, laboratories constructed or renovated under the project would be considered operational when they: 1) conform to ISO 17025: General Requirements for the Competence of Testing and Calibration of Laboratories or other suitable standard. The equivalent standard in China is CNAS/CL01:2006 and the guidelines CNAS/CL10:2012. 2) have qualified staff (at least one postgraduate level analytical chemist in each lab, and more in the provincial level laboratories); 3) have adequate budgets for the operation and management of the laboratories; and 4) initial sampling results and annual updating of risk-based agro products safety monitoring systems are in place (linked to Component B1 and B2). 15. In this regard, the Office of the Steering Group of the Jilin Province Agricultural Product Quality and Safety Project has provided a plan for achieving full operational status for all the laboratories, together with a commitment letter (in the project files), signed by Director General of the Comprehensive Agriculture Development Office (POCAD) of Jilin Provincial Finance Bureau, to allocate the necessary resources. All laboratories have adequate human resources for their current activities and have plans for additional activities and staffing. They have recruited qualified analytical chemists and all have committed operational budgets. 16. Under current Chinese National Regulations, all laboratories must have China Metrology Accreditation (CMA) 48 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) before they can report results for official regulatory programs. The central Jilin Province Agricultural Product Quality Supervision, Inspection and Testing Integrated Centre (Agriculture Committee), the Provincial Aquaculture and Changchun Municipality laboratories are CMA-accredited. Between them they cover a large part of the National and Provincial regulatory surveillance. The work program of the Aquaculture laboratory requires them to carry out a lot of analytical work in the summer. They will therefore relocate to the new site after this year’s work program is completed and will seek reaccreditation for the new site by March 2018. This is an understandable and justified management action by the Aquaculture line agency. The Baishan Laboratory is expecting final CMA accreditation by end of August 2017 and Jilin City Laboratory by end of September 2017. The remaining 5 are expected to be CMA-accredited by no later than June 2018. 17. The Agriculture Committee, Provincial Aquaculture and Changchun Municipality laboratories also have China Agri- product Testing Laboratory (CATL) Accreditation Certificates issued by the Ministry of Agriculture. Regarding China National Accreditation Service for Conformity Assessment (CNAS) to meet the ISO 17025 standard as required by the Project Loan Agreement, the Provincial Agriculture Committee laboratory has already achieved CNAS/CL01:2006 accreditation using CNAS/CL10:2012 guidelines, albeit after the closure of the project in June 2017. Changchun Municipality has also been operating to the required quality standards and awaiting a formal audit from CNAS and together with Baishan laboratory is expected to be CNAS-accredited by end of December 2017. Aquaculture, Jilin City, Liaoyuan, Yanbian and Baicheng Laboratories will be CNAS-accredited by end of 2018 with Songyuan and Tonghua to follow by end of June 2019. This is a considerable step forward in terms of a robust laboratory support system for the food safety and quality regulatory infrastructure in Jilin Province. 18. The Office of the Steering Group of the Jilin Province Agricultural Product Quality and Safety Project has provided a plan for achieving full operational status, including CNAS (ISO17025) accreditation, for all the laboratories, together with a commitment letter, signed by Director General of the Comprehensive Agriculture Development Office (POCAD) of Jilin Provincial Finance Bureau, to allocate the necessary resources to achieve this. All the laboratories have adequate human resources for their current activities and have plans for additional activities and staffing. They have recruited qualified analytical chemists and all have committed operational budgets. 19. In terms of lessons learnt, component B3 faced many unforeseen obstacles. Many of the detailed construction drawings were prepared well before the start of the project implementation in 2010. These had to be revised because of the rapidly changing national regulations for buildings, fire and environmental control and the time required to get clearances from various provincial authorities was lengthy. Many designers were also not familiar with the requirements for Jilin Province. The construction period in Jilin Province was limited because of the severe winter conditions in North East China, causing further delays. Many delays were also caused because of variations in the civil works contracts. A permanently assigned project supervision engineer would have helped to overcome many of these issues. 20. For a large project of this nature, a better option would have been to limit the initial phase to establishing core central laboratories. The expertise of these central laboratories could then be used to develop a network of smaller regional laboratories. On the introduction of modern, robust and cost-effective analytical methods for food contaminants, the laboratories face a significant constraint because many national mandatory testing standards are based on old technology. Although this situation is changing, it needs to be accelerated to allow Jilin Province to implement modern techniques and fully utilize the equipment available in the project-supported laboratories. 49 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) Sub-component B.4: Establishing a baseline inventory for agricultural production for environmental contamination. 21. This sub-component has been completed to a very high standard and could be used as an excellent model for similar work not only in other parts of China but also in other countries. Based on extensive data collection and analysis, relationship between the farmland environmental conditions and the agricultural product quality in Jilin province has been established. The research achievements have been officially transferred to the Jilin Provincial Agricultural Commission in July 2017 to support the establishment of farmland environmental quality platform and forecast system soon. 22. In cooperation with the Provincial Department of Science and Technology, additional environmental baseline studies are also planned for further data collection to improve the database. By adopting the risk-based approach, the transferred achievements will be classified and shared among different functional departments under the Commission to support the restructuring of crop farming and the development of agricultural policies in the province. Sub-Component C.1: Research on agricultural product safety issues. 23. The project financed 64 research projects on topics covering crop and livestock production, food processing techniques and approaches to risk analysis or hazard detection. 62 were completed and reviewed by an expert group. Results are of a very high standard and many have been published including several in peer-reviewed international journals. Sub-component C2: Training. 24. Training of supervision officials, village units and farmers has been completed well above the target; 13068 governmental officials, 2545 village cadres, and 52130 farmers have been trained. University curricula have also been developed but these should be updated regularly to reflect the rapid national and international technical and regulatory advances for food safety and quality. 25. The hand-on analytical training for the laboratory scientists was suspended because of delays in laboratory construction and staffing. Training these staff is vital for successful implementation of the validated analytical methods. Although limited training on the use of new instruments will be provided by instrument suppliers, more robust and detailed training on analytical techniques focused on food contaminants will be needed for qualified laboratory personnel. This should include theoretical and practical, hands-on training on analytical techniques, methods validation, quality control and assurance should be provided to qualified laboratory personnel. The laboratories are urged to organize this as soon as possible. Sub-component C3: Public Awareness. 26. Public awareness activities on quality and safety of food products and good agricultural and processing practices has been carried out at Province, Municipality and County levels through media programs and newspaper articles. 27. An online and telephone support system for farmers and consumers has been developed and is in current use. It has been well received by farmers, food processors and consumers. The system is also used by farmers to market specific produce as good quality and safe food. 50 The World Bank Jilin Agricultural Product Safety and Quality ( P101716 ) 28. The rapid development in the use of information technology and ecommerce has also had a positive impact. In fact, the awareness campaign took advantages of mobile texting and WeChat messaging, which turned out to be more efficient and had much wider coverage than original planned. As indicated by surveys done by the monitoring agency (Jilin Agricultural University), retention rate of food safety knowledge was much higher than PAD estimation. This sub-component contributed to attitudinal and behavioral changes in food safety among regulators, farmers, enterprises, and consumers. The provincial authorities are committed to provide sustainable support for this component to ensure long-term continuation of this activity. It can also be a useful model for other provinces. Component D: Demonstration Models for Safe Agriculture Supply Chains 29. Projects under this component involve food processing and farming cooperatives with the key objective to ensure good quality of the products with provision of advice on good agricultural and processing practices and supply of quality raw materials to the farmers. The producers/farmers are required to follow the good farming practices and standard operation procedures, e.g., the use of high-quality seeds, low toxicity and low residue agro- chemicals. Some of these cooperatives were requested to apply for green and organic certification of their products. Meanwhile, some of the sub-project companies even integrated upwards to invest in producer cooperatives to ensure better quality of products and achieve the coordination of supply chains. 30. All the loan payments, including interest, have been collected on time. During the field visits, it was observed that such project loans play a key role to enable small and medium firms/cooperatives to engage in processing and other value-added activities, which allowed them to capture higher value markets for their products. 31. Based on the data provided by PPMO, the existing sub-projects created 1,667 jobs; and 12,820 farmer households were linked into the market and dynamic value chains. Financial viability of the 10 enterprises is also satisfactory. 51