Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Report Number: ICRR0021894 1. Project Data Project ID Project Name P118597 CN - Integrated Eco. Dev of Small Towns Country Practice Area(Lead) China Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-81570,IBRD-81600,IBRD-81610 31-Dec-2017 148,428,212.35 Bank Approval Date Closing Date (Actual) 24-May-2012 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 150,000,000.00 0.00 Revised Commitment 148,428,212.35 0.00 Actual 148,428,212.35 0.00 Prepared by Reviewed by ICR Review Coordinator Group Kavita Mathur John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO) was "to improve public infrastructure and services to contribute to sound development of selected small towns in Guangdong, Hunan and Gansu Provinces " (separate Loan Agreements for Guangdong, Hunan and Gansu Provinces and Project Appraisal Document para 17). The PDO was not revised. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 20-Dec-2017 c. Will a split evaluation be undertaken? No d. Components The project was expected to finance investments in 28 small towns in Gansu, Guangdong, and Hunan Provinces. The project comprised three components for each province, as described below (PAD pages 5 and 6): Component A: Infrastructure Development (estimate at appraisal US$244.7 million; actual cost US$187.2 million). This component was expected to strengthen the high priority infrastructure in selected towns and included investments in the following areas: (a) Gansu Province - construction or rehabilitation of urban and rural roads; expansion and modernization of irrigation infrastructure; and development of agricultural markets, commercial infrastructure and integrated service centers. (b) Guangdong Province - construction or rehabilitation of urban and rural roads; expansion or modernization of water supply and waste water treatment systems; and carrying out environmental protection activities, including creek rehabilitation; and development of commercial infrastructure and integrated service centers; (c) Hunan Province - construction or rehabilitation of urban and rural roads; construction of wastewater treatment facilities and improvement of solid waste management systems; and carrying out environmental protection activities, including creek protection and embankment rehabilitation; and development of agriculture markets, and integrated service centers. Component B: Institutional Strengthening and Capacity Enhancement (estimate at appraisal US$15.6 million; actual cost US$5.35 million). The component would finance activities to improve the institutional capacity of the project provinces to manage the development and operation of modern and efficient infrastructure, carrying out business development activities, and the provision of support services. These included: (a) carrying out of policy, technology and marketing studies; Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) (b) carrying out of technical, managerial and financial training, workshops and study tours for stakeholders (Government, Cooperatives, Water Users’ Associations, private entrepreneurs); (c) acquisition of equipment for testing, grading and real time monitoring of the quality and safety of marketable products; and monitoring environmental aspects of infrastructure sub-projects; and (d) establishing training and extension services for rural migrants and farmers. Component C: Project Managements and M&E (estimate at appraisal US$8.2 million; actual cost US$11.0 million). This component would support the implementation of the project and finance the establishment and operation of a management information, monitoring and evaluation system in each province. Other changes The project was restructured thrice.  The first in December 2015 to respond to the rapid changes in the three provinces development and planning processes and difficulties in land acquisition and resettlement. The restructuring resulted in the changes in component cost, financing plan, institutional arrangements, implementation schedule, and closing date. The number of participating towns was reduced from 28 to 25. In Guangdong province, Fuxing and Dongli Towns were dropped and Xiantang Town was added. Fuxing Town was dropped as project activities were implemented with domestic funding. In Hunan province, Zhentou Town was dropped because of changes in market conditions which made the proposed investments in horticulture industry irrelevant. In Gansu province, Wenfeng Town was dropped because of unsolvable land and resettlement issues. The project activities in Fucheng Town were aligned with the town master plan. With the dropping of these towns, project funds were reallocated to existing towns.  The second restructuring in May 2017 revised the loan closing date (to December 31, 2018).  The third restructuring on December 2017 revised the results framework (see section 9a below). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: At the time of first restructuring in 2015, the project costs were reduced from US$296.6 million to US$264.2 million. This was due to competitive procurement, which resulted in cost savings in contract values. The actual project cost was US$213.2 million, lower than the revised estimate of US$264.2 million because the number of participating towns was reduced from 28 to 25 (as explained in section 2d above) and the project activities were dropped in these towns. The project team provided the revised cost figures by component. These add up to US$203.56 million, which is the total baseline cost. Total project cost comprises: (a) total baseline cost; (b) contingencies; (c) front-end fee; and (d) interest during implementation. Therefore the total baseline cost (sum of component costs) is less than the total project cost. Financing: The project was financed through an IBRD loan, actual amount disbursed was US$148.4 million compared to the original amount of US$150 million. Since the project was designed to focus on three different provinces without a central coordination body, and each province had selected different financing terms, three separate loans of US$50 million were made for each province. Guangdong canceled Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) US$$344,898 (0.7% of its loan amount), Hunan US$61,005 (0.1% of its loan amount), and Gansu USS$1,165,883 (2.3% of its loan amount) A total of US$1,571,787 was cancelled. Borrower Contribution: The actual Borrower contribution was US$64.7 million, almost half the appraisal estimate of US$146.6 million. The ICR notes (page 34) that since the national program to support the small towns was cancelled, the interest of the Borrower waned and counterpart contribution declined. Also, due to cost savings as a result of competitive procurement the total amount of Borrower contribution was reduced. Dates: The project was approved on 24 May, 2012 and became effective on 5 November 2012. The second restructuring in May 2017 extended the closing date to give participating provinces more time towards achievement of the PDO. The project closed on 31 December 2018, one year after the original closing date. Split Rating. All PDO level indicators were essentially intermediate outcome indicators measuring improving “infrastructure and public services” and there was not indicator to measure the outcome “sound development of these small towns”. Moreover, the PDO level indicators on infrastructure investments were mostly outputs i.e. counting the number of people benefiting from the project investments, rather than measuring the quality or impact of such investments, such as water quality or service reliability. At the third restructuring (approved on December 20, 2017), a new PDO level indicator on beneficiary satisfaction was added to measure the outcomes on improving public infrastructure and services, but it still did not address the issue of missing indicators to measure progress towards meeting the outcome. A few targets were changed in response to project scope. The number of roads constructed were decreased from 140 km to 119 km. The target for the length of water supply pipes was increased from 15.4 meters to 20 meters and the volume of potable water was increased from 6,185 cubic meters to 19,700 cubic meters. In some cases, the appraisal targets were incorrect. For example, the target number of beneficiaries for road investments (PDO indicator 1) in Hunan was incorrectly written as 150,000 given that the planed length of road was only 13.9 km. This was changed to 15,000 (the extra "0" was typing error). Therefore, a split rating assessment will not be undertaken. 3. Relevance of Objectives Rationale Country Context: At appraisal, China was going through rapid urbanization with huge rural-to-urban migration (estimated 300 million migrants) and the task of absorbing the migrants was daunting. Small towns were expected to play a major role in the urbanization process, given their potential for creating jobs. However, small towns faced significant challenges such as deficient infrastructure, and inadequate support services. Previous Sector Experience: The Bank had significant experience in China supporting projects in small towns and urban rural integration. These include the Chongqing small cities infrastructure improvement project, the Sichuan small towns development project, the Chongqing urban rural integration project, and the Ningbo new countryside project. The selected provinces for this project were not included in the previous projects. Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Alignment with Bank Strategy: The project objective was fully aligned with the first and second themes of the Bank's China Country Partnership Strategy (CPS page 13) for FY13–FY16, namely Strategic Themes 1: Supporting Greener Growth and Strategic Theme 2: Promoting More Inclusive Development. The project would contribute to the following outcomes of the CPS:  Outcome 1.2 Enhancing urban environmental services (wastewater and waste management);  Outcome 2.2 Strengthening skills development programs (vocational trainings and workshops);  Outcome 2.3 Enhancing opportunities in rural areas and small towns (improving basic infrastructure, investing in irrigation systems, building farmer’s markets and industry/public service centers); and  Outcome 2.4 Improving transport connectivity for more balanced regional development (improving roads infrastructure). Alignment with Government and Provincial Strategies: The project was highly relevant to China’s development priorities as stated in the 13th Five-Year Plan (2016-2020). The plan emphasizes “moderately prosperous society” and inclusive growth that would be achieved through a number of initiatives including the development of towns, public service provision to towns, and fostering homegrown entrepreneurship. The National New-type Urbanization Plan (2014 – 2020) also stresses the importance of improving infrastructure and public services in towns which can serve the rural areas around it and become driving forces in local economic development. In China, since the provincial plans are aligned with the National development strategies, the project development objective was also consistent with the provincial development strategies of Guangdong, Hunan, and Gansu (ICR page 10). Although, the project objectives were aligned with the Bank Strategy and the Government strategies, and was planned to be jointly designed and implemented with a national Chinese program to support small towns, the national program was cancelled during project preparation, the relevance of objectives is rated substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve public infrastructure and services to contribute to sound development of selected small towns in Guangdong, Hunan and Gansu Provinces Rationale Theory of Change: The project was built on the premise or assumption that physical investments in key public infrastructure (construction/rehabilitation/upgrading) such as roads, water, sewerage, wastewater, irrigation facilities, agricultural and industrial service centers, and vocational training centers, would result in Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) improved infrastructure. Improvement in roads would result in improved access and reduced travel time between farms, industries and service centers and thus contribute to reduced costs. Improved irrigation facilities would increase the area served with irrigation and may increase farm yield and productivity. Construction of agricultural and industrial service centers would increase storage and processing capacity for agricultural and industrial products. The capacity building activities for key stakeholders at provincial, county and town level were expected to result in improved capacity of local governments in providing infrastructure, services, and planning and supporting key industries. The capacity building for (a) migrants; (b) extension training for farmers; and (c) entrepreneurs, was expected to provide additional skilled labor to industry. Thus, the key infrastructure improvements, accompanied with capacity building were expected to contribute to the economic development of small towns in project provinces. The ICR lists a number of assumptions for the theory of change to work. These include (ICR page 7):  Timely completion of infrastructure to demonstrate contributions to the development of towns,  Vocational schools are affordable and curriculum appropriate for migrants and entrepreneurs;  Economy would remain favorable to absorb new labor;  No extreme weather to adversely affect crop production in agricultural towns;  Farmers use extension services, marketing studies, and irrigation to increase yields and crop quality. The results framework did not include indicators to measure “sound development of the selected small towns.” Outputs: Physical infrastructure improvements Guangdong Province (investments in six small towns: Guzhen, Yuecheng, Rucheng, Dongcheng, Xiantang, and Fucheng). In total, 36.5 km of new roads were constructed in all six small towns and 16.8 km were rehabilitated. These improvements benefitted 107,104 people, exceeding the PDO target by 19%. Most of these roads were in industrial parks or around tourist attractions which benefited tourists, small businesses and local residents. Water supply investments were carried out in four towns. Rucheng invested in a water plant and upgraded the town’s existing water supply network and laid new pipes. Guzhen, Xiantang and Fucheng added new water pipes along the newly constructed roads. By project closure, potable water supply was 90,000 m3 exceeding the intermediate outcome target of 18,000 m3 (there was no baseline). The ICR notes (page 12) that this figure is very high, but the team had no independent means of verification (the weaknesses in project M&E are discussed in section 9 below). The number of households with new piped water connections was 11,516, exceeding the target by 82%. The total number of beneficiaries with access to improved water sources was 94,500, exceeding the target by 19%. Five towns (Guzhen, Rucheng, Xiantang, Fuchengwater, Dongcheng) invested in wastewater management. All installed new sewer and drainage pipes. Rucheng also invested in modernizing and upgrading its existing Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) sewage network. In total, 45km of sewer pipes and 37km of drainage pipes were installed and the beneficiaries target was exceeded by 22%. Four towns (Guzhen, Yuecheng, Rucheng, and Fucheng) built Integrated industrial service centers and vocational training centers. The number of users effectively using the integrated centers was 76.7% of the target, with 34,500 beneficiaries, instead of the intended 45,000 (ICR page 12).  Guzhen Town improved the interior of the Laboratory and Service Center of the Green Expo Park, which served the small businesses and farmers in the landscaping industry.  Yuecheng Town built an Industrial and Public Service Center. The Center served businesses in the building material industry and residents with access to local government services.  Rucheng Town built a vocational training center. By project closing, the Center had provided vocational technical training spaces for electric engineering, mechanical engineering, car repairing, hotel service, and entertainment industry.  Fucheng Town built a Public Service Center. Although construction was completed by project closure, only part of the center was open while rest had to be inspected by officials. Therefore, the center served only 2,500 beneficiaries, a small fraction of its target of 16,000 beneficiaries. Capacity building activities  17 policy, technology and marketing studies were completed, against the target of 15.  Workshops and study tours for stakeholders (Government, Cooperatives, Water Users’ Associations, private entrepreneurs) were conducted. 510 staff (compared to the target of 280) from the project management office was trained in project management, procurement, financial management and M&E.  Training and extension services for rural migrants and farmers were conducted. About 24,751 people were trained against the target of 40,000.  As targeted, 5 farmers organizations (producers associations, cooperatives, water user associations) were developed or improved  The project procured equipment for testing, grading and real time monitoring of the quality and safety of marketable products, and to monitor environmental aspects of infrastructure subprojects. Hunan Province (investments in eight small towns: Cangshuipu, Zoushi, Santang, Xinshi, Taizimiao, Dingziwan, Wanbao, and Yueshan). Six towns (Cangshuipu, Zoushi, Santang, Xinshi, Taizimiao, and Wanbao) invested in roads. Most of the roads were in industrial parks serving enterprises, while some were urban streets serving local residents, government entities and small businesses in town centers. In all, 12.5 km of new roads were constructed, and 1.68 km were rehabilitated, largely achieving the target. 37,386 people benefitted from these roads, far exceeding the target of 15,000. Except for Dingziwan Town, the remaining seven towns invested in water supply. In total, 27.8 km of water pipes were installed, exceeding the target by 39%. 6,027 new households were connected to the water supply network (603% of the original target) (As mentioned earlier, this number is very high, and the task team has no independent means to verify it). These outputs benefited 26,386 people or enterprises, far exceeding the target of 4,000. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) New sewage and drainage pipes were laid in the same seven towns that received water investments. In total, 20 km of sewers (exceeding 21% of the target), and 25 km of drains (exceeding 14% of the target) were installed. Zoushi Town purchased sanitation vehicles including waste collection and compaction trucks, a sprinkler truck, and a car with environment information collection and monitoring facilities. Investment in wastewater/solid waste treatment was carried out in Yueshan Town, where a wastewater treatment plant (WWTP) was constructed. The plant was completed just before project closing, and was running at one-third of its design capacity and serving 5,000 beneficiaries (compared to 15,000 estimated at appraisal). This was due to: (a) the plant was in testing mode, and (b) declining population, as most of the young people from the town work and live in cities returning only for Chinese New Year. Therefore, the amount of waste water generated is much lower than appraisal estimates. Yueshan Town built a farmers’ market and Dingziwan Town invested in one building that would be used as a farmer’s market and a service center. The buildings were completed, but were not being used by project closing. The markets built were not in use by project closing, and no vendors were attracted. Capacity building activities. The ICR reports (page 16) that capacity building, institutional strengthening and project M&E was unsatisfactory. Although, the target for the number of studies were met, these studies were general development plans rather than focused studies to address key issues in the specific industrial sector of each town. 470 staff (compared to the target of 420) from the project management office was trained in project management, procurement, financial management and M&E. 75.6% of the target number of beneficiaries were trained. Gansu Province (investments in 11 small towns including Dongwan, Hongshui, Ganquan, Zaojiao, Meichuan, Shizi, Dongzhi, Dangzhai, Hongwansi, Huahai and Qili). All eleven towns undertook road investments, 43 km of new roads were constructed and 1.9 km rehabilitated. For example, roads in Dongwan Town linked vegetable production areas to markets and storage facilities. Newly constructed roads in Ganquan Town served apple and peanut producers to access markets. Urban roads were also built to provide local residents and businesses connectivity to town centers. In total, 67.9 km of water pipes were laid. 13,393 people had access to improved water source, far exceeding the target of 8000. In total, 14.6 km of sewer pipes and 15.3 km of drainage pipes were laid, exceeding intermediate targets. Ten of the 11 participating towns built farmers’ markets. Five towns invested in irrigation facilities such as constructing pump stations and irrigation canals. At project closure, an additional 4,874.5 ha of agricultural land was under irrigation, meeting 83.5% of the target. Across the 10 towns, the number of users effectively using the agricultural markets and land served by irrigation facilities was 45,399, almost double the target of 23,000. Regarding agricultural markets, while number of vendors attracted was less than half of the target, sales revenues was more than 200% of the target. Capacity building activities.  Migrant workers were trained in areas such as fruit planting, handcrafting, carpet weaving, greenhouse management, and pest control techniques.  7 policy, technology and marketing studies were completed, against the target of 5. Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597)  809 staff (compared to the target of 640) from the project management office was trained in project management, procurement, financial management and M&E.  132 farmers organizations (producers associations, cooperatives, water user associations), significantly far exceeding the target of 32, were developed or improved.  The project procured equipment for testing, grading and real time monitoring of the quality and safety of marketable products, and to monitor environmental aspects of infrastructure subprojects. Outcomes The project results framework did not include any indicator to measure the outcome “sound development of small towns”. A few case studies (three in total - one for each province) were carried out that showed development in small towns:  Investments in the industrial park in Santang Town (Hunan province) attracted 22 enterprises, 800 new jobs, and generated additional tax revenues of RMB 30 million a year.  In Guzhen Town (Gunagdong province), land rental fees at the nurseries and gardens at the project site increased from RMB1,000-2,000 yuan/mu before the project to RMB3,000-4,000/mu after the project.  In Dongwan Town (Gansu Province), where investments in irrigation were carried out, vegetable production generated over 75% of household incomes. Also, farmers’ skills improved with regard to branding of vegetable products, large-scale modern farming of vegetables, and using e-platforms for trading. The number of professional agricultural cooperatives also increased from a dozen before the project to more than 300 by project closing. The ICR also reported (pages 22 and 23) that during the project period, the project towns in all three provinces saw faster GDP growth compared with neighboring towns under the same local jurisdiction (same prefecture) see table below. However, the extent to which this performance of the project towns may be attributed to the project interventions is difficult to quantify. To measure the impact of infrastructure and service investments on the beneficiaries, a PDO indicator on beneficiary satisfaction was added at the third restructuring in December 2017. A Project Social Economic Impact Survey (PSEIS) was designed and carried out to evaluate the project’s performance against this indicator. Overall beneficiary satisfaction ratings for project interventions in roads, irrigation, skill training, urban upgrading, markets and service centers, river rehabilitation, and waste/wastewater were 81%, 83%, and 84% respectively in Guangdong, Hunan and Gansu, exceeding the target of 75%. Irrigation and skills training scored the highest with satisfaction ratings of 98% and 92%, respectively, followed by roads and water/wastewater infrastructure investments with satisfaction ratings around 81%, followed by urban upgrading, markets and service centers, and river rehabilitation investments with satisfaction ratings around 76-77% (ICR page 19). Rating Modest Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project assisted in the improvement of public infrastructure and services in selected towns in the three provinces. Beneficiary satisfaction with the improved infrastructure generally met or exceeded the target of 75%. However, as the key outputs were completed very late in the project period and were relatively small in scope, the project’s contribution to the sound development of the small towns suffered from lack of attribution and could not be adequately measured. Some of the targets were not met. For example, the markets built in Yueshan and Dingziwan were not in use by project closing, and no vendors were attracted. The wastewater treatment plant in Yueshan Town is working below capacity. There was no systematic assessment of the impact of capacity building activities. Therefore, the project outcome is rated Modest. Overall Efficacy Rating Primary Reason Modest Insufficient evidence 5. Efficiency Economic Analysis Roads At appraisal, economic analysis for roads in the industrial parks in Guangdong and Hunan was not conducted as these roads served larger zones and were integral part of the basic infrastructure. The ICR notes (page 24) that it was challenging to quantify their unique contribution to the overall economic development of the zones/areas. At completion, with better understanding of the scope, cost-benefit analysis was carried out. This yielded an Economic Rate of Return (ERR) ranging between 9 to 16% (the difference in part reflecting the ERR with and without the project). For roads not in industrial parks, in Hunan the ex-post ERR ranged between 9 and 16% (no ERR was calculated at appraisal). In Gansu, the ex-post ERR was 12%, compared to ex-ante ERR which ranged between 10 to 15%. Irrigation. Gansu main investment was in irrigation sub-projects. The ex-post ERR was 26%, higher than the appraisal estimate, which ranged between 14 to 22%. Agricultural markets No cost benefit analysis was conducted for agricultural markets at appraisal. At completion, the ERR was 14%. This includes average income growth around 3,300 yuan per person and output growth around 30% (ICR page 59). Financial Analysis Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) At appraisal, financial analysis of the sewerage and water supply utilities assessed the tariff levels required to fully cover the system operations and maintenance (O&M) costs. At completion, all project towns had tariff levels exceeding the targets set at appraisal, except for the case of Yueshan Town in Hunan Province (ICR page 25 and 26). Cost savings. The ICR reports (page 10) that there were cost savings in contract values due to competitive procurement. However, the ICR further notes (page 9) there were increase in costs due to land acquisition. Implementation delays and time overrun. In Hunan, there were construction delays, with 9 out of the 11 works being completed in the final seven months of the project. Some of the delays were due to additional time needed for land acquisition and shortages in counterpart funds. The roads investments were economically viable and the ex-post ERR ranged between 9 to 16%; the ex-post ERR for irrigation was 26% ; and for agricultural markets around 14%. However, there were some minor shortcomings relating to construction delays and resulting in time overrun of one year. Efficiency is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives was substantial, efficacy modest, and efficiency substantial. Therefore, the outcome rating is moderately unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Financial Risk. There is low risk for the continued operations of the sewerage and water supply utilities except for Yueshan (see below), where the risk is high. At appraisal, financial analysis of the sewerage and water supply utilities assessed the tariff levels required to fully cover the system operations and maintenance (O&M) costs. At completion, all project towns had tariff levels exceeding the targets set at appraisal, except for the Wastewater Treatment Plant (WWTP) in Yueshan Town in Hunan Province. At project closure, the operation of the Yueshan WWTP was being subsidized by the county-level government. The WWTP was designed to be financially feasible and the town government had planned to charge a wastewater treatment fee to residents. The fee collection looks problematic due to declining population. Without a government subsidy, it is unlikely that the Yueshan Town Government will be able to finance the O&M of the WWTP (ICR page 15). Regarding the agricultural, industrial and commercial infrastructure facilities (such as cold storage and wholesale markets) rent level at the facilities constructed under the project well exceed the O&M cost recovery level recommended at appraisal, and in most cases are above full cost-recovery level. Markets capacity to attract vendors. In Hunan Province, there is a high risk that the markets built in Yueshan Town and Dingziwan Town will not be able to attract vendors. Although these markets had ambitious business plans, by project closing they had not attracted sufficient numbers of vendors. In Gansu, 10 farmers markets were built in town centers to serve as central marketplaces for trading agricultural products. However, at project closure the project markets were facing competition from other trading places, for example, informal or smaller markets near the vegetable and fruit gardens, supermarkets which trade with farmers directly, and online trading platforms. 8. Assessment of Bank Performance a. Quality-at-Entry The project design was ambitious, as the PDO “to contribute to sound development of these small towns” was broad and the time frame of five years was too short for the infrastructure investments to conclusively prove that they influenced the local economy. The project included 28 small towns in three very different provinces (at different development stages and investing in different types of infrastructure depending on local needs) with different institutional capacities and supervision proved to be challenging. At appraisal, the expectation was that the project would complement the national Chinese program to support small towns and therefore would be jointly designed and implemented. However, while the national program was cancelled during project preparation, the Bank still went ahead with the project. Moreover, during preparation, the National Development and Reform Commission (NDRC) had proposed that China’s Center for Urban Development (which focuses on research and policy advice for small towns) would be the central Project Management Office (PMO), but NDRC later changed its mind. At appraisal, the risk of limited capacity of the client to carry out project management, financial management and procurement was correctly identified as “high”. Without a central PMO or leading group, Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) there was little coordination or synergy at the national level to identify and disseminate lessons (ICR page 28). The overall risk rating at appraisal was assessed as “substantial”. Various mitigation measures were proposed such as extensive training and capacity building for local PIUs in financial management and procurement. Other mitigation measures were proposed but barely realized. For example, coordinated study tours and cross-provincial knowledge exchange did not take place until very late in the project and were infrequent. The results framework was weak. All PDO level indicators were essentially intermediate outcome indicators and there was not indicator to measure the outcome “sound development of these small towns”. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The ICR reports (page 30) that the wide geographical spread of the subprojects increased supervision costs for the Bank. The workload for the Bank’s task team was equivalent to three separate projects but with a supervision budget equivalent to a single project. The supervision budget was increased overtime. Due to limited resources and wide-spread project sites, the Bank’s full implementation support missions were able to visit each province only once a year. The rest of the supervision was carried out by phone or sending out small technical missions. Despite these issues, when the project was downgraded to problem project status in June 2014, the Bank team was proactive and advanced the Mid Term Review (MTR) by six months from June 2015 to October 2014 to design an action plan to compensate for the client’s weak capacity. Eventually, the project was restructured only in December 2015 when the total project financing was 18% disbursed. Action plans for addressing recurrent issues (lack of counterpart funding, land acquisition, procurement/contract management, etc.) were developed and technical support to participating towns was ramped up (ICR page 35). However, the PDO was not changed as there was no consensus among the three participating provinces and the project implementation was more than half way through. As a remedial measure it was agreed to conduct a social and economic impact survey to capture project outcomes. The Bank’s support for procurement, financial management and safeguards was satisfactory. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was weak. All PDO level indicators were essentially intermediate outcome indicators measuring improving “infrastructure and public services” and there was no indicator to measure the outcome “sound development of these small towns”. Moreover, the PDO level indicators on infrastructure investments were mostly output indicators I.e. counting the number of people benefiting from the project investments, rather than measuring the quality or impact of such investments, such as water quality or service reliability. At the third restructuring (approved on December 20, 2017), a new PDO level indicator on beneficiary satisfaction was added to measure the outcomes on improving public infrastructure and services, but still did not address the issue of there not being any indicators to measure progress towards meeting the outcome. A few targets were changed in response to project scope. b. M&E Implementation The ICR reports (page 31) that M&E implementation was weak. The Provincial Project Management Offices (PPMOs) paid little attention to the results framework, did not understand how the indicators were supposed to be measured or how to collect relevant data. There was a lack of consistency on data collected between different versions of progress reports and among different provinces. The ICR notes (page 12) that the figures for potable water supply were very high, but the team had no independent means of verification. The ICR does not explain this further. M&E began to improve when PPMOs hired consulting services for M&E at the behest of the Bank. The ICR does not specify when this happened. c. M&E Utilization The project M&E framework and data were not used as a tool by the PPMOs to evaluate their own work (ICR page 31). M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was assigned Environmental Category “B” and five safeguard policies were triggered: Environmental Assessment (OP4.01), Pest Management (OP4.09), Physical Cultural Resources (OP4.11), Indigenous Peoples (OP 4.10), and Involuntary Resettlement (OP/BP 4.12). Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Environmental Safeguards The Environmental Assessment (EA) documents were prepared according to World Bank safeguard policies and Chinese national laws and regulations. The EAs were disclosed through bulletins in local communities, local newspapers, and local government websites in Chinese in November 2011, and at the World Bank’s Infoshop in English on December 21, 2011. During implementation, the requirements of the Environmental Management Plans were included in the bidding documents and the construction contracts of the project. Trainings on environmental mitigation measures were provided by the Bank and external monitoring of environmental impacts was conducted by third-party consulting firms. To facilitate the implementation of environmental safeguards policies, an environmental management system was established in each province. This included the staff from the Provincial Project Management Office (PPMO), local Project Management Office (PMO), contractors, construction supervisors, and an independent external environmental monitoring agency. The ICR reports that the environmental mitigation measures were generally well implemented to address dust, noise, construction wastewater, soil erosion, and construction waste (ICR page 32). Pest management. The Pest Management Plans (PMP) were prepared and disclosed in Guangdong in October 2011 and Gansu in December 2011. The training materials on pest management were prepared in Gansu by the provincial level research institute that specialized in agriculture and pest control. These were well prepared and went beyond China’s domestic standards. With the use of Integrated Pest Management (IPM) approach in the project areas, the pest prevention and control improved. The PMPs were implemented satisfactorily. Social Safeguards Indigenous People. Due to the presence of Yugu and Tibetan ethnic minority groups in project towns in Gansu Province, Indigenous Peoples safeguards policy was triggered. An Ethnic Minority Development Plan (EMDP) was developed and disclosed locally and in World Bank’s Info shop in November 2011. The implementation of EMDP was as follows:  Hongwansi Town completed all activities under the EMDP on time.  Hongshui Town experienced delays in implementing EMDP activities. Training was completed towards the very end of the project. The slow implementation was mostly due to limited capacity of the local Project Implementation Units who were pre-occupied with completing the construction works under the project. Involuntary Resettlement. Land acquisition and resettlement was triggered for subprojects in Hunan and Gansu Provinces. Each province prepared a Resettlement Action Plan (RAP) and the RAPs were disclosed locally and in World Bank’s Info shop on November 21, 2011. External monitoring of resettlement implementation was carried out by third-party consultants. The monitoring concluded that the resettlement had limited impacts on the income, livelihood and agricultural activities of households because only a minor portion of their land was acquired (ICR page 32). Regarding land acquisition, there were delays in the land acquisition process as well as in providing compensation in some towns in Gansu and Hunan, which resulted in significant delays in project activities. Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) In Guangdong, the Bank was informed that all land for project activities was state-owned, so no physical resettlement would be needed. The Bank team still required Guangdong PMO to commission an external evaluator to assess if people or enterprises would be adversely affected by land acquisition due to project activities. The evaluator's report did not find any issues. b. Fiduciary Compliance Financial Management (FM). The ICR reports (page 33) that the project’s FM was moderately satisfactory. All audit reports were unqualified and were submitted to the Bank on time. Minor FM related issues were identified by auditors. These included: delays in the submission of the project’s interim financial reports (though of acceptable quality) and slow disbursements of the three loans. Slow disbursements were due to the borrowers’ lack of experience with Bank policies and procedures and implementation delays in the first couple of years, but disbursement gradually picked up later in the project. Procurement. Procurement of works, goods and consulting services was carried out satisfactorily and in accordance with the Bank’s procurement policy. There were some delays in procurement due to PPMOs and PIUs being unfamiliar with the World Bank procurement policies and procedures but the Bank provided training on procurement procedures and contract management. The procurement was supervised by procurement specialists in the Bank’s Beijing office and post procurement reviews were carried out on a random basis. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome Unsatisfactory Unsatisfactory Moderately Moderately Bank Performance Unsatisfactory Unsatisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Following lessons are adapted from ICR (pages 36 and 37).  Avoid PDOs that are layered, broad and hard-to-measure. Since project success is measured against its intended PDOs, unattainable PDOs set a project up for failure - even if all investments are completed in time and of good quality. The project experience shows that public infrastructure and services require an adequate number of years to show measurable economic impacts. To show measurable impact by the end of the implementation period, the PDO for this project would need to have been formulated in more intermediate terms.  In countries with central and provincial government structures, it is important to secure central government’s commitment and support in designing and implementing multi-province operations. The decision of the National Development Reform Commission to withdraw from the coordination of provincial level activities early on in the life of this project life and the cancellation of the matching national program adversely impacted project performance in terms of implementation and cross learning.  Avoid setting up project implementation units (PIUs) at very low levels of government without sufficient higher-level monitoring and control. In this project, most town-level PIUs proved to be too weak in authority, severely understaffed, and limited in technical capacity and project management experience to be effective implementors of a World Bank project. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well-written and provides a candid assessment of the project’s achievements and implementation challenges. The ratings are based on the evidence. The ICR is consistent with the guidelines. Each safeguards policy is addressed in detail. In the Theory of Change section, the ICR lists a number of assumptions for the theory of change to work. However, with 37 pages, the ICR is too long (the ICR Guidelines recommend a maximum main text length of 15 pages). There is a discrepancy in actual cost figures in the financing table (ICR page 2) which reports US$213.2 million and Annex 3 "Project costs by Component (page 57), which reports US$231.3 million). The project team provided IEG with the updated figures. a. Quality of ICR Rating Substantial Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CN - Integrated Eco. Dev of Small Towns (P118597) Page 18 of 18