ICRR 13419 Report Number : ICRR13419 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/28/2011 PROJ ID : P088116 Appraisal Actual Project Name : Pro-poor Rural Water Project Costs (US$M): US$M ): 22.20 26.76 Reform Project Country : China Loan /Credit (US$M Loan/ ): US$M): 12.47 10.17 Sector Board : ARD US$M ): Cofinancing (US$M): 0 0 Sector (s): Irrigation and drainage (100%) Theme (s): Water resource management (67% - P) Participation and civic engagement (33% - S) L/C Number : Board Approval Date : 03/19/2004 Partners involved : British Department for Closing Date : 12/31/2008 12/31/2009 Institutional Development (DFID) Trust Fund Evaluator : Panel Reviewer : Group Manager : Group : Peter Nigel Freeman George T. K. Pitman IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: The Pro-Poor Water Reform Project (PPRWRP) was financed by DFID, and no Bank PAD was prepared to provide background, context and details of the project . To compensate, the Implementation Completion Report (ICR) provides much more detail concerning preparation than is normally found in a standard ICR . In addition, PPRWRP included a Bank-executed Trust Fund which provided specialized technical assistance and support for Water User Association (WUA) development under the project; results from that Trust Fund are referred to as "output 5" in the Implementation Completion Memorandum for the Trust Fund . The project development objective (PDO) as stated in the Trust Fund Grant Agreement is "to assist the Government of China in reforms in the management of small -scale rural water facilities through the establishment of farmer Water User Associations (WUAs), based on a participatory approach and with special focus on optimizing benefits to poor farmers, women, and disadvantaged groups ". According to the World Bank's Initiating Brief for a Trust Fund (IBTF) and the ICR, the objective of the project was also to be achieved through the World Bank's sector working program on WUAs . They further state that the project aimed to support a participatory form of irrigation management which is socially equitable and environmentally sustainable. The reforms were to be supported and carried out in large, medium and small scale irrigation schemes . While there are small nuances between the PDOs, IEG has taken the Trust Fund Grant Agreement as the basis for this review. Key Performance Indicators (KPIs) were as follows: Extension of improved WUAs in seven provinces in phase 1 and seven provinces in phase 2 by the end of the project (12/31/2008); Improvement of security and equity of water supply for poor farmers and for women and disadvantaged groups; Establishment of WUAs that are participatory, socially equitable and environmentally sustainable; Adoption of a National Policy for WUAs . Intermediate indicators included the number of participatory demonstration and extension WUAs, participation of women in WUAs, benefits of participatory irrigation management under WUAs, benefits of WUAs, and national and lower level policy documents issued and adopted which supported improved participatory WUA development . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project had the following five components : (i) Policy Support and Guidance - (planned US$0.75 million; actual US$0.66 million) to prepare guidelines, regulations, and research issued by the national, provincial, and local levels to provide policy support for establishing participatory, equitable, and sustainable WUAs consistent with overall policy and reforms in water resources management and poverty reduction . (ii) Capacity Building and Training - (planned US$0.32 million; actual US$0,99 million) to strengthen the capacity of (a) agencies to support WUAs, (b) farmers to manage WUAs, (c) women and the poor to participate in WUAs and their management. This includes support for the development and production of WUA training materials, dissemination through media and other channels, WUA training workshops, and specific training programs . In addition, lessons learned would help enable WUAs to be extended throughout China . (iii) WUA Development - (planned US$1.10 million; actual US$1.47 million) to establish (a) demonstration WUAs that would enable lessons to be learned and best practices to be identified, tested, and implemented for WUA development based on participatory, equitable, and sustainable management of small -scale rural water facilities, and (b) extension WUAs that would be used to spread and extend WUAs based on those lessons and practices . (iv) Monitoring and Evaluation - (planned US$1.07; actual US$0.95 million) to develop innovative, computerized systems for the project Management Information System (MIS) and WUA - Monitoring and Evaluation (M&E)s and to establish them at all project counties /sites, used for both reporting and project management purposes during implementation, and for evaluating the quality, impact, and benefits of project WUAs during and after their establishment. Specialized training and technical support in particular on M&E using "control groups" would also be provided. (v) World Bank Sector Work Program on WUAs - to provide specialized technical assistance and support for WUA development in China, including long term WUA strategy development, WUA training, promotion of participation by women and the poor in WUAs, lesson learning, MIS, and WUA -M&E system design, livelihood studies, and promotion of institutional capacity building for future WUA development, WUA and M&E training assistance to other provinces and projects, dissemination of lessons learned, and other topics as needed . The last component on the World Bank Sector Work Program on WUAs was not included in the Grant agreement as it was executed by the World Bank . However, it was included in the ICR because it was considered linked to the project co-financed by DFID and the Government, and administered by the World Bank . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : As indicated above the total project cost was $ 26.8 million including the borrower contribution, but the ICR cost table only accounted for the expenditures incurred under the DFID trust funds . The estimated project costs were indicated in the IBTF . However, these were specified according to the category of expenditures and the breakdown into components was only done for the consultancies . The remaining costs for inception, MTR, ICR, and management ($0.58 million), goods ($1.0 million), works ($0.27 million), and other e.g. workshops and dissemination activities ($2.6 million) were not divided according to component . The total unallocated amount was $1.4 million. The World Bank executed component totalled $ 2.6 million and the World Bank grant administration fee totalled $0.62 million, according to the ICR (Annex 1 p. 28). The total project costs under DFID trust funds including the Bank executed account at project closure was $ 13.4 million, which was seven percent higher than the originally estimated amount in the Project Implementation Plan . The total project cost excluding the Bank executed account was $ 10.2 million which was 11 percent higher than the originally estimated amount. Capacity building and training under Component 2 and the costs for inception, mid term review, ICR, and management exceeded its original cost estimates by three times, because as the project developed the crucial role of training was better understood, and this being a pilot, the allocation for this component was increased substantially. This was possible because of a favorable shift in the dollar /yuan exchange rate. Financing : The project was funded by China's Free Standing Trust Fund Program - DFID grants in the amount of $12.5 million estimated at approval included the Recipient executed account of $ 9.5 million and the Bank executed account of $3.0 million. At project completion, total project cost was estimated at $ 13.4 million including the Recipient executed account of $10.2 million and the Bank executed account of $ 3.2 million. In the project data sheet, the project cost and Credit/Loan amount are specified as $ 9.5 million at approval and $10.2 million at completion, reflecting the Recipient administered portion only . Borrower Contribution: With the Borrower contribution in the amount of $ 13.4 million (including labor), the total project size at project completion was $ 26.8 million. This was 120 percent of the estimate at approval because of exchange rate fluctuations. Actual Government contribution at the end of the project was 40 percent higher than originally planned. Dates: The project was first extended by six months to June 30, 2009 at Mid-term Review to carry out additional WUA dissemination activities including an International South -South Conference on WUAs using the Global Distance Learning Network. A further six months extension to December 31, 2009 was agreed to allow time for full completion of additional technical assistance activities, including training, and final reimbursement of project costs . 3. Relevance of Objectives & Design: Relevance of objectives: Substantial The Country Assistance Strategy (CAS) of January 2003 focused on addressing the needs of the poorer and more disadvantaged people as well as those regions in China that were lagging economically . This was to be achieved through investment lending in the rural development, infrastructure and social sectors, as well as through analytical and advisory assistance and training . The CAS also supported water resource management reform by building this need into water resource projects and by supporting strategic studies, improved water pricing, participatory irrigation management, self-financing water enterprises, and irrigation rehabilitation etc . The Country Partnership Strategy covering the period 2006-2010 envisaged aiming for better approaches to improve rural public services and enhanced land and water resource management, including improved water pricing . The Government of China was also fully committed to the poverty -targeted reform project as evidenced by the high level support it received and subsequent changes it made to fill gaps it perceived in its national water and irrigation policies . Relevance of design: Substantial The project scope and design were consistent with the CAS elements noted above as well as the principles promoted by the financiers, based on their previous involvement in the water resources sector in earlier projects; the design was also customized to meet the needs of China . Since the project was primarily financed through DFID the Bank had only an advisory role in the framing of the objectives and the design . A PAD was not required, and therefore a results framework was not prepared . Performance targets and indicators were specified at the beginning of the project, but they were mostly output indicators focusing on the number of provinces covered or the number of policy documents and guidelines issued etc . While these outputs were relevant for the intended objectives, it would have been more useful for the purpose of evaluating the impact on the ground to assess the effects of the projects on the poor farmers, women, and disadvantaged groups, but this is admittedly very difficult . The project was specifically designed as a "pro-poor" project and targeted benefits for the poor and women . Project areas were selected to cover relatively poor areas and were located in poverty regions based on national and provincial classifications. During phase 2 the project particularly focused on absolutely poor regions in Western China (ICR p. 21). The results were based on a "with" and "without" comparison of WUAs with control group areas to take out the effects of other factors such as the quality of irrigation facilities, prices and general economic progress . The beneficiary survey, which was conducted at the end of the project by the Chinese Academy of Agricultural Sciences supported by the Bank and DFID, showed good results . While it could have demonstrated the impacts on the ground more effectively if the objectives and the project indicators had been better designed to reflect them, it was still a very valuable piece of analysis . The study utilized a number of methodologies to analyze the impacts of the project including the Sustainable Livelihoods Framework, Participatory Rural Appraisal and econometric models . 4. Achievement of Objectives (Efficacy): The objective to assist the Government of China in reforms in the management of small -scale rural water facilities was achieved . Overall, efficacy is rated High . This objective was expected to be achieved through; (i) establishment of farmer WUAs based on democratic participatory irrigation management, to help maximize benefits to poor farmers, women, and disadvantaged groups; and (ii) through the World Bank's sector work program on WUAs . According to the ICR, all PDO and outcome indicators exceeded the target values; had there been a comparison framework at the time of approval it would have been easier to assess the accuracy of this achievements, but even so the results are far better than anticipated . Possibly some of the indicators in retrospect were on the conservative side given the institutional capability in China, but many of the activities were completely new and relatively difficult to accomplish (ICR para 3.2.6). The following Outcomes were achieved by project closure : The targets for extending rural water reform program in seven provinces in Phase I and seven provinces in Phase II were both achieved . Eight national policy documents promoting various aspects of reform of small-scale water facilities were issued for WUAs, exceeding the target of two, and 62 provincial policy documents were issued for WUAs, substantially exceeding the target of 23. A gender strategy was completed and fully integrated into the revised WUA charter . According to the ICR (p. 21) the establishment of farmer WUAs was based on transparent and democratic participatory irrigation management, to help maximize benefits to poor farmers, women, and disadvantaged groups. Democratic water management under the WUAs provided the poor and women with equal access to water regardless of their status, economic condition or physical strength (ICR p. 21). The project provided about 295,000 days of training for women and the poor, over and above the 505,000 days of training for all WUA farmers. With this support participation by women in WUAs grew steadily and by 2008 they accounted for 28 percent of WUA members and 22 percent of WUA leaders. Compared to the control groups, WUAs under the project saved substantial amounts of water at a relatively low cost due mainly to volumetric water charges, reduced water conflict, and generally better local water management which is reliable, flexible and timely (ICR p. 26). As a result of the better water management introduced by the adoption of WUAs the time waiting for and monitoring irrigation water reduced from 5.6 hours a day to 4.5. (ICR p. 40). For project WUAs compared with control groups, for 2008 alone, food production increased by nearly 238,000 tons, household income by Y 1,846 million and water savings were about 198 million cu.m. Overall, some 17,400 people were lifted out of poverty under project WUAs, and the poverty rate in WUA areas fell from 18 percent to 11 percent during the project period (ICR p. 18). In addition, the beneficiary survey (ICR page 40) showed positive impacts on the income of the ethnic minority groups. For example, in Xinjiang the per capita income of WUA households increased from Y 4,240 in 2004 to Y5,653 in 2007, up 40.9 percent. . Farmers needed to spend less time on agricultural production and could thus afford to spend time (up to two hours a day) on activities such as animal husbandry and on maintenance activities. According to the ICR (p. 3 ) the successful roll out of the project pilots has led to a huge surge in the number of WUAs established, which now number tens of thousands across China . Outputs: The total number of WUAs established under the project was 497 compared to the target of 447. The total number of special training sessions for women and poor persons has exceeded the target of 51,829 by almost six times, with 295,464 persons trained. Dissemination of the merits of WUAs was expanded considerably both through the Global Distance Learning Network and through other fora . CDs were produced in English and Chinese with short video presentations of the merits of setting up WUAs. There was also an effort to demonstrate through these presentations how to calculate the water charges . 5. Efficiency (not applicable to DPLs): While the project was a technical assistance and did not require an economic analysis for justification, the major economic benefits from WUAs and the costs incurred for WUAs were calculated at the time the ICR was prepared . According to this analysis, in year 2008 alone, the value of incremental food production and water saved by improved, standardized WUAs under the project was Y 470.5 million, more than double the total project cost of Y203.6 million. These incremental benefits were derived from comparison with control groups where infrastructure and other conditions were the same as for WUAs, so it is clear that the benefits were due to WUAs and their operation rather than the irrigation infrastructure investments or other factors . Compared to the direct costs incurred for WUA establishment under the project of only Y 11.2 million, the benefits are highly significant . The ICR states that these benefits relative to the costs may appear implausibly high, but this is because in all the project WUAs the irrigation facilities were reasonably adequate and did not involve any physical investments . With adequate infrastructure in place, the potential for very high returns exists from relatively small investments through good management and O&M, and inclusive, pro -poor project design. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re-estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal % % ICR estimate % % * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Performance targets and indicators were specified, but they were mostly output indicators focusing on the number of provinces covered or the number of policy documents and guidelines issued etc . Possibly some of the indicators in retrospect may also have been on the conservative side given the institutional capability in China . Based on the information presented in the ICR, which does not include sufficient evidence about the impact on groups specifically targeted by the PDO, a highly satisfactory outcome is not proven . Taking these factors into account the overall outcome rating is satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risks identified in the ICR included the withdrawal of policy support from the Government, discontinuation of institutional support for WUAs, and inadequate capacity for future WUA development . However, these risks have been either mitigated or addressed through the issuance of high level formal national policy statements on WUAs, plans for the establishment of a specific legal basis for WUAs in national legislation or regulations, wider circulation of the policy notes, establishment of "WUA Technical Services Center" for WUA development support, improved monitoring system, and the availability of training and capacity building materials available for replication . The fundamental institutional shift from total government and /or village controlled operation to participatory management and O&M by farmers based on democratic principles, adoption of formal and standardized policies, and provision of proper training and capacity building activities are likely to ensure sustainability of project impacts . The issue related to the financial sustainability of the WUAs was raised during project implementation, and a national agricultural water pricing reform program has been proposed to tackle this issue . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at entry The project incorporated the key elements of the CAS and supported the Government's reform program in the irrigation and water resource management . While the objective could have been more comprehensive, the project design did incorporate the relevant lessons and experience from other World Bank funded projects in the sector. Although there was only one short preparation mission by DFID, this was supplemented by three very intensive Bank preparation missions . The Bank carried out proper due diligence during preparation, demonstrated by the up-front policy and institutional requirements that were spelled out as conditions for the grant, and through three strong technical assistance missions comprising international and local experts on WUAs, water law, anthropology, women in rural development, poverty and social issues, irrigation engineering, institutional development, water resources reform, and related fields . This was only possible because the Bank had access to substantial TA funding; the level of preparation could not have been achieved under a Bank project with a normal preparation budget. A project launch workshop was held at the start of the project which also confirmed the quality at entry. Quality of supervision During project implementation, regular supervision missions were held with highly experienced and qualified domestic and international experts . This contributed to identifying and resolving key issues that are critical for project performance and success . While the ICR acknowledges that the stronger task team focus during Phase two on establishment of permanent arrangements for comprehensive WUA support would likely have enhanced future prospects for improved, sustainable WUA development, the results have achieved or exceeded what the project initially envisaged. Only two ISRs were filed as the project did not enter the Bank's supervision portfolio until 2008, but satisfactory supervision aide memoires were prepared over the project implementation period . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government There was a clear commitment and ownership by the Government to implement this project and then roll it out nationally, as demonstrated in the issuance of the national policy for the standardization and scaling up of the use of WUAs for the farmers in the irrigation and water resource management sector . Circular 502, which was a National Water Policy Statement, was issued and endorsed by three powerful Ministries . Furthermore, the Government provided 40 percent more counterpart funding than it originally planned . Performance was highly satisfactory. Implementing agencies China Irrigation and Drainage Development Center (CIDDC) under Ministry of Water Resources (MWR) and the State Office for Comprehensive Agricultural Development (SOCAD) under Ministry of Finance (MOF), the CPMOs and PPMOs, and the lower level PMOs all demonstrated exceptional commitment to the projects and to WUAs and overachieved on their performance targets . The implementing agencies also made substantial efforts to be innovative in developing new approaches and in solving problems as they arose . This was consistent with the strong support from their leaders . Rating: highly satisfactory. Although the project required a one year extension it enabled widespread dissemination of the findings of the technical assistance funded through savings generated by favorable exchange rates . a. Government Performance :Highly Satisfactory b. Implementing Agency Performance :Highly Satisfactory c. Overall Borrower Performance :Highly Satisfactory 10. M&E Design, Implementation, & Utilization: Design Performance targets and indicators were specified, but they were mostly output indicators focusing on the number of provinces covered or the number of policy documents and guidelines issued etc . While these outputs were relevant for the intended objectives, it would have been more useful for the purpose of evaluating the impact on the ground to better assess the effects of the projects on the poor farmers, women, and disadvantaged groups and give a fuller socio-economic profile of the beneficiaries . Possibly some of the indicators in retrospect were on the conservative side given the institutional capability in China . Nevertheless, the M&E system was designed to identify the impacts and benefits of WUAs quickly and it included the comparison with "control group" areas so that incremental benefits and the problems of WUAs could be clearly identified . Early detection of problems has ensured that the WUA standards are actually implemented resulting in quick resolutions, higher quality, and more sustainable WUAs . Implementation Unlike most Bank-funded projects, M&E was an actual component in its own right and thus received considerable attention during implementation including monitoring and evaluation aspects . A computerized system was developed and linked through the internet, providing full and rapid information and analysis on both project implementation progress and project impacts to PMUs at all levels . This MIS was an integral part of project management, handling procurement, financial management, and grant reimbursements, as well as standard progress monitoring and reporting. Utilization The ICR is relatively lightweight in its coverage of M&E utilization, but discussions with the TTL revealed that the usage by the farmers, local government and irrigation companies was substantial . The information has also been used by the financiers to further promote the concept and application of WUAs . The beneficiary survey findings were useful. a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards : Although it was technical assistance, the project was rated as a safeguard category B project due to environmental, dam safety, and social safeguards that might be triggered according to the ICR (p16). During implementation, suitable measures were taken where necessary to mitigate these safeguards issues . For example, the improvement of WUA facilities was undertaken in the off -season to avoid crop losses, and WUA offices were generally housed in existing buildings to avoid unnecessary land acquisition . Fiduciary : According to the ICR (p. 16), the project complied fully with Bank fiduciary requirements for financial management and procurement. The MIS developed under the project became an effective tool for monitoring project procurement and financial management. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Highly Satisfactory Satisfactory Performance targets and indicators were specified, but they were mostly output indicators focusing on the number of provinces covered or the number of policy documents and guidelines issued etc. While these outputs were relevant for the intended objectives, they did not allow to specifically measure impact of the project on target groups specifically identified in the PDOs (e.g. poor farmers, women, and disadvantaged groups) Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Highly Satisfactory Highly Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The project demonstrates that properly designed and implemented WUAs customized to meet the needs of a particular country can provide substantial economic, social, and environmental benefits and that with strong government support these benefits can be scaled up over wide areas under diverse conditions . In a project involving several ministries and implementing agencies, strong cooperative partnership and mutual support during project implementation can help maintain the focus on the principles and standardization of the WUAs. A sufficient allocation of funds and a wide focus on policy, training, capacity building, and M&E, as well as effective and comprehensive support from the Government at all levels were identified as the key success factors for ensuring efficient and sustainable WUAs . The project further demonstrates that improved WUAs based on democratic water management are pro -poor and provide an effective and low cost means to benefit disadvantaged groups including the poor and women . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The quality of the ICR is generally satisfactory . Since the project was financed primarily by DFID trust funds, it did not require a PAD and therefore the ICR provided additional background information and the context of the project . This made it slightly longer than is the norm . While the results framework and the baseline were not available at the start of the project, reasonable evidence and analysis were provided in the ICR to facilitate the assessment of project performance . Some of the outcomes were essentially output driven, but there was consistency in assessing the outputs throughout the ICR . The report followed the guidelines and drew appropriate lessons from its analysis . Nevertheless, it would have been useful if the cost table 1-1 given in the ICR (Annex 1) reflected the total project expenditures and not just those incurred under the trust funds . The ICR is also inconsistent in its ratings . While the summary table rates Borrower performance as highly satisfactory, the text in the ICR at one point describes Borrower performance as satisfactory . The coverage of M&E utilization was inferred rather than stated . a.Quality of ICR Rating : Satisfactory