ICRR 13309 Report Number : ICRR13309 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 04/13/2010 PROJ ID : P055022 Appraisal Actual Project Name : Irrigation US$M ): Project Costs (US$M): $30.82M $33.86M Development Project Country : Armenia Loan/ Loan /Credit (US$M ): US$M): $29.9M $27.87M Sector Board : ARD US$M ): Cofinancing (US$M): Sector (s): Irrigation and drainage (63%) Central government administration (19%) Sub-national government administration (9%) Law and justice (9%) Theme (s): Rural policies and institutions (29% - P) Rural services and infrastructure (29% - P) Participation and civic engagement (28% - P) Other social protection and risk management (14% - S) L/C Number : C3568 Board Approval Date : 08/30/2001 Partners involved : Closing Date : 03/31/2007 03/31/2009 Evaluator : Panel Reviewer : Group Manager : Group : J. W. Van Holst Christopher D. IEGSE ICR Reviews IEGSE Pellekaan Gerrard 2. Project Objectives and Components: a. Objectives: The project’s development objective (PDO), according to the PAD, is “to contribute to enhancing the profitability and sustainability of irrigated agriculture .� This is close, though not identical, to the statement of the PDO in the Development Credit Agreement (DCA): �to contribute to improvements in the standard of living in rural Armenia through enhanced profitability and sustainability of irrigated agriculture .� This Review uses the PDO as stated in the DCA since it embraces more clearly the longer term aim set out in the PAD : “…[to provide] the basis for stabilizing irrigated agriculture as a predominant source of productive employment, thus helping to prevent economically and socially unjustified rural/urban migration.� This project is part of a long term national program conceived in the mid -1990s to support the rehabilitation and restructuring of Armenia’s irrigation and drainage (I&D) system. In view of the magnitude of the task at hand and of the resources required, the Government opted to implement this program in two phases . The first phase consisted of emergency interventions to secure continuity of system operation, along with pilot /demonstration activities to develop a practical methodology for improving system management . Implementation of this phase was initiated under the Irrigation Rehabilitation Project (IRP), with financial support from IDA and IFAD . This was followed by the IFAD-supported North West Agricultural Services Project (NWASP) and the IDA-supported Dam Safety Project (DSP), which is expected to conclude the emergency intervention phase . The second phase, to which this project corresponds, focuses on restructuring and development, and on building a solid basis for system sustainability through physical upgrading of the infrastructure and empowerment of stakeholders to prevent its future deterioration . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): ( Part A in the Credit Agreement ) Rehabilitation of Critical Irrigation Infrastructure (US$4.33 million at appraisal, 1. (Part US$4.5 million at completion). The objective was to correct important deficiencies in priority sections of Armenia ’s Irrigation and Drainage (I&D) system which remained after the completion of the Irrigation Development Project (IDP). This included further rehabilitation of infrastructure that was severely deteriorated due to delayed repair and maintenance over an extended period . It also included some significant structural improvements to the I&D system that, among other benefits, would lead to major reductions in O&M burdens and costs . 2. (Part B in the Credit Agreement ) Conversion from Pump to Gravity Irrigation (US$13.9 million at appraisal, US$13.8 million at completion). This component aimed at reducing the reliance on high -cost energy-intensive irrigation by supporting the conversion of pumped irrigation schemes to gravity irrigation schemes in those cases where clear technical and economic viability had been demonstrated . 3. (Part C of the Credit Agreement ) Institutional Development (US$9.65 million at appraisal, US$12.6 million at completion). Based on the outcome of an institutional audit of the O&M function made during project preparation, the previous five agencies involved in O&M were to be restructured and strengthened under the project to allow for the effective introduction of participatory irrigation management (PIM). In that framework, responsibility and accountability for O&M would be clearly allocated between the remaining two types of public agencies : (i) a permanent Water Supply Agency (WSA) and the temporary Drainage and Irrigation Management Agencies (DIMAs); and (ii) the Water Users’ Associations (WUAs) which would operate as partners in the management of the national I&D system. A funding mechanism along with appropriate operational procedures was also to be put in place to ensure that irrigation management of the system at the farm level would be participatory and cost effective . This included the creation of two matching grant schemes (one for equipment and the other for rehabilitation ) and provision for technical assistance and training to WUAs and the central WSA . In the original design of the project the area targeted for improved irrigation management through WUAs was a pilot of 40,000 hectares. 4. (Part D of the Credit Agreement ) Project management (US$1.77 million at appraisal; US$2.97 million at completion) with the additional financing; and $ 2.97 million at completion). Management and implementation of this project was by a Project Implementation Unit (PIU) which was already established for the management and implementation of the Dam Safety Project (DSP). The PIU would also assume the responsibility for design and supervision of the quite simple rehabilitation and minor improvement proposals for irrigation systems within WUAs that were to be established under the support of the Institutional Development component . 5. (Part E of the Credit Agreement ) Additional Financing (US$5 million at appraisal; US$3.01 million at completion). The support to WUAs in this component was scaled up through an “Additional Financing� (AF) development credit agreement for $5.0 million which was approved by the Board in July 31, 2007 on the basis of recommendations in the Armenia CAS Progress Report discussed by the Board in March, 2007. The additional funding became Part E of the project and was (as stated in the Financing Agreement ) for the provision of works and consultants ’ services for the rehabilitation of tertiary canals ($4.55 million), training and technical assistance for the WUAs as well as their annual audit ($0.45 million). The closing date for the AF was March 31, 2009. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: One extension to the closing date to allow for the implementation of the Additional Financing (AF) Change in share of component costs financed by the Bank caused by the additional financing which went to component 4 71 percent of total credit was disbursed before restructuring and 29 percent after restructuring Shares of beneficiary contribution to total cost at appraisal and actual cost at completion were 19.3% and 20.1% 3. Relevance of Objectives & Design: The project ’s objectives remain highly relevant to the country ’s growth and poverty reduction priorities and to the Bank Country Assistance Strategies (CAS) CAS ). The Bank’s CASs for FY02-04 and FY05-08 stressed the importance of the rehabilitation of the national I&D system and the need for improved irrigation system management based on WUAs . The FY09-12 Country Partnership Strategy (CPS), although focused by necessity on assisting Armenia in addressing the impact of the global economic crisis, stressed again the important role of irrigation among the array of strategies to address Armenia’s vulnerability to the fallout of the crisis . The high relevance of the project ’s objectives to poverty reduction at a time of crisis was underlined by the fact that the legacy of unsatisfactory irrigation management under the former Soviet Union (FSU) had left the I&D system in a very inefficient state generating little production or employment. The relevance of the project ’s design was substantial . The focus on two core instruments -- namely crucial rehabilitation and new construction in the national I&D system and the development of a legal and institutional framework for the development of WUAs -- was appropriate. The rehabilitation and new construction encouraged farmers to embrace an institutional framework to provide the basis for the development of sustainable WUAs . Negotiations with communities to introduce Participatory Irrigation Practices in the 40,000 ha pilot area to encourage and support the change in irrigation management to WUAs was a substantial challenge and took considerably more time than planned in the PAD . The effort was, however, successful and the subsequent spread of WUAs across the national irrigation system confirmed the relevance of this core design element of the project . At Mid-Term Review it was noted that the performance of the WSA, which is responsible for the head works and the delivery of bulk water supplies to the WUAs, has been consistently weak, “partially due to the fact that the project design did not spell out clearly the actions to be taken under the project ." (ICR page 6). This was subsequently rectified. The results matrix in the PAD did not define intermediate outcomes, although they were defined for the AF . Some outcome indicators were over -ambitious or irrelevant and had to be dropped or revised during implementation. For example, the target for cost recovery of total O&M expenses of 100 percent by end March, 2007 (the closing date for the IDP) was unrealistically ambitious for an evolving irrigation system in which water delivery, production technology and the markets for agricultural products were far from perfect . The final target was subsequently amended to 60%. 4. Achievement of Objectives (Efficacy): To contribute to improvements in the standard of living in rural Armenia through enhanced profitability and sustainability of irrigated agriculture . Overall efficacy is rated Substantial . Increased farmer incomes: the ICR states that, as a result of the rehabilitation and new construction, “Irrigation water supply is more timely and in enhanced quantities, resulting in a significant increase in crop yields, irrigated areas; and in an overall increase in area sown /planted with high value crops (20 - 30 percent from the baseline)� (Annex 2). Increased irrigation area and changed cropping patterns in 2004-2008 contributed to an average increase in farm incomes of 30 percent, thereby achieving the relevant performance indicator target . High-value crops accounted fro 55% of irrigated agriculture in 2002; by 2009 this was 79.2%. There was a strong association between project -supported rehabilitation/construction activities on the one hand, and increased reliability of water delivery, and the financial performance and income sustainability of farmers, on the other. As a result, farmers received additional benefits from irrigation and were consequently more willing to pay for irrigation services. A beneficiary survey conducted in April, 2009, after project closure, revealed considerable satisfaction with project results. Farmers acknowledged substantial improvements in water supplies and about 72 percent saw that changes in the irrigation systems supported shifts to higher value crops . As a result of the above it would be reasonable to assume that increased farmer incomes thanks, inter alia, to a rehabilitated irrigation system, would improve rural living standards . Similarly, sustainability of irrigated agriculture may be assumed to depend to a considerable measure on the robustness of the institutional strengthening supported by the project – in particular of the WSA and the WUAs which carry the responsibility for financing and managing the system . The responsibility for oversight over the irrigation infrastructure was first consolidated under the Water Supply Agency (WSA) that was subsequently split into four regional branches in 2006. The branches became separate legal entities and took the responsibility for operation and maintenance of respective river basin ’s infrastructure and supply of irrigation water to WUAs of those areas . Currently, the regional branches of the WSA are providing water to existing WUAs, and the introduction of Water User Federations is planned remove further responsibilities from WSA, reducing its sphere of action and easing future efforts for its full restructuring . This is being addressed under the follow -up projects sponsored by USAID and the Millennium Foundation The Law of the Republic of Armenia "On Water Users Associations and Federations of Water Users Associations" was adopted in 2002 and this established a new legal -organizational entity - water users association (WUA). The state-owned irrigation systems and property were transferred to water users free of charge in 2003 and WUAs took over operation of 17,969 km of tertiary canals and their structures, and 728 tubewells. Since 2006, subsidies have become more rational and they are allocated directly to WUAs on the basis of a complex indicator that takes into consideration their planned budgets, planned use of energy and technical state of the infrastructure under their management . Irrigation management became more efficient . Electricity consumption was reduced by 16% or 28 million KWh between 2002 and 2008 even though the total irrigated area expanded from 113,400 ha in 2004 to 128,600ha in 2008. In addition higher system operating efficiency led to water savings of 65 million m3/year. Enhanced performance of farmers' irrigation institutions was substantially achieved . The establishment of 52 operational WUAs (target 8-10 WUAs) in all irrigation areas in Armenia (128,860 ha) in the space of 2-3 years on the basis of a 40,000 ha pilot was a major success for the government and the project . The number of communities participating in WUA rehabilitation schemes rose from zero to 103. This was just short of the target of 115, which could not be attained because grant funds were exhausted . After rehabilitation of irrigation structures under the management of the WUAs, farmers are responding enthusiastically to the potential for empowerment embodied in the WUA concept . Farmers satisfaction with the irrigation supply service increased from 7.5% in 2005 to 91.5% in 2009. WUAs have also improved cost recovery . Cost recovery rates rose from 8% for the system as a whole in 2001 to just over 45% (target 60% in 2009) at the end of 2007. Maintaining the satisfactory performance level of the WUAs may be a challenge in the future as subsidies are reduced and restrictions are imposed on their ability to charge for excess water use on some farms . This is particularly so in those areas served by electric pump systems where the current water price is at only half the level required for full cost recovery when all electricity costs are included . 5. Efficiency (not applicable to DPLs): Overall efficiency is rated High . The ICR re-estimated the project’s benefits and costs,and concluded that at completion its overall economic rate of return (ERR) was 43.5 percent. Using a discount rate of 12 percent the net present value (NPV) of the project is estimated at $122.3 million which is about 715 percent of the appraisal estimate . The ICR indicates that the results of this analysis are conservative because the benefits were confined to (i) the 13.7 percent increase in the irrigation area; (ii) diversification of production to high value crops and (iii) a 10-15 percent increase in crop yields due to improved availability of irrigation water . Although not stated in the ICR, benefits could presumably increase if farmers were to intensify their cropping intensity, which would be feasible with increased irrigation water supplies throughout the year . On the basis of farm budget analysis, the ICR estimated that, with improvements in the irrigation systems, model farms with typical cropping patterns (as determined using GIS-based data generated by the PIU ) will have the capacity to almost double family incomes . The farm models used to estimate the income increases for typical farms and the ERR covered three out of the four zones in the irrigation system . The estimated rates of return are high for this project because many of the costs of the irrigation system are sunk capital costs and the costs needed in this project for the development of WUAs were very small in comparison to the sunk costs . Other similar projects may need to invest far more in infrastructure improvements to achieve the same institutional outcomes . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 24.8% 100% ICR estimate Yes 43.5% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Based on sections 3-5, IEG rates project outcome as satisfactory . a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: There are two risks to development outcome : First, although the WUAs have thus far been a success, maintaining their effectiveness will continue to be a challenge. Their performance may be undermined by (a) a government decree of May 2003 that disallows WUAs to charge for “over normative" water use such as on farms which have more porous soils than average; and (b) the possibility that they will be unable to collect the higher water charges needed for pump systems as energy subsidies are reduced . Second, cost recovery, while much improved from the levels recorded prior to the project, was still less than 50% at closure. Putting these risks in the perspective of overall cost to farmers indicated moderate risks : water fees at present are only 6% of the production value (ICR page 17). a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry . This project built on the foundation of previous involvement by the Bank in the irrigation sector in Armenia and also benefitted from the Bank ’s involvement in the institutional reform and rehabilitation of irrigation systems in other countries . The project appraisal document was well prepared and adequately covered issues normally included in appraisal documents . The project risks, estimated as substantial, were, if anything, over-estimated. On the other hand, some of the outcome targets, such as the 100 percent cost recovery by the close of the project were clearly unreasonable and had to be amended during implementation . Quality of Supervision . The Bank supervision team had a proactive policy dialogue with the Government authorities and was effective in finding powerful political allies for supporting the IDP -led reform process. The Bank’s technical input was valued by the Authorities . Continuity was assured by the fact that the current TTL has been associated with the project as either a task team member or as TTL since the project started . at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Government Performance . The institutional reforms introduced with this project created difficulties within the Government because they meant the elimination of some institutions such as the DIMAs and reduced powers for others like the WSA. While the Government showed a slow commitment to the reform program for about two years it ultimately endorsed the changes as the various aspects of the project were implemented . As a result of this endorsement, the management of all tertiary irrigation in the I&D system became the responsibility of the WUAs, a highly positive development . Implementing Agency Performance . The PIU director, who had already handled a previous World Bank project, was a strong leader and had assembled competent staff . The ICR reports that �the PIU was responsible for an exemplary implementation and management role that included the IDP and other projects, being instrumental for a successful sector policy reform process �. The PIU was experienced in the Bank ’s procurement procedures and provided competent financial management for the project . Auditing arrangements and reporting were consistently satisfactory. Initially, the management of the M&E program was not successful, but this was resolved when an M&E specialist was retained (see Section 10 below. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: M&E design : The PAD specified an M&E framework but some of the indicators proposed (e.g. “improvement of the ratio of water entering the system/water delivered to the WUCCs and the WUFs ) were impossible to measure. In addition some indicators were unrealistic (e.g. the 100 percent cost recovery of O&M costs by the project closing date) and had to be amended during the project ’s life. The ICR identified the M&E design and implementation as a weak spot for the PIU until the project ’s mid-term review (MTR). The weakness stemmed from the lack of an M&E specialist in the PIU and an inadequate definition of guidelines for the M&E work . An M&E specialist was hired and by the end of 2005 a new M&E scheme was agreed and some of the indicators changed - as the ICR states - “in recognition of a better understanding of the systems, evolving events, realism, and the need for measurement� (ICR page 12). The cost recovery rate for O&M mentioned above was an important element of the changes in indicators, although not the only one . M&E implementation : Data specified in the M&E framework that were agreed to be relevant and useful were collected during project implementation. The project carried out a beneficiary survey which proved very useful . In addition the M&E team in the PIU developed a comprehensive geographic information system (GIS). M&E utilization : The data collected were used to good effect . For example cost recovery of O&M expenses was an important monitoring tool. The GIS data were used to monitor “areas cultivated, water volumes, losses, energy consumption for pumping and cropping patterns as well as fee collections and the financial situation of the WUAs� (ICR page 8). The ICR also drew attention to the fact that “at present the system does not have the ability to provide summarized reports responding to specific queries; data can only be queried at the level of the individual WUA. These reporting capabilities are being developed as the system is being developed nationwide � (ICR page 9). a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): An environmental Management Plan (EMP) was implemented and a report prepared and publicly disclosed on a hydrological-environmental investigation of the impact of an enlarged water intake for the irrigation system on the Araks river. According to the ICR, no negative impacts were found and there were no other environmental safeguard issues associated with the project . There were no unresolved fiduciary issues in the project although the ICR notes that two project status reports recorded unsatisfactory financial management performance because of delays in the provision of counterpart funds. The ICR notes that "overall, the PIU's internal controls system was capable of providing timely information and reporting on the projects " (ICR page 9). There were two important unintended positive outcomes of the project . The first, and the most impressive, was the increase in the number of WUAs established at 52 throughout the whole of the I&D system in Armenia . This achievement compared with the target of 8-10 WUAs to be established in the pilot area of 40,000 ha. The second was the development of the GIS -based Water Management Software that facilitates water management by the WUAs and the central authorities through accounting and analysis of a number of parameters that reflect the workings of the irrigation system . The availability of this information also improved the quality of the ERR estimates. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: A national "champion" who provides strong leadership for reform coupled with consistent and a dedicated Bank implementation support team establishes confidence and trust which will in most cases lead to successful outcomes. Relevant institutional change is enhanced by a sound understanding of the incentives that need to be provided to those who are stakeholders in the change . Sustained institutional change benefits from active participation by all stakeholders . Successful institutional change will be more easily achieved if winners and losers are clearly identified and brought into the process of change at an early stage with close attention to resolution of potential disputes . Accurate measurement of results is crucial for the understanding, monitorin g and evaluation of outcomes and the GIS information generated in the context of this project proved to be valuable during implementation and will be valuable in the future. 14. Assessment Recommended? Yes No Why? To verify ratings and document lessons, can be a useful input into a thematic study of irrigation development in developing countries. 15. Comments on Quality of ICR: Although the ICR was broadly consistent with the guidelines it had some drawbacks : (a) the style was on many occasions too descriptive and insufficiently evaluative with a sharp focus on outcomes – (for example section 2.3 is too discursive); (b) information on a specific topic is spread throughout the ICR - for example the AF is first mentioned incompletely in section 1.7 and it is then mentioned again in section 2.2 under implementation which is where one finds an implicit description of the additions/changes that are made by the AF which would more logically have been dealt with in section 1.7; (c) there were places in the text where the statements made were not based on the information in the PAD or in the ICR - for example the reference to "improved sector policy" in the second line of section 3.4 was a surprise because there had so far been no mention of sector policy - the explanation provided by the TTL was that this was ongoing work and could not be reported on . Yet it appeared in a section on the justification for the overall outcome rating . On the other hand the quality of the ERR analysis was very good and detailed, though even here it was not clear what characterized the three zones that were analyzed in the farm models except that they were "homogenous." a.Quality of ICR Rating : Satisfactory