ICRR 14364 Report Number : ICRR14364 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/27/2014 Country : Romania Project ID : P043881 Appraisal Actual Project Name : Irrigation US$M ): Project Costs (US$M): 102.98 95.05 Rehabilitation & Reform Project L/C Number : L4717 Loan/ US$M): Loan /Credit (US$M): 80.00 74.09 Sector Board : Agriculture and Rural Cofinancing (US$M): US$M ): Development Cofinanciers : Grant from the Board Approval Date : 07/31/2003 Government of Japan, Closing Date : 03/31/2011 06/30/2012 local communities Sector (s): Irrigation and drainage (94%); Central government administration (4%); Agricultural extension and research (1%); Vocational training (1%) Theme (s): Water resource management (29% - P); Other rural development (29% - P); Environmental policies and institutions (14% - S); Rural services and infrastructure (14% - S); Rural policies and institutions (14% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : J. W. Van Holst John R. Eriksson Christopher David IEGPS1 Pellekaan Nelson 2. Project Objectives and Components: a. Objectives: The Project Appraisal Document (PAD) for the Irrigation Rehabilitation and Reform Project (IRRP) states that this project's development objectives (PDOs) were to "achieve more economic use of irrigation through changes in farmers' and the Land Reclamation Agency's (LRA's) behavior, and (ii) increase agricultural productivity in the project areas" (page 3). The Loan Agreement for IRRP states that the project's objectives are to "increase agriculture productivity in the Project area, promote community coherence and initiative in irrigation management and enhance institutional sustainability in the irrigation sector " (Schedule 2). Given the difference between the two versions of the objectives, the version in the Loan Agreement will be used as the basis for this Review because it is legally binding (and as per the IEG/OPCS harmonized guidelines). b.Were the project objectives/key associated outcome targets revised during implementation? Yes If yes, did the Board approve the revised objectives /key associated outcome targets? Yes Date of Board Approval: 07/08/2010 c. Components: Component 1: Rehabilitation of Irrigation Schemes (appraisal cost US$77.16 million; actual cost US$85.57 . This component absorbed 90 percent of the project investment and was mainly aimed restoring the original million). capacity of economic irrigation schemes through rehabilitation of deteriorated irrigation infrastructure (PAD, page 13). This component comprised the following six activities or sub -components: (a) Rehabilitation of Main Schemes. Phase I sub-projects, namely Sadova-Corabia (SC) and Nicoresti-Tecuci (NT), were identified based on selection criteria specified in the PAD . These criteria were a pumping head of less than 70 meters, potential economic viability, more than 50 percent coverage by Water Users' Associations (WUAs), and more than 80 percent of farms were private. Phase II irrigation schemes were selected during implementation of Phase I, using the same criteria . (b) Rehabilitation of Major Pumping Stations. Rehabilitation of about 20 of the most seriously deteriorated major lift pumping stations serving schemes that were clearly economically viable . Criteria were established to identify these stations, including low pumping head and evidence of demand, such as establishment of Water Users' Associations (WUAs) and use of irrigation water. (c) Water Metering Equipment. Government policy under its the new water pricing system made volumetric charges for water supplied to WUAs . The project was to finance the installation of water meters at the pumping stations of all WUAs which had signed a service contract with the LRA . (d) Monitoring and Control Equipment. The project was to fund the most technically suitable and cost effective improvements in water management facilities in the rehabilitated schemes. (e) Rehabilitation of On-Farm Irrigation Facilities. This sub-component was aimed at responding to WUAs ’ needs and requests to rehabilitate or restore on -farm irrigation facilities, and support their initial functioning . (f) Feasibility Studies for Phase II . Preparation of feasibility studies for Phase II main scheme rehabilitation . Component 2: Support for Institutional Reform in Land Reclamation Sector (appraisal cost US$2.74 million; actual cost US$3.20 million). This component was intended to support implementation of institutional reform and the initial development of the LRA and WUAs. The following five activities were proposed : (a) Support for the Restructuring of the National Company for Land Reclamation (SNIF) and the Initial Functioning of the LRA. This sub-component was to support: (i) Preparation of SNIF Restructuring which involved assisting the Government prepare a plan to manage and privatize SNIF’s non-core businesses; and (ii) Support for LRA Initial Functioning through technical assistance and training and equipment to support the LRA ’ s initial operations and a M I S system, training in maintenance planning, and development of service standards and customer charter. (b) Support for the Ministry of Agriculture, Forestry, Water and Environment (MAFWE). The project would provide assistance to (i) Land Reclamation Directorate (LRD) in the MAFWE for developing procedures for reviewing LRA service charges and performance, and (ii) WUA Monitoring Unit for developing a database and procedures for monitoring development of WUAs . (c) Economic Analysis of the Irrigation Sector, Changes in Response to the New Price and Subsidy Policy . The project would analyze the effects of the new pricing and subsidy policy on the structure of the irrigation sector and to prepare a strategic plan for future development of the sector . (d) Strengthening Agriculture Extension Services and Use of Agro -Chemicals. The project would assist extension services in raising awareness of environmental impact of agro -chemicals. (e) Support for WUAs This sub-component would mainly consist of technical assistance to provide comprehensive training to WUAs. A substantial portion of this sub -component would be parallel financed by the Unites States Agency for International Development (USAID). Support from IRRP would start in 2005 (after the USAID program is completed) as part of Phase II of the project . Component 3. Exploring Technologies for Reducing Irrigation Energy Consumption (appraisal cost US$1.0 million; actual cost US$0.32 million). The long-term future of irrigated agriculture in Romania depends on a reduction in energy costs by reducing pumping head wherever possible . The project would start to address this problem through price and subsidy reform which would lead to the eventual elimination of irrigation at the higher elevations . Component 4. Project Administration (appraisal cost US$4.24 million; actual cost US$5.96 million). This component would finance national staff, international and national consultants, vehicles, and office equipment to support implementation of the project. Project Restructuring Although its objectives were not changed the project was restructured twice. The first was on July 8, 2010 which extended the closing date to March 31, 2012. While there were no changes in the PDOs, the restructuring changed the scope of the investment program in response to increased costs and depreciation of the loan currency which entailed modifications in the results framework including reduction of some output targets . Restructuring also included four new intermediate outcome indicators and the elimination of others in the results framework . The second restructuring on February 8, 2012 was only to extend the closing date by three months to June 30, 2012 to allow sufficient time to complete the project d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : The project's estimated baseline cost was $ 85.4 million. With physical and price contingencies it was $102.98 million. Actual total project cost was $ 95.05 million or 92.3 percent of the estimated cost at appraisal . Financing : The IBRD loan financed and the WU As (restructured as Water Users' Organizations ) financed $74.09 million and $1.55 million respectively which accounted for 78% and 2 percent of total actual costs . An additional amount of $650,000 from grant funds provided by the Government of Japan and administered by the World Bank was allocated to support the development and restructuring of SNIF, technical assistance to WUAs , feasibility studies for irrigation works, and capacity building for project implementation . Borrower Contribution : The Government contributed $19.41 million or 20 percent of total actual project cost Dates : As noted already, the project's closing date was extended twice . The first was from March 31, 2011 to March 31, 2012 and the second was to June 30, 2012. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Rating: Substantial . At appraisal Romanian irrigation systems were part of an extensive infrastructure built up during the previous decades, but still managed as part of a command economy (see Government of Romania, "National Rural Development Programme, 2007-2013", page 48). A Bank financed Irrigation and Drainage Study study completed in 1994 concluded that Romania's irrigation systems were in urgent need for rehabilitation of infrastructure, transformation in patterns of production, reform in irrigation water tariffs and agricultural subsidies, and development of WUAs. As emphasized in the PAD, without reform irrigation systems were projected to become non operational or remain uneconomic. Reforms were also needed to facilitate the country ’s accession to the European Union (EU) which was eventually approved in January 2007. The Government's intention to implement irrigation reforms was elaborated in a letter to the Bank in April 2003 in support of IRRP (PAD, page 99). The project addressed the main agricultural sector reform issues with an emphasis on supporting changes in water pricing and subsidy policies in the irrigated sector, key institutional reforms in the sector, and an investment program designed to rehabilitate irrigation systems which were economically viable . At exit the project remained substantially relevant to the 2009-2013 Country Assistance Strategy which drew attention to this project's importance to the improvement of Romania's agricultural productivity and competitiveness in agricultural exports to the EU (paragraph 87). b. Relevance of Design: Rating: Substantial The PAD did not include a results framework . Instead it provided a "Project Design Summary" which presented the project's objectives and key performance indicators . However, indicators in the framework were only qualitative . In addition, the design summary did not provide a results chain showing intermediate outcomes that would provide for an ongoing assessment of the project's progress during implementation . The PAD noted that "A participatory monitoring (and) evaluation would be carried out during the first six months of project implementation to develop a baseline survey and monitoring indicators " (page 34). At the first restructuring (July 2010) the results framework was modified by including specific indicators and targets for the final outcomes and four intermediate outcomes . The project's original design addressed the Bank's strategic objectives of revitalizing economic activity in rural areas and strengthening community based development (PAD, page 4). Its core design elements of rehabilitation of irrigation systems, reforms of institutions associated with irrigation systems such as the Land Reclamation Company (SNIF), water pricing reforms to stimulate the eventual elimination of subsidized irrigation, and changes in the legacy of over-investment in irrigation infrastructure by providing an exit strategy for uneconomic irrigation schemes were all highly relevant. However, except for the PAD's direct reference to the role of Water Users' Associations, it was silent on the role of "community coherence and initiative in irrigation management " (part of the Loan Agreement's PDO adopted for this Review). This was an ambitious design involving both policy reforms and investments that required considerable collaboration and participation from the Government, irrigation communities and farmers to be successful. Indeed, IRRP was designed more like a program than a project . The project's design was substantially relevant to the need for reform in Romania's irrigation sector . 4. Achievement of Objectives (Efficacy): IRRP aimed to achieve its development objective through three programs : (a) increase agriculture productivity in the project area; (b) promote community coherence and initiative in irrigation management; and (c) enhance institutional sustainability in the irrigation sector . This section will assess whether these objectives were achieved and the extent to which achievements were attributable to the project . . Objective (a) - Increase agriculture productivity in the project area Rating: Substantia l Outputs The project aimed at increasing agricultural productivity through the rehabilitation of irrigation canals and pumping stations, rehabilitation of tertiary irrigation infrastructure, and the provision of field irrigation equipment . Irrigation infrastructure rehabilitated : Compared with a 312 km target, 336 km of canals were rehabilitated. Compared with a target of 20, only nine pump stations were rehabilitated and hence only about 100,000 ha served by pump schemes benefitted compared with a target of 400,000 ha (PAD page 41). - While arrangements for quality assurance by the PMU were in place, some deficiencies in project works were found and repaired by contractors . The ICR stated that rehabilitated infrastructure performed as expected, but maintenance of canals was not adequate - mainly due to lack of funding. Provision of field irrigation equipment : A support program for the WUOs included rehabilitation of tertiary irrigation canals/distribution systems through full rehabilitation or replacement of irrigation equipment and critical spare parts, 140 WUOs were provided with irrigation equipment compared with an original target of 160 and a revised target of 145 (see Restructuring Paper, July 8, 2010, Annex 1), Outcomes Improved productivity . For the project as a whole average yield targets were exceeded by 8 percent for corn and by 18 percent for vegetables (ICR, Data Sheet and paragraph 41). The Project Impact Evaluation (2012) estimated a 7 percent increase in corn yields and a 22 percent increase for vegetables (page 8). But, no targets were mentioned in the PAD or in the ISRs. In the light of the observed stable cropping patterns during project implementation (see Project Impact Evaluation, 2012, page 6) the observed yield increases are assumed to be attributable to the improved irrigation water delivery resulting from irrigation rehabilitation . * The share of high value crops in the project area increased from 59.6 percent in 2005 to 88 percent in 2011, exceeding the target of 70 percent (ICR, paragraph 41). The crops covered by this assessment of increased shares included corn and soybeans which are not usually classified as high value crops . * There were important differences in results between irrigation systems supported by the project. The Sadova-Corabia (SC) and Nicoresti-Tecuci (NT) schemes, contained small farms (e.g. about 3 ha fragmented into a number of parcels on sandy soils in SC ), WUAs were not well managed, and crop productivity targets and farmers' incomes did not improve . In the other three schemes, where the majority of farms were medium to large-scale, yields and incomes met the project's targets (paragraph 13). *The project's Impact Evaluation (2012) concluded that "The IRRP impact cannot be completely assessed since a large part of the rehabilitation works on the main infrastructure was completed in 2011 and other works are still to be completed in June 2012. However, an estimation of this impact on the area irrigated is assessed as moderately satisfactory " (page 123). Irrigation intensity . For the project area irrigation intensity varied during implementation from high levels of 40 percent to lows of 10 and 13 percent in the last two years (less than the baseline of 15 percent), but in general achievements were well short of the target which was based on 80 percent of the area contracted for irrigation (see first restructuring paper ) but only 70 percent of the area actually used for irrigating crops, resulting in a target irrigation intensity for the command area of 56 percent. Irrigation intensity is not, however, a satisfactory outcome indicator for irrigation development in Romania because irrigation is supplementary to natural rainfall . Hence a low irrigation intensity usually reflects above average rainfall which occurred in 2010 and 2011 and irrigation intensities declined sharply (see Aide Memoire for May 7-15, 2012 supervision mission). The ICR adds that during project implementation irrigation intensity was also strongly influenced by irrigation subsidies (Annex 2, paragraph 1). Irrigation intensity and overall productivity are therefore affected by a range of exogenous factors. Area irrigated in terraces below 70 meters elevation . According to the ICR areas irrigated on terraces below 70 meters, increased during the project to over 90 percent of the project area by 2009-2011 exceeding the 80 percent target. This change reduced energy consumed by pumping stations by 40 percent from about 922 kWh per 1000 cubic meters (1997-2004) to 553 kWh per 1000 cubic meters (2005-2011) which led to a saving about US$3 million per year (paragraph 41). There is no known information available about the resulting improvements in the overall productivity of irrigation systems as a result of these changes . In summary, the efficacy of this objective was rated as substantial Objective (b) Promote community coherence and initiative in irrigation management Rating: Modest Outputs Reorganization of WUAs into WUOs had little practical impact . The reorganization of WUAs entailed their registration as WUOs under the jurisdiction of MARD (Ministry of Agriculture and Rural Development - formerly MAFWE), the separation of powers within WUOs between overall management and day to day technical operations, and establishment of WUOs as institutions for the "public good". These characteristics put WUOs at the center of the strategy for community driven irrigation management reform . However WUOs, particularly those involving small-scale farmers, were not supported as well as envisaged . The USAID-funded Development Project which closed in 2005 should have been followed up by specialized units in MARD to provide training on technical and financial management as well as extension services for farmers, and to facilitate the development of the WUAs, was not done. Small -scale farmers did not benefit from WUOs . Improvements in water management by WUOs were not Small- achieved because of "insufficient project support to institutional development and good governance within WUOs in the two schemes rehabilitated in Phase I " (ICR, paragraph 52). As a result the Sadova-Corabia (SC), Nicoresti-Tecuci (NT) schemes representing 50 percent of the project area, where most of the small -scale farmers were located, and where more than 50 percent of the available funds for irrigation rehabilitation were invested, irrigated only 6 to 8 percent of their command area due to the failure of the WUO management . Similar shortcomings in support to WUOs occurred in Phase II of the project (ICR, paragraph 44) Physical Support to WUOs : At project closing, water meters had been installed in 309 WUOs in power pump systems compared with a target at first restructuring of 320 . Office and communication equipment were provided to 136 WUOs compared with a target at first restructuring of about 150. Motor cycles were provided to 127 WUOs covering 286,000 according to the task team leader, compared with a target of 130 covering 305,000 ha. However, it is clear that the WUOs made inadequate use of this physical support . Exploring Technologies for Reducing Irrigation Energy Consumption . This third project component was delayed until the project's end, and not implemented properly (ICR, paragraph 44). Outcomes Establishment of WUOs . At the project's close about 458 WUOs and 12 federations of water users organizations (FWUOs) managed pressure pumping stations (PPSs) and distribution structures without water subsidies on about 1.1 million ha of irrigable land (which included schemes outside the project area ). Development of WUOs . However, as the ICR states, the WUO Monitoring Unit in the MARD "never assumed its agreed role of supervising their activities and facilitating the development of WU Os" since it had nether the required staffing nor territorial representation to perform its facilitating and monitoring duties (paragraph 44). Community Coherence . In the end, for most of the schemes, "community coherence" was not promoted. On balance the outcome of this objective of the project was modest Objective (c) Enhance institutional sustainability in the irrigation sector . Rating: Modest Outputs Institutional reforms did not meet intended goals . The project contributed to identifying and drafting a new institutional and legal framework for irrigation institutions . The National Company "Land Reclamation" (SNIF) was reformed into the Land Reclamation Agency (LRA) which was then renamed National Administration/Agency of Land Reclamation (ANIF) but, according to the ICR, the management of ANIF, drawn to a large extent from SNIF, was resistant to change and supported by the MARD and MPF, slowed down implementation of reforms (ICR, paragraph 62). Effectiveness of reforms of subsidies and water charges was compromise d.. Although the reduction of subsidies and increased water charges was officially approved, subsidies to irrigation charges did not gradually diminish as agreed by the Government so as to allow farmers to adjust to a full cost recovery regime . Rather, subsidies were increased and remained in place until Romania's accession to the EU and, following a three year grace period, were eliminated in 2010 suddenly to conform to EU policy. MARD changed the approach to subsidies not only in conflict with the cost -sharing principle stated in the EC Water Directive, but more importantly, sent mixed signals to WUOs and consequently distorted incentives for farmer behavioral change . Nevertheless, the new water charging formula (annual fee to cover fixed cost and volumetric tariff for variable costs) and the long-term irrigation service and water supply contracts remain in place in spite of the MARD decision to introduce the new water pricing policy without any phasing (para 43). Outcomes Insensitive approach to change in water pricing policy . The policy of continuing irrigation subsidies for three years after EU accession in January 2007 that led farmers to believe that subsidies would continue (See ICR, Annex 6, paragraph 3) and then suddenly withdrawing the subsidy in January 2010 was an insensitive approach. After the withdrawal of subsidies farmers perceived that water tariffs charged by ANIF and the WUOs were too high, triggering reduced use of irrigation (ICR, paragraph 44). While in principle the elimination of subsidies (along with other irrigation policy and institutional changes ) was correct and will lead to the sustainability of the irrigation sector, the manner in which the new water charging formula was introduced was inappropriate and counterproductive, 5. Efficiency: Rating: Modest The PAD estimated economic rates of return (ERRs) for the first phase SC and NT schemes . The estimates were, respectively, 13.0 and 17.6 percent, and 15.3 percent for the overall project . The ICR calculated ERRs for the project's five main rehabilitated schemes, namely SC, TN, TB, TV and C âmpia Covurlui, as well as for the overall project. The overall ERR was 14.7 percent, with considerable differences among the five schemes . The first phase schemes (SC and NT), where small-scale farmers predominated, showed very low ERR (2.8 and 9.6 respectively) with negative net present values (NPVs). For the other three schemes where large -scale farms dominate ERRs were between 20 and 32 percent. Overall these results reflected the different levels of productivity achieved in the different schemes with the poorest performance in those schemes where small -scale farmers predominated. A sensitivity analysis showed that reductions in output prices and area irrigated by 10 percent or a 10 percent increase in production costs would each decrease the overall ERR to 11.3, 12.5 and 11.0 percent respectively. Another aspect of the project's efficiency was the effectiveness of policy changes that were in turn intended to achieve a more efficient irrigation system . While some of the reforms such as institutional and water tariff policy changes were ultimately made by the Government, their implementation was not as agreed . The Impact Evaluation (2012) concluded that the "The project's impact on the efficient allocation of subsidies is unsatisfactory " (page 215). Based on the inefficient result of the irrigation development for the welfare of small -scale farmers and the inadequate implementation of the institutional and water pricing policy reforms, the efficiency of this project was evaluated as modest. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 15.3% 100% ICR estimate Yes 14.7% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of the project's objectives and design was substantial. While the productivity increases in irrigation rehabilitation were also substantial, the promotion of community coherence and initiative in irrigation management and institutional changes (including the approach to water pricing changes ) in irrigation schemes were only modest . The project's efficiency was also modest . On balance the project had significant shortcomings and its outcome is therefore rated as moderately unsatisfactory . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: Despite the eventual implementation of a reformed water tariff policy beyond the project's closing date, the ICR rated "the long-term sustainability of the expensive rehabilitated irrigation structures as uncertain and the risk to the development objectives as substantial" (ICR, paragraph 55). As noted already, the maintenance expenditures for on-farm irrigation systems (which under the new institutional arrangements were owned by the WUOs ) paid by water users were well below the requirements for long -term functionality of the rehabilitated infrastructures . Other weaknesses were: (a) the limited impact of physical improvements in water delivery on increased agricultural production and the adoption of improved cropping patterns, (b) the weak capacity of farmers to pay for the higher water tariffs as subsidies were eliminated; and (c) the inadequate support to small -scale farmers from WUOs to manage, operate and properly maintain irrigation systems (ICR, paragraph 55). These issues raised questions about the sustainability of the project's physical, institutional and policy improvements . Failure to support small-scale farmers' WUOs in SC and NT schemes, and the low irrigation system maintenance budget in the other three schemes, raised significant questions about the project's sustainability . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The ICR states that the Bank team was pro -active and persevering in the project's preparation but not sufficiently effective in formulating the project's design, and laying the groundwork for the reform agenda in the face of strong resistance from a conservative bureaucracy with vested interests in resisting changes (paragraph 56). However, it was also an over ambitious project involving many activities in the project area an extended effort outside the project area where WUOs were supposed to benefit, and substantial changes in institutions and policies. In addition, readiness at entry was not assured because the project (a) failed to define the required capacity building and extension activities for small -scale subsistence irrigation farmers; (b) had no vision for the transformation of traditional small-scale farming systems into commercial and more efficient operations able to pay for unsubsidized water tariffs; and (c) did not include a plan to support transformation of WUAs into WUOs when USAID financing for this activity terminated . These were significant shortcomings . at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Once the new legislation converting SNIF to ANIF was approved, the Bank successfully backed the Government in resisting attempts by an entrenched bureaucracy to reverse the reforms, and worked hard to improve project performance following the project ’s initial unsatisfactory performance (ICR, paragraph 57). The Bank team paid close attention to implementation of the construction activities and the quality of the works and followed up with the PMU to ensure that implementation deficiencies were addressed by contractors . However, the ICR asserted that supervision "should have paid more attention and reacted more proactively to the slow implementation of institutional activities " and the low irrigation system maintenance budget . Also poor performance of small-scale farmers during Phase I of the project "should have been handled through stronger interventions with technical assistance, strengthening extension services, M&E, training, etc ." (paragraph 59). Lack of adequate attention to these issues compromised the results and the project's sustainability . Although the supervision team and Bank management presumably understood the threats to the project resulting from a lack of Government commitment to the project, reflected in its mismanagement of subsidy reduction and stalled implementation due to inadequate allocations of project financing during the global financial crisis years , the necessary strong actions were not taken. The ICR rightly suggested that Bank management should have given consideration to suspending the project because of the Government's backtracking on policy commitments (paragraph 60). In summary there were significant shortcomings in the Bank's supervision performance . Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: Despite the Government's commitment to the project at appraisal, its ownership of the project's objectives was almost non-existent during implementation because vested interests undermined the implementation of reforms . The counterproductive increases in irrigation subsidies during the three years grace period before their compulsory elimination (to comply with EC directives) provided clear proof that the Government failed to change its approach to the irrigation sector . However, the difficult institutional reforms were passed and implemented, and backsliding on those reforms was prevented during the project . Although the project promoted campaigns for the reorganization of WUAs into WUOs and training sessions, they were targeted only at farmers within schemes where pumping height was below 70 m, At the same time MARD subsidies supported investment in the rehabilitation of tertiary infrastructure (managed by WUOs) within schemes where pumping head was above 70 m - contrary to the project's objectives . In addition, ANIF operated ineffectively and paid little attention to farmer’s needs (ICR, paragraph 63). The ICR also mentions problems of inadequate and delayed counterpart funding by the Government which led to persistent funding gaps and as a result payments to consultants were delayed by many months . Overall there were major shortcomings in the Government's performance Government Performance Rating Unsatisfactory b. Implementing Agency Performance: The Project Management Unit (PMU) within the new MARD, was staffed mainly by technical specialists who paid most of their attention to the project's engineering issues . There was more focus on monitoring implementation of civil works, ground water and surface water quality, environmental issues, procurement and installation of equipment for the WUOs than to the impact of the project on agricultural performance . The MARD’s lack of ownership of the project's objectives was reflected in an inadequate staffing in the PMU staffing, the LRA and in the WUOMU, as well as weak oversight of project implementation . The 25 percent salary cuts during the financial crisis also undermined PMU morale and compromised its dedication to project implementation. Frequent management changes in MARD and ANIF signaled instability to all sector stakeholders. According to the ICR the MARD did not exercise much oversight over ANIF or the project and it was not an effective counterpart partner to the PMU and Bank ’s efforts. These issues amounted to significant weaknesses in the implementing agency's performance (ICR, paragraph 66). Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The PAD allocated the responsibility of M&E to the PMU but it did not specify an M&E design . Not surprisingly, therefore, M&E got off to a slow start . The baseline survey (planned for implementation within six months of project approval) was not completed until more than two years after effectiveness, An improved results framework was not formally approved until six years after approval as part of the first restructuring paper . b. M&E Implementation: Neither Annex 1 of the PAD, nor the PDO indicators in the results framework in the first restructuring paper, nor the results matricies in the ISRs included indicators for "increased agriculture productivity" which was the core project objective. The Data Sheet for the ICR does, however, include PDO indicators for agricultural productivity but there is no explanation of their basis and source It also includes other indicators relevant to the PDO, but they are not the same as the PDO indicators in the first restructuring paper . Preparation of project quarterly reports only began in August 2005 which was over one year after effectiveness . Project data existed at the level of thematic teams, but at the ICR states these were largely focused on project outputs such as "infrastructure rehabilitated" and "equipment delivered", leaving large areas of project performance not covered (paragraph 30). Institutional development of WUOs was not well monitored . As result the WUO database, while adequately updated, provided very little information on how WUOs really functioned . However, a Project Impact Assessment (PIA) of WUOs at project closing focused on problems faced by farmers, reasons for reducing irrigated areas, problems with ANIF, participation of users in their WUOs affairs, use of dispute resolution events, and the frequency of meetings between WUO managers and water users . The ICR observed that this type of survey should have taken place earlier for better PMU management and to provide the basis for remedial actions during implementation (paragraph 31). As a result of these M&E flaws, weak WUO performance was not revealed. After the project closed the Impact Evaluation concluded that overall, "The project impact on WUO management and personnel was unsatisfactory " (page 121). c. M&E Utilization: There is no evidence of constructive use of M&E data . The ICR states that "The data collected on outcomes and impacts was not good for detecting even the severe performance problems as those being evident in SC and NT schemes, where more than half of funds were invested without increasing agricultural productivity in the project area or doing a more economic use of irrigation resources through change of farmers ’ and ANIF’s behavior" (paragraph 30). M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: Environment . The project was classified in environmental category B . The safeguards triggered were an environmental assessment and international waters . Attention to environmental issues was limited until the MTR in November 2007, with only one environmental review conducted in 2006. After the MTR satisfactory environmental oversight was provided by a specialist based on an environment monitoring plan developed by the PMU . Comprehensive environmental monitoring reports provided detailed analysis of water and soil quality in the project area which, because of the availability of a baseline before irrigation works started, made it possible to undertake analysis of the project's impact on the environment . The Impact Evaluation concluded that "The impact of the project rehabilitation works did not affect the environment, moreover by increasing the efficiency of the schemes, the impact was positive" and evaluated as "very satisfactory". With respect to international waters, the issue was the possible impact of the project on the Siret and Danube rivers which are international waterways along which Serbia, Bulgaria and Ukraine have an interest as riparians . The PAD claimed that the project was aimed at rehabilitation and renovation and would therefore not expand irrigated areas . In addition the project would not cause appreciable harm to other riparians, and would not be appreciably harmed by the other riparian's water use . The PAD therefore argued that in accordance with World Bank Operations Policy 7.50, paragraphs 7(a) and 8(c) the project was exempted from notifying other riparians (paragraph 7.2 on page 34). Dam Safety . Issues regarding the safety of dams, although not formally triggered, were (according to the ICR) properly handled when a potential construction of a new dam in the proximity of the NT scheme occurred (paragraph 35). Social Issues . The original social assessment for the project, the Quality at Entry Review in 2005 and the Quality Assessment of Lending Portfolio Review in 2008 all identified a number of social and institutional issues from inequality and elite capture to irrigation costs that warranted close supervision . However, these issues were given insufficient attention by the authorities despite their being flagged repeatedly in supervision mission aide memoires. b. Fiduciary Compliance: Financial Management . Supervision of the project’s financial management (FM) performance was carried out regularly. The PMU had a high FM capacity until two years before the project closed and according to the ICR its performance of FM functions was moderately satisfactory up to this point (paragraph 32). Thereafter FM capacity declined when reductions in staff salaries (affecting all government employees ) caused staff departures . Nevertheless, quarterly unaudited financial monitoring reports continued to be adequate, though received with some delays. All audit reports were received by the due dates and none were qualified . Procurement . In spite of poor initial procurement preparation, procurement oversight was effective . But by the end of the implementation period the procurement rating was downgraded to moderately unsatisfactory due to procedural deficiencies in the implementation of the last International Competitive Bidding (ICB) procedure under the project . The most consequential deficiencies were; (i) the PMU’s introduction of substantive changes to the first amendment of the bidding documents, after its review and clearance by the Bank without consulting the Bank; and (ii) the provisions of the bidding documents were not strictly followed by the evaluation committee during the evaluation stage which resulted in substantial modification of a bid after the bid submission deadline . The PMU was requested to submit clarifications, and finally the bid was cancelled as the closing date was imminent. c. Unintended Impacts (positive or negative): The ICR comments that "The poor results may have negative effects over future public investments aiming to develop irrigation in Romania, in spite of offering a good potential in some of the systems if they are well managed (paragraph 53). d. Other: A Stakeholder Workshop was held during the final supervision mission . Overall, participants were highly appreciative of the infrastructure rehabilitation investments . However, they recognized that several factors within the control of the Government adversely affected the results, including flaws in selection of schemes, the lack of MARD and ANIF leadership, commitment and continuity towards the project objectives and the required software support activities. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR extracted many lessons . They can be summarized as: Where difficult reforms and entrenched interests are involved, the Bank should require more concrete evidence of commitment to the project ’s objectives before approval The absence of implementation readiness aggravated an already ambitious and difficult project . Weak M&E contributed to many implementation problems problem, The economic analysis of projects under preparation should be used as an iterative planning and design tool, rather than as a simple requisite to justify pre -defined investments Where insufficient commitment is evident, the Bank should be ready to walk away from the project . When subsidies on water tariffs are phased out in irrigation systems the viability of agricultural production depends on adjustments in cropping patterns and improved technology which is not likely to happen without strong technical support through extension and improved marketing efficiency . Irrigation development projects require good managers and sound institutional arrangements with good multi-disciplinary teams including social and agronomic expertise to succeed . A lesson that could have been added for this project is the importance of a soundly designed and executed M&E system providing timely information and analysis of the project's progress and achievements . The absence of a sound M&E system was a major drawback during project implementation . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: This was a frank, informative, internally consistent and outcome oriented ICR . The lessons derived from the project were well based on the analysis of project performance, although the source of the targets for agricultural productivity mentioned in Section 4 above was not explained. In addition, lessons from the inadequate M&E systems should have been addressed in the ICR . Information from the Impact Evaluation in 2012 could have been included more often than the one reference in the Data Sheet and the two references in the main text in the ICR . There were many more references to the Project Impact Evaluation in the Government's ICR . a.Quality of ICR Rating : Satisfactory