ICRR 13374 Report Number : ICRR13374 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/28/2011 PROJ ID : P035752 Appraisal Actual Project Name : Integrated Irrigation US$M ): Project Costs (US$M): 528.00 774.80 Modernization Project Country : Mexico Loan /Credit (US$M Loan/ ): US$M): 303.00 303.00 Sector Board : ARD US$M ): Cofinancing (US$M): 0.00 0.00 Sector (s): Irrigation and drainage (87%) Crops (7%) Central government administration (4%) Other social services (2%) Theme (s): Export development and competitiveness (25% - P) Water resource management (25% - P) Other rural development (24% - P) Decentralization (13% - S) Rural services and infrastructure (13% - S) L/C Number : L7206 Board Approval Date : 12/09/2003 Partners involved : Closing Date : 09/30/2009 09/30/2009 Evaluator : Panel Reviewer : Group Manager : Group : Flora Janet Nankhuni George T. K. Pitman IEG ICR Review 1 IEGPS1 2. Project Objectives and Components: a. Objectives: The statements of the project development objective (PDO) in the PAD and the Loan Agreement differ, as do both of them from the statement in Annex 1 of the PAD. The text of the PAD (p. 2) says: “The Project’s main objective is to assist the Government of Mexico in its efforts to adopt a new model to improve the competitiveness of irrigated agriculture and the efficiency of irrigation water use, in both Irrigation Districts (Distritos de Riego, DR) and Irrigation Units (Unidades de Riego, UR), that would include the development of partnerships between the public and private sectors as well as effective institutional collaboration within the public sector, plus the strengthening of a demand-driven approach to investment decisions and financing�. The PDO statement in Annex 1 of the PAD says: i) to improve the competitiveness of irrigated agriculture in both Irrigation Districts and Irrigation Units in Mexico, using an integrated approach; ii) to modernize existing irrigation infrastructure and improve water use efficiency; iii) to promote diversification to high-value crops; and iv) to consolidate transfer of irrigation infrastructure to water user's association. The Loan Agreement’s PDO statement (p. 21) says: “to assist the Borrower in its efforts to adopt a new model to improve the competitiveness of irrigated agriculture and the efficiency of irrigation water use in Irrigation Districts and Irrigation Units�. IEG finds the Loan Agreement to be selective since it does not include the other elements of the objective statement on page 2 of the PAD: that is, the development of public and private sector partnerships and strengthening of demand-driven approaches that, from the project design, were clearly intended. This ICR Review uses the objective statement in Annex 1 of the PAD as it is more monitorable. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): i) Modernization and Rehabilitation of Irrigation Infrastructure in DR and UR (appraisal cost:US$406.00 m; actual cost: US$557.70 m, 137% of appraisal cost ). This component included rehabilitation of infrastructure works and purchase, conservation and repair of machinery and equipment . ii) Diversification and Intensification of Irrigated Agricultural Production (appraisal cost: US$87.00 m; actual cost: US$182.00 m, 209% of appraisal cost ). This component included technical and financial assistance for adoption of improved irrigation and farming technologies and equipment, promotion of improved marketing systems and sustainable farm development that also included crop diversification subprojects to be financed by GOM through the SAGARPA (Secretaria de Agricultura, Ganaderia, Recursos Naturales, Pesca y Alimentacion ) Alianza Program. iii) Training and Institutional Development (appraisal cost: US$30.00m; actual cost: US$31.70m, 106% of appraisal cost ). This component included support to improve O&M of transferred irrigation infrastructure, to promote establishment of new service providers, and to strengthen Water Users (WUs) and executing agencies, both at the central and regional levels . iv) Project Management (appraisal cost: US$2.00m; actual cost: US$0.5m, 25% of appraisal cost ). This component included support for Project Coordinating System and for development and operation of the Monitoring and Evaluation System. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The actual project costs were US$ 774.80 m (including contingencies and IBRD front -end fees). This was 147% of the total appraisal project costs of US$ 528.00 m, due to significantly higher GOM contribution than appraisal estimate. Bank financing was estimated to be US$ 303 m of the US$528 m, the rest (US$225 m) was to be financed by the Federal Government of Mexico . Actual Bank financing was US$303 m, 100% of appraisal estimate. Actual GOM financing was US$472 m composed of US$221 m by the Federal Government and US$ 251 m by the State Governments/Water Users. The GOM financing was 210% of the government contribution appraisal estimate . The project closed as scheduled on September 30, 2009. 3. Relevance of Objectives & Design: high One of the objectives of Mexico's National The relevance of the project's objectives was, and remains, high. Development Plan of 2001-2006 was to achieve dynamic, inclusive, and sustainable economic growth that supports competitiveness. Improving water use efficiency and productivity in the irrigation sector was seen as very important for growth. Improving competitiveness of irrigated agriculture would be particularly timely due to the NAFTA agreement that, by 2008, would put Mexico's agriculture in open competition with producers from Canada and the United States. The project was also consistent with four of the five strategic objectives of the 2002-2005 Country Assistance Strategy (CAS): i) accelerated growth through competitiveness; ii ) human capital development by developing public and private sector partnerships and strengthening of demand -driven approaches; iii) balancing growth and poverty reduction with environmental protection by reaching small farmers and improving their incomes through technical and financial support of economically viable and environmentally sustainable investments; and iv ) building an efficient, transparent, and accountable government by transferring irrigation infrastructure O&M to WUs . The Bank's Country Partnership Strategy (CPS) for the 2008-2013 also makes reference to the need to manage water resources in a sustainable manner, particularly in the irrigation sector, that accounts for about 77% of overall water withdrawals. Design relevance was modest . The project adopted an integrated approach to the modernization of both irrigation infrastructure and irrigated agriculture, in line with the objectives of the government program to modernize the irrigation sector and improve the competitiveness of irrigated agriculture . The design also gave special consideration to environmental issues by requiring an Integrated Irrigation Investment Plan (PDR) that would guide each DR's or UR's investment in irrigation modernization and rehabilitation to take into account the overall water balance so as to avoid activities that would worsen the problems of water scarcity and over -exploitation of aquifers. The design also incorporated lessons learned or reinforced from previous Bank involvement in agriculture and irrigation in Mexico, namely: i) integration of irrigation infrastructure with improvements in agriculture production /competitiveness; ii) continued transfer of irrigation infrastructure to WUs; iii ) greater involvement of potential beneficiaries in the decision making process; iv) technical assistance in provision of appropriate technology for improvements in competitiveness of irrigated agriculture; v) decentralization of the National Water Commission (Comisión Nacional del Agua --CONAGUA), development of a mechanism to monitor performance, and incorporation of issues of water resource availability and environmental issues . Design weaknesses, however, included the assumption that SAGARPA ’s involvement at the project preparation stage would result in its involvement at implementation stage . In reality, SAGARPA pulled out of the project at the time of project negotiations when it learned that the Bank loan would not provide extra resources to its normal budget since Bank funding was treated as a within budget item . This is an extraordinarily late stage for lender and borrower to discover such a fundamental design issue that was closely linked to a key objective . Consequently, WUs in DR and UR did not receive from SAGARPA the expected technical and financial assistance to increase agriculture productivity, diversify production and /or improve their marketing capabilities . Although this was not identified as a risk at appraisal, it was critical in limiting achievement of the objective of improving competitiveness of irrigated agriculture. In addition, the project design underestimated the resources that would be required to ensure that the PDRs were prepared for each DR's and UR's investment resulting in only 55% of the original target being actually prepared. SAGARPA's withdrawal also contributed to the failure to target smaller farmers, which was an intention of the project (ICR, p. 11). However, even had there been collaboration between SAGARPA and CONAGUA as anticipated, “it is questionable if indeed small farmers with limited financial resources and limited organizational capacity would have been reached by the Project . This raises the question of whether a design focusing on creating synergies between programs operating in the rural space in Mexico was sufficient to address the needs of the small farmers� (ICR, p. 11). 4. Achievement of Objectives (Efficacy): The four parts of the PDO statement -- i) to improve the competitiveness of irrigated agriculture in both Irrigation Districts and Irrigation Units in Mexico, using an integrated approach; ii) to modernize existing irrigation infrastructure and improve water use efficiency; iii) to promote diversification to high-value crops; and iv) to consolidate transfer of irrigation infrastructure to water user's associations -- are rated as follows: i) Improve the competitiveness of irrigated agriculture in both Irrigation Districts and Irrigation Units in Mexico, using an integrated approach: modest . The outcome/impact indicator in the PAD's log framework was imprecise: "Improvement of Financial Efficiency Indicators of Irrigated Agriculture" although no specific financial efficiency indicator was mentioned. The ICR argued that the increases in output per hectare of 44-48 percent in the DR and UR and the modest increases in water productivity indirectly contributed to improved competitiveness of irrigated agriculture. This is, however, only indirect and partial evidence. The project aimed to achieve competitiveness of irrigated agriculture by promoting high value crops which would contribute to successful integration of the Mexican economy in NAFTA (PAD p. 6). SAGARPA’s withdrawal meant that activities directly related to improved competitiveness of irrigated agriculture, particularly diversification into high value crops, were not implemented (ICR p. 15). The ICR commented that in hindsight the objective should have been reformulated to lower the expectations regarding competitiveness and crop diversification, but this was not done. ii) Modernize existing irrigation infrastructure and improve water use efficiency: modest There were two parts to this objective: (a) Modernize existing irrigation infrastructure: substantial. 427,100 hectares of irrigation infrastructure were successfully rehabilitated and modernized (308,000 hectares in DR and 119,100 hectares in UR) against a target of 360, 000 hectares at appraisal (260,000 in DR and 100,000 in UR (PAD p. 26). The ICR reported higher targets of 458,00 hectares as per the Loan Agreement that were revised to 455,000 hectares in November, 2005. The PAD's target was 119% achieved, and the revised target 94% achieved. Adoption of improved on-farm irrigation application systems in 157,100 hectares during the period 2003-2006 (127,300 hectares in DR and 29,800 hectares in UR) against a target of 128,000 hectares at appraisal or revised target of 125,000 hectares as of November 2005. The original target was 123% achieved. (b) Improve water efficiency: modest. Based on annual evaluations of CONAGUA's programs in DR, conducted by the Mexican Institute of Water Technology during the period from 2004 to 2006, the increases in water conveyance and distributional efficiency ranged from 26%-34% and 37%-49% respectively in areas directly benefiting from the project. Therefore, according to the ICR, the targets of 26% increase in water conveyance and distribution efficiency and 18% increase in application efficiency were achieved in DR. Based on evaluations of CONAGUA's programs in UR, conducted by the Post-Graduate College during the period from 2004-2006, the increases in overall water efficiency of UR directly benefiting from the project ranged from 20% to 29% when the source is groundwater and 14% to 37% when the source is surface water. Thus, according to the ICR, the achievement rate in the UR was at least a 75%. These improvements resulted in water savings estimated at 1,134 million cubic meters (MCM) per year in DR and 49 MCM in UR. However, the ICR noted that the efficiency improvements do not represent "real" water savings in the hydrological regime and have not translated into the stabilization of over-exploited aquifers because they do not necessarily reduce net extraction of water. According to the annual evaluations conducted by the Colegio de Post-graduado , the idle land in the UR has been reduced from 26% to 17% in the case of schemes depending on surface water and from 40% to 26% in the case of schemes depending on groundwater. The same evaluations also reveal that the area under second crop has increased from 9.5% to 13.0% in the case of schemes depending on surface water and from 6.5% to 9.5% in the case of schemes depending on groundwater. The PAD's intention of using the PDRs to address issues of water scarcity and over-exploitation of aquifers was not realized partly because the design underestimated the time and financial resources that would be needed to finalize the PDRs. Only about 40% of them were finalized in the first year of implementation (ICR page 10), while most of them were completed after project closure and have been integrated into CONAGUA's future DR's and UR's investment plans. However, with respect to water use efficiency, the ICR finds (p. 18) that the DR have reaped only slightly higher returns per cubic meter of water and per hectare of land in real terms, and that these increases can be only partially attributed to the project. Surprisingly, an overall set of basin hydrology models were not used to obtain a true estimate of overall water use efficiency combining surface and ground water use. IEG was advised that this is to be tackled under the next project. In the absence of this, assessing changes in overall system water efficiency is not possible due to the linkages between surface and ground water. A relatively higher weight is placed on the water efficiency improvement sub-objective because modernization of existing irrigation infrastructure is a means to achieving efficiency and eventually productivity and competitiveness goals. The overall rating for objective (ii) is therefore modest. iii) Promote diversification to high-value crops: negligible. Areas sown and harvesting high-value crops in DR maintained by CONAGUA increased very modestly from 18.9% to 19.8% and 18.7% and 19.5%, respectively, between 2001-2002 and 2006-2007. This translates into 33,290 hectares under diversification into high value crops. However no targets were set. The ICR assumed a target of 128,000 hectares that was set for intensification and diversification activities, indicating a 26% achievement rate for this indicator. Beneficiary survey results support lack of progress in this area, as only about 10% of the beneficiaries interviewed diversified to high value crops, while 90% retained their traditional crops and knew little about the production and commercialization of vegetables and other high value crops. The main reason for lack of improvement in this indicator is the withdrawal of SAGARPA. This weakness is given high weighting since it relates closely to a project outcome and to the key objective (i). Crop intensity only modestly increased in DR from 5% of area under the second crop to 8% during the period 2002-2003 and 2007-2008 and in UR it increased from 9.5% to 13% in schemes depending on surface water and from 6.5% to 9.5% in schemes depending on ground water. Comparison of 2001-2002 and 2006-2007 statistics also shows that average yields in DR maintained by CONAGUA increased annually by 0.2 ton per hectare. Irrigated water productivity (year 2002 Mx$/m3) also improved by 111% in DR and 123% in UR (when compared to without project situations). No target was set for these indicators. iv) Consolidate transfer of irrigation infrastructure to water user's association: substantial. The project supported full transfer of irrigation infrastructure to water user organizations to one DR (DR019 Tehuantepec covering 44,074 hectares) and partial transfer to another (DR003 Tula covering 51,925 hectares), against a target of full transfer of irrigation infrastructure to three DRs by project closure. The ICR reported that after closure, full transfer of irrigation infrastructure to a third DR occurred (DR111 Baluarte-Presidio). 27 Limited Liability Societies (SRLs) and 392 User Associations (ACUs) covering 101,160 hectares and 24,486 water users (WUs) were established in UR during 2003-2006. In DR, 15 ACUs covering about 67,465 hectares and 11,100 WUs were set up. The target of 15 SRLs to be established was thus substantially exceeded. There are no specifics on targets for the other reported achievements. 105 integrated investment plans (PDRs): 83 in DR and 32 in UR in SRL, were formulated, against a target of 192 at project appraisal (revised to 85 in November, 2005). Although the revised target was exceeded, the original goal was only 55% achieved. Participation of WUs in the O&M of irrigation infrastructure was enhanced. Water fees paid by WUs cover 100% of the required operation and management costs and about 65% of the required maintenance costs in case of DR. Average contribution of WUs toward investment costs under this project, is 32%. The ratio of annual revenues to required operation, maintenance, and management costs has increased from 77.5% in 2001 to 82.5% in 2008 in DR. Similar analysis could not be done for UR's due to CONAGUA’s inability to access UR's financial data. No targets were set for these indicators. 5. Efficiency (not applicable to DPLs): The ERR was estimated at 20.1% at the time of project appraisal. The benefit-cost ratio was estimated at 1.33 and the NPV at Mx$2,856.4 million. At closure, the ERR was re-estimated at about 17%, based on detailed crop budgets and representative farm models in 15 DR and 10 UR. The ERR analysis did not take account of water efficiency . The B/C ratio was1.29 and the NPV Mx$2,017 million. Efficiency is rated substantial . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 20.1% 100% ICR estimate Yes 17% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Although relevance of objectives and efficiency are rated high, design relevance was modest, one objective was rated substantial, two modest, and one negligible, amounting to moderate shortcomings in Outcome . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: Risk to Development Outcome is moderate According to the PAD (p.19), "the transfer of irrigation infrastructure to water user's associations, which includes also the transfer of O&M responsibilities, together with measures to ensure the financial sustainability of these associations, would guarantee the sustainability of project investment ." Although full transfer to the WUs in the three DR had not been completed by project closure, it was, according to the ICR (footnote 30), achieved subsequently . The ICR also reports that the WUs are covering 100% of operation and management costs and about 65% of maintenance costs. This is considered likely to continue since Mexico already achieves a comparatively high rate of O&M cost recovery. The ICR mentions that "increased water efficiency in the irrigation sub -sector and institutionalization of the PDR to better orient both existing and future investments and non -structural actions in the sector will contribute to the technical sustainability of the project results ..." (p. 25).The unexpected challenges in PDR preparation resulted in reducing the targeted number of PDRs to 85 from the 192 originally planned. This was mainly due to resource constraints. The ICR nonetheless reports that CONAGUA has institutionalized the use of PDR for making investment decisions in the sector and guiding their implementation . Discussions with the project team revealed that, after project completion, CONAGUA finished the preparation of PDRs for each DR and UR, implying that whatever improvements in water efficiency were achieved are likely to be sustained . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry : moderately unsatisfactory The Bank understood well the problems of the sub -sector and tried to complement the improvements in irrigation infrastructure with agricultural productivity enhancing activities through an effective institutional collaboration between CONAGUA and SARGARPA. QAG rated QAE as moderately satisfactory . According to Regional comments on the draft ICR Review, this was an "ambitious project in a very difficult sector which tried to achieve cooperation between various governmental entities, and which tried to take irrigation investments to a higher level." However, the Bank failed to identify the potential risk of SAGARPA's withdrawal from the collaborati on, which limited project outcome. The Bank was not realistic in estimating the budgetary resources and the time needed to formulate participatory PDRs . While not an objective, the PAD had laid emphasis on reaching small farmers, e.g., "..the project would ensure that there will be equal access to project benefits by small and large farmers alike" (ICR page 6), but the design failed to ensure that small farmers would be reached (see Section 11). M&E was very poorly designed. In addition the PAD was ambivalent about the need for the Borrower to implement the Bank's Environmental Impact Assessment (EIA) process for sub-projects (the ICR, footnote 19, page 13 states that because PAD said "the EMP actions will be linked to the Mexican environmental legal framework and authority� [this] made CONAGUA believe that there were no deviations between the EMP ’s requirements and the country ’s legal framework"). Quality of Supervision : moderately unsatisfactory Bank supervision missions were regular and instrumental in identifying critical problems such as the need to formulate the PDR within an overall framework of environmental sustainability and in applying the lessons of international best practices . However, the ICR notes that the composition of supervision missions could have been improved. For instance, no sociologist was included in the teams despite the potential social impacts of the project, while only one mission included an environmental specialist . The Bank could also have been more pro-active in finding a solution to SAGARPA's withdrawal from the project, which, due to its fundamental impact on the attainment of project objectives, justified a restructuring . M&E was poorly implemented. Supervision did not fully apply the procedures set out in the Environmental Management Plans (EMP). In particular EIAs were not prepared for all relevant sub-projects because the Borrower thought its own procedures were compliant when they were not. Similarly a Pest Management Plan (PMP) was not carried out. Finally, the Bank team should have been more vigilant in addressing equity issues arising from the project (see Section 11). at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government Performance . The Government remained committed to the project through the activities of CONAGUA, even when the much needed collaboration between CONAGUA and SAGARPA did not materialize beyond the project preparation stage . Government performance is, however, rated moderately satisfactory because it did not employ any other mechanism to ensure SAGARPA played the expected role in project implementation or to encourage collaboration with CONAGUA . Implementing Agency Performance . This is rated moderately satisfactory on balance, but with some weaknesses. CONAGUA, the implementing agency, is technically very solid, has considerable experience in implementing Bank-financed projects, and an excellent track record in carrying out procurement and financial management related tasks. CONAGUA could have performed better in designing a well functioning M&E system . The project suffered from lack of baseline data and incomplete definition of targets . Delays were also experienced in the formulation of the PDR for each DR and UR. The Implementing Agencies should have been more proactive in trying to ensure that smaller, poorer farmers benefited from the project . a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: M&E Design : negligible . There was lack of specificity in defining some indicators . For example, according to Annex 1 of the PAD, competitiveness of irrigated agriculture would be measured by improvements in financial efficiency. However, there was no specific indicator to measure financial efficiency or to relate it to improvements in the competitiveness of irrigated agriculture . The PAD also failed to specify how promoting diversification to high-value crops would be measured . Most indicators had no baseline and no quantified targets . Control groups were not identified. The ICR noted that the issue of CONAGUA's M&E capacity was overlooked . The PAD had indicated that a M&E system was already in place at the time of project approval, but it failed to indicate its scope and extent and its linkages with CONAGUA and SAGARPA's own M&E systems . The ICR also pointed out that no social and environmental indicators were included in the M&E framework . M&E Implementation : negligible During the course of the first three supervision missions, discussions were held with CONAGUA on the extent and scope of the M&E system to be used during implementation, including institutional responsibilities. Very late, around Mid-2005, a final comprehensive list of indicators was agreed with CONAGUA . Despite this, quantified targets were still missing for important indicators . M&E Utilization : modest . Because of the delay in putting in place the whole list of indicators to monitor project performance, the project could not utilize the M&E system until after mid 2005. Once baselines were established, the project revised several of its targets in November 2005 (ICR p. 4), making them more realistic. The ICR indicated that M&E shortcoming had been partially rectified by the time of the MTR and the final review conducted by the Borrower . Nevertheless, a number of targets remained missing and could not be used to inform decision making . a. M&E Quality Rating : Negligible 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Environment . The project was classified as Category B for the purposes of OP 4.01. The ICR reports that compliance with environmental safeguards policies was moderately satisfactory, citing the fact that EIAs were not carried out as stipulated in five DRs classified as “type 1� areas where aquifers are critically over-exploited. In these areas the works involved improvements to existing infrastructure, and Mexican legislation, which CONAGUA was guided by, does not require EIAs for each subproject in these circumstances . The ICR states that this is not a critical failing for three reasons . First, the ICR reports (pages 13-14) that " It should be noted that the PDR prepared under the Project for these five DR have specifically addressed sustainability issues, and have included structural and non -structural measures to reduce water consumption to sustainable levels. In these cases, the implementation of the structural measures was guided by the PDR . Thus, actions supported by the Project in these DR are expected to improve environmental outcomes in the medium term and achieve more than the EMP requirements ." Second, according to the ICR (page 15), “this will ensure that negative externalities associated with irrigation technological improvements are fully addressed ." Third, CONAGUA is a technically solid institution with long experience of working with the Bank . The PAD describes it as an entity possessing "adequate environmental standards and regulations critical to securing compliance with Bank safeguards policies � (page 14). The Project did not implement a Pest Management Plan (PMP). The ICR notes (page 14): "Since CONAGUA followed the Mexican procedures with regard to environmental assessment, pest management safeguard procedures were not implemented as per the PMP . However, since international standards concerning the use of pesticides are applied in Mexico, the rigor of health and safety measures have been assured ." Social safeguards . No resettlement or land acquisition occurred as a result of the project so that social safeguard policies were not triggered. Fiduciary . According to the ICR, financial management compliance was, “in general,� satisfactory. Financial management reports were timely and satisfactory to the Bank, and CONAGUA ’s financial management team were found to have adequate experience and expertise . However, in 2005 disbursements were delayed by late reporting and provision of required documentation, including the 2005 project audit report. Apart from this, the ICR reports that compliance with auditing requirements was satisfactory and that auditors ’ opinions were always unqualified . No procurement issues arose. Unintended Impact . Poor project design, combined with the implementation of measures to improve water conservation and sustainability, and to enforce water rights, have caused a number of social equity issues to arise, especially in situations where there was over -exploitation, or over-allocation, of water resources : Many smaller farmers were not able to take full advantage of the potential offered by the project due to lack of awareness of what the project had to offer, limited understanding of the use of the new water -saving technologies, and inability to meet the stringent (50%) cofinancing requirements. The PDRs did not adequately address adverse impacts on vulnerable populations including those depending on informal access to water. The project did not take account of the socio -economic differences among the different regions in the country, and WUs in richer, northern states benefited more from the project than their counterparts in poorer, southern ones. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Design relevance was modest, one Satisfactory objective was rated substantial, two modest, and one negligible, amounting to moderate shortcomings in Outcome . Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Quality-at-entry and supervision had Satisfactory Unsatisfactory significant shortcomings -- see section 8. Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: Drawing from the ICR with some adaptation, the main lessons are : Modernization of irrigation infrastructure and on -farm technology improvements is a necessary but not sufficient condition to increase competitiveness in irrigated agriculture . Irrigation improvements need to be complemented with agricultural productivity enhancing activities and support to farmers and producers, particularly small ones, in the areas of organization, financial management, and commercialization . The complementary activities should also include technical assistance to farmers . In cases where a separate institution is responsible for implementation of complementary activities, as was the case with SAGARPA, the parameters of engagement and its commitment need to be established during project preparation . Careful design of the M&E system is necessary prior to project approval including baselines and controls . In cases where the target population is diverse, it is important to establish a baseline of social indicators . Where implementation requires collaboration between two institutions, as was the case with SAGARPA and CONAGUA in this project, a joint M&E system needs to be set up where both institutions work within the same areas and report on the same indicators . Modernization of irrigation projects located in over -exploited aquifers and/or over-allocated surface water resources, should be done within an integral water resources management framework, where water balance and other users including environmental water needs are taken into account . PDRs are an essential tool to achieve this but they need to be adequately budgeted for, updated to define priorities in the sector, and accompanied by detailed social assessments, to avoid negative effects of conservation and enforcement actions on vulnerable populations; mitigation measures need to be identified To maximize project benefits for all groups, targeting mechanisms need to be put in place including eligibility criteria and special financing terms to small farmers and producers . Great care has to be taken at appraisal to ensure that the Bank's safeguard policies are not negated by ambivalence. In this case the Borrower was led to believe incorrectly that its own safeguard policies could substitute for the Bank's. To compound the problem the Bank did not adequately supervise safeguard compliance. 14. Assessment Recommended? Yes No Why? To check the status of transfer of irrigation structures to WUs, to check if the improvements in the efficiency of irrigation water have started generating "real" water savings, to assess whether the project is contributing to stabilization of aquifers and to better understand the impact of the project on income distribution . The assessment would also examine the issue of compliance with safeguards and equity issues . 15. Comments on Quality of ICR: The ICR did a good job of consolidating information from the PAD including the annexes and from the Loan Agreement to identify baseline data against which to measure achievement of objectives . The ICR also noted inconsistencies in the data provided by the PAD and the Loan Agreement and the lack of adequate articulation of the objective statement as presented in the main text of the PAD (it used the more monitorable statement in Annex 1 of the PAD as did this Review). The data is generally well presented and the lessons drawn from the analysis . Greater clarity on safeguard issues was needed . There is a minor date error for target and revised values on pages iii and vi of the Data Sheet. a.Quality of ICR Rating : Satisfactory