ICRR 12603 Report Number : ICRR12603 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 04/18/2007 PROJ ID : P060718 Appraisal Actual Project Name : Renewable Energy US$M ): Project Costs (US$M): 31.3 21.7 For Agriculture Project (gef) Country : Mexico Loan/ US$M): Loan /Credit (US$M): 8.9 8.9 Sector Board : ENV Cofinancing (US$M): US$M ): 22.4 12.8 Sector (s): Renewable energy (50%) Central government administration (45%) Animal production (5%) Theme (s): Other rural development (33% - P) Rural services and infrastructure (33% - P) Climate change (17% - S) Environmental policies and institutions (17% - S) L/C Number : Board Approval Date : 12/21/1999 Partners involved : Closing Date : 06/30/2004 06/29/2006 Evaluator : Panel Reviewer : Group Manager : Group : Robert Mark Lacey Peter Nigel Freeman Alain A. Barbu IEGSG 2. Project Objectives and Components: a. Objectives: The project’s objectives were: (a) to promote the use of renewable energy for productive purposes in Mexico’s agricultural sector by removing barriers and reducing implementation costs; and (b) to reduce greenhouse gas emissions in the agricultural sector. The objectives were to be met through: (i) providing farmers currently without electricity with reliable electricity supply for productive purposes in a least-cost and sustainable manner using renewable energy technologies; (ii) increasing the productivity and income of farmers currently without electricity by supporting productive investments and improved farming practices; and (iii) catalyzing the penetration of renewable energy technologies in the agricultural sector. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): The project consisted of a set of interrelated and mutually supporting activities which were implemented concurrently: a campaign promoting the use of renewable energy by farmers; studies to identify the potential market for renewable energy systems in the agricultural sector; installation and demonstration of renewable energy systems; technical assistance in the maintenance and operation of these systems; establishment of specifications and certification systems; a pilot program to test vendor financing of energy systems and components in four states; technical assistance for agricultural extension personnel advising farmers on the proper operation of renewable energy systems; and institutional strengthening for training technicians, extension personnel, vendors and the project implementation unit – the Trust Fund for Shared Risk (FIRCO) -- in project management (including monitoring and evaluation). Revisions. There were no revisions to the project components. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost. Total project costs at completion were US$21.73 million, about 70 percent of the US$31.3 million estimated at appraisal. The GEF grant of US$8.9 million was meant to cover 28.5 percent of total project costs. It ended up being fully disbursed and accounting for 50 percent of costs. The cost savings of US$9.57 were concentrated in demonstration systems (US$2.87 million), vendor financing (US$2.09 million), and technical assistance (US$3.5 million). Project management costs, mainly incurred by FIRCO, were also nearly US$1 million less than anticipated. The vendor financing pilot component did not receive the expected response from States, and little vendor financing in fact materialized. Demonstration costs savings were due to lower installation and parts costs, reflecting in part economies of scale from increased supply. Although technical assistance was successfully implemented, and indeed amplified, only 81 extension workers were ultimately required, 30 less than originally envisaged. This is the only explanation provided in the ICR for the lower costs of this component. Financing. Since the full amount of the grant was disbursed, all the cost savings accrued to the counterpart financing from the grant beneficiary. Dates. At the Borrower’s request, the original closing date of June 30 2004 was extended by two years, and the project closed on June 29, 2006. This was mainly due to start-up delays. The project was under the auspices of a national agricultural and rural development initiative known as the Alianza para el Campo. Prior commitments initially prevented the Alianza program from accompanying the project’s investments. After these initial delays, implementation proceeded satisfactorily. 3. Relevance of Objectives & Design: The project was highly relevant to Mexico’s efforts to further the development of its agricultural sector by raising farm productivity while simultaneously reducing farmers’ dependence on carbon-emitting internal combustion engines for electricity generation. The design was appropriate to meet the project’s objectives. The project was fully consistent with the three core themes of the Mexico Country Assistance Strategy: (i) social sustainability (the Alianza program focuses on poorer farmers); (ii) removing obstacles to sustainable growth; and (iii) more effective public governance. It also contributed to the “win-win� investment opportunities, stressed in the Bank’s environmental strategy, where both economic gains and environmental benefits may be generated through an integrated approach to development. 4. Achievement of Objectives (Efficacy): Despite lower expenditure than anticipated at appraisal, and despite the fact that this was the Bank’s first experience of promoting productive uses of renewable energy, the project achieved its development objectives. An estimated 2,312 farmers, who previously had no electricity, were provided with a reliable supply in a least-cost and sustainable manner, primarily through photovoltaic-energy (PV) water pumping systems. Prior to the project, between 1994 and 2000, 195 renewable energy systems had been installed in rural areas with the support of USAID and the US Department of Energy. Over the life of the project (2000-2006), a further 2,312 systems were installed. 1,545 PV pumps were set up, compared to an appraisal target of 1,050. A significant reduction in the cost per watt of water pump systems was achieved – systems sold in 2004-5 were about 25 percent cheaper than those installed prior to the project. The effects on farmer productivity and income can only be roughly estimated, based on the evaluation of three beneficiary farms. These showed that the rate of increase in farm incomes (after adjusting for income received directly from the project) more than doubled. In the first year of implementation, the project avoided 36,292 tonnes of carbon emissions, 21 percent higher than the original target figure of 30,000 tonnes. The project helped to build capacity in both FIRCO and the Ministry of Agriculture in the promotion of renewable energy use. Thanks to the achievement of project objectives, especially in the light of the experimental nature of the operation, results were widely disseminated in the Bank and the GEF (through the renewable energy toolkit and the golden plough award) and more widely in Mexico and beyond via renewable energy congresses and the renewable energy colloquium held in Mexico City in 2006. Knowledge was also disseminated to private industry and to academia as well as to direct beneficiaries through extension services. 5. Efficiency (not applicable to DPLs): The target population was narrowed down to between 80,000 and 100,000 during implementation from the appraisal estimate of 600,000 farms without electricity. This was anticipated in the PAD which stated that early market research would be carried out to define the target population more precisely taking account of ability to purchase systems, access to reliable water supply and other factors. To assess project benefits at the farm level, four models were developed, three of them bovine livestock models (most of the target population were livestock farmers in arid areas). These models were used at appraisal to assess anticipated financial returns at the farm level. At completion, mature data was available up to the end of 2003, and showed a considerably lower rate of return on investments than that foreseen, especially for the predominate arid livestock beneficiaries: Arid livestock Arid mixed Temperate Tropical Appraisal 44% n.a. 19% 35% Completion 18% 20% 17% 26% The ICR supplies no explanation for the lower than rates of return except to say that PV water pumping systems are profitable relative to those using conventional energy when the investment life is long, when relatively low volumes of energy (less than 1,500 watts) are required, and when the beneficiary is located a substantial distance from the grid. These last two conditions tend to limit profitable investments to watering livestock or producing other, non-perishable products. As a consequence, PV systems can increase incomes and productivity while yielding relatively low on-farm returns. The overall economic rate of return, taking account of off-farm support costs, is estimated to be 15 percent compared with 30.9 percent anticipated at appraisal. FIRCO’s support costs were less than forecast. These results are still higher than the normally used opportunity cost of capital of 12 percent (not mentioned in either the PAD or the ICR). ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 31% 100% ICR estimate Yes 15% 100% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The project’s development objectives remain relevant and were achieved. The economic and financial rates of return, although less than anticipated, are still above the normal estimate of the opportunity cost of capital. The project has enabled the development of a new understanding of how to implement renewable energy projects, and is included as a case study in the Bank’s renewable energy toolkit. a. Outcome Rating : Satisfactory 7. Rationale for Risk to Development Outcome Rating: The main risk to sustainability of the project’s outcome is the possibility that the government may discontinue subsidizing investments in renewable energy applications in agriculture. This risk is significant given the recent advent of a new government, and that future PV system investments will require continued subsidy. There are two other risks –projected increases in farm production may fail to materialize (for example, due to a fall in output prices), and the rate of system breakdown and maintenance costs may be higher than anticipated. The second of these risks is low, but the first is considered moderate. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: Project design was sound, and quality-at-entry satisfactory. Bank staff worked closely with implementing agencies and implicated their staff in project preparation. The definition of the precise target population during implementation rather than preparation was appropriate and foreseen at appraisal. Supervision was diligent and flexible. During implementation, for instance, assistance to farmers was extended beyond help with installation, as originally envisaged, to include operation and application of the systems. It was also realized that the impact of the PV systems would be considerably enhanced if farmers were simultaneously helped to diversify cultivation and to market their crops. at -Entry :Satisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Satisfactory 9. Assessment of Borrower Performance: Despite temporary disruptions, such as those following the change of government in 2001, the Borrower’s commitment to project objectives remained strong. Difficulties were always resolved. National Financiera (NAFIN) which was responsible for channeling external funding, performed diligently. The implementing agency, FIRCO, also performed well and met the challenge of temporary reductions in technical staff in 2001. Implementation benefited from FIRCO’s decentralized structure – a specialist working on the project was assigned to each of its 28 state offices, and this facilitated coverage,the tailoring of activities to local needs and building the institutional underpinning for sustainability. a. Government Performance :Satisfactory b. Implementing Agency Performance :Satisfactory c. Overall Borrower Performance :Satisfactory 10. M&E Design, Implementation, & Utilization: 18 key indicators were developed at appraisal; these were appropriate and most of them were used to measure the project’s impact. Separately, FIRCO is able to measure the carbon emission reductions resulting from the introduction of PV systems. Field activities were closely monitored throughout, with information being collected by each of FIRCO’s 28 regional offices. The offices prepared energy programs for their states, and also training events and demonstrations. These fed into the national workshops and programs. A web-based information system was developed. Both the MTR and the Borrower’s report (prepared by FIRCO) contain a thorough assessment of project achievements. One drawback of the indicators was that they did not enable the regular monitoring of changes in beneficiary income or types of beneficiary. a. M&E Quality Rating : Substantial 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Environment. The project was rated category B since no major negative environmental impact was anticipated. Indeed, the project was intended to benefit the environment, and complied throughout with all relevant safeguards policies. The concern raised during preparation over possible pollution leading from the use of batteries to store power was addressed by ensuring direct power usage in all cases and no storage. Water depletion was mitigated by ensuring that beneficiary farms had an adequate supply of groundwater and sufficient wells to supply the pumping equipment. Resettlement No resettlement was associated with the project. Fiduciary compliance was assured by NAFIN which managed project disbursements. The agency also made sure of compliance with auditing requirements. NAFIN staff is trained on both Bank regulations and eligibility and fiduciary issues associated with GEF projects. All work was carried out to a high standard. FIRCO successfully implemented the agreed action plan for strengthening its financial management system. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Risk to Development Moderate Significant Of the three main risks identified – Outcome : (i) discontinuance of government subsidy, (ii) higher than anticipated breakdown of installed systems, (iii) failure to achieve anticipated on-farm productivity increases -- (i) is considered significant and (iii) moderate, while (iii) is considered low. Overall, therefore, the risk must be considered significant. Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The main lessons are: 1. The choice of an agricultural sector institution (FIRCO) as the implementing agency, rather than an energy sector entity, was highly beneficial since it ensured that renewable energy provision remained demand-driven and could be tailored to the needs of farmers. 2. The impact of the PV systems is considerably enhanced if farmers are simultaneously helped to diversify cultivation and to market their crops. 3. The establishment of vendor financing systems in new sectors and remote geographic areas requires careful design, and must take full account of already operating replication systems. 4. An effective financing scheme for renewable energy use for productive purposes in rural areas will likely require continued subsidized financing. 5. In future renewable energy projects, it would be useful to include indicators which measure changes in beneficiary income and in category of beneficiary. 6. The project shows the benefits of flexibility during implementation, especially in an innovative operation of this kind. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: Substantively, the ICR contains a full discussion of all the elements required to evaluate the impact of the project. It suffers, however, from careless proof-reading. Significant parts of the text are missing, including paragraphs from important sections such as the discussion of project outcome. It would have been useful to have analyzed why the completion rates of return (both economic and financial) were lower than foreseen at appraisal. a.Quality of ICR Rating : Satisfactory