IEG Report Number: ICRR14914 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 05/31/2016 Country: Rwanda Project ID: P097818 Appraisal Actual Project Name: Rw: Sustainable Project Costs (US$M): 8.30 8.22 Energy Development Project (gef) L/C Number: Loan/Credit (US$M): Sector Board: Energy and Mining Cofinancing (US$M): Cofinanciers: AFREA Board Approval Date : 10/15/2009 Closing Date: 01/31/2014 02/06/2014 Sector(s): Public administration- Energy and mining (56%); Other Renewable Energy (17%); Hydropower (17%); General energy sector (10%) Theme(s): Climate change (69%); Rural services and infrastructure (26%); Micro; Small and Medium Enterprise support (5%) Prepared by: Reviewed by: ICR Review Group: Coordinator: Fernando Manibog John R. Eriksson Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The over-arching goal of the Sustainable Energy Development Project (SEDP) is to strengthen and consolidate the Rwandan renewable energy (RE) market over the long term, as indicated in both the Project Appraisal Document (PAD and the Implementation Completion & Results Report (ICR) for this Global Environment Facility (GEF)-financed project. Over the project's implementation time period, the GEF Grant Agreement dated February 8, 2010 indicates in Schedule 1 that: "The objective of the Project is to improve the policy and institutional framework of the renewable energy and energy efficiency subsectors and increase private sector participation in the renewable energy subsector." This IEG ICR Review bases its assessment on the specific, measurable and time-bound objective as stated in the GEF Grant Agreement. An identical formulation of the project objective is stated in Schedule 1 of the Trust Fund Grant Agreement for the co-financier, the Africa Renewable Energy and Access Program, dated February 8, 2010. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: Aside from GEF funds, the project was also financed by the Africa Renewable Energy and Access Program (AFREA), which is funded by the Netherlands Government and executed by the Africa Energy Unit of the World Bank. The financing shares between GEF and AFREA are indicated below under each component. The project did not support any infrastructure components. Component A: Strengthening of Renewable Energy Policy, Strategy and Management (US$3.5 million, of which GEF US$1.7 million and AFREA US$1.8 million) This component intended to support the Ministry of Infrastructure (MININFRA) in streamlining and coordinating activities related to the preparation of operational guidelines for RE project design, implementation, and increased private sector participation. Specific activities included: (i) preparation of the RE policy, strategy and monitoring & evaluation (M&E) of the RE sector; (ii) Establishment of the Lake Kivu Monitoring Unit; (iii) roll-out of an Energy Entrepreneurship incubation program for RE small and medium enterprises (SMEs); and (iv) provision of RE advisory services and technical support. Component B: Efficient Utilization of Biomass Resources (US$1.4 million, of which GEF US$0.9 million and AFREA US$0.5 million) This component sought to increase biomass end-use efficiencies so as to reduce the unsustainable use of firewood and charcoal, through activities focused on increasing the efficiency of charcoal production and the use of improved cooking stoves. Component C: Sustainable Development of Micro Hydro Resources (US$1.2 million, of which GEF US$0.9 million and AFREA US$0.3 million) This component was meant to enable private sector investment in micro-hydro power plants, by addressing the lack of: (i) credit access from local banking institutions; (ii) capacity to prepare detailed feasibility studies; and (iii) clarity of tender procedures or other principles for identifying and selecting investors. Activities included: (i) private sector-led micro-hydro expansion; (ii) technical assistance to ensure the long-term operation of existing micro-hydro plants commissioned by the Government; and (iii) establishment of a micro-hydro helpdesk. Component D: Solar Energy (US$1.55 million, of which GEF US$0.35 million and AFREA-US$1.20 million). Activities under this component aimed at increasing private sector capacity to develop the market for solar systems by: (i) developing national installation and user guidelines for institutional photovoltaic applications; (ii) establishing conducive frameworks and build capacity of local private firms to participate in international tenders for the Rwandan market; and (iii) supporting the development of a commercial Solar Home System (SHS) market for rural areas. Component E: Energy Efficiency (EE) Strategy Development (US$0.65 million from GEF) This component aimed to reduce technical losses in the electricity grid network of the Rwanda Electricity Corporation (RECO) and promote EE in major electricity users, through: (i) a grid audit of the Energy Water and Sanitation Authority (EWSA) grid network; (ii) EE audit of street lighting, water pumping facilities and public buildings; and (iii) preparation of the EE policy including actions to reduce energy use and increase efficiency for the various electricity consumer categories. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: At appraisal, the total project cost was estimated at US$8.3 million. The actual project cost at closure was US$8.22 million. The actual project costs were financed by GEF (US$4.47 million) and AFREA (US$3.75 million). The Government did not make any contribution. There were three restructurings, which were all administrative extensions of the AFREA Trust Fund, which had a different closing date than the GEF grant. The midterm review was scheduled originally for July 2012 and was conducted during that month. The project was scheduled originally to close on January 31, 2014 and closed on that date without any extensions. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High Government Strategy. At appraisal, the project's objectives were highly relevant to the Government's strategy. The Government's Economic Development and Poverty Reduction Strategy (EDPRS 2008-2012) featured--as one of its key priorities--the rapid expansion of electricity access, as well as the improvement in the quality and the lowering the cost of economic infrastructure, especially transport, power, and communications. The project fit well into the Government's priorities by enhancing private sector investments in RE and increasing the efficiency of household biomass use. At present, the project's objectives remain highly relevant vis-a-vis the Government's energy sector priorities, which are focused on reducing the cost of electricity, increasing energy efficiency and energy diversification, and strengthening institutional capacity to facilitate private participation. Bank Strategy. At appraisal, the project's objectives were substantially relevant to the Bank's FY2009-2012 Country Assistance Strategy (CAS) for Rwanda, given the CAS' primary objective of promoting sustainable growth and private sector development, while engaging in and supporting the Government's EDPRS. At present, the project's objectives remain substantially relevant to the Bank's Country Partnership Strategy (CPS) for Rwanda 2014-2018, given the CPS' key objective (among others) of "‘accelerating economic growth that is private-sector driven and job-creating. This includes IDA investments and analytical work in energy (to tackle high costs and low reliability which are major barriers to enterprise development)." b. Relevance of Design: Substantial Three key design features support a substantial rating for relevance of project design: (i) the project's close integration with the Government's related programs; (ii) the project design's application of lessons learned from RE experiences in neighboring countries and other global RE programs; and (iii) coordination of project activities with other donor-funded RE projects. The project's design was well coordinated with the Government's parallel programs intended to mainstream RE development, notably the national-level Energy Sector-wide Approaches (SWAp), The project design's emphasis on technical assistance (TA) complemented the grid extension efforts under the Electricity Access Scale-up and SWAp Development Project (EASSDP). Thus, while the latter financed infrastructure, the project focused on RE sector sustainability and pulling in the local private sector into RE investments. Good and best practices were incorporated into project design, based on experience from GEF-supported projects in Uganda and Sri Lanka, the Global Village Energy Partnership (GVEP), and similar projects in Rwanda itself. Lessons from the World Bank's Renewable Energy Toolkit were also utilized. Specific project components were designed taking into account the lessons from photovoltaic solar experience in Kenya and Tanzania, which highlighted the role of the private sector, the importance of stakeholder participation, the need to focus on setting the right policies, and the importance of using market-driven, flexible approaches. The mistakes in Tanzania of the top-down approach to promoting improved cooking stoves were also factored into project design. Parallel experience with donor activities were also considered, e.g., the USAID study to design a cooking stoves program, the GTZ Micro Hydro Project and work on a national biomass strategy; and upstream studies under the Urgent Electricity Rehabilitation Project (UERP). The Results Framework (PAD, Annex 3, pages 33-38) was also analytical and well-articulated. An effort was made to outline the results chain, from specific activities, to intermediate outcomes, up to the achievement of final outcomes, while signaling the project's contribution to long-term outcomes. Baselines and yearly targets were also specified, together with the allocation of monitoring and evaluation (M&E) responsibilities. The ICR (page 11) suggests that more analytical work could have been done to establish the linkages along the causal chain from outputs to outcomes, while also recognizing the difficulties of measuring the avoidance of CO2 emissions as a direct result of the project's technical assistance components and activities (no infrastructure component was supported by the project). 4. Achievement of Objectives (Efficacy): The objective of the Project is "to improve the policy and institutional framework of the renewable energy and energy efficiency subsectors and increase private sector participation in the renewable energy subsector." High Outputs  Operational guidelines for RE project design and implementation were developed, with inputs from the private sector.  The Government prepared a RE policy and strategy covering solar, hydro, geothermal and wind technologies.  The Ministry of Infrastructure (MININFRA) prepared a new sector policy that integrates RE and energy efficiency activities in sector development.  Guidelines for the installation and use of institutional PV systems were developed.  A PV Solar Market Assessment was carried out and identified 11 distribution outlets (compared to the original target of 4 outlets) in the major towns outside the capital Kigali. Solar PV suppliers and dealer networks were established outside Kigali.  Private PV companies/technicians and technical schools received capacity-building support and other technical assistance.  A Solar Association was created. Solar equipment was promoted.  The concept of a RE business incubation program was promoted to local energy entrepreneurs through a short, practice-oriented course on “Entrepreneurs Skill & Renewable Energy Development” and the Kigali Institute of Science and Technology (KIST).  Several micro-hydro sites were being developed (4.5 MW under construction) and additional sites totaling about 5MW were at various stages of feasibility study and financial closure by the time the project closed. Outcomes  The project helped the Ministry of Infrastructure (MININFRA) and the Energy, Water and Sanitation Authority (EWSA) to streamline and coordinate activities related to renewable energy and energy efficiency.  The Government implemented various incentives to attract the private sector, such as a feed-in tariff for RE, light-handed regulations, and standardized Power Purchase Agreements (PPAs) to ease the regulatory requirements for the development and operations of micro hydropower plants.  The project reinforced the institutional and regulatory framework for RE, the Government's investment promotion and awareness-raising capacity, and the preparation of pre-investment analysis (such as the EWSA grid audits), all of which provided the requisite environment for sector development and increased private sector participation.  The organization of solar technology distribution was improved by bringing into sharper focus the distinct roles of Government as regulator and investment promoter, as well as RE commercial lenders, project developers, and service providers.  Public and media awareness about the quality of solar systems and complementary private sector activities has been raised. The Stove Performance Inventory Report of the UN Foundation (2012), for example, assumes that at least 30% of the new households using improved cooking stoves can be attributed to the awareness-raising campaign of the project.  10 local firms were registered as eligible participating firms for the Solar Rwanda Program. About 450 solar water heater systems were installed, with an estimated capacity of 1,200 m2, translating into an annual energy saving of about 400 MWh.  By project closing, about 45 enterprises were involved in the RE business (solar and pico/micro hydro) and 69 were in businesses related to biomass-efficient utilization (improved cook stoves, biogas and efficient charcoaling), according to results of surveys conducted in December 2013. Survey results also indicated that these enterprises employed about 1,460 personnel, which is significantly above the original project target of 75.  The project has facilitated about 9.5 MW of micro hydro sites, representing annual energy savings of about 400 MWh from the installed solar water heaters, biomass savings of about 30-39% by using improved cooking stoves, and 30% of wood savings by using improved carbonization techniques. In the Kigali area, 82 percent of households were reported as using improved cooking stoves, compared to the original target of 75 percent.  Scale-up has been put on track by having successfully built linkages among (i) companies selling high-quality equipment, (ii) trained and qualified technicians, (iii) consumer groups, and (iv) other possible stakeholders.  The increased private sector participation in the RE business was complemented by other donor-funded efforts, notably: (i) GIZ’s Private Sector Participation in Micro-hydro Power Supply for Rural Development project (PSP Hydro); and (ii) GIZ and the Government's National Domestic Biomass Program.  By project closing, avoided CO2 emissions were estimated at 0.560 million tons CO2 equivalent, attributed to the use of solar water heaters instead of grid electricity and increased efficiency of improved cooking stoves. At project closing, 4.5 MW of micro hydro power plants were under construction; upon commissioning, an additional 8.20 million tons CO2 equivalent is expected to be avoided. This is an achievement level that is 80% of the original target at closing, but this is likely to be achieved or exceeded when the 4.5 MW of micro hydro plants, which were already being constructed at project closing, are commissioned. (The ICR does note that the impacts of some of the components were difficult to unequivocally quantify in terms of avoided CO2 emissions, although all of them had a positive impact for this indicator.) 5. Efficiency: Substantial Cost Efficiency. The project's outcomes were fully achieved or more than achieved for 4 of the 5 targets. The target related to the reduction of CO2 emissions was achieved at 80% of the original target, but is highly likely to be achieved or exceeded once the 4,5 MW of micro hydro plants (already under construction at project closing) come on stream. The project provided technical assistance and does not lend itself to conventional economic and financial analyses. Using GEF's incremental cost methodology, the PAD (Annex 9) analyzed the project “with and without GEF funding" and demonstrated (i) how barriers to RE would be removed and (ii) how the project would result in the expansion of the solar, micro-hydro, and improved stoves markets, as well as the reduction of CO2 emissions. Administrative and Implementation Efficiency. The project was implemented within the allocated GEF and AFREA funding without any cost overruns. All activities were completed by the original project closing date, without necessitating any extensions of the GEF funding (the 3 restructurings were related to extensions of the AFREA funding, which its special procedures necessitated, without impacting on the implementation timetable). Procurement was satisfactory overall, having benefited from early training and close Bank supervision. a. If available, enter the Economic Rate of Return (ERR)/Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The relevance of project objectives at appraisal and at present is high. The relevance of project design at appraisal and at present is substantial. Achievement of the project objective is high, with outcomes achieved on budget and on schedule. Efficiency is substantial, based on GEF's incremental cost analysis methodology. The overall project outcome is rated satisfactory. Overall Assessment The available evidence supports the conclusion that the project successfully strengthened the legal and regulatory frameworks for both renewable energy and energy efficiency, notably through the development of guidelines, enunciation of a strategy to govern the sector for the next 10 years, use of standardized Power Purchase Agreements, and adoption of the RE feed-in tariff -- thus creating an enabling environment for private sector participation. More specifically, the project resulted in transparent market regulation and guidelines to facilitate small, distributed power production and distribution services, while building local private sector capacity to plan, design, implement and operate the small power plants. Its support for capacity-building for the private sector resulted in the participation of a number of private firms in Rwanda's solar RE market (e.g., 11 solar energy distribution outlets, 10 local firms registered for the Solar Rwanda Program). The project complemented and was well integrated with other RE initiatives. A distinguishing feature of this project is its close coordination and parallel processing with a Bank energy SWAp operation, thus significantly enhancing Government commitment, stakeholder participation particularly the private sector, and broad-based buy-in to timely implementation -- which is a good/best practice that could be emulated in other RE operations in the Bank. a. Outcome Rating: Satisfactory 7. Rationale for Risk to Development Outcome Rating: There is strong Government and private sector ownership of the project's legal/regulatory, institutional, financial and social aspects. In particular, the Government remains strongly committed to promoting private sector participation. To this end, it has initiated several actions to promote private investments, including the unbundling and corporatization of EWSA into separate electricity and water companies, in order to improve the financial sustainability of sector and reduce the perceived risk by the private sector. Other actions ensure project sustainability, such as: (i) regular review of the feed-in tariff; (ii) Government commitment to scale up use of improved cooking stoves and alternative energy technologies such as biogas digesters; and (iii) increased private involvement in micro hydro development, low-cost solar lighting products, and other RE businesses. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: In addition to selecting realistic development objectives, the Bank's quality at entry was most evident in the project team's work on to project design. The Bank also provided adequate resources, in terms of staff weeks and budget, to ensure that preparation and appraisal work achieved quality standards, while maintaining a good working relationship with the Borrower. However, there were some minor shortcomings in project preparation that led to slow implementation at project start-up. First, although the PAD indicated that weak sector institutions may lead to delays, steps were not taken during project preparation to strengthen institutional capacity, which led to a slow start of implementation. Second, procurement activities could have been taken to more advanced stages during project preparation in order to enable timely start-up of activities once the project was declared effective. Quality-at-Entry Rating: Moderately Satisfactory b. Quality of supervision: The frequency of supervision missions twice a year was adequate. Supervision quality--including reporting on the outcome indicators--was of a high standard. With part of the team based in the country office, implementation issues were identified and addressed in a timely manner. Documentation was adequate and issues were quickly brought to the attention of management. The Bank team had the requisite multi-disciplinary skills, and did not change significantly during the project implementation period. The team was supported by specialists to address specific issues. Procurement was monitored regularly, and post-reviews were conducted during supervision. Financial management supervision was adequate; financial management arrangements and external audit were reviewed regularly. The Bank's supervision included environmental and social due diligence. The quality and candor of supervision reports were satisfactory. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government's high level commitment to the project is clearly evident from its policy to increase private sector investment in RE. To this end, the Government has approved several laws and policies that define the emerging sector structure and institutional framework. This includes the Government's policy to increase private sector investment especially in generation and off-grid electricity distribution, for which it has signed a number of Memoranda of Understanding for investments. Other key initiatives have included: (i) the adoption of feed-in RE tariffs, and (ii) the unbundling and corporatization of the utility (EWSA) into separate electricity and water companies to ensure improved financial sustainability of the sector. Government Performance Rating Satisfactory b. Implementing Agency Performance: The implementing agency MININFRA performed slowly especially during the start of project implementation. There were delays recruiting the project coordinator, and a lack of a clear allocation of responsibilities within MININFRA for coordinating the project's various components and numerous activities, which led to a slow start of project implementation. Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: In line with the project's purely technical assistance (TA) content, the performance indicators were keyed to sector management, planning, studies, and institutional development targets. An M&E Plan was prepared. Per GEF requirements, the results framework included a specific indicator on avoidance of CO2 emissions, although this indicator was difficult to measure directly from the TA activities and outputs. b. M&E Implementation: The Project Implementation Unit (PIU) regularly collected the performance data required by the M&E plan. The indicator of avoided CO2 emissions was not monitored during implementation, as it was difficult to attribute a direct reduction of CO2 emissions to the project's specific TA activities. During the ICR preparation, the CO2 emissions reduction was assessed in relation to Component 2: Efficient Utilization of Biomass Resources. c. M&E Utilization: M&E data was used to monitor progress. The M&E results informed decision-making. For example, after the Renewable Energy Feed-In Tariff (REFIT) was approved in 2012, the continuous monitoring of the micro hydro projects in the pipeline that have reached financial closure led to the revision of REFIT in 2014. M&E Quality Rating: Substantial 11. Other Issues a. Safeguards: The project was rated Category B under the safeguards policy on Environmental Assessment (OP4.01), thus requiring a partial Environmental Assessment (EA). The rationale is that while there were no investment components in the project, the guidelines and other products from the TA activities could have led to downstream risks. For example, promoting micro hydro investments could have potential environmental and social implications; similarly, the scaling-up of more energy-efficient cooking stoves and charcoal kilns could have some environmental implications. To avoid, manage or mitigate these potential adverse impacts downstream of this project, the Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF) that had already been adopted for the Urgent Electricity Rehabilitation Project was updated and utilized. In addition, an Environmental Management Framework (EMF) was prepared specifically for the micro hydro subprojects. None of the project activities triggered the EMF provisions by project closing. b. Fiduciary Compliance: The ICR (page 12) indicates that "financial management arrangements under the project were implemented in an adequate manner and maintained throughout the life of the project. The audits of the implementing agency and project financial statements were submitted on a regular basis and with no qualifications. Overall, financial management systems were considered moderately satisfactory." Procurement. Procurement processes were satisfactory, according to the ICR (page 12). The PIU staff received procurement training at implementation start-up, complemented by Bank team during supervision. Thus, despite a slow start in preparing procurement documents, the PIU performed its procurement functions well. c. Unintended Impacts (positive or negative): d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Satisfactory Risk to Development Moderate Moderate Outcome: Bank Performance: Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately There were delays in project Satisfactory implementation due to the slow recruitment of the project coordinator and the unclear allocation of responsibilities within MININFRA for coordinating the project's components and activities. Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR provided useful lessons that are well grounded in the project's implementation experience. These are paraphrased below: Government leadership and focus on results is paramount to achieving the expected impacts from technical assistance. For example, the Rwandan Government acted upon the recommendations or studies done under the project, such as the renewable energy feed-in tariff, light-handed regulations, and standardized Power Purchase Agreements (PPAs). These actions have greatly reduced the transactions cost for micro hydro developers by shortening the time required to negotiate PPAs and having prior information about the price of the generated energy. Realism is needed when funding from multiple donors and co -financiers is involved in a project . Projects with multiple sources of financing need to be structured such that the different administrative procedures governing funding are fully taken into account, optimized, and leveraged in a manner that minimizes the risks of mismatching funding with project activities. The AFREA funds, for example, could have been considered as parallel financing instead of co-financing, given the risk that any non-extension of the AFREA Trust Fund could have negatively impacted the implementation of the co-financed GEF activities. An Incubation Program is a useful approach to build the local capacity of private companies . An entrepreneur incubator can help in developing business plans and feasibility studies, as well as train local technicians and trainers. This could be instrumental equipping small and medium start-up companies with the basic framework to understand the RE business, and take advantage of local skills. Using public resources in awareness -raising and promotion campaigns is a key tool in promoting increased uptake of new products , especially those offered by small and medium businesses. The use of improved cooking stoves, for example, can have a huge impact, but in most cases the end-users may not be aware of such products. Project complexity and the number of activities need to be well balanced with the targeted project impacts . Operations that target policy and institutional development need to be addressed in an integrated manner, as the lack of one intervention or wrong sequencing could hinder the achievement of the intended outcome. Realism is also needed with regard to what the project and other complementary operations can directly influence. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is well-prepared, candid, and focused on results. There is adequate evidence to support its assessment and ratings. The complementarity with other Bank and donor operations, and the overall sector context, were well articulated. A positive and commendable feature that is not usually seen in ICRs is Table 1, which delineates the proposed risk mitigation measures (at appraisal) and the ICR's assessment of the actual effectiveness of those mitigation measures during project implementation. The ICR complies with the OPCS guidelines for preparing ICRs. However, the M&E section could have given more information on the different experiences encountered between the solar and biomass components of the project. The discussion of project results related to energy efficiency is also weak. a.Quality of ICR Rating : Satisfactory