Executive Summary of Evaluation Terminal Evaluation of the Russia Sustainable Energy Finance Name of Evaluation Program (RSEFP) Date of Final Report of Evaluation July 2013 Author of Final Report of Evaluation Ravi Kantamaneni, ICFI, ravi.kantamaneni@icfi.com Date of this Executive Summary June 30, 2014 Ravi Kantamaneni, ICFI, ravi.kantamaneni@icfi.com; Dilyan Author of this Executive Summary Donchev, IFC-C3P, DDonchev@ifc.org Number of pages of this Executive Summary 6 Modifications from original Executive Summary Without Modification Executive Summary Approved for public Rolf Behrndt, IFC-CFMDR, RBehrndt@ifc.org; Anna Hidalgo, IFC- disclosure by (name) on (date) ECRIO, AHidalgo@worldbankgroup.org Background The Russia Sustainable Energy Finance Program (RSEFP) was launched in 2005 to stimulate investment in energy efficiency (EE) and renewable energy (RE) projects by supporting financial institutions (FIs) in building sustainable energy (SE) lending businesses. The program has provided credit lines to Russian FIs as well as technical advisory services to help finance projects, build institutional capacity, and promote EE and RE across the Russian commercial and industrial enterprise landscape. With the completion of the Programs’ 7 year (2005-2012) implementation mandate, the governance framework of the RSEFP state that an independent terminal evaluation to assess the Program’s progress and achievement of results is required. The Terms of Reference (TOR) and the Guidelines for GEF Agencies in Conducting Terminal Evaluations serve as the basis for this terminal evaluation. Per GEF guidelines, IFC is conducting a Terminal Evaluation of RSEFP to analyze and assess the achievements and progress made towards the original objectives, and document lessons learned to improve the design and delivery of future programs. The conceptual foundation for the evaluation is presented in Appendix C, which describes the linkage between the TOR questions (Appendix G) and the GEF evaluation areas:  Relevance: The extent to which the program structure, design and product was suited to local and national development priorities.  Effectiveness: The extent to which the actual program outcomes are commensurate with the original or modified program objectives.  Efficiency: The extent to which program results have been delivered value with the least costly resources possible (without carrying out a full financial audit).  Sustainability: The likely ability of the program to continue to deliver benefits for an extended period of time after their completion. Page 1 of 6  Additionality: The extent to which projects would not have been possible without program technical and financial support.  Catalytic impacts: The extent to which the program contributes to leveraging or catalyzing the investment of new and additional funds.  Monitoring and evaluation (M&E) systems: The extent to which the program has met the minimum requirements for M&E design and implementation. The assessment is based on the insights gathered through the stakeholder interviews and document reviews, with findings rated against the GEF evaluation areas. Objectives The key goal of the Russia Sustainable Energy Finance Program (RSEFP) is to establish a sustainable market capacity in Russia to develop and finance commercial projects in the areas of energy efficiency and new energy sources (including renewables). The program has five primary objectives:  Objective 1: Establish/monitor the IFC-GEF investment facility  Objective 2: Support SE project development by partner FIs and their clients  Objective 3: Improve market awareness, understanding of EE  Objective 4: Strengthen capacity of local energy service companies  Objective 5: Policy and legal support to EE investment projects Analysis RSEFP, initially designed as a 5-year program, was granted two extensions: from June 2010 to June 2012, and again, from June 2012 to December 2012. Below are some of the main achievements of the program over its lifetime:  By the end of the program, 14 partner FIs had financed 342 projects valued at US$289 million with expected GHG emission reductions of 559,000 tons of CO2 per year.  IFC signed credit line agreements for a total value of US$148 and US$77 million of IFC financing was invested in projects. As such, each US$1 IFC invested resulted in an additional US$2.76 of project financing from partner FIs.  The program received US$456,000 for the total value of its Advisory Services (AS) provided to FIs. Page 2 of 6  The program organized around 200 workshops, training events, seminars, conferences for a total of about 3,200 participants from various audiences including: end-users, vendors, FIs, journalists, policy makers, etc.  RSEFP published best practice guides, technical briefs, market studies and success stories.  The program communicated about SEF and program activities through a large array of media channels (including newspapers, TV and radio), developed awareness campaigns, organized study tours, and produced promotional films. RSEFP contributed to the development of Federal Law No. 261-FZ “On Saving Energy and Increasing Energy Efficiency, and on Amendments to Certain Legislative Acts of the Russian Federation” which was adopted by the State Duma in November 2009. RSEFP experts were then invited to participate in the development of several sub-laws. Key Findings The following table presents a high-level overview of the findings for each GEF evaluation area. Relevance Satisfactory RSEFP credit lines and advisory services have played an important role in developing the SEF market by remaining flexible and addressing the key barriers to market transformation through de-risking policy, lowering financial barriers and raising market awareness. Effectiveness Satisfactory The program exceeded initial targets and expectations and has helped build momentum in the field of SEF by effectively leveraging funds from other sources, and supporting the evolving policy debate, leading to the adoption of Federal Law No 261- FZ and its bylaws. Charging fees for AS has been successful; although some FIs have noted it a barrier to participation. Promoting the ESCO business model in a difficult market context has yielded few concrete results. Efficiency Highly Satisfactory The results of RSEFP activities indicate that the program was cost-effective in building capacity and facilitating EE and RE investments. At the project-level, the majority of investments are less than 0.01 tCO2 per US$ of loan disbursed, which are less efficient from a GHG reduction perspective. However, these projects have payback periods of 5 years (on average), which highlights the additionality of the program’s SEF activity. Page 3 of 6 Sustainability Moderately likely Short-term outlook for continued SEF activity without the support of credit facilities is poor. Regulatory uncertainty and a lack of confidence in the long term direction of government policy remain important risks. Technical skills and knowledge built in FIs has enabled them to move forward and undertake SEF activities independently. After the closure of RSEFP, a clear need exists for an organization to continue to provide capacity building and outreach activities. Public (GHG reductions) and private (energy savings) impacts have exceeded targets, but this performance and the long term benefits are very dependent on one single energy project. Additionality Highly Satisfactory Most projects would not have been implemented without RSEFP support due to the overall lack of market capacity and adapted financial products. RSEFP remains a distinctive and additional offering due to continuing market barriers. Catalytic Not Rated (per GEF guidelines) impact The key catalytic impact of the program has been its influence on the design and adoption of Federal Law No. 261-FZ on energy efficiency. However, the impact of RSEFP and market adoption of EE will be limited until detailed regulations and implementing policies are established. The RSEFP model is being replicated within Russia, and in other geographic regions. Monitoring Satisfactory and Effective collaboration between AS and IS and a “One IFC” approach to SEF have been evaluation key factors of success. A formal process for tracking program performance and reporting to donors was established; however, areas to improve granularity, consistency and transparency exist. Conclusions and Recommendations A series of lessons learned arising from RSEFP is set out below along with specific recommendations which could be applied to future programs. LESSON LEARNED 1: Continuing financial market uncertainties limit the short-term outlook for SEF products without credit facility support. RECOMMENDATION 1: At least in the short term, IFC should continue to offer credit facilities to support greater SEF penetration in the Russian market. Page 4 of 6 Long term liquidity issues in the Russian financial market, continuing effects of the financial crisis, policy uncertainty, and continuing short-term business mentality make it difficult for FIs to offer longer term (3-5 years) lines of credit for SEF products without support from IFC or similar programs. LESSON LEARNED 2: A holistic strategy of de-risking policy, lowering financial barriers, and raising awareness has helped transform the Russian EE market, but significant EE market penetration is still years away. RECOMMENDATION 2: A suitable organization should be identified to ensure continued EE outreach and dissemination of SEF knowledge after RSEFP closes. The program’s objectives were well defined to address the basic tenets of market transformation. Nonetheless, greater EE market penetration in Russia is still several years away. As such, continued outreach and dissemination of EE knowledge through a public or private organization will be required to maintain EE and SEF momentum. LESSON LEARNED 3: A “One IFC” approach to SEF is crucial to ensuring the successful adoption of SEF by partner FIs. RECOMMENDATION 3: A protocol should be developed to detail coordination/reporting requirements between AS and IS. This should be undertaken at the start of the program to ensure key responsibilities and lines of communication are defined, and internal capacity is sufficiently built. Over the course of the program, a good working relationship was established between AS and IS, which was instrumental in engaging with FIs effectively and presenting a seamless approach: credit lines and AS. Nearly 80% of SEF business generated has been through FIs that have initially established a credit line and advisory services relationship with RSEFP. LESSON LEARNED 4: Charging fees for AS has been successful although some FIs have noted it a barrier to participation. RECOMMENDATION 4: AS should continue to be provided on a fee basis; although, the transparency in AS costs per client might need to be strengthened. Since 2009, AS has been charging a full fee service. This fee structure has been met with broadly positive FI sentiment and helped ensure FI management buy-in and commitment to SEF. However, it has dissuaded some banks from participating in the program. LESSON LEARNED 5: A comprehensive process for reporting on program activities and performance was established; however, there are still areas for improvement. RECOMMENDATION 5: RSEFP should encourage FIs to provide more robust and detailed project data, including actual energy savings achieved by projects. Generally, the granularity, transparency and consistency of the information/indicators tracked by the program could be improved; in particular, the data concerning the projects financed which relies heavily on the quality of the information provided by FIs. Page 5 of 6 LESSON LEARNED 6: RSEFP has been successful in developing SE projects, in particular, those with longer payback periods. RECOMMENDATION 6: Program targets should be clearly prioritized in order to ensure that activities support the principle aims of the program Over the course of the program, 342 SE projects have been implemented, with each US$1 of IFC finance producing $2.76 of project financing from partner FIs and end users. Almost 60% of projects demonstrate emission reductions of less than 0.01tCO2/US$. While such projects are less efficient from a GHG reduction perspective, they are more likely to demonstrate longer payback periods (i.e., 5 years on average). Therefore, while facilitating SEF and GHG reductions are clearly linked, it is important that the program indicates which targets are to be prioritized (e.g., maximizing SEF or GHG reductions) so that the principle objectives of the program are not jeopardized. LESSON LEARNED 7: AS proved to be an effective, flexible tool to engage with private and state FIs to extend the reach of the program. RECOMMENDATION 7: Since the overall objective of SEF is to facilitate investment in EE and RE, IFC should consider making exceptions to enable AS to work with state banks in future programs, if relevant to the market. Over US$460 of SEF loans were disbursed for every US$1 of AS spent. Similarly, although no formal AS agreements were established with state banks within the program period, AS has effectively engaged with Gazprombank since 2006, and Vnesheconombank (VEB), who have signed an AS agreement in early 2013 after IFC provided an exception to enable AS to support a state bank. Copyright and Legal Disclaimer The material in this publication is copyrighted. IFC encourages the dissemination of the content for educational purposes. Content from this publication may be used freely without prior permission, provided that clear attribution is given to IFC and that content is not used for commercial purposes The findings, interpretations, views, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Finance Corporation or of the World Bank or the governments they represent. The views expressed in this publication are those of its authors and do not necessarily reflect the views of the wider institution. Some of the information used in this document may come from publicly available sources such as company websites and publications. The Lessons of Experience series does not represent a commitment by IFC to require projects it finances to take certain or all of the actions specified in this publication. Instead, any issues arising in an IFC-financed project will be evaluated and addressed in the context of the particular circumstances of the project. Page 6 of 6