Executive Summary of Evaluation Midterm Review (MTR) report for the IFC “Russia Residential Name of Evaluation Energy Efficiency Program” (RREEP) Date of Final Report of Evaluation June 7, 2013 Seth Landau, Vladimir Zhuzhe, Ksenia Petrichenko, and Anne Author of Final Report of Evaluation Maassen, ECO, United Kingdom Tel +44-20 30 120 130, info@ecoltdgroup.com Date of this Executive Summary July 10, 2014 Luis Domenech, IFC Development Impact Department Author of this Executive Summary ldomenech@ifc.org , Olga Y. Ioffe Kasher, IFC F&M GP - Europe & CA OIoffe@ifc.org Number of pages of this Executive Summary 6 Format modifications and additions from final report. No Modifications from original Executive Summary content modifications Executive Summary Approved for public disclosure by (name) on (date) Anna Hidalgo, IFC-ECRIO, AHidalgo@WorldBankGroup.org Background IFC “Russia Residential Energy Efficiency Program” (RREEP) addresses the great need for energy efficiency (EE) in the Russian Federation’s residential housing sector, which is the second largest sector in terms of its greenhouse gas (GHG) emissions in the country. The program is funded primarily by the Finnish Government and the Global Environment Facility (GEF) and is expected to run from March 2010 until November 2015. Improving the EE of the residential housing sector could lead to an estimated reduction in energy consumption of 51 percent by 2020, leading to an estimated reduction in CO2 emissions from residential buildings of 150 million tons per year by 2020. Unlocking the market potential for energy efficiency and GHG reductions, however, has been slow because of several significant market barriers, including insufficient market awareness and an unfavorable legal and financial enabling environment. Objectives To assist in addressing the above-mentioned barriers, the IFC developed RREEP, which is structured under three Objectives: Objective 1 “Development of an appropriate legal and regulatory environment”, dealing with legislative and regulatory barriers; Objective 2 – “Development of financial products”, targeting financial market barriers and Objective 3 “Raising awareness and increasing market capacity” targeting awareness and knowledge barriers This document is the Midterm Review (MTR) report for the RREEP. The MTR provides an opportunity to support the sound planning and adaptive management of the project by the IFC while facilitating the reporting of progress and impacts to its donors for the implementation period of 1 March 2010 to 1 April 2013. Page 1 of 6 Analysis The key finding that emerges strongly from the review is that RREEP is on track to create a market for EE investments using bank lending in the multi-family housing sector in the Russian Federation, however, this is taking longer than originally envisioned as a result of slower than expected legislative reforms. Legislative barriers have taken substantially longer to address than was expected at the time of project design, and overcoming them has proved to be essential to unlocking the market. In agreement with the Terms of Reference of the MTR, the Reviewers assessed RREEP across five dimensions that are in accordance with standard international evaluation principles and criteria and minimum requirements. By using these review criteria, the Reviewers were able to arrive at a consolidated picture of RREEP’s progress since inception in March 2010, until the timing of the MTR in April 2013. Recommendations have been made for the next implementation period and lessons learned summarized. Review findings by criteria, recommendations and lessons learned are summarized below. Key findings by review criteria  Program Design and Relevance – RREEP’s original design has proved to be relevant to local circumstances and addresses key sector needs and market barriers. The Program is timely, and fits well with IFC’s organizational strengths and priorities. The three Project Objectives are: (1) Addressing key legal and regulatory issues, in particular the framework for EE in residential buildings and legislation enabling local Home Owner Associations and Housing Management Companies to access finance for the purpose of improving EE in multi-family buildings; (2) Develop and market financially-viable residential EE products by working with 3-5 Russian banks; and (3) Increase capacity and awareness about residential EE among key market stakeholders. These objectives are generally appropriate to address the barriers and the Program strategy is consistent with the needs of key stakeholders. Some targets and timeframes in the original design have proved to be too optimistic, as Project risks identified in the original project design did not take adequate account of the risks associated with slow development of the legislative frameworks, and related delays in the development of bankable borrowers. There are some issues related to the design and tracking of indicators – most importantly that there is no outcome-level indicator on capacity building.  Effectiveness – RREEP has been effective in bringing about important legislative changes, exhibiting a high degree of stakeholder inclusiveness and approval that make elements of Program implementation stand out as best practice. Based upon a scale used by the GEF for evaluating such projects, the Reviewers rate the Implementation Progress as “Highly Satisfactory” given that all Objectives and activities of the Program are in substantial compliance with the original/formally revised implementation plan for the Program. However, given significant shortcomings in meeting targets for investment volumes and GHG reductions, RREEP’s Development Objective Rating is rated “Moderately Satisfactory” at this stage. The notable aspect of the implementation and managerial arrangements of the Program is that the GEF-funded portion of the Program is being jointly implemented by IFC and the EBRD. This unique partnership of two IFIs has turned out to be Page 2 of 6 especially effective and the impact of the two GEF Implementing Agencies has likely been greater than it would have been with just one involved. This is especially the case related to the development of the legislative framework. The relatively slow pace of progress towards overall development objectives – target investment volumes and GHG reductions (see further ‘Results and achievements’), should not be taken as a reflection of the Program’s effectiveness but as related to an overly optimistic assessment during project design of the pace of development of legislation, which was logged as part of the RREEP team’s effective adaptive risk management practice.  Efficiency – RREEP has achieved a good balance between impact and financial resources within the benchmark of what would be expected from a technical assistance program in a market as big as Russia. Inputs into training, information dissemination and regulatory support have been of high quality and timely. Significant increases in residential demand and prospective local financial institutions interested in on-lending are necessary to scale-up the investment projects pipeline to levels that are needed for meeting Program targets.  Results and achievements – The Program’s mid-point achievements and progress toward longer- term Impact-level indicators, including the sustainability of these Impacts are as follows: Objective 1 (Development of an appropriate legal and regulatory environment) is being accomplished at the national level though requires additional work at the regional level to improve the regulatory framework. Objective 2 (Development of financial products) is being accomplished for the most part (in particular training and information dissemination activities), though investment targets are not being met. All Objective 3 (Raising awareness and increasing market capacity) Output-level targets have been achieved. The Objective 3 Outcome-level target of 2000 formal loan applications will require significant scaling up in the coming two and a half years. In the absence of the Program, it is very likely that the Law on Capital Repairs would have missed key market-friendly elements such as building-level bank accounts and decision-making powers. Interviews further confirmed that remaining substantial legislative barriers, principally those that prevent HOAs and HMCs from being ‘bankable’, are likely to be addressed within the next few years. The scale of the potential market is so vast that the legislation passed marks a breakthrough in making it possible to invest in EE in buildings and potentially having huge indirect impacts on GHG emissions.  Sustainability – RREEP is likely to have significant sustainable impacts on the entire market for building-level EE improvements in the Russian Federation, due mostly to the success in adjusting the legislative framework, as well as other Program activities. Significantly, the full implementation of the Law on Capital Repairs will cause a drastic shift in the institutional structure of payments and implementation of multi-family building repairs. It would appear that awareness raising and capacity development through national-level institutions with outreach to sub-national actors (e.g. Home- owners Associations – HOAs – and financial institutions) is highly likely to continue in the post- implementation period. However, given the large scale of the market, additional work at the regional level will be critical for generating sustainable impacts on all levels of project activities – in particular work on regional legislation and regulation (Objective 1), availability of financing products (Objective 2) and sub-national stakeholders’ awareness and capacity (Objective 3). Conclusions and Recommendations Recommendations on Program design include: Page 3 of 6  To make the Program goal realistic either the targets should be reduced, or, preferably, in the opinion of the Reviewers, the duration of the Program should be extended by at least 1 year and ideally to 3 years if funding can be identified.  Additional technical assistance should be mobilized to support cooperation with Regional authorities, given their importance in the current policy framework.  An Outcome-level indicator for capacity building should be developed and tracked.  In Program reporting, the specific legislative barriers that need to be addressed and that have been removed due to Program inputs should be made explicit and matched to the numerical indicators.  The Program should consider tracking impacts from Program activities on increased apartment-level loans for energy efficiency interventions.  The budget for the risk-sharing mechanism may need to be re-allocated. Within the GEF project budget, ~USD 3.38 million was meant for IFC to set up a first-loss guarantee mechanism for banks involved in the sector. This mechanism has not been developed to date. Aside from returning the money, the IFC has the following options: 1. Move forward with bi-lateral agreements between IFC and partner banks agreements on the risk facility/ 1st loss agreement – as described in the GEF Request for CEO endorsement; 2. Establish regionally-based risk-sharing funds that could work with numerous partner institutions – with funds to be returned to the IFC/GEF; 3. Establish regionally-based risk-sharing funds that could work with numerous partner institutions – with funds to be non-reimbursable; 4. Set up a fund for the purchase of receivables/ forfaiting; and 5. Reallocate some funds for additional technical assistance at the regional level. All of these options are possible as described within the GEF Request for CEO endorsement (the basis for receiving the funds), but any change undertaken should involve notification of the GEF Secretariat. Re-allocation of the budget to non-reimbursable mechanisms such as technical assistance at the regional level or grants to regional funds would likely need approval from the GEF Secretariat and agreement with EBRD if the budgetary change is less than 10% of the project grant (i.e. USD 513,656). If the amount is more than 10% of the project grant, this will require a revised Request for CEO endorsement and circulation to the GEF council. Recommendations on Program implementation include:  Technical assistance should be (re-)deployed to focus on the most promising investment prospects with selected Regional Authorities (e.g. 2 to 3 regions) and perhaps in partnership with the largest and best-run Housing Management Companies (HMCs).  Awareness raising activities should closely track emerging needs for technical assistance of partner banks as they arise. Partner banks’ promotional activities could be supported through the development of targeted messages based on target audience focus group research. Page 4 of 6  Related to awareness-raising, the Reviewers recommend focusing on quality of the attendees to HOA/HMC workshops and trainings and on continuing to leverage existing networks to distribute information.  Indicators, in particular those required for GEF reporting (e.g. quality of life of tenants, capacity improvements), should be tracked more consistently, e.g. using ‘before’ and ‘after’ questionnaires.  The levels of investment being made via partner banks should be tracked against Program plans closely which will make it clear when additional intervention – additional awareness raising, altered technical assistance, etc. – is necessary.  For calculations of energy and GHG savings, the use of district heating-specific emissions factors and inputs from feasibility studies/due diligence would enhance the methodology for more effective ex- post estimations. Lessons learned The review process generated several useful lessons:  A project that heavily depends on developments beyond the project’s control (e.g. development of legislation or regulation) is inherently risky. However, in some cases this cannot be avoided as legislation must be addressed before the private market can be unlocked. In such cases care must be taken to take into account that developing legislation can take substantially longer than expected - this should be reflected in the risk analysis and mitigation strategy.  Cooperation with others, in this case the EBRD, can prove very effective in addressing legislative barriers.  Local champions are valuable in strengthening the demonstration of market potential.  In the multi-family residential sector, public outreach through leveraging existing networks is effective for generating larger-scale dissemination.  Flexibility in a project’s design, for example when implementing regulatory support and in developing investment mechanisms, may be necessary to ensure continued relevance of the project. Copyright and Legal Disclaimer The material in this publication is copyrighted. IFC encourages the dissemination of the content for educational purposes. Content from this publication may be used freely without prior permission, provided that clear attribution is given to IFC and that content is not used for commercial purposes The findings, interpretations, views, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Finance Corporation or of the World Bank or the governments they represent. Page 5 of 6 The views expressed in this publication are those of its authors and do not necessarily reflect the views of the wider institution. Some of the information used in this document may come from publicly available sources such as company websites and publications. The Lessons of Experience series does not represent a commitment by IFC to require projects it finances to take certain or all of the actions specified in this publication. Instead, any issues arising in an IFC-financed project will be evaluated and addressed in the context of the particular circumstances of the project. Page 6 of 6