EXECUTIVE SUMMARY OF EVALUATION FOR PUBLIC DISCLOSURE FROM: C3P Terminal Evaluation of China Utility Based Energy Efficiency Program (CHUEE) Redacted Version 1. The Executive Summary of the Evaluation entitled Terminal Evaluation of China Utility Based Energy Efficiency Program (CHUEE) has now been redacted for public disclosure in accordance with IFC’s 2012 Access to Information Policy, following the Procedure for Development, Management and Disclosure of IFC Evaluations effective on 04/01/2014. 2. The attached redacted version reflects the following adjustments:  Redaction of sensitive or confidential information related to clients  Minor typographical corrections. 3. Questions on this document should be addressed to Dilyan Donchev (DDonchev@ifc.org) or William Beloe (WBeloe@ifc.org) Page 1 of 7 Executive Summary of Evaluation Terminal Evaluation of the China Utility Based Energy Efficiency Name of Evaluation Program (CHUEE) Date of Final Report of Evaluation July 2013 Author of Final Report of Evaluation Ravi Kantamaneni, ICFI, ravi.kantamaneni@icfi.com Date of this Executive Summary June 30, 2014 Ravi Kantamaneni, ICFI, ravi.kantamaneni@icfi.com; Dilyan Author of this Executive Summary Donchev, IFC-C3P, DDonchev@ifc.org Number of pages of this Executive Summary 6 Modifications from original Executive Summary With Modification(Please refer to cover letter) Executive Summary Approved for public William Beloe, IFC-GFGA1, WBeloe@ifc.org; Anna Hidalgo, IFC- disclosure by (name) on (date) ECRIO, AHidalgo@worldbankgroup.org Background The China Utility-Based Energy Efficiency Finance Program (CHUEE) was launched in 2006 to stimulate investment in energy efficiency (EE) and renewable energy (RE) projects by supporting financial institutions (FIs)1 to build a sustainable energy lending business in China. The Program provided investment services (IS) in the form of a risk sharing facility (RSF)2 to Chinese banks as well as technical advisory services (AS) to help finance projects, build institutional capacity, and promote EE across the Chinese commercial and industrial enterprise landscape. With the completion of the Program’s 6 year (2006-2012) implementation mandate, the governance framework of CHUEE state that an independent terminal evaluation to assess the Program’s progress and achievement of results is required. The Terms of Reference (TOR) and the Guidelines for Global Environment Facility (GEF) Agencies in Conducting Terminal Evaluations serve as the basis for this terminal evaluation. Per GEF guidelines, IFC is conducting a Terminal Evaluation of CHUEE to analyze and assess the achievements and progress made towards the original objectives, and document lessons learned to improve the design and delivery of future programs. The conceptual foundation for the evaluation is presented in Appendix B, which describes the linkage between the TOR questions (Appendix F) and the GEF evaluation areas:  Relevance: The extent to which the program structure, design and product was suited to local and national development priorities.  Effectiveness: The extent to which the actual program outcomes are commensurate with the original or modified program objectives.  Efficiency: The extent to which program results have been delivered value with the least costly resources possible (without carrying out a full financial audit).  Sustainability: The likely ability of the program to continue to deliver benefits for an extended period of time after their completion. 1 The term “FI” is used interchangeably with “banks” in this report. 2 The name of the Risk Sharing Facility changed over the life of the program, from Loss Sharing Agreement (LSA) to Loss Sharing Facility (LSF) to Risk Sharing Facility (RSF). For simplicity, only RSF is used in this document. Page 2 of 7  Additionality: The extent to which projects would not have been possible without program technical and financial support.  Catalytic impacts: The extent to which the program contributes to leveraging or catalyzing the investment of new and additional funds.  Monitoring and evaluation (M&E) systems: The extent to which the program has met the minimum requirements for M&E design and implementation. The assessment is based on the insights gathered through the stakeholder interviews and document reviews, with findings rated against the GEF evaluation areas. Objectives The key goal of the China Utility-Based Energy Efficiency Finance Program (CHUEE) is to overcome perceived market and technical barriers in China’s banking sector to develop and finance commercial projects in the areas of energy efficiency and GHG reductions. The program has three primary objectives:  Objective 1: Support EE project development by end users, partner utilities/EE equipment suppliers and ESCOs  Objective 2: Support partner FIs in building their EE lending business  Objective 3: Improve market awareness and public understanding of EE business Analysis Within the first year of the original CHUEE program, it was determined that there was a much larger under-funded EE market in China than expected, and therefore IFC sought to provide a second, larger facility (CHUEE II) to existing and new partner banks and to modify the program structure and priorities.  By the end of the program, three participating banks provided CHUEE loans totaling US$783 million, exceeding the end-project target of US$533 million  178 EE/RE projects financed, leading to the estimated avoidance of 19.02 million metric tons of carbon dioxide equivalent (MTCO2eq) annual greenhouse gas (GHG) emissions  The loans at two of the three participating banks are performing with no default; two defaults by the same client have been recorded at the remaining participating bank  Participating banks have gone on to build Sustainable Energy Finance (SEF) portfolios outside of the CHUEE program: o Bank A reported its SEF portfolio have reached over US$20 billion; more than 1000 companies have accessed finance via the bank’s SEF products o Bank B reported its SEF portfolio of US$16 billion o Bank C has reported an SEF portfolio of US$1.6 billion  Three comprehensive market studies completed, and program results have been widely disseminated  CHUEE III started in January 2013 with a focus on small and medium enterprises (SMEs) Key Findings The following table presents a high-level overview of the findings for each GEF evaluation area: Page 3 of 7 Relevance Highly Satisfactory CHUEE has played an important role in addressing two key market barriers to SEF in the Chinese banking sector: perceived market risks and technical risks. The RSF and AS provided a unique service offering that transformed the Chinese SEF market. The flexibility of CHUEE I and II allowed the program to successfully adapt over time to reflect market realities and the changing macro-economic context. Effectiveness Satisfactory The CHUEE program was effective in supporting the development of EE projects through diverse channel partners. CHUEE supported participating FIs to build their SEF portfolios through the unique combination of RSF and AS. Market awareness and public understanding of the EE business was also achieved. Efficiency Highly Satisfactory The results of CHUEE activities indicate that the program was cost-effective in facilitating EE investments and developing SEF portfolios for participating FIs. Sustainability Likely Short-term outlook for continued SEF activity without the support of RSF facilities is excellent. The government’s support of SEF lending and EE improvements is has important implications for program impact sustainability. Technical skills and knowledge built through AS has enabled FIs to undertake SEF activities independently. Public (GHG reductions) and private (energy savings) impacts have exceeded targets. Additionality Moderately Satisfactory Most EE projects supported by the program would have been implemented without CHUEE-supported loans, with the exception of some projects through completed by ESCOs and for SMEs. The very low proportion of loans overdue may indicate that the program could have taken on more significant risks. Catalytic Not Rated (per GEF guidelines) impact CHUEE had a catalytic impact on Chinese SEF stakeholders, particularly in the case of the ESCO relationship with both participating and non-participating FIs. The CHUEE program influenced the implementation of the Green Credit Guidelines in China. The CHUEE model is being replicated within China, and in other geographical regions. Monitoring Satisfactory and M&E procedures and evaluation reports provided good feedback, but evaluation recommendations could have been taken into account earlier. Some aspects of the program were not properly documented. Better indicators to track sustainability effects could be developed. Page 4 of 7 Conclusions and Recommendations A series of lessons learned arising from CHUEE is set out below along with specific recommendations which could be applied to future programs. LESSON LEARNED 1: Although CHUEE had many objectives and targets, the CHUEE program team chose to strategically prioritize GHG reductions and loan volume over specifically allowing SMEs access to finance. This early decision was an important tradeoff that resulted in the large GHG reductions and loan volume that made the program so successful. RECOMMENDATION 1: Program design should have a specific, well-aligned objectives and clear expectations. The CHUEE team made a conscious effort to prioritize GHG reductions and commercial viability over supporting SMEs, and this allowed for the program to meet and exceed GHG reduction and loan value targets. The program was considered a success by donors and other stakeholders as a result. In future programs, the program design should include well-aligned objectives that influence the selection of FI partners, other channel partners, targeted sectors, etc. LESSON LEARNED 1: Although CHUEE had many objectives and targets, the CHUEE program team chose to strategically prioritize GHG reductions and loan volume over specifically allowing SMEs access to finance. This early decision was an important tradeoff that resulted in the large GHG reductions and loan volume that made the program so successful. RECOMMENDATION 1: Program design should have a specific, well-aligned objectives and clear expectations. The CHUEE team made a conscious effort to prioritize GHG reductions and commercial viability over supporting SMEs, and this allowed for the program to meet and exceed GHG reduction and loan value targets. The program was considered a success by donors and other stakeholders as a result. In future programs, the program design should include well-aligned objectives. LESSON LEARNED 2: ESCOs are important market partners to address finance barriers and impact sustainability for SMEs. RECOMMENDATION 2: Include ESCOs as main project channel partners in future program design to facilitate the participation of SMEs. ESCO involvement in CHUEE RSF-supported loan applications was clearly linked to smaller project sizes, which suggests that ESCOs allowed for greater participation of SMEs in CHUEE I and II. ESCOs provide a shared savings energy performance contract model that reduces finance risk of EE projects for SMEs. ESCOs also provide technical assistance for both technology and finance aspects of the SEF market. LESSON LEARNED 3: The issue identified in the original program concept regarding SMEs’ lack of access to funding due to traditional lending criteria was not fully addressed. RECOMMENDATION 3: Take more financial risk to increase additionality by adjusting the lending criteria used for CHUEE I and II in future program design. Although one of the original program targets was to support smaller projects and SMEs, this was dropped in favor of maximizing GHG emissions reductions, and demonstrating commercial viability. FIs also preferred this approach as it enabled them to offer higher value loans to larger companies with less credit risk. As such, while CHUEE I and II successfully demonstrated the viability of large SEF portfolios Page 5 of 7 at participating FIs; the program did not address the issue of SMEs lacking access to finance as indicated in the original program targets. LESSON LEARNED 4: EE market transformation was an important program impact, but it was difficult to measure because FIs did not want to share details on their non-CHUEE loans. RECOMMENDATION 4: A clearer definition of transformational impact is required, with more rigorous M&E resources and more effective performance indicators. CHUEE had a catalytic impact on Chinese SEF stakeholders; however, the impact was hard to evaluate due to data deficiencies. For example, participating FIs were hesitant to share detailed information about non-CHUEE SE loans with IFC. Also, it was difficult to obtain SEF loan portfolio information for non-participating banks. Additionally, CHUEE influenced the implementation of the Green Credit Guidelines in China; however, without effective M&E frameworks the impact of this effort is hard to quantify. LESSON LEARNED 5: Communication and record-keeping improves market development effectiveness. RECOMMENDATION 5: Improve communication lines and record-keeping both internally and with channel partners including EE equipment suppliers, ESCOs, and end-users. There were gaps in the program documentation that affected the comprehensiveness of the evaluation. The utility company’s exit from the program was not properly documented, although CHUEE management was able to give a full explanation during an interview. However, full documentation would be helpful for lessons learned and future program design. Over time, the CHUEE PMO built up more AS services in response to the mid-term reports and PAC suggestions, which resulted in increased awareness of the program for all market players. However, end-user and ESCO contact details were not kept up to date, indicating that there was no regular personalized communication between CHUEE PMO to at least a few end-users and other market partners. IFC was therefore unable to access end-users and market partners for terminal evaluation interviews, which would have been useful for understanding end-user and channel partner impact. LESSON LEARNED 6: A rule change during the course of the program significantly reduced the potential size of the SEF loan portfolio compared to the original project design. RECOMMENDATION 6: Avoid making mid-program rule changes that reduce the program’s ability to meet its targets, or adjust targets as required. A change in program design limited the total number of projects financed by CHUEE I and II. On the advice from the M&E group, CHUEE loan targets were not adjusted mid-program to take into account this structural change that reduced the program’s ability to meet its targets. Therefore, the number of loans disbursed was below target (178 of 260; i.e. 68% of target). In future, targets could be adapted to account for program design changes. LESSON LEARNED 7: Modifying the original program design to allow for more diverse end-users resulted in greater program outreach than would have occurred by working with a utility as the key market partner, which has different objectives than the CHUEE program. RECOMMENDATION 7: Future programs should avoid designating loan end-users in advance using a set model and instead focus on developing a more diversified end-user base with the help of channel partners that share similar objectives. Page 6 of 7 The original program design included a utility company as a key channel partner and its customers making up a significant proportion of CHUEE RSF loan end-users. However, the objectives of the chosen utility and CHUEE were not aligned. The program structure was modified to include other channel partners such as ESCOs and equipment vendors which shared similar objectives to the CHUEE program. Through these channel partners, diverse end-users were selected that better met the criteria set by CHUEE and the FIs. This early decision by CHUEE management resulted in wider-reaching program with a greater impact on the market as a whole. LESSON LEARNED 8: The flexibility of the CHUEE strategy, structure, and target segments allowed for FIs to gain confidence in the SEF business model with existing clients. RECOMMENDATION 8: Maintain some flexibility in program strategy and structure to allow FIs to test the SEF business models with existing clients before pushing FIs to focus on priority target sectors. FIs participating in CHUEE I and II tended to start SEF businesses with their traditional client groups with whom they have existing relationships and can properly test and gain confidence in the SEF business model. An SEF program can encourage FIs to find new groups of borrowers, such as SMEs, and new types of products only after the FIs have developed confidence in the SEF business model. The flexibility of the CHUEE I and II strategy, structure, and target segments allowed for this process to occur and for banks to achieve and exceed GHG reduction and loan volume targets. Subsequent programs are helping FIs to work in strategic sectors such as SMEs, green buildings, etc., now that the SEF business model has been successfully demonstrated. LESSON LEARNED 9: The RSF alone did not convince participating FIs to increase risk tolerance or lend to non-traditional borrow groups. AS was instead the key factor allowing participating FIs to initiate or expand their SEF businesses. RECOMMENDATION 9: RSF requirements should include expanding to non-traditional borrower groups or other criteria if this is a key program target. The RSF added the most additionality when participating FIs lent to non-traditional borrower groups such as ESCOs and SMEs. There is not enough evidence that the RSF encouraged FIs to consistently reduce collateral requirements or lend to non-traditional borrowers. If program objectives require that participating FIs aggressively pursue non-traditional borrower groups, this should be included in the RSF requirements. In CHUEE I and II, AS seemed to be the key factor to ensuring that FIs initiate or expand their SEF business to new borrower groups. Copyright and Legal Disclaimer The material in this publication is copyrighted. IFC encourages the dissemination of the content for educational purposes. Content from this publication may be used freely without prior permission, provided that clear attribution is given to IFC and that content is not used for commercial purposes The findings, interpretations, views, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Finance Corporation or of the World Bank or the governments they represent. The views expressed in this publication are those of its authors and do not necessarily reflect the views of the wider institution. Some of the information used in this document may come from publicly available sources such as company websites and publications. The Lessons of Experience series does not represent a commitment by IFC to require projects it finances to take certain or all of the actions specified in this publication. Instead, any issues arising in an IFC-financed project will be evaluated and addressed in the context of the particular circumstances of the project. Page 7 of 7