Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Report Number: ICRR0022126 1. Project Data Project ID Project Name P122178 Turkey SME Energy Efficiency Project GEF Country Practice Area(Lead) Turkey Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-82420,IBRD-82430,IBRD-82440 28-Sep-2018 204,578,555.57 Bank Approval Date Closing Date (Actual) 27-Mar-2013 30-Sep-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 201,000,000.00 0.00 Revised Commitment 201,000,000.00 0.00 Actual 201,000,000.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ihsan Kaler Hurcan Fernando Manibog Ramachandra Jammi IEGSD (Unit 4) P132189_TBL Project ID Project Name P132189 Turkey SME Energy Efficiency Project GEF ( P132189 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-14579,TF-14580,TF-14581,TF-14582 3578555.57 Bank Approval Date Closing Date (Actual) 27-Mar-2013 Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 3,640,000.00 Revised Commitment 0.00 3,578,555.57 Actual 0.00 3,578,555.57 2. Project Objectives and Components DEVOBJ_TBL a. Objectives To support energy efficiency financing to small and medium enterprises (SMEs) in Turkey, the World Bank (the Bank) provided loans to three financial intermediaries (FIs) to establish credit lines. Additionally, the Bank provided a Global Environment Facility (GEF) grant for technical assistance and risk sharing to the three FIs and policy support to the Ministry of Energy and Natural Resources (MENR). The three FIs were the state- owned banks of Halk Bank, Vakif Bank and Ziraat Bank. Three loan agreements (LAs) and four grant agreements (GAs) were signed on May 6, 2013. The project objective in all agreements (LAs, p.5; GAs, p.6) was identical: “to improve the efficiency of energy use in small and medium enterprises in the Republic of Turkey, by scaling-up commercial bank lending for energy efficiency investments.” The objective in the project appraisal document (PAD, p.8) did not include the reference to “the Republic of Turkey”. The PAD (p.8) defined the global environment objective of the project as “to reduce Greenhouse Gas emissions through the removal of barriers to energy efficiency (EE) financing in the small and medium enterprises”. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components According to the loan and grant agreements, the project had four components. The first three components were identical but with separate headings for each FI. These three components are given as one component below. The second component, which was the fourth component in the loan and grant agreements, was to support the Ministry of Energy and Natural Resources (MENR) in the enhancement of the enabling environment for energy efficiency market development. Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) A. Credit Lines to Halk Bank, Vakif Bank and Ziraat Bank. (Appraisal cost: US$294.95 million; actual cost: US$271.06 million) 1. The establishment and operation of a credit facility within the three FIs for the financing of subprojects through the provision of sub-loans to SMEs. 2. The establishment of a loan loss reserve fund within the FIs to assist in defraying risks associated with new energy efficiency loan products and in addressing SME collateral requirements. 3. Enhancement of the capacity of the FIs in project implementation. B. Energy Efficiency Policy and Project Management Support to Ministry of Energy and Natural Resources (MENR). (Appraisal cost: US$10.94 million; actual cost:US$10.91 million) 1. Enhancement of the enabling environment for, and the fostering of, broader energy efficiency market development through, inter alia, the following activities to be carried out by MENR: (a) Market development and information dissemination, including: (i) raising awareness of, and providing training and disseminating information on, opportunities for energy efficiency and the success of schemes used in the credit lines established under this project; (ii) carrying out market studies and assessments and developing options for future investment programs beyond the SME market; and (iii) establishing a dialogue with stakeholders. (b) Strengthening of the energy efficiency and regulatory regimes, including: (i) reviews of energy efficiency policies and developing recommendations for improvement especially in the SME sector; reviews of energy efficiency incentive and informational programs (including impact assessments) and developing recommendations to improve utilization and impact of those programs; and reviews of institutional arrangements to strengthen the energy efficiency policy and implementation function in all sectors; and (ii) staff training. 2. Project implementation support to MENR. Revised Components In July 2016, the first component was restructured to extend sub-grants to energy service company (ESCO) subprojects, which were already eligible for risk coverage under the GEF loan loss reserve fund. (Restructuring Paper, Report No: RES23019, p.13). e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost was originally estimated at US$305.89 million. In September 2019, the project closed with a total cost of US$281.97 million. Financing: At appraisal, the International Bank for Reconstruction and Development (IBRD) loan was estimated at US$201.0 million, which was to be equally distributed to the three FIs to provide financing to SMEs for energy efficiency projects. The GEF grant was estimated at US$3.64 million. By project closing in Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) September 2019, the project had fully disbursed the IBRD loan and the actual disbursement of GEF funds was US$3.58 million. Borrower contribution: At appraisal, the contribution of the FIs was estimated at US$56.25 million and the MENR’s contribution at US$5.0 million. Sub-borrowers were expected to contribute US$40.0 million. At project closing, the FIs’ actual contribution was US$8.86 million. The ICR (p.9) states that because of the weak investment climate, the number of subprojects was not high enough to utilize both the credit lines and the FIs’ own funds during project implementation; hence, the FIs’ contribution was lower than estimated. The MENR’s contribution was US$4.0 million. Sub-borrowers contributed US$64.53 million, which partially compensated for the FIs’ low contribution. Restructurings: There were three project restructurings.  First Restructuring (July 21, 2015): The subproject eligibility criterion on Debt Service Coverage Ratio (DSCR) was lowered from an average 1.2 to 1.1 for the sub-loans over US$1.0 million and removed for sub-loans below US$1.0 million. Because of the economic slowdown and the SMEs’ fluctuating revenues from year to year due to their small sizes, if they could not meet the criterion even for one year, some otherwise financially viable SMEs would not be eligible to apply for a loan. Since the Bank’s due diligence confirmed that the FIs had adequate internal credit appraisal methods, it was agreed to lower the DSCR to allow more SMEs to be eligible for the loan. Additionally, the SME definition was revised to match the then official definition in Turkey. Lastly, some indicators were revised as follows: (i) The unit of the target value of the “Associated GHG reductions from project investment” indicator was corrected from “1,000 CO2-e/year” to reflect the unit of measure in the PAD, “Tons of CO2-e/year,” which reads as “tons of carbon dioxide equivalent per year”; (ii) The name of the “Number of SMEs attending awareness raising activities” indicator was corrected to read as “Number of SME personnel attending awareness raising activities” to measure the number of people, not the SMEs; and (iii) Due to the introduction of Core Sector Indicators, the name of the “Estimated energy savings (from project investments)” indicator was changed to “Projected lifetime energy savings (MWh)”.  Second Restructuring (July 26,2016): In March 2016, the Mid-Term Review (MTR) identified weak investment climate, competing financing sources available to the SMEs, depreciation of the Turkish lira and the lack of necessary preconditions for the development of ESCO-type deals as challenges for project implementation. To address these issues, following changes were introduced at the this restructuring: (i) The definition of SME (an enterprise with sales less than EUR50 million and employees fewer than 250) and Mid-cap Company (an enterprise that is not an SME and with employees fewer than 1,500) were adjusted to account for the depreciation of the Turkish lira; (ii) Vendor companies were added as an eligible category for sub-borrowers with a maximum sub-loan amount of US$10 million; and (iii) Due to no progress on ESCO deals, the loss coverage ratio in the Loan Loss Reserve Fund was increased from 20 percent to 50 percent; it was clarified that the size of the company in an ESCO deal could be any size as long as the end beneficiary is an SME or a Mid-cap Company; the DSCR requirement was removed on subproject sponsors that had entered into energy performance contracts. The target values of four intermediate indicators were revised based on the MTR’s projections: (i) The number of loans given using alternative business models was decreased from 45 to 15; (ii) The percentage of active loans to women-owned businesses from 25 percent to 10 percent; (iii) The amount of energy efficiency (EE) investments using the EE screening toll from US$225 million to Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) US$50 million; and (iv) the number of SME personnel attending awareness raising activities from 4,000 to 750. The target value of the “Projected lifetime energy savings” was also revised from 307,000 MWh/year to 6 million MWh for projected lifetime. There was a minor calculation mistake in the new target value; the target should have read 6.14 million MWh with a projected lifetime of 20 years (ICR, p.9).  Third Restructuring (March 14, 2018): The cofinancing requirement for the FIs was removed. According to the PAD (p.10), the FIs were to contribute an amount equal to 25 percent of the IBRD loan, once 50 percent of the IBRD loan was committed. This requirement was to be revisited at the MTR if the commitment rate did not reach 50 percent of the IBRD loan, which did not materialize; therefore, upon the request of the FIs, the Bank removed the cofinancing requirement. Additionally, the limit for sub-loans to Mid-cap Companies was increased from 20 percent of the loan proceeds to 27.5 percent due to increased demand. The target values of three indicators were increased: (i) The projected lifetime energy savings from 6 million MWh to 7.5 million MWh; (ii) the amount of associated greenhouse gas (GHG) reductions from project investments from 154,400 tons of CO2 equivalent per year to 220,000 tons; and (iii) the number of loans given using alternative business models from 15 to 60 due a sharp increase in these loans after the changes made in the second restructuring. On the other hand, the target values of four indicators were decreased: (a) The volume of bank funding to SMEs from US$121 million to US$106 million; (b) total value of EE investments from US$292.6 million to US$230 million; (c) percentage of active loans to women-owned business from 10 percent to zero; and (d) the amount of EE investments using EE screening tool from US$50 million to US$12 million. Lastly, the project closing date was extended by one year from September 28, 2018 to September 30, 2019.The extension was required to compensate for the initial slow project implementation caused by corporate restructuring in two of the FIs, depreciation of the Turkish lira, the weak investment climate and limited experience of the FIs and the SMEs with energy efficiency investments (Restructuring Paper, Report No: RES23019, p.4). Dates: The project was approved on March 27, 2013 and became effective on July 22, 2013. The original closing date was September 28, 2018. In the third restructuring, the closing date was extended by one year. The project closed on September 30, 2019. The reasons for closing date extension have been outlined in the third restructuring entry above. 3. Relevance of Objectives Rationale The project objective to improve energy use efficiency in SMEs is highly relevant to the country context. Turkey imports 75 percent of its energy needs as oil and natural gas. Therefore, enhancing energy security by decreasing the country’s dependence on imported energy is a strategic goal for the Turkish government. Although the energy consumption per capita in Turkey is lower than the per capita consumption in advanced economies, the energy intensity in Turkey is substantially higher. In other words, Turkey uses more energy to produce one unit of gross domestic product compared to advanced countries. Improving energy efficiency is an integral part of the country’s energy strategy and policy; the government introduced Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) the Energy Efficiency Law in 2007, and energy efficiency was included in various other policy documents, such as the Electricity Market and Security of Supply Strategy (2009), the National Climate Change Strategy (NCCS, 2010-2020), the National Climate Change Action Plan (NCCAP, 2011-2023), the Energy Efficiency Strategy (2012), and the 11th Development Plan announced in 2019 (ICR, p.8). According to the National Energy Efficiency Action Plan (2017-2023), Turkey aims at decreasing its primary energy consumption by 14 percent, which will require US$10.9 billion for energy efficiency investments. Since the SMEs produce about 60 percent of the gross domestic product in Turkey, improving energy efficiency in these enterprises would directly contribute to the achievement of the targets set by the National Energy Efficiency Action Plan. Additionally, improving energy efficiency is critical for the achievement of the commitment of the Turkish government to reduce greenhouse gas emissions by 21 percent by 2030 compared to the business as usual scenario (ICR, p.8). The project objective is also highly aligned with the current World Bank strategy as defined in the Country Partnership Framework (CPF) FY18-21. The third focus area of the CPF, i.e., Sustainability, includes the objective of “increased sustainability of infrastructure assets and natural resources”. This objective was defined based on the findings of the Systematic Country Diagnostic finalized in February 2017 that improving the energy consumption was critical for the country’s competitiveness and sustainable economic growth with positive impacts on climate change (CPF, p.23). The Bank strategy envisions follow-on or scaled-up projects in energy efficiency based on client demand and defines energy efficiency as supporting the energy security agenda under Objective 7, i.e., improved reliability of energy supply and generation of green energy. Enhancing SMEs’ access to finance is another goal of the Bank strategy under the Objective 2: Enhanced access to finance to underserved segments within the first focus area of Growth (CPF, p.15). The Bank has been an important development partner in Turkey in the energy sector since the 1960s. At the time of the project preparation, the Bank had already provided US$1 billion two other state-owned FIs, i.e., Turkiye Sinai Kalkinma Bankasi and Turkiye Kalkinma Bankasi, for the financing of renewable energy and industrial energy efficiency investments under the Private Sector Renewable Energy and Energy Efficiency Project (PAD, p.7). The Bank implemented energy efficiency projects in India, China, Ukraine and Uzbekistan. Therefore, the Bank had sufficient sector and country experience, but this project was the first one targeting the SMEs in Turkey. The three FIs, i.e., Halk Bank, Vakif Bank and Ziraat Bank, had the institutional capacity to implement the project utilizing their extensive branch networks throughout the country. Overall, the project objective was adequately challenging and appropriately pitched for the development status and capacity in the country. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) To improve the efficiency of energy use in small and medium enterprises in the Republic of Turkey, by scaling-up commercial bank lending for energy efficiency investments. Rationale Theory of Change The project sought to address the financing barrier and four other market barriers to energy efficiency investments in the SME sector (PAD,p.5): (i) Lack of knowledge among banks and SMEs about EE opportunities, project performance and risks;(ii) high transaction costs for small SME energy efficiency investments; (iii) financing constraints due to high collateral requirements; and (iv) limited institutional capacity to identify, prepare bankable energy efficiency projects. The project inputs—IBRD loan, GEF grant and technical assistance—were expected to reduce or remove these barriers through the availability of loans with long maturity, trainings, awareness raising activities, the development of standard technical product lines (to streamline the project application process; hence lower transaction costs), introduction of alternative business models (equipment leasing and ESCOs to lower high collateral requirements), the Loan Loss Reserve Fund (to help FIs mitigate the risks associated with new loan products and lower high collateral requirements) and policy support to the Ministry of Energy and Natural Resources. These activities were expected to facilitate the financing of at least 200 SME energy efficiency projects under the IBRD loan and scale up financing from the three FIs. The expected outcomes were energy savings, reduction in greenhouse gasses and decrease in Turkey’s dependence on imported energy. This change theory had some shortcomings. The assumption that the FIs could identify energy efficiency subprojects in SME clients and develop a sufficient project pipeline was optimistic, because the FIs lacked experience in the appraisal process; moreover, the business models the project was to introduce, and the willingness of the SMEs to take loans for energy efficiency investments could decrease if the investment climate was to weaken. Furthermore, it would not be realistic to expect the project to overcome those barriers with a limited intervention in terms of loan size and project scope (200 SMEs out of more than 4 million in the country). Yet, the project should have been expected to have a demonstration impact by encouraging the FIs to extend more loans to energy efficiency investments and the SMEs to undertake such investments beyond project closing. Overall, despite some shortcomings, the causal pathways from inputs to outcomes were valid and direct, and the outcomes achieved could be fully attributed to the project’s interventions. Outputs The project fully disbursed the loan amount to three FIs and the project activities resulted in the following outputs.  At about US$268 million, the actual total value of energy efficiency investments was higher than the revised target of US$230 million, but lower than the original target of US$292.6 million. The difference was because of lower contribution of the FIs than expected. The FIs were to provide US$50.25 million to finance subprojects, but they contributed only US$2.86 million due to insufficient project pipeline to utilize both the IBRD loan and the FIs’ funds during project implementation (ICR, footnote 2, p.9). On the other hand, the SMEs contributed US$64.53 million, which was US$24.53 million more than US$40 million estimated at appraisal.  At appraisal, the target for number of energy efficiency loans given was 200; the project closed with 325 loans given to 283 SMEs and 42 Mid-cap Companies.  The number of loans using alternative business models. i.e., ESCOs, and leasing and vendor credit, was 110. The original and revised targets were 45 and 60, respectively. The changes made in the Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) second structuring, such as eligibility for sub-grants, resulted in a sharp increase in the number of subprojects with alternative business models, which contributed to an overall increase in the loans.  The volume of Bank funding as lines of credit to SMEs was about US$157 million against the original target of US$121 million.  The amount of the investments using the energy efficiency screening tool that was developed to conduct initial assessments of potential energy efficiency subprojects and to determine project eligibility with the agreed criteria was US$16 million against the original target of US$225 million. The target was revised twice to US$50 million and US$12 million. The project team commented that the technical assistance consultants hired under the project had assessed the projects and determined their eligibility; therefore, the screening tool had been used in fewer subprojects than estimated at appraisal.  The project financed energy efficiency investments in 17 sectors. The subprojects in the textile and clothing industry and the metallurgical industry constituted about 40 percent of the loans given to energy efficiency improvements. The share of three other industries, i.e., chemical industry, energy and electrical industry and food industry, was about 25 percent (ICR, p.34). There was no target set for industries, but at project preparation machinery and equipment, metal products, food and beverage, textiles, trade and services, pulp and paper, and hotels and other commercial buildings were identified as target subsectors, although other sectors would also be eligible for financing (PAD, p.10).  Under technical assistance support 836 persons from the SMEs attended awareness raising activities. The original target was 4,000 which was revised to 750 after Mid-Term Review because of the sufficient demand created for the energy efficiency investment loans.  As originally planned, the project supported the Ministry of Energy and Natural Resources (MENR) through technical assistance as follows (ICR, p.10): (i) public awareness on energy efficiency, such as baseline assessment, communications strategy, awareness materials, online access to the screening tool on MENR’s website); (ii) a policy gap analysis to identify shortcomings in the policy and regulatory framework for energy efficiency in SMEs; (iii) evaluation of MENR programs to support SME energy efficiency; (iv) ESCO market development activities; and (v) a market assessment for energy efficiency in public buildings, which led to a follow-on project.  Consultants hired under the project held energy efficiency workshops to the staff of the FIs including those in local branches in different regions of the country. The consultants were involved in portfolio screening and provided on-the-job training, too (ICR, p.10). There were no targets set for these activities. Outcomes The availability of the IBRD’s longer maturity funds with attractive pricing was critical in addressing the financing barrier within the project’s limited intervention. The technical assistance activities were sufficient to increase awareness in energy efficiency and to create demand from SMEs for energy efficiency financing. As a result, the project financed 325 energy efficiency investments, and these investments are expected to save 10.7 million MWh of energy for a 20-year subproject lifetime. The original target was 6.14 million MWh, which was revised to 7.5 million MWh at the third restructuring in March 2018. These energy savings would result in about 400,000 tons of carbon dioxide equivalent per year reduction in greenhouse gas emissions, which was higher than the target of 220,000 tons. These end target values were revised upward at the third restructuring based on the actual project implementation, which demonstrated that actual energy savings and reduction in greenhouse gases per US dollar-invested was higher than originally estimated at appraisal based on a sample of representative projects. In addition to savings in electricity, some investments resulted in lower fuel Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) consumption (coal and natural gas), which led to higher energy savings and higher reduction of the greenhouse gas emissions despite a lower total value of energy efficiency investments than estimated at appraisal (ICR, p.9). The revision of the first component to extend sub-grants to ESCO subprojects was also critical in increasing the number of such projects and achieving the outcome targets in energy savings and greenhouse gas reductions. On the other hand, the project was to mobilize financing from the FIs in the amount of US$50.25 million, but their contribution stood at US$2.86 million. Although the FIs’ contribution requirement was subject to review at the Mid-Term Review, during which it was removed, the contribution of these three large banks to the financing of energy efficiency subprojects under the project was negligible. This raises concerns about the sustainability of the energy efficiency financing by these three FIs beyond project closure. The SMEs and the ESCOs contributed US$67 million of equity to subprojects, which was estimated to be US$40 million at appraisal. The evidence is insufficient to conclude whether the project was successful in leveraging more equity from the SMEs and ESCOs than estimated, because it is also possible that these companies might have preferred utilizing their own equity if the loans extended under the project were not financially attractive for every SME and ESCO. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale By extending longer maturity funds with attractive pricing and providing technical assistance to increase awareness in energy efficiency and create demand from SMEs for energy efficiency investments, the project was successful in addressing the financing barrier and helping three FIs increase their lending for SMEs energy efficiency investments. The project financed 325 subprojects, and these resulted in achieving higher projected lifetime energy savings than estimated at appraisal. The achievement could be attributed to the project’s intervention. Overall, the efficacy of the achievement of the project objective to improve the efficiency of energy use in small and medium enterprises by scaling-up commercial bank lending for energy efficiency investments is rated Substantial, but with serious concerns about the demonstration impact of the project and the sustainability of such financing by the FIs as a mainstream business activity after project closure. Overall Efficacy Rating Substantial 5. Efficiency Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Economic Analysis Since the project was a financial intermediary operation, the subprojects to be financed were not known at appraisal. Therefore, for economic analysis, a sample of representative projects already financed by the FIs through their other credit lines and meeting the eligibility criteria for the project was chosen to calculate the economic rate of return. The analysis included economic benefits from energy savings and environmental benefits priced at US$10 per ton of carbon dioxide mitigated (PAD, p.57). Reduced cost of operation and maintenance was not included in the analysis, which made it more conservative. The economic lifetime of the equipment financed by the project was assumed to be 15 years. The assumptions about electricity tariff and gas tariff were realistic. The analysis resulted in an economic internal rate of return (EIRR) of between 14.9 percent and 34.9 percent assuming the investment would not result in a production capacity increase—in other words, there would be no incremental increase in sales, hence, no additional economic benefits, because of the investment—and a net present value (NPV) of between about US$32,000 and US$990,000 at a discount rate of 10 percent (PAD, p.58). The EIRR of the projects for which benefits from capacity increase were included ranged from 13.3 percent to 81.1 percent and the NPV from US$2,490 to US$9.7 million. The EIRRs provide a better comparison than the NPVs due to the differing sizes of the subprojects. At project completion, same methodology was used to conduct an economic analysis, with the difference that the environmental benefits were priced at US$40 per ton of carbon dioxide mitigated per the current Bank guideline. For economic analysis, three subprojects financed by the project were chosen—one from each FI in the mining, metallurgy and textile sectors. The calculations resulted in EIRRs of 16.2, 28.9 and 39.2 percent for three projects—benefits from production capacity increase were included in the subprojects that yielded higher EIRRs. NPVs were not calculated. When environmental benefits from carbon dioxide reduction are excluded, the EIRRs at 11.2, 21.8 and 30.1 percent, respectively, were still high enough to justify the subprojects’ viability Financial Analysis The same assumptions were used in the financial analysis excluding benefits from carbon dioxide reduction. At appraisal, the analysis resulted in a financial internal rate or return (FIRR) ranging from 13.4 percent to 30.5 percent for subprojects without capacity increase, and from 14 percent to 67 percent for subprojects with capacity increase. At project completion, the three FIs provided FIRRs for all subprojects financed by the project. The average FIRRs calculated by the two of the three FIs were 22 and 24 percent. The FIRR calculated by the third FI ranged from 9 to over 200 percent, most of the subprojects having an FIRR in the 30-50 percent range (ICR, p.12). The FIRRs of the three projects, for which the Bank project team calculated EIRRs, were 10, 52.3 and 54 percent. Cost-effectiveness Analysis A cost-effectiveness analysis was conducted at project closing only by calculating the cost per unit of energy saved and the cost per unit of carbon dioxide emission reduced. The energy savings for a 20-year period was estimated at 10.7 MWh. When the total project cost of US$268.4 million is divided by the estimated energy savings, the investment cost per unit of energy saved is US$25 per MWh, which is 2.5 US cents per kWh. Assuming a 15-year period for energy savings, this increases to 2.9 US cents per kWh. Both investment costs per unit of energy saved is much lower than the cost of electricity in Turkey—which is between 7 and 8 US cents per kWh for SMEs. Similarly, the investment cost per unit of carbon dioxide emission reduced was calculated at US$45 per ton of carbon dioxide equivalent. The ICR (p.37) notes that the results were comparable to—but slightly above—the results of similar energy projects financed by the Bank, such as the Ukraine Energy Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Efficiency Project with US$19.2 per MWh for energy savings and US$63.4 tons of carbon dioxide equivalent for carbon dioxide reduction, and the Turkey Private Sector Energy Efficiency Project with US$17 per MWh and US$26 per ton of carbon dioxide equivalent. Operational and Administrative Efficiency The absence of a project pipeline had an adverse impact on the project’s efficiency during the early stage of implementation (ICR, p.15). The FIs preferred to postpone the marketing activities after the hiring of the technical assistance consultants, whose hiring process started after project effectiveness and lasted for about one year. Two of the FIs did not have experience with the Bank’s procurement guidelines. The Bank could have supported the FIs in strengthening the FIs’ capacity in contract management and procurement according to the Bank guidelines starting from an early stage of project implementation (ICR, pp.16-17). Due to the frequent reorganizations on the Ministry of Energy and Natural Resources (MENR) side, the management of a portion of the GEF grant was inefficient. The addition of the MENR as one of the project implementing entities increased the complexity of the project. Subproject eligibility criteria were included in the loan agreements. Therefore, changes in the eligibility criteria required project restructurings, some of which could have been avoided if they had been included in the operational manuals. Despite shortcomings in the operational and administrative efficiency, the project was successful in economically converting resources and inputs to results. Overall, the efficiency of the project is rated Substantial. Note: The following table could not be completed, because a range was given for the EIRRs at both appraisal and project closure. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Turkey is heavily dependent on imported energy, and the government strategy aims at decreasing this dependency through, energy efficiency investments. SMEs produce 60 percent of the gross domestic product in Turkey, and the credit availability for SMEs’ energy efficiency investments was limited; the project objective was highly relevant to the country context. The project objective was also aligned with the Bank strategy. Therefore, the relevance of objectives is rated High. Despite serious concerns about the demonstration impact of the project and the sustainability of energy efficiency financing by the FIs as a mainstream business activity after project closure, the project, through its limited intervention by extending longer maturity funds with attractive pricing and technical assistance in increasing awareness and creating demand for energy efficiency investments, was successful in achieving the targets set for energy savings and greenhouse gas reductions, and the achievement of these outcomes could be attributed to the 325 subproject financed by the project. The efficacy of the achievement of the project objective is rated Substantial. There were shortcomings in the operational and administrative efficiency of the project, but the project was successful in economically converting project resources and outputs into expected results. The efficiency of the project is rated Substantial. Overall, the outcome of the project is rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Economic and political uncertainties pose as a substantial risk to the sustainability of development outcomes. The Turkish economy has experienced frequent downturns since 2015. In addition to the economic slowdown in the European Union, which is the largest trading partner of Turkey, the regional conflicts and the coronavirus pandemic negatively impacted tourism revenues and adversely affected the Turkish economy. The chronic current account financing needs, long overdue structural reforms and the sharp volatility in foreign exchange rates have been the underlying reasons for economic uncertainties. Although the recent shift from the parliamentary government system to the presidential system seems to have brought some level of political stability, the future of the new system is not clear since the opposition parties have been advocating for a return to a parliamentary government. If these economic and political uncertainties lead to recurrent economic downturns, the SMEs’ production can fall resulting in lower energy savings from the energy efficiency investments financed under the project. SMEs’ low interest in energy efficiency investments is a substantial risk for the demonstration impact of the project. The SMEs are not sufficiently knowledgeable about the concept of energy efficiency, which is confused with renewable energy (ICR, p.74). The energy efficiency awareness is very low among the SMEs. For SMEs, the data and documentation requirements of energy efficiency investments are complex and time consuming. The investment loans can be more attractive to SMEs over energy efficiency loans due to pricing. The alternative business models are not well-established to be attractive solutions for SMEs’ energy efficiency investments. Unless there is a demand-side market transformation, scaling up energy efficiency in SMEs will not be possible. Similarly, a weakening in the commitment of the FIs to energy efficiency projects is a risk to scaling up energy investment financing. As a result of the project, the FIs strengthened their capacity in energy efficiency financing, but there are some barriers to mainstreaming this business model. Technical assistance consultants hired under the project played a critical role in supporting the FIs in screening and appraising Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) energy efficiency subprojects. Some FIs don’t have in-house capacity to technically appraise the projects (ICR, p.63). They would need to hire technical experts to continue with the business. Deposits in Turkish lira are short maturity, which does not match the longer maturity needed for energy efficiency financing. Unless the FIs have access to longer maturity funds with attractive pricing, such as the IBRD loans, they might find it difficult to scale up energy efficiency financing when there are other more attractive business opportunities. All three FIs noted in their ICRs that they would be interested in financing renewable energy projects, because the SMEs were more aware of the benefits of renewable energy, such as solar energy, and the demand for renewable energy financing was high (ICR, pp. 51, 63 and 74). 8. Assessment of Bank Performance a. Quality-at-Entry The strategic relevance of the project was high; energy efficiency is critical to decrease Turkey’s dependence on imported energy while achieving the country’s commitment in reducing carbon dioxide emissions. The approach of the project was also relevant; the project provided credit lines to three FIs with wide branch networks and strong SME client bases. The project benefited from the experience gained and lessons learned in other similar Bank-financed energy efficiency projects, such as selection of strong banking partners with a demonstrated interest in energy efficiency financing business, standardization of assessments to lower transaction costs, clear and transparent eligibility criteria and implementation of alternative business models gradually becoming complex (PAD, pp.14-15). The technical aspects of the project were adequately considered; investments to be financed by the project would be expected to consist of the replacement or upgrading of industrial equipment. Regarding safeguard policies, the project had adequately outlined the subproject scope to exclude those that would qualify as Category A in environmental assessment or trigger involuntary resettlement or international waters safeguard policies. Fiduciary aspects of the project were sound. Implementation arrangements were in place. The three FIs had limited experience with Bank projects, but they had the potential to improve their capacities to implement the project. The M&E framework was sufficient to assess the achievement of the project objective and test the links in the results chain, but there were minor shortcomings in its design (see section 9.a M&E Design below). Major risks were considered, and mitigation methods were identified, but the risk related to economic downturns was overlooked, “which led to slow uptake of the credit lines by the FIs and the SMEs” (ICR, p.16). A subproject pipeline could not be developed during appraisal because the FIs did not want to start marketing the energy efficiency financing products and identify subprojects before project effectiveness. The subproject eligibility criteria were included in the loan agreements rather than the operational manuals; therefore, when eligibility criteria were changed, this required a project restructuring that diminished the administrative and operational efficiency of the project. The Bank sufficiently identified, facilitated the preparation of, and appraised the project in order to achieve the planned development outcome, but there were minor shortcomings in risk assessment, M&E design, eligibility criteria and subproject pipeline preparation. Overall, the Bank performance in ensuring quality at entry is rated Satisfactory. Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR (p.19), supervision missions were held approximately every six months, which produced 13 Implementation Status and Results Reports. During the six-year project implementation period, there were three task team leader changes. All the task team leaders had been involved in the project since the appraisal; therefore, task team leader changes ensured continuity of focus on the development impact of the project rather than disrupting project implementation. The project team closely supervised the project and took proactive measures based on the M&E data to accelerate project implementation, which was stalled in the early stages. Deadlines were introduced by which the FIs would have committed a certain proportion of the loans, such as 20 percent by April 30, 2015 and 50 percent by the end of 2015. Introduction of such milestones improved project implementation (Restructuring Paper, Report No: RES19936, pp.3-4). Upon the request of the FIs, the project team revised the eligibility criteria to increase the number of subprojects. The project team successfully addressed the implementation challenges and bottlenecks identified at the Mid-Term Review through a project restructuring by amending the subprojects and subproject sponsor criteria and introducing GEF-funded sub-grants as incentives for ESCO subprojects (Restructuring Paper, Report No: 23019, p.3). These changes played a critical role in increasing the number of ESCO subprojects. The project team adequately supervised the safeguard and fiduciary aspects of the project, but more support could have been provided to the FIs and the MENR, which had different levels of experience in contract management procedures and the Bank’s procurement guidelines (ICR, pp.16 and 82). The success of the project team’s role in ensuring the continuity of bank loans for SME energy efficiency investments depended on factors beyond the control of the project, such as the chronic current account deficit financing requirement, fluctuations in foreign exchange rates, short maturity of loans, and SMEs’ preference of investment loans over energy efficiency loans due to pricing and complex data and documentation requirements (ICR, p.75). Overall, project team was substantially successful in identifying and resolving impediments to the achievement of the development outcome and the Bank’s fiduciary role. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objective was simple and clearly specified. The theory of change, given the project’s limited intervention in a large SME sector, was sound. The achievement of the project objective to increase Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) efficient use of energy was adequately captured by the “projected lifetime energy savings” indicator. This indicator was specific, measurable and achievable, and the methodology to measure energy savings by each subproject was well-defined, which included data measurement at commissioning (PAD, p.17). These ex-post assessments of subprojects were critical for verifying that the energy saving estimates were actually achieved (ICR, p.18). On the other hand, there were 12 intermediate outcome indicators and some of them were not relevant to capture the contribution of the project activities and outputs toward achieving the expected outcome, such as number of active loan accounts and portfolio at risk, but they had to be included in the results framework as core indicators. The results framework lacked indicators to capture the project’s impact in addressing barriers to scaling up energy efficiency financing. The project’s attempt to include a gender specific intermediate outcome indicator was laudable, yet it was not clear how the 25 percent target set for percentage of active loans to women-owned business in year 3 of project implementation was determined (ICR, p.18). The effectiveness of the technical assistance activities in strengthening the institutional capacity of the FIs in energy efficiency financing was not captured by the results framework. b. M&E Implementation The three FIs used the same M&E format to collect data at the subproject level. This system was applied to all 325 subprojects financed by the project to collect data for baseline and post-project production and energy use, loan and investment amounts, and energy savings and greenhouse gas emissions reduction (ICR, p.18). There were no major revisions in the results framework, but target values of some indicators were adjusted in accordance with the progress during project implementation including the outcome level indicator of projected lifetime energy savings (see Restructurings in section 2.e above). The M&E functions are well-embedded in the three FIs’ operational procedures, and they are highly likely to be sustained after project closing. c. M&E Utilization The FIs regularly and punctually reported the M&E findings to project stakeholders (ICR, p.16). The M&E findings enabled the Bank’s project team to proactively intervene to accelerate the project implementation. The project team and the FIs utilized the M&E findings to make changes in the subproject eligibility criteria and reallocation of GEF grants to accelerate disbursement. This resulted in a sharp increase in the number of loans disbursed through alternative business models. The ICR adequately used the M&E findings to provide evidence for the achievement of outcomes and also in analyzing the efficiency of the project. But given the large size of the SME sector in Turkey, it is not clear whether the M&E findings can influence subsequent interventions in the near future or not. Despite some moderate shortcomings in its design, the M&E system as designed and implemented was sufficient to assess the achievement of the project objective and test the links in the results chain. Overall, the project’s M&E quality is rated Substantial. Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Category FI under Environmental Assessment (OP/BP 4.01) and did not trigger any other safeguard policy. Environmental Assessment (OP/BP 4.01): Since the project was to be implemented by financial intermediaries, subprojects’ environmental impact would be assessed separately according to the national regulation and the Bank guidelines. The subprojects were expected to consist of mostly rehabilitation or renovation activities, or small-scale construction works, therefore, they were anticipated to be Category B or C due to their limited and reversible impact on environment. Category A project and projects triggering other safeguard policies would not be eligible for financing under the project. The Operational Manuals included detailed environmental review procedures and the manuals were disclosed in Turkey and in the Bank’s Infoshop. The ICR (p.19) notes that “The participating FIs screened environmental impacts, worked with their clients to prepare subproject technical and environmental documents, and supervised implementation as required.” One of the borrowers, i.e., Ziraat Bank, reported that the subprojects had no adverse environmental or social impacts, and the subprojects were classified as Category C (ICR, p.74). b. Fiduciary Compliance Financial Management Financial management arrangements were adequate (PAD, pp.37-38). The FIs established the accounting and reporting systems to comply with the project requirements and opened designated accounts by the effectiveness of the project. The Ministry of Energy and Natural Resources (MENR) did not have the financial management capacity. An action plan was agreed to bring the financial management arrangements of the MENR to an acceptable level by effectiveness. There were no significant problems or internal control weaknesses. The FIs had proper accounting and reporting systems in place, but the FIs were sometimes late in submitting the interim un-audited financial reports and completing internal audits. The staff in charge of financial management had other responsibilities, which increased their workload. In some cases, staff required support in managing financial reporting tables and reviewing the draft audit reports, which resulted in “some back-and-forth to maintain the financial management arrangements at an acceptable level” (ICR, p.19). The external audit of the project was in compliance with the international standards and it was unqualified. All project funds were accounted for at project completion. On the other hand, according to the MENR, the Bank provided effective guidance to the ministry starting only from the last quarter of 2017 in the preparation of financial reports and overcoming complicated financial issues such as tax refunds ((ICR, p.83). Procurement Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) The FIs were familiar with the Bank procurement procedures, but they needed support to implement procurement in accordance with the Bank’s guidelines in the early stages of project implementation. The MENR had difficulties in recruiting and retaining specialists with experience in the Bank’s procurement procedures (ICR, p.83). As the ICR (p.17) notes the FIs and the MENR could have benefited from more training prior to the tendering and implementation phase. The procurement under the GEF grant was compliant with the Bank’s guidelines, and the FIs’ private sector procurement methods and commercial practices adequately met the Bank’s requirements (ICR, p.19). c. Unintended impacts (Positive or Negative) None. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons This review has drawn two lessons incorporating material on lessons listed on pages 21-23 of the ICR. Flexible eligibility criteria can facilitate the development of a sufficient subproject pipeline. The Debt Service Coverage Ratio criterion was high for some otherwise-financially-viable SMEs to meet in every year to be eligible for financing. The number of subprojects increased after this criterion was lowered from 1.2 to 1.1 and removed for sub-loans below US$1.0 million. Furthermore, the inclusion of the eligibility criteria in the Loan Agreements required project restructurings when there was a need to revise the criteria. Although this did not have a substantial adverse impact on the operational and administrative efficiency of the project, some restructuring could have been avoided if the eligibility criteria had been included in the operational manuals. Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Subgrants can be critical, rather than the loan loss reserve funds, in demonstrating the successful implementation of the ESCO model but might not be sufficient for its sustainability. The experience of the project confirmed the findings of other projects with similar guarantee and credit enhancement schemes that loan loss reserve funds are generally insufficient to encourage the FIs to lend to small ESCO firms with weak balance sheet or limited experience. Following the introduction of GEF-funded sub-grants as incentives for ESCO subprojects at the second restructuring, there was a sharp increase in the number of these subprojects. The project was successful in achieving the target set for alternative business models, but due to the complexities and transaction costs of this model, absence of grants or similar incentives might hamper the sustainability of this model after project closing. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain An assessment of this project would contribute to the planned energy efficiency evaluation. 14. Comments on Quality of ICR The ICR is concise and tightly presents the performance of the project. It is candid in explaining the shortcomings of the project. The narrative is internally consistent; there is a logical linking and interrogation of the various parts of the report. The ICR sufficiently analyzes the evidence and links the evidence to findings. The evidence is restricted to the achievement of the indicators but sufficient to support the outcome rating. The ICR is also consistent with the Bank guidelines. The project cost table and additional tables in Annex 3 are detailed and useful to understand how the project funds were used. The results chain figure on page 4 of the ICR broadly explains the theory of change. On the other hand, the sections on safeguards and financial management could have benefited from a more detailed evaluative discussion. The last section of the ICR presents mostly findings rather than lessons and recommendations drawn from the experience gained during project implementation. a. Quality of ICR Rating Substantial Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Turkey SME Energy Efficiency Project GEF (P122178) Page 19 of 19