Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Report Number : ICRR0022924 1. Project Data Project ID Program Name P157571 Power Sector Financial Recovery Program Country Practice Area(Lead) Armenia Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Program Cost (USD) IBRD-86150 30-Jul-2021 29,999,595.50 Bank Approval Date Closing Date (Actual) 27-Apr-2016 30-Jul-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 30,000,000.00 0.00 Revised Commitment 30,000,000.00 0.00 Actual 29,999,595.50 0.00 Prepared by Reviewed by ICR Review Coordinator Group Kishore Laxmikant Dileep M. Wagle Ramachandra Jammi IEGSD (Unit 4) Nadkarni 2. Program Context and Development Objectives DEVOBJ_TBL a. Objectives The proposed Program Development Objective (PDO) was to support the Borrower's efforts to maintain adequate and reliable electricity supply through improvement of the financial standing of the state- owned power generation companies and the private power distribution company. (Loan Agreement, Schedule 1 and Project Appraisal Document, para. 61). The Borrower was the Republic of Armenia (RoA). For the ICRR, the PDO is parsed into the following objectives: Page 1 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Objective 1: To maintain adequate electricity supply through improvement of the financial standing of the state-owned power generation companies and the private power distribution company. Objective 2: To maintain reliable electricity supply through improvement of the financial standing of the state- owned power generation companies and the private power distribution company. The original PDO remained unchanged during implementation. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components (Reference PAD paras. 42 to 47 and Annex 2, and ICR paras. 17 and 18). The program scope, activities, and classification into Results Areas, as defined in the PAD, are given below. Changes in scope, activities, indicators, and targets during implementation are discussed later below in the sections on restructurings. The planned scope and activities were classified into four Results Areas (RAs) along with associated Disbursement Linked Indicators (DLIs). The DLIs are discussed later below in Section 3 on Relevance of Objectives.  Results Area 1: Elimination of non-core cash outflows of state-owned power generation companies: (Bank loan amount allocated at appraisal: US$9.83 million,33% of the total; disbursed amount from Bank loan US$9.83 million): The objective was to be achieved through: (a) a decision by the shareholders of the state-owned power generation companies to prohibit non-core business- related expenditures, and (b) liquidation of Haigasard CJSC (Close Joint Stock Company), a 100 percent state-owned company, historically used to park liabilities of privatized companies in the power sector and to channel funds to finance non-core business expenses.  Results Area 2: Rationalize the current assets and liabilities of the power-generation companies:. (Bank loan allocated amount allocated at appraisal US$11.54 million, 39% of the total; disbursed amount from Bank loan US$11.54 million). This was to be achieved through: (a) replacing principal amounts of expensive commercial loans of state-owned power generation companies with concessional IBRD resources under the Program; (b) improving the collection of receivables from ENA (Electricity Network of Armenia), the single buyer distribution company; and (c) repayment of YTPC's (Yerevan Thermal Power Company) payables for natural gas from Gazprom Armenia CJSC using cash from the recovery of receivables from ENA and disbursements under the IBRD loan.  Results Area 3: Setting of tariffs reflecting changes in the cost of electricity supply: (Bank loan amount allocated at appraisal - US$4.00 million, 13% of the total; disbursed amount from Bank loan US$4.00 million). This was to be achieved through: (a) amendment of ENA's license to allow adjusting the margin by the full size of the loss (profit) incurred due to mismatch between actual and Page 2 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) forecast cost of electricity purchased by ENA: and (b) revision of the tariff-setting methodology to allow adjusting the tariff of the large thermal plants by the full size of the natural gas purchase- related loss (profit) incurred due to fluctuation of the exchange rate between the AMD and the US$.  Results Area 4: Maintaining the generation capacity of the gas turbine at YTPC's Combined- Cycle Gas Turbine (CCGT) plant: (Bank loan amount allocated at appraisal US$4.55 million, 15% of the total; disbursed amount from Bank loan US$4.55 million).This was to be achieved through replacement of some turbine and compressor parts of YTPC's existing CCGT plant to preserve its generation capacity and allow it to supply the required amount of electricity. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Program Cost: The estimated program cost at appraisal was US$43.20 million. The actual cost at completion was US$54.70 million, US$11.50 million higher than planned. Financing: At appraisal, the Program was to be supported by an IBRD loan of US$30 million. At completion, the loan was fully disbursed. Borrower Contribution: At appraisal, the planned Borrower contribution through YTPC was US$13.20 million. The actual amount contributed was US$24.70 million, US$11.50 million higher than planned. Dates: The Project was approved in April 2016, became effective in July 2016, and the planned closing date was July 30, 2021. The Project closed on the originally planned closing date. Mid-Term Review: A Mid-Term Review (MTR) was carried out in March 2019. Restructurings: The Project underwent one Level 2 restructuring in June 2021. (Disbursement US$29.42 million or 98% of the loan amount). The restructuring responded to the Government of Armenia's (GoA) request to modify the target for the final DLI (2.3) linked to Results Area 2 (rationalizing the current assets and liabilities of the power generation companies). Two changes were made at the restructuring as follows:  The target for the final DLI (DLI 2.3) was changed from "Haigasard's liquidation is completed and registered by the authorized state agency" to " first public tenders to sell Haigasard's assets have closed". The rationale for this change is provided under the discussion of DLIs later in Section 3.  The relatively small (1.6 percent) residual amount remaining under the IBRD loan was reallocated from YTPC to ANPP (Armenian Nuclear Power Plant). 3. Relevance TBL Page 3 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) a. Relevance of Objectives New Rationale (Reference PAD paras. 1 to 12 and ICR paras. 1 to 5). Country and Sector Context: In the four years leading up to appraisal in 2016, Armenia went through a period of slow growth and sharply increasing government debt. The Government debt as a percentage of GDP (Gross Domestic Product) increased sharply from 36 percent in 2012 to 52 percent in 2016 financed through Eurobonds and concessional support from international financial institutions, including the World Bank. The increase in debt was driven by government stimulus in response to the slow-down; the stimulus focused on infrastructure and social projects. In parallel, the Government of Armenia (GoA) decided to put in place a program of targeted subsidies for electricity consumption after an electricity price increase of 16 percent in 2015 resulted in street protests. Poverty and inequality continued to remain high. The electricity sector had been healthy up to 2010, with cost recovery tariffs and elimination of subsidies, but its financial performance declined thereafter. The financial viability of state-owned power generation companies ANPP (Armenian Nuclear Power Plant) and YTPC (Yerevan Thermal Power Company) began to deteriorate starting 2011 due to a number of factors. In addition to the impact of the economic slowdown, these factors included (a) non-core business related borrowing, lending and expenses; and (b) insufficient recovery of exchange rate losses related to gas purchases permitted in the tariff formula. Sector health was also impacted by the inability of ENA (Electricity Network of Armenia), the private sector distribution company and single buyer, to recover losses stemming from tariff-related forecasting errors in the price of electricity purchased. The financial difficulties of ANPP and YTPC impacted their ability to maintain adequate levels of power generation thereby affecting the cost of electricity supply in Armenia. Furthermore, financial constraints impacted the ability of ENA to maintain and modernize its infrastructure, thereby endangering reliability of electricity supply. These issues were starting to affect the competitiveness of the Armenian economy and potentially lead to social and health issues for the population. Alignment with national priorities: (PAD paras. 39 to 41 and ICR paras. 4 and 11): The program development objective (PDO) was consistent with the national priorities prevailing at the time of appraisal (as specified in the GoA's National Concept of Energy Security 2013 and the Armenian Development Strategy for 2015-2024) and continue to remain so under the current national priorities. Armenia's Development Strategy for 2021-2025 emphasizes the importance of reliable electricity supply and financial viability of the power sector. To address the situation, the GoA approved in February 2016 a Financial Recovery of the Power Sector Program. The program included 10 specific remedial measures and was estimated to cost US$43.2 million. The 10 remedial measures under the program included:  prohibiting non-core expenses of state-owned power generation companies (ANPP and YTPC);  reducing their payables for gas;  reducing their receivables from ENA;  refinancing their commercial loans with concessional finance;  allowing ENA to recover cost-forecasting related losses through its tariff;  allowing the power generation companies to recover foreign exchange related losses on gas purchases through tariffs;  liquidating Haigasard, the entity used for channeling the generation companies' non-core expenses; Page 4 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571)  legal collection of debts owed to the generation companies by state-owned enterprises;  rescheduling the generation companies' debts to Gazprom Armenia;  overhaul of the gas turbines at YTPC's CCGT plant to allow it to operate at capacity. The PDO remains aligned with Armenia's Energy Sector Development Strategic Program to 2040 (approved in 2021). The Vision under the Program includes energy to be 'safe and reliable' and the sector to be 'energy independent ' and 'attractive to investors'. The Program includes 10 action areas of which the PDO contributes to 'Development of Generation Capacity, 'Development of the Distribution Network'. 'Electricity Market', and 'Management of State-Owned Enterprises'. Alignment with the Country Partnership Strategy/Framework: (CPF for FY19 to FY23): At appraisal, the PDO was consistent with the priorities under the then prevailing Country Partnership Strategy (CPS) for FY13 to FY17, particularly the engagement area 1.3 of the CPS (improved access, quality, and sustainability of key infrastructure). The PDO remains consistent with the latest Country Partnership Framework (CPF) for FY19 to FY23. The PDO is consistent with the CPF's Strategic Objective 3 of enhanced connectivity and access to reliable infrastructure (transport, energy, digital). The CPF noted (para. 24) that the average interruption frequency (one of the indicators of electricity supply reliability) was 2.5 times higher than for comparator utilities, thus impairing the country's competitiveness. The PDO of increasing reliability of electricity supply is fully aligned with this priority. Under the GoA sector strategy and the current CPF, the focus has shifted towards renewable energy and strengthening of transmission and distribution networks. However, achieving these objectives would require achievement of adequate and reliable electricity supply as targeted under the Program's PDO which continues to remain relevant and aligned with the current priorities. Relevance of the PDO: Given the country and sector context, the PDO was relevant at appraisal, and continues to be relevant in regard to the currently prevailing national and CPF priorities. It was pitched at the right level to address the selected sector-related priorities. Rationale for PfR Support: The Program was designed to support the GoA's Financial Recovery of the Power Sector Program approved in February 2016. Through providing a World Bank loan of US$30 million (out of the total program cost of US$43.2 million), the PfR enabled the Bank to support and incentivize key aspects of the GoA program. The PfR instrument was selected (over other alternatives including a Development Policy Loan or investment operation) since it was assessed to better motivate the GoA to undertake reforms underpinning the DLIs through realistic actions largely under its control. The PfR supported the GoA's program by providing financial incentives and support for eight out of the ten financial recovery measures under the GoA's program. The support included a US$300,000 World Bank executed grant from ESMAP (Energy Sector Management Assistance Program) for capacity building and advisory support for key activities supported under the Program. This included assistance to the PSRC (Power Sector Regulatory Commission) for improving its tariff-setting methodology. The PfR-supported improvements in the financial condition of the sector were expected to also help improve the sustainability and viability of other parallel World Bank supported operations for improving the power transmission networks in Armenia. Prior Bank experience in the sector: Prior to the PfR Project, the Bank had supported transmission-sector related interventions in Armenia through the Electricity Supply Reliability Project (P116748) and its Additional Financing and the Electricity Transmission Network Project (P146199). The PfR operation itself was supported by tariff-related triggers and prior actions in the Third Development Policy Financing Operation (P153234) approved in 2015. Page 5 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Rating Relevance TBL Rating High b. Relevance of DLIs New DLI RELEVANCE TBL DLI 1 DLI Decision at the shareholders' meetings of ANPP and YTPC prohibiting non-core business related expenditures, lending and borrowing by the above companies. (Indicative loan allocation US$2.832 million or 9.5% of the loan amount). Rationale This DLI was linked to Results Area 1. Deterioration in the financial performance of the two state-owned power generation companies (ANPP and YTPC) was assessed to be impacting their ability to maintain adequate levels of power generation. The companies' non-core business related expenditures, lending, and borrowing were assessed to be among the principal contributors to their deteriorating financial performance. Decisions by the respective shareholders to prohibit such non-core activities were to be the first step towards improving the financial performance of the two companies. Rating High DLI 2 DLI Liquidation of Haigasard. (Indicative loan allocation US$2 million or 6.7% of the loan amount). Rationale This DLI was linked to Results Area 1. Haigasard was the entity used by the GoA to channel the non-core activities of the two power generation companies. With the planned prohibition of non-core activities, there was no future role for Haigasard. Over time, it had built up accumulated receivables and payables. Liquidation was essential to settle accounts. The DLI was supported by the following Disbursement Linked Results (DLRs): DLR 2.1: Decision taken to liquidate the company. DLR 2.2: Revaluation of assets and liabilities is completed. Page 6 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) DLR 2.3: Liquidation is registered by the authorized state agency. Completion of the targeted liquidation of Haigasard was delayed due to protracted court proceedings. Under a restructuring in June 2021, the target for DLR 2.3 was revised to "first public tenders to sell Haigasard's assets have closed". Rating Substantial DLI 3 DLI No new non-core business related expenditures, borrowing and lending incurred by ANPP. (Indicative loan alloaction U$2.5 million or 8.4% of the loan amount). Rationale This DLI was linked to Results Area 1. As discussed above, elimination of non-core related expenditures and liabilities was assessed to be a key requirement for improving ANPP's financial performance. Progress under the DLI was monitored at six-month intervals in the period January 2017 to December 2020. Rating High DLI 4 DLI No new non-core related expenditures, borrowing and lending to be incurred by YTPC. (indicative loan allocation US$2.5 million or 8.4% of the loan amount). Rationale This DLI was linked to Results Area 1. As discussed above, elimination of non-core related expenditures and liabilities was assessed to be a key requirement for improving YTPC's financial performance. Progress under the DLI was monitored at six-month intervals in the period January 2017 to December 2020. Rating High Page 7 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) DLI 5 DLI Reduction in the principal amount of outstanding commercial loans of ANPP. (indicative loan allocation US$4.26 million or 14.2% of the loan amount). Rationale This DLI was linked to Results Area 2. ANPP had accumulated a high amount of commercial loans at relatively high interest rates, resulting in high interest liabilities and payments that reduced the funds available for efficient operations. Reduction of commercial loans and their replacement by concessional finance from the Bank loan was assessed to be an essential requirement for improvement of ANPP's financial performance. Progress under the DLI was monitored at six-month intervals in the period July 2016 to June 2017. Rating High DLI 6 DLI Reduction in the principal amount of outstanding commercial loans of YTPC. (Indicative loan allocation US$2.3 million or 7.8% of the loan amount). Rationale This DLI was linked to Results Area 2. YTPC had accumulated a high amount of commercial loans at relatively high interest rates, resulting in high interest liabilities and payments which reduced the funds available for efficient operations.. Reduction of commercial loans and their replacement by concessional finance from the Bank loan was assessed to be an essential requirement for improvement of YTPC's financial performance. Progress under the DLI was monitored at six-month intervals in the period July 2016 to June 2017. Rating High DLI 7 DLI Reduction of YTPC's payables for natural gas. (Indicative loan allocation US$4.982 million or 16.6% of the loan amount). Rationale Page 8 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) This DLI was linked to Results Area 2. Over time YTPC had built up an accumulated backlog of payables for natural gas. Reduction of these accumulated payables was assessed to be a key requirement for improvement in YTPC's financial performance. Progress under the DLI was monitored at six-month intervals between May 2016 and December 2020. Rating High DLI 8 DLI ENA's margin for each new tariff period fully reflects the loss of revenue accrued due to the difference between the forecast and actual cost of electricity purchased by ENA during the preceding tariff period. (indicative loan allocation Us$2 million or 6.7% of the loan amount). Rationale This DLI was linked to Results Area 3. The existing tariff adjustment procedures did not provide for timely and adequate compensation being provided to ENA to cover lost revenues due a mismatch between forecast and actual costs of electricity purchases. This needed to be corrected under the PfR program. Progress under the DLI was monitored at six-month intervals between July 2016 and December 2020. Rating High DLI 9 DLI Tariffs for YTPC and Hrazdan TPP (Thermal Power Plant) for each new tariff period fully reflect the loss of revenue resulting from the difference between the forecast and actual cost of natural gas purchases due to exchange rate fluctuations between the AMD and the US$ during the preceding tariff period (Indicative loan allocation US$2 million or 6.7% of the loan amount). Rationale This DLI was linked to Results Area 3. The existing tariff adjustment procedures did not provide for timely and adequate compensation being provided to YTPC and the Hrazdan TPP to cover lost revenues due a mismatch between forecast and actual costs of natural gas purchases caused by exchange rate fluctuations during the preceding tariff period. This needed to be corrected under the PfR program. Progress under the DLI was monitored at six-month intervals in the period July 2016 to December 2020. Rating Page 9 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) High DLI 10 DLI Electricity to be supplied by YTPC, (Indicative loan allocation US$4.55 million or 15.1% of the loan amount). Rationale This DLI was linked to Results Area 4. Adequate power generation by YTPC was important for the Armenian economy. YTPC, along with ANPP, generated about 42 percent of the power in Armenia, and was among the lower cost producers. Technical assessments had determined that, to maintain the generation capacity of YTPC, investments were needed to replace some parts in the turbines and compressors at the Combined Cycle Gas Turbine (CCGT) plant in YTPC. The Program was to provide financing for the required investments. The DLI was to be used to assess the maintenance of power generation capacity at the YTPC plant. In regard to maintaining adequate generation of power, it was assumed that ANPP, with the help of the additional financing provided under the Program, would be able to maintain its generation capacity. No specific physical interventions were targeted for ANPP under the Program. Progress under the DLI was monitored annually in the period 2016 to 2020. Rating Substantial OVERALL RELEVANCE DLI TBL OVERALL RELEVANCE RATING Rationale The relevance of objectives is rated High as they were aligned with, and continue to be aligned with, the national-level and Country Partnership Strategy/Framework priorities at the time of appraisal and at present. The relevance of the DLIs is rated High because they individually, and collectively, helped to achieve the PDO (adequate and reliable power supply) by improving the financial performance of the two state-owned power generation companies (ANPP and YTPC) and the private power distribution company (ENA).  DLIs 1, 2, 3, 4, and 9 contributed to improving the financial performance of ANPP and YTPC by (i) elimination of new non-core business activities and (ii) improvements in the tariff methodology. This helped increase the funds available to the companies for ensuring adequate power generation, a part of the PDO.  DLIs 5, 6 and 7 contributed to improving the financial performance of ANPP and YTPC by helping to reduce the amount of expensive commercial loans and the companies' payables to the natural gas supplier company., thereby increasing funds available to the companies for improving operations.  DLI 8 contributed to improving the financial performance of ENA by supporting improvements in tariff methodology which prevented loss of revenue to ENA through mismatches between actual and forecasts of power purchase. This increased the revenues available to ENA for providing reliable power supply, a part of the PDO. Page 10 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571)  DLI 10 focused on ensuring that generation capacity at YTPC was maintained. This was essential for ensuring adequate power supply, a part of the PDO. Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To maintain adequate electricity supply through improving the financial standing of the two state-owned power generation companies and the private power distribution company. Rationale Both national level and WBG priorities recognized the importance of adequate and reliable power supply for the Armenian economy and population. Financial issues were seriously constraining the operations of (i) the two state-owned power generation companies (ANPP and YTPC), together accounting for about 42% of the total power generation, and (ii) the private power distribution company (ENA), which was the single buyer and sole distribution company, to provide reliable electricity supply. The Theory of Change (TOC) was that the Program would provide inputs through technical assistance and financing to help the Ministry of Energy and Natural Resources (MENR) the targeted companies (ANPP, YTPC and ENA), the Power Sector Regulatory Commission (PSRC) , and the Settlement Centre to carry out activities that would result in outputs including: (i) elimination of non-core business activities in ANPP and YTPC; (ii) reduction in commercial loan amounts in ANPP and YTPC; (iii) reduction in YTPC's gas payables; (iv) improvements in the tariff methodology to enable ANPP, YTPC and ENA to more fully cover revenue deficits resulting from differences between forecasted and actual costs; and (v) ensuring adequate supply of electricity from the YTPC CCGT (Combined Cycle Gas Turbine) plant. The intermediate results would be improvement in the financial standing of the companies including (i) reduction of cash deficits at ANPP and YTPC; (ii) reduction in revenues lost in ENA due to mismatches between forecasted and actual costs; and (iii) reduction in revenues lost in the thermal power plants (TPPs) due to exchange rate fluctuations. The outcomes would be (i) total annual electricity supply by ANPP is maintained; (ii) total annual electricity supply by YTPC is maintained; and (iii) average interruption frequency is maintained. The combined outcome would be provision of adequate and reliable electricity supply. Key assumptions underlying the TOC and the Results Chain were that: (i) the targeted companies (ANPP, YTPC and ENA) have the operational and management capacity to maintain efficient operations; (ii) the companies would direct the cost savings and/or additional revenues from the Program interventions to improving and maintaining efficient operations; (iii) the GoA and PSRC would continue to follow a cost Page 11 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) recovery policy for the sector through the improved tariff methodology; and (iv) the GoA would retain its overall commitment to follow results-based approaches. Outputs and Intermediate Results: (based on ICR paras. 36 to 43 and Annex 1 - Results Framework). The outputs and intermediate results for Objective 1 fall under the four results areas under the Program: Results Area 1: Elimination of non-core cash outflows of state-owned power generation companies. Results Area 2: Rationalizing of the current assets and liabilities of the power generating companies. Results Area 3: Setting of tariffs reflecting changes in the cost of electricity supply. Results Area 4: Maintaining the generation capacity of the gas turbine at YTPC's CCGT plant. Intermediate Results Indicators (IRIs) (some of which were also DLIs as indicated below): The IRIs related to actions taken to improve the financial standing of ANPP and YTPC which was assessed to be essential for achieving the PDO of maintaining adequacy of electricity supply. The targets for all the IRIs were achieved as indicated below. IRI 1 and 2: (DLI 3 and DLI 4): No new non-core business related business expenditures, borrowing and lending:  ANPP (baseline - no; actual - yes, by 2017; target achieved).  YTPC (baseline - no; actual - yes, by 2017; target achieved). IRI 3: (DLI 2): Liquidation of Haigasard: The original target was "Haigasard's liquidation is completed and registered by the authorized state agency". This could not be achieved by Program closing due to delays caused by protracted court proceedings. Under the June 2021 restructuring, the target was revised to "first public tenders to sell Haigasard's assets have closed" (baseline - not achieved; original target - not achieved; revised target - achieved). IRI 4 and 5: Reduction of cash deficits of ANPP and YTPC enabled by the Program.  ANPP (baseline - deficit of US$18.5 million; target - no deficit; actual - no deficit; target achieved).  YTPC (baseline - deficit of US$ 84.5 million; target - US$ 44.8 million; actual US$ 29.8 million; target achieved). IRI 6 and 7: (DLI 5 and 6): Reduction of the principal amount of outstanding commercial loans of ANPP and YTPC to be refinanced under the Program.  ANPP (baseline - 0; target - reduction by US$ 8.5 million; actual - reduction by US$ 8.9 million; target achieved).  YTPC (baseline - 0; target - reduction by US$ 4.6 million; actual - reduction by US$ 4.6 million; target achieved). Page 12 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) IRI 8 and 9: Recovery of ANPP's and YTPC's receivables from ENA.  ANPP (baseline 0; target - US$ 15 million; actual - US$ 15 million; target achieved).  YTPC (baseline - 0; target - US$ 25 million; actual - US$ 25 million; target achieved). IRI 10: Reduction of YTPC's gas payables (baseline - payables of US$ 12.16 million; target - US$ 3.86 million; actual - US$ 0 million; target achieved). IRI 12: Elimination of losses at YTPC due to exchange rate fluctuations not accounted for in tariff setting. (baseline - not eliminated; actual - eliminated; target achieved). IRI 13: Percent of tariff related questions and inquiries (gender disaggregated) responded to by the PSRC within stipulated service standards for response. (target - 100%; actual - 100%; target achieved). IRI 14: Overhaul of the gas turbine at YTPC's CCGT plant. (baseline - not completed; target - completed; actual - completed; target achieved). Achievement of outcomes: The Program's targets in regard to improvement of the financial standing of ANPP and YTPC were achieved as indicated above. The targeted outcome of maintaining an adequate supply of electricity was achieved. The outcome was to be measured by two PDO level indicators: PDOI 1: Total annual electricity supply by ANPP is maintained: (baseline - 2,300 GWh; target - 2,300 GWh; actual - 2,752 GWh; target achieved). PDOI 2: Total annual electricity supply by YTPC is maintained: (baseline - 1,400 GWh; target 1,400 GWh; actual - 1,706 GWh; target achieved). Rating of efficacy of Objective 1: Based on the achievement of the targets for the PDO indicators and for the IRIs, the efficacy of Objective 1 rated High. Rating High OBJECTIVE 2 Objective To maintain reliable electricity supply through improvement of the financial standing of the two state-owned power generation companies and the private power distribution company. Rationale The overall Theory of Change (TOC) was provided earlier under Objective 1. Specifically in regard to Objective 2, improvement of the financial performance of ENA was assessed to be essential for enabling ENA to maintain reliability of electricity supply. The Program was to provide inputs through technical assistance and financing to MENR, ENA and PSRC to carry out activities the outputs from which would include improvements in the tariff setting methodology in regard to distribution tariffs. The intermediate result would be reduction in revenues lost by ENA as a result of differences between forecasted and actual costs Page 13 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) not covered by the existing tariff methodology, thereby increasing the funds available to ENA for efficient operations. The outcome would be that reliability of electricity supply would be maintained as reflected in the levels of average interruption frequency in the Program area. Key assumptions underlying the TOC and the Results Chain were as provided earlier under Objective 1. An additional assumption was that the incentives provided under the Program would be sufficient to motivate ENA to maintain reliable electricity supply. Outputs and Intermediate Results: (based on ICR paras. 44 to 53 and Annex 1 - Results Framework). IRI 11: Elimination of loss at the private distribution company due to differences between forecasted and actual costs of electricity purchased (baseline - not eliminated; actual - eliminated; target achieved). IRI 13: Percentage of tariff-setting related questions and inquiries (gender disaggregated) responded to by PSRC within the stipulated service standards for response. (target -100%; actual - 100%; target achieved). Achievement of outcomes: The outcome was to be assessed on the basis of a single PDO indicator (PDOI 3) "average interruption frequency in the project area is maintained". However, there were flaws in the measurement and baseline for PDOI 3 as defined at appraisal (ICR para. 45). These had to be adjusted to correct errors in regard to the baseline definition and measurement methodology, and to enable a better comparison between pre- and post- Program reliability levels. The PAD had defined the PDOI as "total number of customer interruptions a year divided by the total number of customers in the project area". A search for the period data did not yield the figure corresponding to the baseline data (ICR para. 45). Instead, it was found that the baseline figure in the PAD (1.10 for 2013) was actually the quarterly figure for interruption data for 2013, not the annual figure. During Program implementation, the figures used were the average of quarterly figures for a full year as provided by the PSRC. Also, there was a flaw in the baseline figure in the PAD - the figure was for the year 2013 and not for 2016 (the year preceding start of implementation). After carrying out the necessary adjustments, it was determined that the more appropriate baseline figure is 1.71 (not 1.10 as provided in the PAD) (ICR para. 45). Based on the foregoing, achievement of targets under PDOI 3 are as follows: (original PAD baseline 1.10; original PAD target 1.10; corrected baseline - 1.71; corrected target - 1.71; actual 1.99; target not achieved). The average frequency of interruptions increased (reliability deteriorated) during the Program period from 1.71 in 2016 to 2.16 in 2018, before starting to improve to 2.01 in 2019 and 1.84 in 2020. However, in 2021, it deteriorated again to 1.99 (compared to 1.84 in 2020 (ICR para. 48). It is not clear from the ICR specifically what factors led to the deterioration in reliability. Rating of efficacy for Objective 2: In rating the efficacy of Objective 2, the ICRR has taken into consideration the views expressed in the ICR (paras. 50 and 51) that (i) the improvement in reliability levels in 2019 and 2020 can be attributed to ENA's investments and the impact of the Program, (ii) without the Program interventions, the reliability would have deteriorated further; and (iii) it would take more time for the investments by ENA to fully translate into improved reliability However, as discussed above, the original as well as the revised (corrected) targets for the relevant PDO indicator were substantially underachieved. Moreover, the flaws in the baseline data and measurement methodology reported in the ICR were not formally corrected during implementation and the revised targets were not a part of the project restructurings. Based on the foregoing, the efficacy of Objective 2 is rated Modest. Page 14 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Rating Modest OVERALL EFFICACY Rationale As discussed earlier in Section 4, (i) the efficacy of Objective 1 is rated High based on the achievement of targets for the relevant PDO indicators as well as Intermediate Results Indicators, and (ii) the efficacy of Objective 2 is rated Modest reflecting the substantial underachievement of the original as well as revised targets for the relevant PDO indicator. In addition, the Program substantially achieved the original and corrected Core Indicator targets of number of direct beneficiaries: (original baseline 3,018; original target 3,038; corrected baseline 2,926; revised target 2,926; actual 2,963; original target substantially achieved; revised target achieved). Number of female beneficiaries: (original baseline 1,618; corrected baseline 1,550; revised target 1,550; actual 1,569; revised target achieved). On balance, the Program's overall efficacy is rated Substantial. Institutional strengthening: Although it was not an explicit Program objective, strengthening of the GoA's institutional capacity to plan, monitor and implement the Program was an important part of the Program. Capacity building and advisory support were extended to MENR, ANPP, YTPC, ENA, and PSRC to help them plan and implement the Program. This included an ESMAP grant provided in conjunction with the Program to support: (i) good practice approaches to tariff setting methodologies; (ii) assessment of social impacts of tariff increases; (iii) development of mitigation strategies for impacts on vulnerable consumers; and (iv) strengthen consumer relations and communication strategies. Other areas supported under the Program included improvement of fiduciary and financial management; social assistance; prevention of fraud and corruption; citizen engagement, including gender focus. Rating Substantial 5. Outcome The Program's outcome is assessed based on the ratings for (i) relevance and (ii) efficacy. Page 15 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) As discussed in Section 3, relevance was rated High based on ratings of High for relevance of objectives and High for relevance of DLIs. As discussed in Section 4, overall efficacy was rated Substantial based on ratings of High for Objective 1 and Modest for Objective 2. Based on the foregoing, the Program's outcome is rated Satisfactory. Outcome Rating Satisfactory 6. Risk to Development Outcome Technical risk: This risk is assessed as Moderate. The companies (ANPP, YTPC, and ENA) are well-run with adequate experience and capacity for technical management of operations. However, their capacity for continued efficient operations would depend upon the timely and sufficient availability of funds to be able to maintain and operate their facilities. This would be influenced by the GOA's tariff setting policies for the electricity sector. While the GoA is currently following policies for full cost recovery tariffs, these would need to be sustained over time in the face of changing economic and social environments. Financial risk: This risk is assessed as Moderate. The GoA is committed at present to improving the financial viability of the power sector utilities, including setting of cost recovery tariffs. While these policies are expected to continue in future, social and political considerations, including the possible impact of external factors (e.g. international natural gas and oil prices) could influence timely setting of tariffs at cost recovery levels, and result in periods of insufficient cost coverage, Institutional risk: This risk is rated Moderate. While the Program has contributed substantially to institutional strengthening, including in MENR and PSRC, sustainability of the gains will require the continued commitment of the agencies to implement the improved methodologies and procedures. The ICR reports (para. 104) that the GoA (i) is embarking on a major electricity sector reform program starting February 2022 which is expected to further strengthen the commercial aspects of electricity sector operations and (ii) has put in place a technically sound strategy for the long-term development of the power sector in the form of an Energy Sector Development Strategic Program for 2040. The risks discussed above could be mitigated depending upon successful execution of the planned strategy and program. 7. Assessment of Bank Performance a. Quality-at-Entry Page 16 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) (Reference ICR paras. 92 to 102). For the World Bank, the Program was the first Bank-supported PfR operation in the power sector in Armenia. Nevertheless, the Bank team was able to draw on the experience gained under previous and ongoing Bank-financed projects in the sector. The strategic relevance and approach were generally sound and the Program was well-aligned with the national and WBG CPF priorities (prevailing at appraisal as well as the latest). The Theory of Change and the Results Chain were adequately conceived. Technical, financial, and economic aspects were well-considered. The design took into account poverty, gender, and social aspects, including mitigation of impacts of tariff increases on the more vulnerable sections of the population. Assessments were carried out in regard to (i) the GoA program for financial recovery in the sector; (ii) fiduciary systems; and (iii) environmental and social systems. Due diligence included ensuring that the planned Program support to ANPP (a nuclear power generation company) was in compliance with existing WBG policies and guidelines. Policy and institutional aspects were well-considered, and relevant capacity building and advisory support were to be provided through a complementary ESMAP grant. The risk assessment was generally adequate and mitigation measures were identified. The DLIs and IRIs were relevant to achievement of the PDO, and appropriate arrangements and protocols were put in place for independent verification and validation of results. The M&E design was adequate overall. However, one shortcoming was that, as reported in the ICR (para. 45), there were flaws in the original baseline data and measurement methodology in regard to a key PDO indicator (average interruption frequency) related to reliability of electricity supply. Also, as acknowledged in the ICR (para. 95), although the license of the private power distribution company (ENA) was amended under the Program as part of the distribution reform, there was a missed opportunity in not requiring ENA to maintain the then prevailing reliability level as a performance requirement. The ICR acknowledges (para. 95) that such a requirement may have helped the Program to meet the PDO-level indicator target for reliability of electricity supply and thereby to augment the quality at entry of the Program. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The Bank supervision team focused on development impact and was proactively involved in the Program's implementation over the duration of the program. This included timely interactions as necessary with MENR, PSRC and the companies supported under the project to anticipate and resolve Program related issues. In the case of the private power distribution company (ENA), despite the lack of leverage with ENA on reliability issues, the team worked with MENR and PSRC to ensure that the required periodic tariff adjustments were made, thereby increasing the funds available to ENA for its operations. The team also followed up diligently on the progress being made in the institutional capacity building and advisory assistance provided through the complementary ESMAP grant. In regard to the targeted liquidation of Haigasard, the original target could not be achieved due to protracted court proceedings which were beyond the control of the team. Nevertheless, the team ensured that progress was being made by resetting the target to to ensure that the public tenders for sale of Haigasard's assets were issued by the Program's closing date. One area that was not adequately addressed during implementation was the correction of the baseline data and measurement methodology regarding the PDO relating to reliability of electricity supply. Any shortcomings in this regard could have been formally addressed through project Page 17 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) restructurings. The ICR (para. 79) acknowledges that it is unclear from the ISRs and other available information to determine what proactivity, if any, was pursued to try and correct this situation. The Bank team carried out a total of nine supervision missions over the Program's four-year implementation period. The team was well-staffed with environmental. social, and fiduciary specialists. Supplementary fiduciary related missions were carried out as required to review and ensure continuing adequacy of fiduciary, financial management, and procurement related actions relevant to evaluating achievement of the Program's results. The team benefited in terms of continuity in there being a single Task Team Leader (TTL) over the entire implementation period. The team's reporting was regular and candid with Implementation Status and Results Reports (ISRs) being filed in a timely manner. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 8. M&E Design, Implementation, & Utilization a. M&E Design (Reference ICR paras. 76 to 80) The PAD included a well-developed Theory of Change and Results Chain. The TOC was based on the assessment that an improvement in the financial standing of the two state owned power generation companies (ANPP and YTPC) and the private power distribution company (ENA) was essential for achievement of the PDO of maintaining adequate and reliable electricity supply. The DLIs and IRIs adopted in the Results Framework to monitor the Program's progress were mostly related to improvement of the financial standing of ANPP, YTPC and ENA. They were generally clearly defined and measurable. Given that there were multiple agencies and sources involved (MENR, PSRC, ANPP, YTPC, ENA, and Settlements Centre), the institutional responsibilities for data collection and reporting were well defined. The results verification and validation arrangements and protocols were well specified. The Independent Verification Agents (IVAs) included independent auditors, independent law firms, financial consultants, and the Settlements Centre. However, the ICR reports (para. 77) that there were some shortcomings in design in regard to the single PDO indicator relating to reliability of electricity supply: (i) the baseline data and measurement methodology at appraisal were flawed and needed to be corrected (discussed earlier in Section 4 on Achievement of Objectives); and (ii) as designed, the PDO indicator was not suitable for attribution of the reliability results to the Program outputs. Also, the PDO indicator related to assessing the responsiveness of the PSRC to inquiries from the public was not suitable for assigning attribution. Page 18 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) b. M&E Implementation During implementation, the M&E data were regularly collected and analyzed to assess the progress in achieving targeted results and the compliance of the GoA administrative agencies with agreed actions. The reported results were subject to verification and validation by independent auditors and reviewers under arrangements and protocols agreed with the Bank. One weakness that persisted through the implementation period was that the necessary corrections were not formally made (e.g. under restructurings) to correct the flaws in regard to the PDO indicator related to reliability of electricity supply. The ICR (para. 79) acknowledges that it is unclear from the ISRs and other available information to determine what proactivity, if any, was pursued to try and correct this situation. c. M&E Utilization The M&E data were regularly reviewed and utilized to assess implementation progress including evaluating off fulfilment of disbursement linked conditions, with the help of qualified and independent verification agents including auditors, law firms, financial consultants, and the Settlements Centre. Rating of M&E: Based on the foregoing, on balance, M&E quality is rated Substantial. M&E Quality Rating Substantial 9. Other Issues a. Safeguards (Reference ICR paras. 81 to 88). Environmental and Social Safeguards Environmental: At appraisal, the PAD did not specify whether any environmental and social safeguards policies were triggered. The ICR does not provide any information in this regard. The ICR reports (para. 81) that the environmental impacts of the Program were very limited as the Program focused primarily on the legal and regulatory framework and financial restructuring. In regard to the four Results Areas under the Program, the activities under Results Areas 1, 2 and 3 did not include any physical works and carry environmental implications. Under Results Area 4, replacement of key components in the CCGT at YTPC had associated physical activities with resulting environmental impacts. The ICR reports (para. 81) that, according to the national legislation, the works were subject to an Environmental Impact Assessment (EIA) and the YTPC received a positive conclusion on May 5, 2016. The EIA did not identify any hazardous waste resulting from the repair works. The ICR confirms that (i) the repair works were conducted following the health and safety guidelines applicable to such activities and (ii) no accident or incident occurred during the repair works. Page 19 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Social: The overall social impact of the Program was expected to be positive since maintaining adequate and reliable electricity supply would benefit all electricity consumers including vulnerable and low-income households. At appraisal, it was anticipated that the Program could result in two possible negative social impacts: (i) retrenchment of 40 employees of the liquidated Haigasard CJSC and (ii) the effect of possible tariff increases on vulnerable and low-income consumers. Retrenchment: The liquidation of Haigasard was done according to a Liquidation Plan which included a phased retrenchment of its employees over a period of several years. The retrenchment was to be governed by the Armenian Labor Code 0f 2004 and monitored by the State Labor Inspectorate. The ICR reports (para. 85) that, while the Haigasard retrenchment process was not formally subject to the World Bank safeguard policies, it did comply with the IFC's (International Finance Corporation) Performance Standards on labor and working conditions which requires: (i) an analysis of alternatives to retrenchments to be conducted; (ii) non-discrimination in retrenchment; (iii) consultation with workers and their organizations in planning retrenchments; and (iv) compliance with local laws and collective bargaining agreements. Social impact of tariff increases: The impacts of required tariff increases on vulnerable and lower-income sections of the population were being mitigated by the GoA under the GoA's existing Family Benefit Program which is a means-tested unconditional transfer mechanism to mitigate the impacts of increased energy prices on consumers. The Program provided technical assistance to the PSRC (Power Sector Regulatory Commission) for further strengthening of the tariff methodology, including mitigating impacts of tariff increases on vulnerable and lower-income sections of the population. b. Fiduciary Compliance (Reference ICR paras. 89 to 97). Fiduciary assessments conducted during Program preparation had concluded that the overall fiduciary and governance framework was adequate to support the implementation of the Program (ICR para. 89). The assessments found that the Program's fiduciary systems and institutions provided reasonable assurance that the financing under the Program was likely to be used for the intended purposes with due regard to the principles of economy, efficiency, effectiveness, transparency and accountability. Procurement: Country systems were to be used for procurement. The country procurement framework was assessed and found to be conducive to competitive, economic, efficient and transparent procurement. However, the Program risk assessment was found to be substantial due to the heavy reliance on direct contracting. (ICR para. 90). The ICR does not provide any information regarding how procurement was carried out, and, if there were any issues during procurement, how they were handled. The ICR does not report the rating for Procurement in the last ISR filed before project closing. Financial Management: The ICR confirms (para. 91) that the FM arrangements under the Program, including planning and budgeting; financial reporting; accounting; internal controls; flow-of-funds; external audits, continued to be adequate and acceptable to the World Bank. Required audit reports and FM reports were received generally in time with no significant areas flagged by the auditors for attention. The rating for FM in the last ISR filed before project closing was Moderate. Page 20 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) c. Unintended impacts (Positive or Negative) The ICR does not report any significant unintended impacts. d. Other Gender: The Program was not gender-tagged. The Program recognized, however, that women, in particular, disproportionately carry the responsibility for household work, and are thus more vulnerable to the impacts of unreliable and costly electricity supply. The citizens' engagement measures introduced under the Program especially encouraged questions and inquiries from women by way of feedback on the tariff setting processes, and the responses received were monitored. The customer database included in the Results Framework used data disaggregated by gender. Poverty Reduction and Shared Prosperity: The overall social impact of the Program has been positive as it helped to prevent shortages in supply of electricity through the financial strengthening of the supported companies. Maintaining adequate power supply through ANPP and YTPC helped avoid possible substantial increases in distribution tariffs that may have occurred otherwise if the shortages were sought to be met through higher cost alternatives (e.g. the Hrazdan TPP) for power generation. Such higher tariffs would have disproportionately affected the vulnerable and lower-income sections of the population. The negative social impacts of the Program, including the retrenchment at Haigasard and the impact of tariff increase for ENA, were being mitigated by the GoA under the GoA's existing Family Benefit Program which is a means- tested unconditional transfer mechanism to mitigate the impacts of increased energy prices on consumers. 10. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory While IEG rates the Quality of Supervision as Satisfactory, it rates the Quality at Entry as Moderately Satisfactory due to Bank Performance Satisfactory Moderately Satisfactory weaknesses in design of the Results Framework. Under IEG guidelines, this results in a rating of Moderately Satisfactory for Bank performance. Quality of M&E Substantial Substantial Quality of ICR --- Substantial 11. Lessons Page 21 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) The ICR (paras. 101 to 106) lists a number of lessons derived from the experience during Program implementation from among which IEG derives the following lessons relevant to Programs undertaken in similar environments. The PfR instrument can be a suitable choice for tackling the long-term, structural nature of challenges leading to poor financial performance in the energy and infrastructure sectors. The Program was among the first of several World bank operations utilizing the PfR instrument to support the financial recovery of infrastructure sector utilities. Financial distress is common in client country power sectors and not just in state-owned enterprises. The successful use of the PfR instrument under the Program showed how a smaller operation targeting financial recovery actions can generate substantial reform impact through a granular and well-sequenced DLI approach. Success of PfR operations requires stakeholders' buy-in and commitment through a combination of actions. These include deep analytical engagement at the design stage; strong Government commitment to the Program objectives; realistic theory of change with an achievable timetable for the DLIs; and DLIs, including those related to the private sector, that are within the control of the Government and/or implementing agencies. Continuity in oversight and support from a stable supervision team - in this case over a six-year period - was a major factor in ensuring he Program's success. Standard electricity reliability indicators may not be the most appropriate measures of project outcomes. The Program relied on system-average interruption frequency (SAIFI) as a PDO indicator even though this indicator takes time to show any improvement; is relatively to measure adequately; and can be affected by many external factors outside the Program's control (e.g. weather variations). Future programs aiming to improve reliability should also consider using indicators that can be more immediately linked to Program results. While private participation in infrastructure provision is desirable on considerations of fiscal prudence, operational efficiency, and quality of service provision, care needs to be taken to craft the right set of incentives for balancing public and private interests. In the case of the Program, while the private power distribution company benefited financially from the incentives provided under the Program, the lack of a clear incentive and penalty structure in its license undermined the ability of the GoA to ensure that the PDO indicator target for reliability was met. Programs that include the closure of state-owned companies should include provisions for addressing labor issues, including retrenchment, that may occur as a direct and indirect consequence of the World Bank financed intervention. While PfRs rely on national systems and laws, any PfR that includes measures resulting in retrenchment should review the appropriateness of the relevant provisions in the national systems and ensure their augmentation, if needed, in line with the principles in the Bank's Environmental and Social Framework (ESF). 12. Assessment Recommended? Yes ASSESSMENT_TABLE Page 22 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Power Sector Financial Recovery Program (P157571) Please Explain The Program was the Bank's first PfR operation in the power sector in Armenia. It addressed issues related to power generation and distribution. The Bank has also financed, and is financing, projects to improve power transmission systems in Armenia. An assessment of the PfR, in conjunction with assessments of the transmission projects, would enable a holistic assessment of the impact of Bank-supported power sector interventions in Armenia. 13. Comments on Quality of ICR The ICR is well-written, clear, candid, and generally complies with the relevant OPCS guidelines (except in regard to length - 37pages compared to the recommended length of 15 pages or less). The ICR presents an adequate theory of change with a diagrammatic presentation of the results chain from activities to outcomes. The analysis and conclusions are generally supported by evidence but constrained to some extent by weaknesses in the Results Framework and the M&E system e.g. the key PDO indicator related to reliability of electricity supply. The ICR includes a number of lessons derived from the experience in the Program's implementation that are relevant for similar PfR operations undertaken in comparable environments. a. Quality of ICR Rating Substantial Page 23 of 23