Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004486 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-H8660) ON A CREDIT IN THE AMOUNT OF SDR 3.40 MILLION (US$5 MILLION EQUIVALENT) TO THE UNION OF THE COMOROS FOR THE ELECTRICITY SECTOR RECOVERY PROJECT February 4, 2019 Energy and Extractives Global Practice Africa Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective January 31, 2019) Currency Unit = Comorian Franc (KMF) KMF 428.2635473 = US$1 US$1 = SDR 1.400700 FISCAL YEAR July 1–June 30 ABBREVIATIONS AND ACRONYMS AfDB African Development Bank AM Aide-Mémoire CFO Chief Financial Officer CPS Country Partnership Strategy DGEME Direction Générale de l'Energie, des Mines et des Ressources en Eau (Directorate General for Energy, Mines and Water Resources Ministry of Energy) EDA Electricité d’Anjouan (Power Utility in Anjouan) EIRR Economic Internal Rate of Return FCV Fragility, Conflict, and Violence GDP Gross Domestic Product GoC Government of Comoros HFO Heavy Fuel Oil HIPC Heavily Indebted Poor Countries ICR Implementation Completion and Results Report IMF International Monetary Fund ISR Implementation Status and Results Report IT Information Technology KPI Key Performance Indicator LV Low Voltage M&E Monitoring and Evaluation MA-MWE Gestion de l’Eau et de l’Electricité aux Comores (Water and Electricity Power Utility in Comoros) MD Managing Director MIS Management Information System MoE Ministry of Energy MV Medium Voltage NPV Net Present Value O&M Operation and Maintenance PAD Project Appraisal Document PASEC Energy Sector Support Project PDO Project Development Objective PIU Project Implementation Unit PPF Project Preparation Fund PSIA Poverty and Social Impact Analysis RDP Recovery and Development Plan SC Steering Committee SCH Société Comorienne des Hydrocarbures (Petroleum Company in Comoros) SOE State-owned Enterprise YIF Youth Innovation Fund Regional Vice President: Hafez M. H. Ghanem Country Director: Mark R. Lundell Senior Global Practice Director: Riccardo Puliti Practice Manager: Sudeshna Ghosh Banerjee Task Team Leader(s): Jan Friedrich Kappen ICR Main Contributor: Paulette Castel TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL ............................................................................................................... 5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION.................................................................. 10 OUTCOME .................................................................................................................... 12 A. RELEVANCE OF THE PDOS ........................................................................................................... 12 B. ACHIEVEMENT OF PDO (EFFICACY) ............................................................................................. 12 C. EFFICIENCY .................................................................................................................................. 17 D. JUSTIFICATION OF OVERALL OUTCOME RATING ........................................................................ 18 E. OTHER OUTCOMES AND IMPACTS.............................................................................................. 19 KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION......................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ................................................................................. 22 BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ................................................................ 24 B. ENVIRONMENTAL, SOCIAL AND FIDUCIARY COMPLIANCE ......................................................... 25 C. BANK PERFORMANCE ................................................................................................................. 26 D. RISK TO DEVELOPMENT OUTCOME ............................................................................................ 27 LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 36 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 38 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 39 The World Bank KM — Electricity Sector Recovery Project (P131659) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P131659 KM-Electricity Sect. Recovery Proj(FY14) Country Financing Instrument Comoros Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency The Union of Comoros MA-MWE Project Development Objective (PDO) Original PDO The Project Development Objective is to contribute to the improvement in the electricity sector's commercial and financial performance. Page 1 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 5,000,000 5,000,000 4,495,613 IDA-H8660 Total 5,000,000 5,000,000 4,495,613 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 5,000,000 5,000,000 4,495,613 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 06-Sep-2013 13-Mar-2014 27-Oct-2015 30-Apr-2017 30-Apr-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 02-Mar-2017 2.54 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial Page 2 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 11-Feb-2014 Satisfactory Satisfactory 0 02 23-Aug-2014 Satisfactory Satisfactory .53 03 02-Apr-2015 Satisfactory Satisfactory .61 04 29-Nov-2015 Satisfactory Moderately Satisfactory 1.04 05 09-Jun-2016 Satisfactory Moderately Satisfactory 1.34 06 29-Dec-2016 Moderately Satisfactory Moderately Satisfactory 2.19 Moderately 07 17-Jun-2017 Moderately Unsatisfactory 2.54 Unsatisfactory 08 15-Jan-2018 Moderately Satisfactory Moderately Satisfactory 3.71 SECTORS AND THEMES Sectors Major Sector/Sector (%) Energy and Extractives 100 Public Administration - Energy and Extractives 30 Other Energy and Extractives 70 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 26 Business Enabling Environment 26 Investment and Business Climate 13 Regulation and Competition Policy 13 Public Sector Management 74 Public Administration 74 State-owned Enterprise Reform and 74 Privatization Page 3 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Haleh Z. Bridi Mark R. Lundell Senior Global Practice Director: Jamal Saghir Riccardo Puliti Practice Manager: Lucio Monari Sudeshna Ghosh Banerjee Task Team Leader(s): Mustafa Zakir Hussain Jan Friedrich Kappen ICR Contributing Author: Paulette Castel Page 4 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. Context at Appraisal Context 1. The Comoros was in 2013—and remains today—a fragile country with considerable development challenges, although the country’s poverty is relatively low compared to its regional peers. The Comorian economy was and remains small, undiversified, and constrained by poor quality of public services including in electricity. In 2014, around 18 percent of the population lived below the international poverty line of US$1.90 per capita per day (2011 purchasing power parity exchange rate). The Comoros also features high electricity access rates—at above 601 percent—compared to a regional average of 43 percent in Sub-Saharan Africa. However, in the three decades since its independence in 1975, the Comoros has experienced 20 coups or attempted coups and frequent changes in government. A strong separatist movement in Anjouan contributed to political fragility. Institutional capacity was and continues to be weak, hampering the Government’s ability to implement development projects and deliver basic services. The Comoros’ gross domestic product (GDP) per capita was US$645 in 2015, and the country ranked 159 out of 188 countries on the 2015 Human Development Index. 2. At appraisal in 2013, the country had just emerged out of a program of international debt relief and, therefore, the commercial and financial performance of its state-owned enterprises (SOEs) was a high priority for the Government and international financial institutions. At appraisal, the Comoros was on a moderate growth path, emerging from a long period of economic and institutional instability; however, growth was constrained by a severe crisis in the electricity sector. After more than a decade of elevated tensions, a negotiated resolution between the separatist island of Anjouan and the Union was reached in 2001, followed by the popular approval of a constitutional revision in May 2009. With gradual political normalization during 2009–13, growing remittances from the Comorian diaspora (close to 20 percent of GDP), and the support of the International Monetary Fund (IMF)-World Bank Joint Initiative for Heavily Indebted Poor Countries (HIPC), economic growth reached an eight-year high of 3.5 percent in 2013 and was expected to continue. However, growth was constrained by extremely unreliable and expensive electricity services. The two state-owned electricity providers, Water and Electricity Power Utility in Comoros (Gestion de l’Eau et de l’Electricité aux Comores, MA-MWE, providing electricity to the islands of Grande Comore and Mohéli) and Power Utility in Anjouan (Electricité d’Anjouan, EDA, serving only the island of Anjouan), were unable to supply uninterrupted electricity services for more than a few hours per day. As a result, most Comorian households were dependent on kerosene and candles for lighting and commercial charging stations for mobile phone charging and had to spend a significant part of their income on disposable batteries to power radios and other small appliances. Health centers could not guarantee quality medical care; water was often unreliable due to power cuts (even in the capital); and businesses requiring continuous power supply remained uncompetitive because unreliable supply 1Population density has helped the Government of Comoros’ (GoC) past electrification efforts and at appraisal electricity access was relatively high compared to much of the Sub-Saharan African countries. At appraisal, the access rate in Grande Comore was estimated at 65 percent, with Anjouan ranging at around 50 percent, and the more thinly populated Mohéli lagging with only about 20 percent of the population connected to the grid. Page 5 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) despite electricity tariffs, at over US¢20 per kWh, were significantly above the regional median of US¢15 per kWh.2 3. Poor service delivery and high cost of electricity were the result of highly limited institutional capacity and decades of fragility that impeded investment in the sector: (a) High cost of electricity supply. Due to a combination of high system losses, poor collection rates, poor governance, and costly generation, the islands’ cost of electricity service—at an estimated US¢ 61 per kWh—were the second highest in Sub-Saharan Africa,3 imposing a heavy burden on both the economy and the government budget, 10 percent of which was spent on the utility’s fuel subsidies at project appraisal. (b) Poor commercial performance of the SOEs, MA-MWE, and EDA. At project appraisal, the utility had a billing rate of 55 percent of power generated and a collection rate of only 58 percent of electricity billed, meaning that over two-thirds of electricity generation remained unpaid. The resulting severe financial distress, including the steady decline of cash flow available for fuel purchases and basic operation and maintenance (O&M), created a vicious cycle as the resulting deterioration of service quality with increasingly long and frequent power cuts further eroded collection rates with disgruntled customers becoming ever more reluctant to pay their electricity bills. (c) Poor governance and lack of sector planning. As a result of weak sector planning and project implementation as well as the absence of a coherent legal and regulatory framework for private sector participation, the GoC had failed to make significant advances in reducing the cost and share of thermal power production in the country’s energy mix. (d) Dependence on expensive, small-scale diesel generators and inadequate diesel supply arrangements. In the absence of appropriately sized and lower maintenance units to cover baseload, the Comoros’ variable O&M cost for power generation exceeded US¢ 40 per kWh, the highest in Sub-Saharan Africa. In addition, the country had to buy fuels in small quantities and lacked adequate storage facilities and reliable logistics. (e) Poor condition of electricity transmission and distribution infrastructure. In addition to outages resulting from the utility’s maintenance-intensive generation infrastructure, the power system experienced high levels of outages and service interruptions from a saturated and poorly maintained transmission and distribution network. 4. As the World Bank’s first lending engagement in the Comoros’ energy sector, the project represented a priority in the World Bank’s overall engagement given the macroeconomic significance of the power sector’s financial woes and constraints on growth. The project was designed to support the GoC’s strategic objectives of improving the reliability of electricity supply and, at the same time, of solving the problem of the unsustainable financial burden that the sector had on public finances. The Comorian authorities had recognized that the acute power supply situation was the result of four interlinked factors: 2 Trimble, Chris, Masami Kojima, Ines Perez Arroyo, and Farah Mohammadzadeh. 2016. “Financial Viability of Electricity Sectors in Sub-Saharan Africa: Quasi-Fiscal Deficits and Hidden Costs.” Policy Research Working Paper 7788. World Bank, Washington, DC. https://openknowledge.worldbank.org/handle/10986/24869. License: CC BY 3.0 IGO. 3 Ibid. Page 6 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) poor billing and collection practices, an inefficient fuel supply chain, and the inability to carry out the necessary periodic maintenance of generators. Without addressing these factors, sustained economic recovery would be impossible. The operation was a key component of a broader package of the World Bank’s assistance associated with the Comoros Second Development Policy Grant (P122941) approved in October 2012 in support of reforms structured around two broad policy pillars: (a) fostering public transparency and accountability, civil service management, anti-corruption, and fisheries and (b) addressing economic and social vulnerability emanating from weak performance in the energy sector and weak natural disaster risk management. 5. Improved commercial and financial performance of the electricity SOEs was high on the GoC’s agenda at the time of appraisal and the Government engaged multiple donors in the sector to assist with this agenda. With the assistance of a World Bank energy sector diagnostic prepared in 2011–12, the GoC developed its strategy for the energy sector in the Sector Policy Note (Document de politique de l’énergie électrique et des produits pétroliers de l’Union des Comores) which was adopted in July 2012. The note highlighted the need for (a) comprehensive corporate governance reforms at MA-MWE; (b) investments in renewable generation including micro-hydro, wind power, and geothermal energy to reduce the reliance on thermal power generation; (c) reforms of the sector legal and regulatory framework including on tariffs and purchasing power parity; (d) upgrading of petroleum storage facilities to improve safety and security of supply. The African Development Bank (AfDB) took the lead role in the sector and was expected to finance the physical infrastructure, mostly through the purchase of new generators and rehabilitation of the grid. The World Bank was requested to lead on improving both utility and sector governance. The European Union was involved in the development of small-scale solar capacity in Mohéli and, together with the French Development Agency (Agence Française de Développement), was ready to fund any additional activities, as needed. Theory of Change (Results Chain) 6. The project was designed to contribute to the World Bank’s twin goals of poverty reduction and shared prosperity by targeting nontechnical losses and sector governance to improve the delivery of modern energy services and freeing up fiscal resources for spending on human development. It was also designed to restore an acceptable level of electricity provision in the shortest possible time frame. Based on the diagnostic of the sector and, given that investment in physical infrastructure was expected to be covered by financing from AfDB, the World Bank team decided to target both the key governance issues of the sector: (a) the utility’s massive revenue losses caused by electricity theft and unpaid bills as well as (b) fiscal strain from a lack of coherent long-term investment planning. Figure 1 summarizes the results chain linking the project’s activities to the objective of improving the commercial and financial performance of the electricity sector and expected short-, medium-, and long-term outcomes. 7. Investments under the project focused almost exclusively on MA-MWE—the utility that covers Grand Comore and Mohéli—because it was in a much worse financial and commercial state than EDA (which covers Anjouan). At the time of appraisal, EDA was smaller than MA-MWE and performed significantly better in terms of its commercial performance, with losses at appraisal of around 40 percent and collection rates above 90 percent (compared to 45 percent losses and 55 percent collection rate for MA-MWE) and was therefore not the main focus of the project. Page 7 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) Figure 1. Theory of Change of the Project Reduced poverty and shared prosperity Higher-level Objectives Reduced fiscal burden of electricity subsidies and freed-up Improved electricity service delivery fiscal resources for poverty alleviation and other social needs Improvement in the electricity sector's commercial and financial performance Outcomes Improved collection rate Reduced system losses Reached direct project Adopted new governance (MA-MWE) (MA-MWE) beneficiaries (MA-MWE) arrangements Adoption and implementation of new energy Targeted measures by MA-MWE and/or EDA to reduce non-payment, Outputs & Results policy, legal and regulatory framework, tariff electricity theft and technical system losses reforms, and improved planning capacity • Purchase and installation of management information • Design of an energy policy including review of sector system (MIS) and hardware governance arrangements • Purchase of and installation of network and metering • Drafting of a legal and regulatory framework Inputs/activities equipment • Development of planning capacity and provision of tools for • Poverty and social impact analysis and its implementation communications and awareness campaign • Study of energy subsidies and tariffs Project Development Objective (PDO) 8. The PDO was to contribute to the improvement in the electricity sector’s commercial and financial performance. Key Expected Outcomes and Results Indicators 9. The four original outcome indicators and the expected outcomes were as follows: • Results Indicator 1: ‘Collection rate of MA-MWE improved’ (from 55 to 70 percent), defined as cash collected as a percentage of billed electricity at MA-MWE • Results Indicator 2: ‘Total losses reduced’ (from 45 to 37.5 percent), defined as billed electricity as a percentage of produced electricity at MA-MWE • Results Indicator 3: ‘Adoption of new governance arrangements for the sector’, defined as the adoption of new utility governance arrangements and tariff and subsidy framework • Results Indicator 4: ‘Direct project beneficiaries’ (from 0 to 200,000), 50 percent of which female, defined as the (then) current and prospective household electricity consumers of MA-MWE who all benefit from improved availability and reliability of service Page 8 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) Components 10. The scope of the three original project components as approved by the Board is summarized in Table 1. Table 1. Project Component Scope and Financing (US$, millions) Original Revised No. Technical Scope Actual Total/IDA Costa Cost 1 Assistance to the electricity state-owned enterprises’ 3.50 4.15 3.91 commercial and financial performance recovery (covering mostly MA-MWE, with some support to EDA) 1a Commercial and financial technical assistance. Hiring of 1.60 — — external experts to act as senior managers and lead the commercial and financial restructuring of the utilities 1b Commercial and financial performance enhancement: 1.90 — — 1. Purchase and installation of management information system (MIS) and hardware at MA-MWE and EDA (US$1.2 million) 2. Purchase of and installation at MA-MWE of approximately 80 units of network metering equipment, 4,000 boxes to protect customer meters (during implementation changed to 4,000 prepaid meters), and fuel metering equipment for fuel tanks (US$0.4 million) 3. Poverty and social impact analysis (PSIA) to identify the potential impacts of the enforcement of payment of electricity mitigation measures (US$0.10 million) 4. Communications and awareness campaign to promote awareness and address negative practices both within MA-MWE and among its customers (US$0.20 million) 2 Electricity sector governance. Finance technical 0.90 0.1 0.04 assistance, capacity-building activities, and studies related to key aspects of sector governance: 1. Design and implementation of an energy policy including review of sector governance arrangements (US$0.20 million) 2. Drafting and implementation of a legal and regulatory framework (US$0.30 million) 3. Development of planning capacity and provision of tools for its implementation (US$0.20 million) 4. (a) Study of the energy subsidies in the sector and development of a new subsidy/tariff structure for electricity (US$0.10 million) 4. (b) Framework for price and tariff adjustments (US$0.10 million) 3 Project management. Support the overall coordination, 0.60 0.75 0.57 management, and monitoring functions of the project Page 9 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) Original Revised No. Technical Scope Actual Total/IDA Costa Cost carried out by the project implementation specialists Total 5.00 5.00 4.52 Note: a. The project included no counterpart financing, so the total project cost equals the IDA allocation. B. Significant Changes During Implementation 11. In 2016, the team identified additional funding needs to fully implement Component 1. Specifically, the resident experts hired by the project alerted the recipient that supplementary funds would be needed to cover cost overruns in Component 1 related to additional hardware (enterprise application servers) and software (multiyear extension of maintenance guarantee) needs. Moreover, the experts underlined that the project’s impact could be much enhanced if additional funds were available to (a) double the number of prepaid meters to be installed though the project and (b) establish a national energy project management team led by international experts to oversee and accelerate the stalled implementation of critical sector investments in sector infrastructure. Consequently, the team agreed with the Government that an additional financing would be sought from the World Bank to address these needs and enhance both likelihood and sustainability of PDO achievement. However, with the continuation of highly disruptive Government-imposed changes of MA-MWE’s senior management team, including in December 2016 (the simultaneous dismissal and replacement of all senior directors), the World Bank’s management decided in early 2017 that circumstances were not conducive for the proposed additional financing. Instead, the team was instructed to proceed with a simple restructuring including the reallocation of funds from Component 2 to accommodate the most pressing and PDO-critical additional funding needs for Component 1. 12. In line with the above guidance from both Country Management Unit and Global Practice (GP) management, the project underwent a level 2 restructuring on March 2, 2017. The restructuring changed the PDO-level results indicators and reallocated funds between components (details below). The PDO remained unchanged during project implementation. Revised PDOs and Outcome Targets 13. The PDO remained unchanged, but the outcome targets no longer included any changes related to the adoption and implementation of sector governance reforms because funds were reallocated from uncompleted activities under Component 2 to Components 1 and 3. Revised PDO Indicators 14. Two changes were made to the PDO-level results indicator. • Results Indicator 2 (‘Total losses reduced’). The restructuring added two breakdowns of the results indicator, adding detail regarding the (estimated) technical and commercial losses that together make up the total system losses. • Results Indicator 4 (‘Adoption of new governance arrangements for the sector’). Dropped because the GoC decided that the key measures associated with this project—the Page 10 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) development of a new legal and regulatory framework—were going to be undertaken under an AfDB-funded project. Revised Components 15. Funds were reallocated between the three components to account for a change in scope and the extension of the project closing date by one year from April 30, 2017 to April 30, 2018. • Revised Component 1 (US$4.15 million). Funds were reallocated from Component 2 to Component 1 to fully complete the implementation of the utilities new MIS (activity 1 under Component 1b in table 1) and extend the tenure of four internationally recruited senior managers supporting the commercial and financial operations of the utility (Component 1a in table 1). • Revised Component 2 (US$0.10 million). All activities not launched before restructuring were dropped (activities 1, 2, 3, and 4b, as specified in table 1). Only the study of the energy subsidies in the sector and development of a new subsidy/tariff structure for electricity remained (activity 4a in table 1). • Revised Component 3 (US$0.75 million). Funds were reallocated from Component 2 to Component 3 to cover the cost of extending the operations of the Project Implementation Unit (PIU) for 12 months in line with the closing date extension. Other Changes 16. The restructuring extended the project’s closing date by 12 months, from April 30, 2017 , to April 30, 2018. Rationale for Changes and Their Implication on the Original Theory of Change 17. The restructuring was critical for the achievement of the PDO and the efficient use of project resources. (a) Closing date extension. Frequent changes in the utility’s senior management team slowed the progress of key procurement processes. The contract award for the implementation of the MIS was consequently much delayed, having taken place only in June 2016. This date coincided with the third change of MA-MWE’s commercial director, who was unfamiliar with the proposed MIS and the challenges of its implementation. Extending the contracts of the senior managers providing technical assistance to the utility’s team was therefore deemed essential to bolster knowledge transfer and ensure the completion of vital utility reforms including MA-MWE’s new operational procedures, accounting and commercial management systems, as well as the implementation of information technology (IT) infrastructure critical to achieve the PDO-level results indicators. (b) Dropping of governance-related activities and associated PDO-level indicator. The Government opted for the project’s governance-related activities to be financed by AfDB. This narrowed the mechanisms through which the project could achieve the PDO (see Page 11 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) sections III and IV for details). The remaining activities were still able to achieve the PDO- level targets that most immediately measured MA-MWE’s commercial and financial recovery (Results Indicators 1 and 2 in table 2), as detailed in section III. OUTCOME A. Relevance of the PDOs Rating: High 18. The PDO was to contribute to the improvement in the electricity sector's commercial and financial performance. This PDO remained unchanged throughout the project’s lifetime. 19. Reducing the quasi-fiscal deficit from SOEs was a high priority for the GoC at the time of appraisal, as it had just emerged out of international debt relief. In 2013, the IMF and the World Bank decided to support comprehensive debt relief of US$176 million for the Comoros, after having declared in June 2010 that the Comoros had reached the decision point under the Enhanced HIPC Initiative, making the East African island nation eligible for comprehensive debt relief under the HIPC and the Multilateral Debt Relief Initiative.4 The strengthening of the commercial and operational management of the Comoros’ large SOEs, including MA-MWE, was part of the reform program tied to the support of the IMF-World Bank Joint Initiative for HIPC and therefore a high priority for the Government and development partners. 20. Strengthening service delivery in the power sector by means of improving the sector’s commercial and financial performance remains a top priority for the GoC and development partners at the time of the Implementation Completion and Results Report (ICR). The PDO remains consistent with the latest Country Partnership Strategy (CPS, FY2014–2017, extended to FY2019, Report No. 82054-KM).5 The CPS asserts that the acute power supply situation is due to several factors “all linked to the operational and technical performance of MA-MWE” (p. 23) and aimed to tackle the commercial and financial performance of MA-MWE under Pillar 2 (‘Shared growth and increased employment’), CPS Objective 9 (‘Improvement in the commercial and financial performance of the electricity sector’). The GoC Strategy for Accelerated Growth and Sustainable Development 2015–2019 (Stratégie de Croissance Accélérée et de Développement Durable) explains that the country is struggling to attract investors partly because of recurrent shortages of water and electricity. The relevance of the PDOs is therefore rated High. B. Achievement of PDO (Efficacy) Rating: Substantial/High against Pre-/Post-Restructuring Assessment of Achievement of Outcomes 21. The operation exceeded its objectives and overachieved on all three PDO-level indicators that remained after restructuring (see table 2). The project’s achievement of outcomes is therefore rated as ‘High’ against the post-restructuring Results Framework. The project’s pre-restructuring shortcomings relate to unsatisfactory progress on Component 2 before the restructuring (Results Indicator 3 in table 2) 4See https://www.imf.org/external/pubs/ft/survey/so/2013/CAR010813A.htm?id=304903. 5The extension of the CPS is recommended by the Performance and Learning Review FY2014–2019 of the CPS (No. 125363- KM). Page 12 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) and the failure to meet certain pre-restructuring intermediate results indicators (Results Indicators 7, 9, and 10 in Table 2). Table 2. Achievement of Project Outcomes No. Indicator Unit Baseline Target Actual at ICR Comment Results Indicators 1 Collection rate of MA- % 55 70 77.4 Overachieved by MWE improved significant (sliding 12 months margin average); defined as cash collected as a percentage of billed electricity at MA-MWE 2 Electricity losses per % 45 37.5 36.8 Overachieved year in the project area; defined as billed electricity as a percentage of produced electricity at MA-MWE and EDA 3 Adoption of new n.a. No Yes No Dropped during governance restructuring arrangements for the sector; defined as the adoption of new utility governance arrangements and tariff and subsidy framework 4 Direct project Number 0 200,000 354,6136 Overachieved by beneficiaries (from 0 (of which 50% (of which 50% significant to 200,000), 50 female) female) margin percent of which female, defined as the number of household electricity consumers of MA-MWE and EDA Intermediate Results Indicators Component 1 5 Commercial and Yes/No No Yes Yes Fully achieved financial recovery plan Implemented 6 Commercial and Yes/No No Yes Yes Fully achieved financial software procured, installed, and operational 7 Metering equipment, Text n.a. 80 network Only prepaid Dropped during protection boxes, and meters and meters restructuring, fuel metering installed 4,000 installed but results 6 This number assumes the same number of household members per connection as at the state of appraisal. Page 13 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) No. Indicator Unit Baseline Target Actual at ICR Comment protection related to boxes, 1 fuel prepaid meters metering were captured in equipment separate indicator. 8 Prepaid meters Number 0 4,000 4,000 Fully achieved; installed added during restructuring Intermediate Results Indicators Component 2 9 Energy sector policy Yes/No No Yes No Dropped during drafted and adopted restructuring 10 New planning and Yes/No No Yes No Dropped during decision-making restructuring process for investments adopted 11 Subsidies and tariff Yes/No No Both Subsidy study Achieved; tariff study delivered completed completed study dropped during restructuring 22. The project’s main contributions and impacts are summarized in the following paragraphs. The discussion of contributions and impacts is organized by results indicators. 23. MA-MWE’s commercial and financial performance improved significantly between 2015 and 2017 compared to a baseline trajectory. Most importantly, as a result of the project, the utilities’ combined cash recovery index increased by more than 50 percent. Total cash recovery increased from 30.25 percent to 48.90 percent (49.40 percent for Grand Comore and 43.00 percent for Mohéli), which is 5.2 percentage points higher than the expected level of 43.70 percent based on the project’s combined targets on collection and electricity losses. These financial performance improvements were the result of the following achievements: (a) Collection rates (Results Indicator 1 in table 2) sharply improved from an average of 55.0 percent at appraisal to 77.4 percent in 2017. For MA-MWE, the commercial results for the year 2017 show a collection rate at 78.3 percent on Grand Comore and 67.6 percent on Mohéli, averaging 77.4 percent for the company overall. The overall result is 7.4 percentage points higher than the target. These improvements in bill collection were the result of the following: (i) Changes in commercial procedures, including the process of solving disputes resolutions over arrears with large medium voltage (MV) customers; clients’ awareness of payment deadlines and impact of nonpayments on MA-MWE operations through the use of local radios, dialogue with consumer associations, and the setup of an SMS reminder system; and the reduction of bottlenecks and long queues at payment centers though the installation of high-speed printers. These changes were developed and overseen by the externally hired senior managers funded by the project. Page 14 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) (ii) Installation of 4,000 mobile payment system-compatible tamper-proof prepaid meters improved collection compared to previously used prepaid meters that were notoriously easy to corrupt and manipulate. (iii) Institutionalization of direct payment by the GoC to MA-MWE of the administration’s and public services’ electricity usage, which resulted in significant decline in outstanding receivables to the public sector. This was part of a performance contract facilitated by the World Bank between MA-MWE and the GoC during the implementation of the project. (iv) Debt settlement and settlement of cross-arrears between the Government-owned fuel supply company Société Comorienne des Hydrocarbures (Petroleum Company in Comoros, SCH), MA-MWE, and the State, which enabled more efficient financial planning of the SOEs with the establishment of regular and transparent subsidies from the GoC. (v) Effective implementation of a modern MIS7 at MA-MWE and related staff training.8 The MIS provides seamless traceability over the entire commercial business cycle process, the streamlining and automation of billing and collection processes, and better financial planning and revenue projections. The MIS improved bill collection rates and speed and gave the utility the ability to cope efficiently with the much higher volume of electricity sales in 2016 and 2017.9 The acquisition and operationalization of the MIS was overseen by the externally hired senior managers funded by the project. (vi) A load management system capable of recording load shedding events and of identifying which populations are affected by power cuts allowed for a better communication with the clients in informing them when the controlled shedding will occur and to better solve grid-related blackouts. This in turn improved client relations and bill payment discipline. (vii) Better financial planning and analysis as well as bookkeeping and payroll functions from project-funded staff training as well as a combined update and upgrade of the utility’s accounting software that has fallen in disuse since 2009. (b) Total system losses (Results Indicator 2 in table 2) declined by almost a quarter from 45.0 percent at appraisal to 36.8 percent in 2017. For MA-MWE, the commercial results for the year 2017 show a system loss rate at 36.8 percent on Grand Comore and 36.4 percent in Mohéli, averaging 36.8 for the company overall. This result is 0.7 percentage points lower than the target. These improvements were the result of10 7 In October 2017, the last module of the MIS became fully operational. 8 Training of MA-MWE’s staff focused on how to use and administer the MIS (8 training modules with 25 training sessions in total) led to better results in billing and collection as well as improved client relations because less incorrect invoices were being sent out to customers 9 The contract of the MIS update extends beyond the project’s closing date (out of Government resources) and covers after -sales service through April 2019. 10 Related support to EDA, not measured by the results indicator, was provided in the form of a vehicle to facilitate meter reading activities in remote locations. Page 15 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) (i) Changes in commercial procedures, including improvements in the process of monitoring large MV customers, staff work ethics focused on ensuring that billing schedules are respected, and meter and connection anomalies are systematically reported, universal use of secured online peripheral devices in local offices for real- time recording of payments, and mobile offline payment system to facilitate and securely record collection in remote locations by MA-MWE staff. All the above process changes were developed and overseen by the externally hired senior managers funded by the project. (ii) Deployment of the MIS (see above) at MA-MWE, allowing for automated, consistent, and timely billing and a reduction of commercial losses. Moreover, the new processes and planning functions introduced by the MIS allowed MA-MWE to systematically optimize the meter reading tours of its agents to ensure exhaustive meter coverage and reduce data-entry logjams and errors. Furthermore, the change from the point of delivery billing to the use of a unique client-specific identifier allowed for the introduction of targeted campaigns to reduce meter fraud and electricity theft. 24. The project reached an estimated 354,613 beneficiaries (Results Indicator 4 in Table 2), compared to an original objective of 200,000. This was made possible by the utilities’ continued expansion of access during the project period. 25. Other impacts of the projects not captured by the Results Framework include the following: (a) Government subsidies required to maintain adequate supply reduced as a result of the improvement of MA-MWE’s commercial performance. The State’s combined explicit and implicit subsidies to the utility—that is, the sum of direct subsidies and residual losses absorbed by the utility was equivalent to 0.4 percent of GDP on average during 2015–17— were much lower than the level of 1.7 percent of GDP on average during 2011–13 observed at appraisal. (b) The additional cash flow available for fuel purchases allowed MA-MWE to almost double power supply over the period of implementation. Power supply increased from 39 GWh to 69 GWh, bringing power supply much closer to the economy’s needs and reducing the share of unserved demand. 26. Three activities originally planned under Component 1 were dropped as a result of the reduction in project funding due to the depreciation of the SDR against the euro (the currency of most contracts). These dropped activities were the purchase and installation of (a) MIS and hardware at EDA, the smaller of the two utilities; (b) approximately 80 units of network metering equipment at MA-MWE; and (c) fuel metering equipment for fuel tanks. 27. Progress on the PDO-level indicator capturing the sector governance component was limited up until it was dropped during restructuring. The project’s achievement of outcomes against the pre- restructuring Results Framework is therefore only rated ‘Substantial’. Before restructuring, the project carried out a study on subsidies and cross-arrears as well as an evaluation of the training needs of the Direction Générale de l'Energie, des Mines et des Ressources en Eau (Directorate General for Energy, Mines and Water Resources Ministry of Energy, DGEME), EDA, and MA-MWE teams (AM 2016). A Steering Page 16 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) Committee (SC) was established in November 2014 (Arreté n. 1487/MPEEIA/CAB) and a performance contract was signed in March 2015 and amended in March 2016 to include the GoC’s commitment to regular payment of fuel subsidies to MA-MWE and refrain from intervening in the competitive recruitment of the utility’s senior directors. However, from the beginning, progress on activities under Component 2 was slow mostly due to the limited institutional capacity of the core team of the Ministry of Energy (MoE) for the power sector which consisted of only two full-time staff. Following elections, the ministry’s new team remained understaffed, and consequently all the project’s governance-related activities under Component 2 continued stalling. The stalemate now included the execution of the critical performance contract and the launch of a critical tariff analysis and sector studies, for which all tender documents had already been completed with the support of the World Bank team. At the same time, the new government’s continued disruptive changes of the MA-MWE senior management team resulted in significant additional delays in procurement. Consequently, at restructuring, all remaining activities and associated key outcome indicators were dropped. Intermediate results achieved at that time were, however, not lost because most of the analytical and training activities were taken up by AfDB’s Energy Sector Support Project (PASEC). Justification of Overall Efficacy Rating 28. The project’s efficacy is rated ‘High’ against the post-restructuring Results Framework because all three post-restructuring results indicators (1, 2, 4 in Table 2) were overachieved and there were no shortcomings against the expected results. 29. The project’s efficacy is rated ‘Substantial’ against the pre-restructuring Results Framework because of shortcomings in the form of unsatisfactory progress on Component 2 before the restructuring. Specifically, Results Indicator 3 in table 2 was not achieved. The same is true for certain pre- restructuring intermediate results indicators (Results Indicators 7, 9, and 10 in Table 2). C. Efficiency Rating: Substantial Assessment of Efficiency and Rating 30. The project economic efficiency exceeded expectations, with an estimated economic internal rate of return (EIRR) of 18 percent and a net present value (NPV) of US$1.2 million at 12 percent discount rate (US$3.8 million at 6 percent discount rate). This compares to an EIRR estimate at appraisal of 16 percent and an NPV of US$0.6 million at 12 percent discount rate. 31. In economic terms, the main project benefit is improved energy efficiency as the project’s impact on revenues collection raises users’ actual price paid per kWh. This increase in the actual price of electricity incentivizes more efficient use of electricity.11 As in the economic analysis at appraisal, the evaluation of economic benefits assumes that out of the change in the collection rate, two-thirds was the result of a purely financial transfer from existing consumers to MA-MWE and one-third was the result of more efficient use of energy (thereby freeing supply for users which would not have been served 11 At the time of appraisal, there was significant evidence that widespread fraud and nonpayment for electricity supply is resulting in inefficient use of electricity given the limited incentives to conserve (for example, for public lighting and private lighting which constitutes a significant share of total demand and which was inefficient). The sector situation combined widespread load shedding and suppressed demand for a large proportion of users, with wasteful use by others. Page 17 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) otherwise). Overall growth in the consumption of electricity in the Comoros slowed to 4 percent per year during 2013–17 compared to 9 percent per year during 2009–13, consistent with gains in end-use efficiency. Not measured by the analysis is the additional benefit from the project’s contribution to a significantly higher power supply by enabling MA-MWE to acquire larger volumes of fuel for production. 32. The lack of data separating improvements in energy efficiency from other improvements in the bill collection rate does not represent a shortcoming in the monitoring and evaluation (M&E) framework. This is because it is technically not possible to evaluate the exact contribution of efficiency gains to the increased collection rate. The economic analysis therefore has to rely on reasonable assumptions, based on the assumptions used at appraisal. 33. The project achieved significant commercial and financial improvements with limited financial resources. Financially, the main benefit of the project is that restoring payment discipline for electricity supply results in an incremental transfer of revenue from electricity users to MA-MWE. This in turn made MA-MWE less dependent on Government subsidies and avoided crowding out other Government spending. D. Justification of Overall Outcome Rating Overall Rating: Satisfactory 34. The project’s overall outcome is rated ‘Satisfactory’, based on a split rating t hat is applicable due to the restructuring, which narrowed the project scope. Because the project’s restructuring narrowed the scope of outcomes and results indicators, the ICR uses the ‘split rating’ method, which yields an overall outcome rating of ‘Satisfactory’. The scope of the project narrowed with the removal of the governance-related measures and indicators. Against the original results indicators and targets, the project would be rated ‘Moderately Satisfactory’, whereas it is rated ‘Highly Satisfactory’ against the revised targets. Weighted by disbursement, the split rating methodology yields an overall rating of ‘Satisfactory’. 35. The ‘Satisfactory’ rating is consistent with the fact that the project overachieved on all three PDO-level objectives that remained post restructuring and achieved all intermediate outcome indicators. Commercial and financial results indicators were overachieved, as was the number of beneficiaries reached. The intermediate outcome indicators were also largely achieved, with only minor shortcomings. This indicates that the theory of change of the project was sound and implementation successful. 36. The ‘Satisfactory’ rating is also consistent with the fact that the project was resilient to disruption and achieved its key objectives despite exceptionally difficult circumstances of project implementation in a volatile governance and fragile country context. Table 3. Split Rating of Project Outcomes Against Original PDO Against Revised PDO Overall Indicators (2013–17) Indicators (2017–18) Relevance High Efficacy Substantial High Efficiency Substantial Page 18 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) Against Original PDO Against Revised PDO Overall Indicators (2013–17) Indicators (2017–18) Overall rating Moderately Satisfactory Highly Satisfactory Rating value (1) 4 6 Disbursed (US$ million) (2) 2.54 1.98 4.52 Weight (% Disbursed) (3) 56 44 100 Weighted value (1 × 3) 2.24 2.64 4.88 Final rating Satisfactory 37. The rating represents an improvement compared to the last Implementation Status and Results Report (ISR) rating on ‘Progress toward PDO’ because data on the full year of 2017 show that the SOEs’ performance increased significantly during the last year of the project. Ratings of implementation progress and progress toward the development objective were rated relatively low during the project’s last few ISRs because of delays in the submission of critical financial reports as well as commercial and performance data, in the absence of a MA-MWE administrative and financial senior manager. This represented noncompliance with the Government’s obligations under the project and made it difficult for the project to evaluate progress against the PDO-level objectives. The last ISR rated the project ‘Moderately Satisfactory’ in January 2018. This situation was only resolved in the last months of the project. Data on the full years of 2016 and 2017, which became available only in 2018, showed that the project’s activities significantly increased MA-MWE’s commercial and financial efficiency. The ICR’s rating of the project’s overall outcomes is therefore one level above the last ISR. E. Other Outcomes and Impacts Gender 38. With support from the World Bank’s AFR Gender and Energy program and the Youth Innovation Fund (YIF), the project increased women’s awareness of and, promoted women’s advocacy about, the value of formal electricity connections and the importance of paying electricity bills on MA-MWE capacity to supply reliable energy.12 The initiative emerged from two findings. First, based on the project’s PSIA finding that indicates nearly 60 percent of registered electricity theft cases in 2015 were filed against women. This is mainly a consequence of the Comorian matrilineal system under which women own the family house and hence electricity service contracts are registered in their names. Second, given their social status resulting from matrilineal traditions and their relative lack of implication in the political arena, Comorian women are widely perceived as impartial and well suited to promote transparency and civic values in their respective communities. Capitalizing on this opportunity, the World Bank’s Gender and YIF team worked with the client to establish program for women advocacy against electricity theft and fraud. A multi-stakeholder team composed of grassroots women’s associations, civil society, private sector representatives, customer support syndicates, and MA-MWE staff worked together to train more than 100 women and reach over 5,000 customers. Because of the face-to-face grassroots 12The initiative was supported by Energy Sector Management Assistance Programme and the World Bank’s Collaborative Leadership for Development initiative (CL4D) Page 19 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) advocacy approach of the women’s group, the number of outstanding bills in the pilot regions of Hamanvou and Bambao dropped in a period of 100 days from 3,834 to 777 customers13. Institutional Strengthening 39. Technical assistance and investments under the project have resulted in significant efficiency gains in the utilities’ administrative and commercial operations. Key institutional improvements are summarized in section III. Mobilizing Private Sector Financing 40. Private sector financing in the power sector in the Comoros remains a distant prospect as of today, but the project contributed to making MA-MWE a more credible offtaker. Credible private investment requires conducive macroeconomic and legal frameworks as well as financially sound and well-managed utilities. The project contributed to the latter through the measures taken under Component 1. However, much remains to be done, especially with regard to sector governance, before private sector participation in the Comoros’ power sector can be realistically considered. Poverty Reduction and Shared Prosperity 41. Project interventions were poverty sensitive and benefited the poor and vulnerable. The key channels of impact were the following: (a) Reduced power rationing improved poor households’ living standard as, according to the PSIA, 60 percent of poor households’ demand in Moroni had been suppressed in 2013–14 against a mere 39 percent for the better-off. (b) Based on advice from the externally hired resident expert, the utility deliberately spared the tariff group with the lowest per household consumption, which predominantly consists of the poorest and most vulnerable customers, from the systematic procedures of power cuts for unpaid bills. (c) Improved billing and collection helped avoid a tariff increase to all MA-MWE customers, a measure that would have had a negative impact on poor households. (d) MA-MWE’s financial improvement and consequently lower need for public financing will contribute to higher spending in currently underfinanced social sectors such as health and education. Other Unintended Outcomes and Impacts 42. In combination with lower global fuel prices (see paragraph 57), the project’s achievements added significant fiscal space by reducing the need for budget transfers to the SOEs while maintaining the utilities’ ability to operate. The sum of the GoC’s subsidies to the utility and MA-MWE’s losses 13 World Bank’s AFR Gender and Energy program report by Camilla Gandini (2018). Page 20 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) equivalent to 0.4 percent during 2015–17 were much lower than the level of 1.7 percent during 2011–13. This freed up fiscal resources to spend on human development and other GoC priorities. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. Key Factors During Preparation 43. A recently completed sector assessment provided the groundwork for a project design that was highly relevant, simple, and yielded quick results. Scope and design of the project benefited from the MA-MWE Recovery and Development Plan (RDP), prepared during appraisal and adopted soon after the beginning of the project by the MoE. The RDP, with its focus on the massive commercial losses and nonpayment as critical levers to utility performance, provided a detailed road map to tackle the root causes of the electricity crisis in the Comoros. Moreover, the RDP became the basis for the performance contract between MA-MWE and the GoC adopted a year later. 44. Most risks were adequately identified. With the exception of risks related to the Government- imposed frequent changes of MA-MWE’s senior management and insufficient ownership of sector reform activities, the project’s appraisal adequately identified the most important risks to the project outcome. As explained in detail in the following paragraphs, this helped mitigate several important risks either fully or partially, including the utilities’ implementation capacity, stakeholder risks, and project complexity. 45. Implementation readiness was insufficient despite significant efforts that included the use of the Project Preparation Fund (PPF). Funds from the PPF were to be used before effectiveness to support the GoC in the completion and official adoption of the RDP and recruit a group of international senior experts to support MA-MWE’s management on all critical utility reforms. However, due to procurement delays, the RDP could not be finalized before the end of 2014 and the process of recruiting international experts to assist MA-MWE management team started only in 2015. 46. The World Bank team proposed a simple and pragmatic ‘stand-alone’ project design in consideration of the fragility, conflict, and violence (FCV)-context, the client’s limited institutional capacity, and the small overall budget. Accordingly, at the Project Concept Note and Quality Enhancement Review stages, the original project setup was narrowly focused on the most critical challenges of the utility’s operational and financial recovery while leaving out many related and important, but somehow less urgent, sector governance challenges. However, during project development and negotiations, the client requested to extend the project focus to include a component to be implemented by the MoE, which would facilitate several ambitious sector reforms including the establishment of a conducive legal and institutional sector framework for public-private partnerships and a restructuring of the country’s fuel supply chain. The team accommodated the requests by allocating funding for a separate coordinator for Component 2 to be appointed by the GoC and a number of sector governance-focused project activities. 47. Institutional arrangements were designed to ensure client buy-in and accountability through streamlined decision-making processes. With the overwhelming majority of project activities focused on the introduction of state-of-the-art billing and MIS as well as the reengineering of critical business processes, the team considered close supervision and strong ownership by the utility’s senior management as the most important success factors for implementation. Therefore, it was considered crucial for the PIU not only to be located at the utility’s headquarters but also to be headed by MA-MWE’s Managing Director (MD). Similarly, to streamline implementation and decision-making processes, the Page 21 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) original project design intended for the international resident experts to be hired under the project to be provided with the managerial and decision-making authority needed to effectively lead the implementation of utility reforms under Component 1 while being accountable for the achievement of agreed key performance indicators (KPIs) in their respective domains. B. Key Factors During Implementation Factors Subject to Government and/or Implementing Entities Control 48. Frequent Government-ordered changes of the MA-MWE senior management team jeopardized implementation and compromised the utilities’ institutional independence. Following a costly and time- consuming search and recruitment of suitable candidates, the ability of the international resident experts to fill their dual mandate of leading the company’s operational recovery and building local management capacity by working side by side with the MA-MWE team of senior directors was critically impeded by Government interferences. During project implementation, three MDs and an interim managing committee as well as three Chief Financial Officers (CFOs), three commercial directors, and two technical directors succeeded each other at the helm of MA-MWE. As the project’s institutional arrangements determined for MA-MWE’s MD to also be the head of the PIU, the latter was also subject to the disruption created by these changes which in turn created significant procurement and disbursement delays. The persistent lack of continuity at the MA-MWE’s senior management also affected the morale of the international resident experts hired by the project (ISR Seq. 8) as their efforts to foster knowledge transfer and strengthen local managerial skills through cross-training and working in tandem with MA-MWE’s senior directors were wiped out with every change of their local counterparts. As a result, some resident experts asked to significantly reduce time spent at MA-MWE while others refused to renew their contracts. 49. International resident experts were not given the agreed executive mandate. Adding to the above challenges and contrary to the explicit provisions of the Project Appraisal Document (PAD), the international resident experts were never granted the critical managerial and decision-making authority to effectively lead the implementation of reforms to achieve the agreed objectives of the RDP in their respective domains. Therefore, as experts were only provided with an advisory mandate, they could not ensure continuity or shield ongoing reforms of the RDP from the disruption created by the frequent change of MA-MWE’s senior management. Consequently, the implementation of the RDP suffered major delays and remained incomplete. 50. Excessive cuts to electricity subsidies led to severe financial difficulties in 2015 and the reemergence of power shortages due to lack of access to funds for fuel. Delays in the execution of the Government’s commitment in the RDP (in January 2015) to provide fuel subsidies result ed in excessive budget constraints and MA-MWE incapacity to use its full production capacity to meet demand. This was resolved through the amendment of the performance contract between MA-MWE and the GoC in 2016 which introduced a provision for the monthly disbursement of the sector subsidy in the form of ‘free’ fuel deliveries from SCH. 51. Government leadership on the sector’s governance framework fell short of commitments made during project preparation. After a first meeting of the SC in June 2015 and the signature of the amendment of the performance contract between MA-MWE and the GoC in March 2016, the Government did not monitor the execution of the contract or called for new meetings of the SC. This delayed several Page 22 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) key activities planned under Component 2. Furthermore, the GoC’s decision to replace insufficiently maintained generators with a set of small, low-efficiency marine propulsion engines operated by an independent contractor with no incentive to minimize fuel consumption reversed the much-needed reduction of per kWh fuel cost. Instead of consolidating the project’s financial outcomes, the new generators led to the doubling of MA-MWE’s fuel bill and the further degradation of financial results (ISR Seq. 8). 52. Opposition to reform from stakeholders benefiting from the status quo—namely utility employees and some consumer groups—was anticipated during preparation and was successfully mitigated. The risk rated High at appraisal did not materialize as the project prioritized ‘win-win’ reforms focused on revenue protection and efficiency, while the frequent change of MDs made any of the RDP’s more challenging and controversial reforms, including the reduction of the utility’s personnel and overhead cost, virtually impossible. With regard to the reduction of theft and fraud, the project’s activities including women’s advocacy groups against electricity theft, local radio campaigns, and ongoing dialogue with consumer associations helped thwart any major backlash. Moreover, the PSIA completed by the project contributed to a better understanding of the social impacts of (a) electricity outages, (b) socially unbalanced increases in tariffs, and (c) undifferentiated enforcement of arrears payment. The findings of the PSIA helped senior managers successfully advocate against unpopular measures that would only have had an insignificant impact on revenues while strongly affecting the poorest and most vulnerable households. 53. Coordination and complementarity with other donors’ projects were not as successful as anticipated. Some activities originally under the scope of the project were shifted to other donors during implementation. This was in part because the high-level SC could not play its role of coordination of donors’ activities as effectively as expected at appraisal. The AfDB PASEC program for the rehabilitation of generation and distribution network was considerably delayed and restructured in 2015. Therefore, the expected mutual reinforcement of the project’s commercial and financial results and AfDB’s technical improvement (rehabilitation of generation, fuel storage, and distribution network) never materialized. Neither did the GoC planned transition of small-scale and diesel-based thermal generation capacity toward cheaper heavy oil-based production happen: at the time of ICR, the works for the installation of a heavy fuel oil (HFO) power plant had barely started with no official plan for the creation of the needed HFO supply chain. Factors Subject to World Bank Control 54. Institutional arrangements to secure the buy-in of the utility’s senior management had major shortcomings which could only be partially addressed. To secure the needed strong ownership of the project’s utility reforms and related investment activities including in IT systems and business process reengineering, the project’s institutional arrangements at appraisal required for the PIU to be headed by MA-MWE’s MD. While this arrangement secured the strong buy-in of the utility’s senior management, it had three shortcomings: (a) the duties and responsibilities of MA-MWE’s MD often left little time for the MD and resulted in limited availability of the MD to play his role of PIU coordinator; (b) the frequent changes of the utility’s MDs created delays and disruption as newly appointed MDs had to first become familiar with their new responsibility as head of the utility jobs before becoming effective leaders of the PIU; and (c) while MA-MWE’s MD could legitimately lead implementation of all utility reform-related activities, the GoC did not consider this arrangement as viable solution for the implementation of broader sector governance activities. The risks resulting from the frequent changes in MA-MWE and PIU leadership Page 23 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) could only be partially mitigated by the project’s senior experts’ efforts to smoothen the transition and accelerate the learning phase of new MDs. In contrast, the GoC’s concerns regarding the ownership of sector governance activities were fully addressed by the appointment of a full-time project coordinator from DGEME to oversee implementation of Component 2. 55. Implementation delays required a project extension to bring PDO-critical activities to a close. As explained in detail earlier, recurrent changes of the MA-MWE MD, who also acted as head of the project’s PIU, significantly delayed procurement and financial management procedures and led to the need to restructure and extend the project by one year. 56. The risk of implementation delays and disruption from excessive project complexity in a context of fragility and low capacity was anticipated during project preparation and further mitigated during implementation. This risk, rated Substantial at appraisal, only partially materialized and was further mitigated through the restructuring including the cancellation and reallocation of funds for activities planned under Component 2. By focusing on a limited number of PDO-critical activities which were further reduced during implementation, the World Bank team successfully mitigated the risk of critical delays and disruption resulting from project complexity and low implementation capacity. Factors Outside the Control of Government and/or Implementing Entities 57. Global fuel prices fell during project implementation which, even though it had no immediate impact on the PDO-level indicators, contributed to improved cash flow and fuel availability for the utilities. Fuel prices, which are a major determinant of the SOEs’ financial performance due to their reliance on diesel-fueled power generation, fell from an average of about US$110 per barrel during appraisal to below US$50 in 2015 and 2016 before rising again to about US$60–80 during the time of the ICR. The fall in oil prices was partially offset by the continued devaluation of the local currency and did not immediately affect the PDO-level results but certainly contributed to improved cash availability for the SOEs. Other major macroeconomic and political risks identified during project preparation, including a severe deterioration of the macroeconomic environment or flaring up of conflict and instability, did not materialize. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. Quality of Monitoring and Evaluation (M&E) Rating: Substantial M&E Design 58. The design of the M&E framework captured most relevant results but would have benefited from including intermediate steps on MA-MWE’s financials and reforms of the utility’s governance framework. Commercial performance and governance were adequately captured in the results indicators, but financial performance was only indirectly covered. The M&E framework would have also benefited from capturing the performance contract signed by MA-MWE and the MoE. Further, the results indicator for project beneficiaries was wrongly defined in the PAD as household connections, whereas the target figure related to individual beneficiaries (that is, a much larger number than household connections). Page 24 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) M&E Implementation 59. Measurement of project progress along the indicators of commercial performance was adequate and timely, despite the limited data availability in the Comoros. All other indicators were qualitative and discrete (mostly ‘Yes/No’) and measured by the PIU and team continuously without much effort as part of project implementation. M&E Utilization 60. The M&E framework, much of which relied on data that were not regularly collected previously, significantly improved data availability for decision making in the sector. This is particularly true for the utility’s commercial performance indicators. The lack of monitoring indicators on financial results did not affect the project’s monitoring as ISRs and Aide-Mémoire (AM) paid close attention to all critical factors that could jeopardize progress on improving the utility’s financial position. The M&E framework also helped inform the World Bank’s decision making on the project on time. Justification of Overall Rating of Quality of M&E 61. The overall quality of M&E is rated Substantial. This reflects that there were only moderate shortcomings in the M&E framework and its implementation and utilization, which most importantly relate to (a) the absence of indicators relating the performance contract signed by MA-MWE and the MoE, (b) the inadequate target for—and measurement of—the number of beneficiaries (PDO-level indicator), and (c) the insufficient coverage of financial indicators in the results indicators. B. Environmental, Social and Fiduciary Compliance 62. No safeguards policies were triggered by the project. 63. Financial management arrangements were compliant, even though the project faced delays in the submission of project audits. The financial management risk was rated Moderate at appraisal but Substantial during the project and Unsatisfactory at the last ISR (Sequence 8) because of delays in the submission of project audits and concerns regarding the risk of failure to implement the project’s budget plan if no backup project management and financial management persons with the authority to sign off and grant continuity in day-to-day PIU operations were found. These issues were resolved in time, however, and at closing, all audits were received, and financial management arrangements were considered compliant. The latest independent audit of the project’s financial statements, expenses, and balance sheet received an unqualified opinion. 64. Procurement was compliant but source of significant delays in implementation. The PIU’s procurement staff was trained in the World Bank guidelines for contract preparation, bidding, and implementation and no issues of noncompliance were identified during project execution. The main sources of delays in procurement were the frequent changes of the head of the PIU, the lack of staff capacity, and the GoC’s multiple levels of approvals required for most contracts. Page 25 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) C. Bank Performance Rating: Satisfactory Quality at Entry 65. Overall, the simple and focused project design adequately reflected the country context at appraisal, the key bottlenecks in the sector, and the intended activities and results. The project design was based on prior analytical engagement and took into account (a) the fragile country environment, (b) the limited available IDA resources, (c) the agreements between the GoC and development partners on the focus and scope of their respective interventions, (d) the client’s limited implementation capacity, and (e) the need for a highly flexible and adaptive project design. 66. The design reflected best practices for operations in FCV contexts. These included (a) a limited and realistic set of initial objectives to facilitate a robust and focused client ownership of the project; (b) a deliberately lean program focused on the bare essentials of utility operations, specifically billing and bill collections; and (c) close attention during preparation to implementation arrangements, including the project’s readiness to implement. 67. Governance-related risks were underestimated at the design stage and the project’s objectives in terms of policy reforms should have been reflected as intermediate indicator rather than as PDO- level indicators. While the project supported the technical work that was meant to provide the foundations for sector governance reforms, the Results Framework reflects their implementation as a PDO-level indicator, even though reform implementation was beyond the control of the project. It would have been more adequate to reflect the outcomes of the reform component in the intermediate results indicators. 68. In hindsight, the project was overambitious in the expectations on procurement and implementation timelines. Considering the limited project resources and the weak implementation capacity of both MA-MWE at the utility level and DGEME at the sector governance level, a less ambitious approach focusing on only one of the two governance levels would have been more adequate and allowed more effective project implementation through greater focus. This is particularly true for Component 2, whose successful implementation, similar to the close support of critical activities under Component 1 by senior experts, would have required the backing of DGEME’s limited and frequently changing team of two full-time staff with international experts. Quality at Supervision 69. The World Bank responded to emerging issues through just-in-time advice, technical assistance, and—eventually—a level 2 project restructuring and one-year extension. Semiannual supervision missions were conducted throughout the project with appropriate staffing. Emerging issues were identified on time and time-bound action plans were recorded in the ISRs and the AM to ease follow-up. As explained earlier, the World Bank repeatedly alerted the client and argued for constructive solutions about the disruption and delays in procurement and managing decisions resulting from the frequent changes of the MA-MWE senior management team and at the head of the PIU, the negative impact of the cutbacks of state fuel subsidies on electricity supply in 2015 and 2016, MA-MWE’s financial vulnerability due to the lack of incentives for the independent contractor of the new diesel generators to minimize fuel consumption, and high electricity cost of production and possible additional World Bank’s engagement in Page 26 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) system rehabilitation and solar photovoltaic installations. Restructuring came at the right time to recover project performance. 70. The World Bank’s presence in the Comoros increased significantly during project implementation, facilitating closer implementation support. While the project started off with only a local operations specialist and consultant, the World Bank’s country office continuously expanded during project implementation. At project closing, the country office team included four staff led by a Resident Representative which much improved the effectiveness and quality of the World Bank’s policy dialogue with the client. 71. Two transitions in team leadership on the World Bank side were managed well. The Task Team Leader changed twice during the project’s lifetime, once during preparation and once during implementation, but the transitions were managed well and did not negatively affect project progress. 72. Overall, the project was adequately supervised despite exceptionally difficult circumstances of project implementation in a volatile and disruptive governance context. During the project’s lifetime, the team dealt with four different project coordinators, four MDs, four commercial directors, three technical directors, and three CFOs of the utility. The changes of the MD and project coordinator were particularly disruptive because they not only slowed down ongoing utility reforms but also led to month- long interruptions of procurement activities as the Government insisted that only MA-MWE’s MD could be accredited to sign contracts and procurement documents. 73. Collaboration across GPs in the World Bank contributed to the project’s implementation. Cross- sectoral expertise was brought to the project in the context of SOE debt management and public expenditure management, which helped improve financial management at MA-MWE. Justification of Overall Rating of Bank Performance 74. World Bank performance is rated ‘Satisfactory’ as there were only minor shortcomings in Quality at Entry and Quality of Supervision. At entry, the project was realistic, simple, and well designed with mitigations measures that effectively helped reduce implementation risks. During implementation, the World Bank was proactive in supporting the PIU daily operations and ensuring adequate project progress. The only shortcomings relate to the overambitious project scope and timeline defined at approval. D. Risk to Development Outcome Rating: High 75. The project’s key achievement, the successful implementation of MA-MWE’s new MIS and commercial management system, is at risk due to the insufficient one-year duration of the IT provider’s maintenance warrantee. This critical issue was identified about a year before project closing. With almost two years of implementation delays and the client’s limited capacity, the MIS’s maintenance warrantee clearly needed to continue beyond the April 2018 closing date of the Electricity Sector Recovery Project (ESRP). Without a sufficiently long warrantee contract in place, any minor incident may force the utility to roll back the newly installed IT system and infrastructure. Therefore, the team worked with the client, the IT provider, and the World Bank’s procurement experts to find a solution allowing the project to fund an extended maintenance warrantee lasting four years beyond the project’s closing. Unfortunately, the Page 27 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) initially proposed solution, the advance payment of the warrantee extension against the provision of a bank guarantee by the software provider to secure satisfactory performance, was not in line with the World Bank’s operational policy. At the submission of this ICR, the Bank was still analyzing the issue to eventually provide the client with an alternative solution of this pressing issue. 76. The departure of international experts at the end of the project also puts the sustainability of the results at risk. The international experts were critical for application of best practices to operational and commercial procedures in the utilities. Their departure without replacement, due to the unavailability of Government funds to continue funding their position, therefore represents a risk to the sustainability of the project’s outcomes. 77. Furthermore, several external factors might be detrimental to the utilities’ commercial and financial performance in the future: (a) The country’s limited progress in terms of institutionalizing good sector governance represents a major risk to the development outcome and may reverse some of the achievements through incoherent investment planning and implementation. (b) The continued reliance on expensive fossil fuel power generation makes the utility’s commercial and financial performance vulnerable to cuts in Government fuel subsidies (as it happened in 2015). (c) The utilities’ continued shortfall of adequately trained staff and the continued lack of transparent and competitive recruitment of senior managers will make it difficult for the utilities to continue the path of commercial and financial performance improvements. 78. The risk to the development outcome is only partially mitigated through the World Bank’s post- completion engagement. A follow-on investment project is under preparation, with an IDA allocation of US$20–36 million. The project intends to provide funding for the transition from a costly fossil fuel-based generation infrastructure to the increased use of renewable energy sources as well as network upgrades to further improve MA-MWE’s financial and operational performance. LESSONS AND RECOMMENDATIONS 79. The analytical engagement of the World Bank along with strong client buy-in as the basis for interventions is particularly important in fragile and low-capacity countries. MA-MWE’s RDP, prepared during appraisal and adopted by the GoC soon after the beginning of the project, ensured that both scope and design of the project were targeting the most critical levers for project success. Moreover, the key parameters of the RDP subsequently served as the basis for the performance contract between MA-MWE and the GoC adopted a year later. Without the analytical foundation and official adoption of the RDP, the project would neither have the needed relevance nor the client buy-in to successfully pursue the most critical utility reforms. 80. Simple, well-targeted interventions based on proven solutions to critical operational challenges can have significant positive impacts within a short time frame, even in the most difficult country context. The simple project design and rigorous shedding of non-PDO-critical activities during restructuring was instrumental in achieving results within a difficult governance context. As stated earlier, Page 28 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) an even more focused approach exclusively aimed at utility governance may have allowed a faster and more effective achievement of the PDO. 81. If adequately communicated and targeted, measures to reduce electricity theft and nonpayment can yield large financial benefits without triggering significant opposition from vested interests. The project’s support of targeted communication campaigns was key in MA-MWE’s successful reduction of nontechnical losses. 82. Women’s groups can be effective in fostering positive change by appealing to social norms and individual responsibilities, especially in an FCV context with weak enforcement of property rights and chronically poor payment behavior for public services. The engagement and empowerment of women’s groups to lead campaigns against electricity theft and meter fraud was particularly powerful within the context of the matrilineal Comorian society. 83. In fragile and low-capacity environments, any activities aiming at sector governance reforms have to be carefully targeted and accompanied with close and highly adaptive implementation support. Within this context, the broad focus of Component 2 at appraisal with its limited US$0.9 million funding envelope simultaneously aiming at (a) the review of sector governance arrangements, (b) defining of the sector’s missing legal and regulatory framework, (c) development of DGEME’s sector planning capacity and IT systems, and (d) the development and implementation of tariff reforms may appear overambitious. 84. If it is not explicitly addressed, the lack of provisions for IDA-funded warrantee contracts and maintenance services to outlast a project’s closing date can become a major threat to project sustainability. The apparent lack of options for the World Bank to finance long-term warrantee contracts or maintenance services beyond a project’s closing date is a major limitation that has to be addressed appropriately during project preparation. With warrantees of MIS and related hardware typically lasting 5–10 years, projects must either be extended accordingly or governments have to credibly ringfence the necessary funds in their budgets. For typical FCV environments such as the Comoros, the latter is clearly not an option due to the perceived risk of Government default on contract payments. 85. Most FCV countries require teams to carefully and pragmatically consider the political economy implications of the projects’ proposed institutional arrangements and budget allocations. It was within this context, and to accommodate the client’s desire to maximize the GoC’s budget control, that the project team agreed to enhance the allocation for Component 2 of the project and include funding for a ministry-appointed coordinator. Although the implementation of Component 2 was ultimately unsuccessful, the modification of institutional arrangements and resulting budget reallocation was a necessary compromise to overcome critical bottlenecks during negotiations and secure the needed client ownership for implementation. 86. The FCV context may require teams to consider alternative procurement arrangements. Even for projects such as the ESRP, which exclusively included small contracts and a limited number of goods and service to be procured, typical procurement arrangements including the reliance on a locally recruited PIU may not be viable and prone to delays and misprocurement. In the case of the ESRP, the team should in retrospect have considered the use of internationally recruited fiduciary experts, a tender agent, or the United Nations Office for Project Services to support at least the most critical procurement and financial management activities. Page 29 of 40 The World Bank KM — Electricity Sector Recovery Project (P131659) 87. Effective South-South knowledge sharing can help teams dramatically reduce the lead time needed to develop the tailored services and solutions and find the often highly specific expertise needed to support our clients. Regarding the ESRP, lessons learned and ongoing cross-support from the World Bank’s Haiti Electricity Loss Reduction Project (P112164) was instrumental for developing the activities of Component 1 and identifying the most suitable international experts to lead their implementation. 88. In an FCV context, the intervention of resident experts can make the difference between project success and failure. However, their intervention must be well prepared and carefully managed as numerous and complex challenges have to be addressed. These include the definition of a robust executive mandate including decision-making authority and detailed KPIs, the identification of suitable candidates, the recruitment and embedding of experts with the client organization, and robust provisions for expert retention and effective knowledge transfer. Teams tend to underestimate the complexity of these challenges while overestimating the capacity of the client’s PIUs to address these, and the ESRP was no exception. For the ESRP, the added value of the resident experts’ intervention was much reduced by the categorical refusal of the GoC to grant experts more than an advisory mandate. Moreover, the lack of a proper managerial mandate made the intervention of experts much more vulnerable to the continuous change of MA-MWE’s senior management team. Regarding the lengthy recruitment efforts, the ESRP team, after numerous unsuccessful attempts to hire financial and technical resident experts through the PIU, decided to use the services of a specialized recruitment agency. This decision, though it created significant additional cost, was effective in identifying candidates and negotiating contracts much more rapidly than previous efforts by the local PIU. Moreover, the engagement of a recruitment agency was also more effective in retaining the needed experts or quickly exchanging them if needed. . Page 30 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: The Project Development Objective is to contribute to the improvement in the electricity sector's commercial and financial performance. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Collection rate of MA-MWE Percentage 55.00 70.00 70.00 77.40 improved (sliding 12 mts average) 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Overachieved by significant margin Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Electricity losses per year in Percentage 45.00 37.50 37.50 36.80 the project area (sliding 12 mts average) 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Electricity losses per year in Percentage 18.00 18.00 18.00 18.00 the project area- Technical (sliding 12 mts average) 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Page 31 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) Electricity losses per year in Percentage 27.00 19.50 19.50 18.60 the project area- Non- Technical 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Partially achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 200000.00 200000.00 354613.00 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Female beneficiaries Percentage 50.00 50.00 50.00 50.00 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Overachieved by significant margin A.2 Intermediate Results Indicators Component: Component 1 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Intermediate indicator for Text No Commercial and Commercial and Implemented Component 1: Commercial Financial Recovery Financial Recovery and Financial Recovery Plan Plan implemented. Plan implemented. implemented. 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Page 32 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) Comments (achievements against targets): Fully achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Intermediate indicator for Number 0.00 1.00 1.00 0.00 Component 1: Fuel metering equipment 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Dropped during restructuring; only pre-paid meters were installed. The results related to pre-paid meters were captured in separate indicator. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Intermediate indicator for Number 0.00 4000.00 4000.00 4000.00 Component 1: Installation of prepaid meters 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Fully achieved; added during restructuring Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Intermediate indicator for Text No Yes Yes Yes Component 1: Commercial and Financial Software 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 procured, installed and operational Page 33 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) Comments (achievements against targets): Fully achieved; Launch of CMS for end of August 2017 Component: Component 2 Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Intermediate indicator for Text No Both completed Subsidies study Subsidies study Component 2: Subsidies and completed completed Tariff Study delivered 05-Sep-2013 30-Apr-2017 30-Apr-2018 30-Apr-2018 Comments (achievements against targets): Achieved; Tariff study dropped during restructuring Page 34 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) B. KEY OUTPUTS BY COMPONENT The PDO is to contribute to the improvement in the electricity sector's commercial and financial performance. 1. Collection rate of MA-MWE improved (Target: 70 percent) Outcome Indicators 2. Electricity losses per year in the project area (Target: 37.5 percent) 3. Direct project beneficiaries (Target: 200,000) Intermediate indicators for Component 1: 1. Commercial and Financial Recovery Plan implemented (Target: Yes) 2. Fuel metering equipment installed (1) Intermediate Results Indicators 3. Prepaid meters installed (4,000) 4. Commercial and Financial Software procured, installed, and operational (Target: Yes) Intermediate indicators for Component 2: 1. Subsidies and Tariff Study delivered (Target: Yes) Outputs under Component 1: 1. Commercial and Financial Recovery Plan adopted and under implementation 2. 4,000 Pre-paid meters installed Key Outputs by Component 3. Customer Management System procured, installed, and operational in 2018 4. Numerous changes to the institutional setup and operational practices of MA-MWE. Outputs under Component 2: 1. Cross arrears/subsidy study completed (Tariff study dropped during restructuring) Page 35 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Mustafa Zakir Hussain Task Team Leader Fabrice Bertholet Team Member Alexandra Planas Team Member Tamou Moussa Team Member Lu Ha Team Member Antoissi Said Ali Said Team Member Arnaud Braud Team Member Helene Bertraud Lawyer Wolfgang Chadab Disbursement Joseph Byamugisha Financial Management Sylvain Auguste Rambeloson Procurement Specialist Supervision/ICR Jan Friedrich Kappen Task Team Leader Sylvain Auguste Rambeloson Procurement Specialist Maharavo Harimandimby Ramarotahiantsoa Financial Management Specialist Paul-Jean Feno Environmental Safeguards Specialist Andrianjaka Rado Razafimandimby Social Safeguards Specialist Tamou Mohamed Moussa Team Member Leonard Ewang Ngumbah Wolloh Team Member Ewa Katarzyna Klimowicz Team Member Joern Huenteler Team Member Paulette Castel Team Member B. STAFF TIME AND COST Page 36 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 3.475 25,570.84 FY13 18.413 188,189.06 FY14 4.013 30,316.71 Total 25.90 244,076.61 Supervision/ICR FY14 3.300 99,115.59 FY15 9.300 106,752.12 FY16 10.737 145,334.61 FY17 18.014 208,910.12 FY18 5.887 101,312.98 FY19 .793 4,611.40 Total 48.03 666,036.82 Page 37 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) ANNEX 3. PROJECT COST BY COMPONENT Percentage of Amount at Approval Actual at Project Closing Components Approval (US$, millions) (US$, millions) (US$, millions) Component 1: Assistance to the 0.00 4.15 0.00 electricity state-owned enterprises’ commercial and financial performance recovery Component 2: Electricity sector 0.00 0.10 0.00 governance Component 3: Project management 0.00 0.75 0.00 Total 0.00 5.00 0.00 Page 38 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) ANNEX 4. EFFICIENCY ANALYSIS 1. Analytical framework. Project efficiency is assessed through economic cost-benefit analysis. (a) Economic cost. In line with the appraisal-stage economic analysis, the economic cost of the project is assumed to be equal to the project cost (excluding taxes and customs duties) plus additional expenses equivalent to 50 percent of project costs. (b) Economic benefit. In line with the appraisal-stage economic analysis, the economic benefit is assumed to be the marginal economic value of reduced electricity waste as a result of the project activities. In the Comoros, widespread fraud and nonpayment for electricity supply is resulting in inefficient use of electricity given the limited incentives to conserve (for example, for public lighting and private lighting, which constitutes a significant share of total demand and which was inefficient). Widespread load shedding and suppressed demand for a large proportion of users coincide with wasteful use by others. In line with the appraisal- stage economic analysis, the analysis assumes conservatively that out of the change in the collection rate, two-thirds were purely financial transfer from existing consumers to MA- MWE (electricity that is still consumed but now paid) and one-third was the result of more efficient use of energy (that is, consumers reduced their consumption as the project’s impact on revenues collection raises users’ actual price paid per kWh and incentivized more efficient use of electricity). Not measured by the analysis is the additional benefit from the project’s contribution to a significantly higher power supply by enabling MA-MWE to acquire larger volumes of fuel for production. 2. Assumptions. The main assumptions of the efficiency analysis are summarized in Error! Reference source not found.. Table 4.1. Main Assumptions for the Economic Analysis Variable Value Geographical scope MA-MWE’s distribution grid coverage (Mohéli and Grand Comore) Base year 2014 Time horizon / project 20 years lifetime Discount rate 12 percent (as in appraisal-stage analysis)/6 percent Disbursement in year i, multiplied by additional expense factor to account for Economic cost in year i estimated additional expenses (same as in appraisal-stage analysis) Additional expense factor 1.5 (same as in appraisal-stage analysis) Electricity production year i × Improvement in collection rate in year i compared Economic benefit in year i to base year (%) × (1/3) × Marginal economic cost of electricity (US$ per kWh) (same as in appraisal-stage analysis) Electricity production per Actual data for 2014–17; no growth assumed from 2018 onwards as a year conservative estimate Page 39 of 40 The World Bank KM-Electricity Sect. Recovery Project (P131659) Variable Value Marginal economic cost of US$0.23 per kWh (Trimble et al. 2016)14 electricity (US$ per kWh) 3. Results. The project economic efficiency exceeded expectations, with an EIRR of 18 percent and an NPV of US$1.2 million at 12 percent discount rate (US$3.8 million at 6 percent discount rate). This compares to an EIRR estimate at appraisal of 16 percent and an NPV of US$0.6 million at 12 percent discount rate. The results are summarized in Error! Reference source not found.. Table 4.2. Key Results of Economic Efficiency Analysis at ICR and at Appraisal Result Unit At ICR At Appraisal NPV (12% discount rate, as in appraisal-stage analysis) US$ million 1.2 0.6 NPV (6% discount rate) US$ million 3.8 1.4 EIRR % 18 16 4. Threshold values. The project would retain a positive NPV as long as, ceteris paribus, (a) The additional expense factor is below 2.5 (1.5 assumed in ICR and appraisal-stage analysis); (b) The marginal economic cost of electricity during 2018–30 is US$0.15 per kWh or above (compared to US$0.23 per kWh in 2014); and (c) Electricity production per year during 2018–30 is 34 GWh or more (compared to 69 GWh in 2017). 14 Trimble, Christopher Philip, Masami Kojima, Ines Perez Arroyo, and Farah Mohammadzadeh. 2016. “Financial Viability of Electricity Sectors in Sub-Saharan Africa: Quasi-fiscal Deficits and Hidden Costs (English).” Policy research working paper WPS 7788, World Bank Group, Washington, DC. http://documents.worldbank.org/curated/en/182071470748085038/Financial- viability-of-electricity-sectors-in-Sub-Saharan-Africa-quasi-fiscal-deficits-and-hidden-costs. Page 40 of 40