Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Report Number : ICRR0021435 1. Project Data Project ID Project Name Rwanda Electricity Access Scale-up P111567 Proj. Country Practice Area(Lead) Additional Financing Rwanda Energy & Extractives P126483,P126489 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-46510,IDA-51930 31-Jan-2014 124,496,525.55 Bank Approval Date Closing Date (Actual) 15-Oct-2009 30-Mar-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 70,000,000.00 0.00 Revised Commitment 130,000,000.00 0.00 Actual 124,496,525.55 0.00 Prepared by Reviewed by ICR Review Coordinator Group Victoria Alexeeva Fernando Manibog Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The project development objective was “to improve access to reliable and cost-effective electricity services for households and priority public institutions” (Financing Agreement, page 6; Project Appraisal Document, page 8). b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 19-Feb-2013 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components 1. National Grid Rollout (appraised at US$68 million, of which IDA US$58 million and OPEC Fund US$10 million; IDA additional financing (AF) of US$45 million; actual US$105.19 million). The component targeted areas identified in the National Electricity Access Program (NEAP) access prospectus to be suitable for grid connection following a least cost prioritization model. The investment program had the following subcomponents: (i) Grid intensification in urban and peri-urban areas to finance a grid intensification program for already-connected urban areas; the target was to increase the number of consumers from 110,000 to 210,000 households by end-2012. This subcomponent would support Rwanda Electricity Corporation (RECO) to procure the required poles, conductors, transformers, meters, and other connection hardware and local installation services. The project would start with “quick wins,” focusing on the larger cities such as Kigali, Cyangugu, Huye, Kibuye, Giseniy, Gitarama, and Ruhengeri, where the “unserved” potential was believed to be the highest. As the resource envelope permits, the project would then extend to peri-urban areas.(ii) Electrification of new districts and sectors, including medium voltage (MV) extensions and low voltage (LV) reticulations would finance extension of the existing national grid to previously unconnected areas identified in the NEAP prospectus where there were major clusters of potential demand close to existing grid supply points. Connections would be made using appropriate technologies to minimize investment costs and the need for maintenance. Investments would include poles, conductors, transformers, and other needed hardware as well as planning, engineering and installation services. (iii) Backbone MV/HV grid strengthening, improved operational capacity, and service network expansion would address persistent supply issues in the Rwandan national grid, such as supply bottlenecks and low or fluctuating voltage levels, to reach the required transfer capacity and supply quality for the targeted areas in the rollout program. Investments would be focused on continued rehabilitation and capacity improvement of the existing grid infrastructure, started under the IDA-funded Urgent Electricity Rehabilitation Progam, including both upgrading the existing Kigali underground cable network and meeting rehabilitation needs as identified in the RECO energy audit to be completed in 2009. With customer numbers set to triple, support would also be provided to RECO to extend its service network to new areas. This would include establishment of new customer service centers, procurement of equipment and vehicles for improved maintenance, reduced response times to customer complaints, and unforeseen service interruptions. 2. Green Connections (appraised at US$5.2 million (only IDA); IDA AF US$5.0 million; actual US$3.6 million) was to finance a range of activities to improve affordability for the consumers and reduce the need Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) for additional generation resources. The subcomponents were: (i) Efficient lighting to improve affordability and reduce impact on supply, each new customer would receive a package of energy-efficient CFLs (compact fluorescent lights) when the new meter is installed. (ii) Energy efficiency. The program would support incentives to promote the use of solar hot water heaters sold through private dealers. The incentive scheme would introduce a voucher system to provide partial subsidies for households (up to 40 percent of an estimated capital cost of US$1,200 per unit) and a credit scheme to cover part of the remaining investment cost. (iii) Solutions adapted for low-income households. The program would support initiatives to buy down connection costs further for the poorest households, including mainstreaming the use of low-cost “ready boards” for poor households to reduce the need for internal wiring, and to enable households in dwellings previously not eligible for a connection (such as thatched roof houses and temporary constructions) to qualify. (iv) Productive users would support efforts to increase productive use of electricity in the targeted areas, which would stimulate economic growth and job creation and increase affordability. Project support would include spatial mapping and targeting connections to rural market centers and other clusters of commercial activities, such as coffee-washing stations, tea plantations, and sites for agro-processing. It would fund specific outreach and information campaigns to newly connected areas to accelerate the use of electricity for productive purposes, with special focus on Small Medium and Micro Enterprises (SMMEs) and local artisans. 3. Technical Assistance, Capacity Strengthening, and Implementation Support (appraised at US$6.8 million (only IDA); IDA AF US$10 million; actual US$25.86 million). This component was designed to finance technical assistance and capacity building to ensure adequate management of the NEAP and Sector-Wide Approach (SWAp). The program focused on three main areas of support: (i) RECO Institutional Strengthening and Program implementation; (ii) Development of a SWAp for the Energy sector; (iii) technical assistance to develop strong sector institutions and strategies. Revisions to components during project implementation: • At the time of additional financing approved in February 2013, activities under all three components were scaled up (see sub-section (e) below for detail). • During project restructuring in May 2016, Component 2- Green Connections was dropped. Procurement of solar water heaters (SWH) under sub-component (ii) was delayed due to slow decision making related to procurement decisions and lack of understanding of the World Bank procurement procedures (ICR, para 98). As the Project Restructuring Paper (PRP, 2016) specifies the final draft bidding documents for the design, supply, installment, and commissioning of solar water heaters were submitted to the Bank in May 2015 (one year prior to project closure), and as there was not sufficient time to complete this activity, it was agreed with the Government that the activity would be dropped and the undisbursed funds (US$8.17 million equivalent) would be reallocated for activities under Component 1. Regarding the progress towards achievement of the target on compact fluorescent light bulbs (CFLs), the Government policy did not allow imports of incandescent bulbs, implying that all new households connected to the grid would use CFLs (the utility had a parallel project distributing compact fluorescent lamps for all new customer connections). As per PRP (2016), due to this Government policy, the target of 90 percent had already been exceeded (reaching 100 percent)-without project intervention. The ICR Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) (para 100) also adds that the project had procured sufficient numbers of ready boards that were in stock and did not require more during the lifetime of the project. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated total project cost of US$80 million was increased to US$140.17 million through additional financing to scale up project activities. The actual total project cost was US$134.5 million. Financing: The project was financed through an original IDA credit in the amount of US$70 million, supplemented by an additional financing (AF) of US$60 million approved on February 19, 2013 to scale up the ongoing activities, to a total US$130 million. The IDA credits were disbursed at US$124.5 million at project closure. A parallel co-financing loan of US$10 million was provided as appraised by the OPEC Fund for International Development (OFID) for the national grid rollout under component A. Borrower contribution. The project did not plan for the government contribution but, as the project team subsequently clarified, it introduced a principle (through a covenant) of the Government contributing to the electricity access program through reinvesting consumer connection fee contributions. As reported by the project team, the Government continued to abide by this covenant. Dates: The project closing date was extended by about four years from January 31, 2014 to March 30, 2018. Overall, the project was restructured four times as follows: 1. In February 2013, following fast progress on achieving the access targets, additional financing (AF) in the amount of US$60 million was approved to scale up the activities under all three components. The AF would support connecting an additional 48,000 households to the electricity grid as part of Phase 2 of the Rwanda Electricity Access Rollout Program (EARP). In addition, the AF was to support scaling up of "green connections" associated with energy efficient solutions; increased uptake of solar products such as solar water heaters and solar photo-voltaic systems; low-cost products such as ready-boards to make it affordable for low-income households to get connected to the electricity grid; and productive uses of electricity to promote increased use of electricity by SMEs (Project Paper on a Proposed Additional Credit, 2013). The project closing date was revised from January 31, 2014 to June 30, 2016 in line with additional activities. The outcome targets were revised upwards. 2 . In May 2016, the following changes were made: (a) Component 2 was dropped from the project (see reasons above in section 2d). (b) The implementation agency was changed from the Electricity, Water, and Sanitation Authority (EWSA) to the Electricity Development Company Limited (EDCL). (c) Funds were reallocated between the disbursement categories. (d) Outcome indicators related to solar water heaters (SWHs) installed and estimated energy reduction from energy efficient investments were Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) dropped since related activities under Component 2 were cancelled. (ii) The indicator ‘Households connected to electricity’ was renamed ‘Households connected to electricity under the Government EARP program’, to reflect the relationship between the indicator and the Government electrification program. (iii) The core indicator ‘People provided with access to electricity by household connections’ was added to the Results Framework. (e) The project closing date was extended to June 30, 2017. 3 . In May 2017, the project was restructured to extend the closing date for an additional five months to November 30, 2017. This was motivated by the availability of funds (US$3 million) released from cancellation of two nonperforming contracts of an engineering, procurement, and construction (EPC) company terminated in March 2017 (Restructuring Paper, May 2017). 4 . In November 2017, the project was restructured to extend the closing date by an additional four months to March 30, 2018, to allow completion of a ‘purchase of trucks’ contract due to delays in the approval process. 3. Relevance of Objectives Rationale At appraisal in 2009, only 6 percent of Rwandan households (110,000 customers) had access to electricity from the grid, despite the high density of population (estimated at about 9 million and more than 320 individuals per km2). The project was to support the Rwanda Electricity Access Rollout Program (EARP-the Program) that presented a five-year plan (2009-2013) under Phase 1 for extending electricity access to achieve Rwanda’s targets for the electricity sector set under the Economic Development and Poverty Reduction Strategy (EDPRS). The total number of electricity connections to be increased was from around 110,000 connections to 350,000 connections, tripling access to about 16 percent of households by 2013 and at least 50 percent of identified public institutions in health, education, and local administration. The project was to be delivered through the Rwanda Electricity Sector Access Program Investment Prospectus (IP-2009) funded by the Energy Sector Management Assistance Program (ESMAP) that used geospatial prioritization for grid compatible areas, prioritizing new grid connections depending on demographic and cost factors, including proximity to the existing grid, inter-household distances, social infrastructure, access to road networks, and the ability of consumers to afford electricity. Together with contributions pledged by the Government of Rwanda (GoR) and the Rwanda Electricity Corporation, contributions from customers, and major donors such as African Development Bank (AfDB), Arab Bank for Economic Development in Africa (BADEA), Belgium, European Union (EU), Netherlands, Japan, OPEC Fund for International Development (OFID), Saudi Fund, the World Bank, and others, a total of US$357 million was mobilized for the Program (PAD, Annex 1). The EARP was set up under a sector-wide approach (SWAp) concept that was initiated by Rwanda in 2008 in the energy sector to increase aid effectiveness by reducing fragmentation of donor aid flows in the sector. The World Bank was a leading donor and the main contributor to the Program. Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) The project was scaled up in 2013 as part of Phase 2 of the EARP to support increasing the national access rate to at least 35% by 2017 (Project Paper on a Proposed Additional Credit, 2013). The project development objectives remained aligned with the GoR’s priorities in the energy sector and its long-term development strategy-Vision 2020. It also remained relevant to the most recent WBG Country Partnership Strategy (CPS) for Rwanda (FY2014-2018), which prioritized increased generation and access to electricity under its pillar of accelerating economic growth (CPS, Annex 1: Results Matrix). It emphasized that increased access to electricity was key to the countries’ poverty reduction, economic growth, and broad- based welfare gains, and also highlighted the need for considerable institutional strengthening in the sector. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve access of electricity services to households. Rationale Theory of change: The project’s theory of change was that through investments in the national grid roll-out, support to energy efficiency and affordability measures (e.g., efficient lightning, use of solar water heaters), and capacity strengthening for sector planning and financing, access would increase to reliable and cost- effective electricity services for households and priority public institutions. In particular, the infrastructure investments (including rehabilitation of transformers and installation of meters), complemented by energy efficiency measures (such as developing strategy, incentive structure and strengthening mechanisms for promotion of energy-efficient technologies), institutional development of ministries/regulators and utilities/operators, and developing customer feedback systems and establishing of service standards, would lead to improved sector planning and financing framework, improved customer service orientation of utilities/operators, improved network capacity and efficiency, and increased adoption of energy-efficient technologies/options. These activities were to lead to improved access to reliable and cost- effective electricity services measured through principal outcome indicators focused on the overall increase in connectivity and the decrease in monthly interruptions and average cost per connection. Energy efficiency investments were to lead to load reduction that was to be monitored at the intermediate results level (PAD, Results Chain, Figure A3.1, page 54). The results framework, however, had weak linkages between the project activities, outputs, outcomes and objectives. The substantial support for the sector reform and institutional development activities under the project were not captured at the outcome level. No activities were planned to improve the reliability of electricity services that was specifically stated in the objectives. The ICR (para 51) clearly indicates that the project did not have any direct influence over upstream investment decisions to improve reliability. Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Split Rating: A split rating is not used for the assessment of the achievement of the objective, as the project outcome targets were revised upwards in line with additional investments. Targets for energy efficiency at an intermediate outcome level were dropped with the cancellation of component 2. Outputs • 7,136 km of transmission and distribution lines were expanded under the project from the baseline of 1,800 km in 2008, significantly overachieving the target of 4,200 km (original target was 3,000km). The ICR reports a number of factors that contributed to a faster connection rate, i.e., (i) the project financed the extension of the existing national grid to previously unserved areas identified in the Investment Prospectus (IP-2009) that used a geospatial prioritization for grid compatible areas prioritizing new grid connections depending on demographic and cost factors, including proximity to the existing grid, interhousehold distances, social infrastructure, access to road networks, and the ability of consumers in each planning cell to afford electricity. Once the MV backbone was built, the PIU focused on connecting all users within a 5 km distance of the backbone capturing many households and productive users. (ii) The local contractors installed the LV line drops and house connections on the long MV lines, enabling a faster rollout of the program; and (iii) a new connection policy was introduced in 2017 that allowed connections without an advance payment and deduction of the connection fee from payments for purchase of units of electricity, which alleviated constraints to connections and consumption by poorer households (ICR, page 19). •850 transformers were rehabilitated or replaced, below the revised target of 985 (the original target was 40% of the overall number). Under the institutional component, the following activities were implemented: • Joint Annual Sector Performance Report was approved by SWG, as planned. • A grid electrification study, 'The Electricity Network Planning & Design', was produced, commonly referred to as the SOFRECO Report, and according to the project team, it was a key tool in the implementation of the Government Program (2012-2018). • The following sector-reform studies were prepared: (i) The EWSA Financial Performance Review and Outlook; (ii) A study to establish and advise on the legal framework for new companies to replace EWSA; (iii) A study to undertake the Energy Sector Functional and Organizational restructuring of EWSA; and (iv) A Study on Asset Verification and Separation between the two EWSA successor entities. According to the project team, the above studies became a blueprint for sector horizontal unbundling that led to the creation of Rwanda Energy Group (REG) and its subsidiary companies (Electricity Utility Company Limited Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) – EUCL and Electricity Distribution Company Limited – EUCL), incorporated under progressive Company Law. • The two studies – (i) A study to undertake scoping for Management Information System (MIS) and (ii) The review of proposed strengthening projects for the EWSA HV and MV Networks – provided guidance on capacity building and strengthening of the newly created unbundled electricity utility. According to the project team, these studies also informed project designs of interventions funded by other donors and the Government. • Capacity building was carried out for project staff and staff of the Implementing Entity in various areas of specialization including Finance, Project Management, Procurement, Environmental and Social Safeguards and Construction Management. Outcome Access to electricity services was provided to 275,714 households under the project, as compared to the original target of 100,000 and revised to 148,000. Under the overall Program, electricity access was expanded to 810,923 households. The total EARP contribution to percentage electrification rate during the project lifetime was 28.5%, of which 10% was through the project funds. Overall, at project closure in 2018, the grid electrification rate was 31% in Rwanda. The ICR (page 17) reports that Rwanda’s annualized increase in access (percentage points) between 2010 and 2016 was the highest among the world’s 20 least electrified countries and much higher than the world average (3.28%, compared to the world average of 0.64%). The ICR also reports positive socio-economic outcomes in electrified households that were assessed as part of the impact study carried out between 2014 and 2016. The results showed a decrease in total energy expenditure, increased income, increased asset values, including for rural communities showing a general improvement in quality of life and increase in incomes. Rating High PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Improve access of electricity services to priority public institutions. Rationale Outputs Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) The same as under Objective 1 Outcome The achievement of access expansion to public institutions was measured at the program (EARP) level, as overall country connection rates. The access to priority public institutions improved as follows: schools connection to electricity increased from 21% in 2009 to 77.2% in 2018, slightly below the revised target of 80% (original target was 50%); connection of administrative buildings increased from 38.7% in 2009 to 94.5% in 2018, below the targeted 100% (original target was 52%); connection to electricity of health centers increased from 38.4% to 92.1%, not achieving 100% (original target was 50%). The ICR (page 20) attributes a small shortfall in realizing the targets to lack of funding for the GoR electrification program. While all public infrastructure (health centers, schools, and public administration buildings) within a 5 km distance from the line were connected to the grid, other institutions were not reached beyond it. Overall, according to the ICR (page 20), 100% of the administrative offices in the country were connected to electricity by a combination of grid and off-grid methods at project closure. The impact study 2014-2016 has shown such results as increased time spent in school and improvement in health outcomes. With respect to gender, there was a decrease in mortality rate during childbirth in health centers that had been connected to electricity; increase in the number of females ages 16 and above visiting health facilities; enrollment of girls in schools with electricity; performance of girls in the final examinations improved; and there was increase in the female teaching population. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Improve reliability of electricity services. Rationale Outputs • The same as under Objective 1. Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) •Energy efficiency activities (i.e., compact fluorescent lights, solar water heaters), were cancelled under Component 2 that aimed to lead to load reduction (GWh); the relevant indicator was dropped. Outcome At the time of project closure, the rolling average of monthly interruptions per km of MV line was 61 percent higher than the baseline (ICR, page 23). The rolling average was 0.102 at closure (based on 730 interruptions per month and 7,135 km of MV line, as compared to the baseline of 0.06338 (based on 270 interruptions per month and 4,260 km of MV line). The revised target set was a reduction of 35% (0.0412), while the original target was 25%. This indicator was measured at the Program (EARP) level, relating to the country grid expansion program. The ICR (page 21) notes that the indicator monitored outages on the whole network and could not insulate the effects of the project implementation works from other activities being carried out on the electricity network. The indicator was not well designed, as clarified by the project team, due to data collection limitations at appraisal. At that time, reliability measurement tools based on SAIDI (System Average Interruption Duration Index) and SAIFI (System Average Interruption Frequency Index) were not available. However, in 2015, the utility installed a SCADA system, which allowed the recording and reporting of accurate customer-level measurements starting in 2016. The ICR and project team discussed with and provided IEG with more recent information, consisting of SAIDI and SAIFI data from July 2016 to December 2018. In its April 15, 2019 note, the ICR team indicated that the data "shows that the annual cumulative number and frequency of outages have steadily decreased, clearly demonstrating a continuously improving service reliability trend." The ICR also refers to results from the multi-tier framework energy survey (World Bank 2018), which found that 68.4 percent of the grid-connected households in Rwanda lie within a higher Tier 4 (on a scale of 1-5), with more than 14 disruptions a week or more than 2 hours disruption a week and daily availability of 16–23 hours a day. Data from the survey indicate that only 5.7 percent of the population in Rwanda are constrained to relatively lower tiers due to low reliability, i.e., limited evening availability (less than 4 hours) and quality (such as inadequate voltage) (ICR, para 53). Given the trend that is evident from the latest available data, the achievement of Objective 3 is rated substantial, following IEG's guidelines (page 35) that the project almost fully achieved its objective or intended outcomes, or is likely to do so. Rating Substantial PHREVDELTBL PHEFFICACYTBL Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Objective 4 Objective Improve cost-effectiveness of electricity services. Rationale Outputs • The same as under Objective 1. • 60,000 low cost ready boards were delivered and installed to low-income households reducing the need for internal wiring. The rest of the activities were cancelled under Component 2 that aimed to improve affordability for the consumers and reduce the need for additional generation resources. While energy-efficient CFLs (compact fluorescent lights) were delivered under the Government Program, without project support, the support was cancelled for an incentive scheme to promote the use of solar water heaters sold through private dealers by providing partial subsidies for households and a credit scheme to cover part of the remaining investment cost. In addition, the support for productive users was also cancelled. The project team subsequently clarified that the project did not monitor the productive users as initially intended, but all customers, including productive users, were connected to the grid within a 5 km radius. Outcome At project closing, the average cost per connection was US$ 411.12, around 60 percent lower than expected cost (target was US$ 1,090). The indicator was a program-level (EARP) indicator. During the implementation of the project between 2009 and 2012, unit connection costs ranged from US$300 (for fill-in connections) to US$840. At the time of the AF in 2012, the target was set at +9 percent from the original target (US$1,090). The scale-up of electrification implied considerable reductions in connection costs. In addition to the expansion of connections that drove connection costs down, the ICR (para 56) says that competitive procurement processes and using concrete poles and wooden poles instead of steel lattice poles and single wire earth return (SWER) technology on some of the MV lines helped cost reduction. Rating High PHREVDELTBL PHOVRLEFFRATTBL Rationale The project significantly overachieved its access targets for households and was close to the access target for public institutions. The most recent data (2016-2018) on monthly interruptions per km of MV line -- based on more accurate SAIDI and SAIFI measurements made possible by the installation of a SCADA system -- show an improving service reliability trend. The average cost per connection dropped dramatically during project Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) implementation, due to the expansion of connections. In light of these positive achievements, the overall outcome rating is rated substantial. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic analysis At appraisal, the proposed financing was expected to fund about 100,000 grid connections during a five-year period (2009–2013). An economic net present value (ENPV) was estimated to be US$273 million at a discount rate of 12 percent and the economic internal rate of return (EIRR) of the project was 40 percent. The economic benefits were derived from the consumer surplus from electricity consumption estimated for newly connecting households. The consumer surplus was estimated at US$0.46 per kWh for households consuming less than 20 kWh per month and US$0.26 per kWh for households consuming above 20 kWh per month. For new commercial customers, the analysis used a consumer surplus of US$0.27 per kWh, derived from the cost of backup self-generation. In 2013, the World Bank approved AF US$60 million to support the connection of an additional 48,000 households to the electricity grid as part of the second phase of the EARP. The EIRR estimated at this stage was 59 percent and the ENPV at a discount rate of 10 percent was US$75 million. The AF was also designed to support scaling up of ‘green connections’ associated with energy efficiency solutions, including uptake of solar products such as SWHs and CFLs. The economic benefits estimated during the AF were derived from (a) benefits from new household connections, (b) savings for households derived from the use of CFLs, and (c) savings from installing SWHs in public and commercial buildings. At project closure, the economic analysis estimated an EIRR of 55 percent and the NPV, at a discount rate of 10 percent, at US$112 million. The ICR notes (Annex 4) that the excel model used to calculate the economic analysis during appraisal was not available, thus, the ex-post economic analysis was prepared using various sources as well as the information available from the two existing economic analyses (the one prepared for appraisal and for the AF). The analysis identified the main benefits of the project to be derived from (a) increased electrification (households and institutions), (b) savings in electricity connection costs, and (c) savings from green connections (increased use of CFLs and SWHs). However, the ICR (Annex 4, page 63) erroneously estimates savings derived from the use of (a) efficient CFLs (131 kWh per household per year) and (b) SWHs (3,240 kWh per building per year). It reports that " all the connected households had received CFLs and 2,837 SWHs had been delivered until 2016 (after that the component was discontinued)"; no related activites were delivered under the project. While the benefits primarily accrued from the increased electrification among households, the estimated EIRR should be lower due to nonattainment of benefits under Component 2 that was cancelled. Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Operational/ administrative efficiency While overall the project was scaled up adding another two years to implementation, other closing date extensions were due to delays in effectiveness, weaknesses in design, and procurement issues that resulted in cancellation of the whole component. The ICR also attributes some delays to a slow decision-making process at the PIU. In addition, the efficient use of US$26 million is questionable under the institutional building component that has no apparent benefits reported. On balance, the overall efficiency rating is substantial in consideration of a high economic rate of return from grid roll-out investments, albeit marginally due to significant operational and administrative inefficiencies. * In Table a below, (a) the ex-post EIRR is not reported due erroneous estimation of savings derived from energy efficiency activities (CFLs and SWHs) under Component 2, which were not implemented under the project. It would be lower than the reported 55 percent, though not significantly. (b) The coverage/scope for the ERR estimated at appraisal is the whole Program, therefore indicated as not applicable. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  40.00 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the project development objectives is rated high. Efficacy is rated substantial. The project significantly overachieved its access targets for households and was close to the access target for public institutions. The monthly interruptions per km of MV line -- based on the most recent and more accurate SAIDI and SAIFI measurements -- shows the likely trend of improvements in the reliability of electricity services. The average cost per connection dropped dramatically during project implementation due to the expansion of connections. Efficiency is rated substantial in consideration of a high economic rate of return from the grid roll- out investments and operational inefficiencies that led to cancellation of a full component. Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) The overall project outcome rating is satisfactory, based on high relevance, substantial efficacy, and substantial efficiency. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The main risks are: • Funding gaps to cater to increased electricity access to households and priority public institutions. The required estimated funding to meet the ESSP (2018/2019–2023/2024) targets are approximately US$1 billion, including a combination of funding from the GoR, development partners, and private sector. The risk that this funding is not available in its entirety to meet the targets of the current ESSP/NST poses a risk to PDO outcomes not being maintained in the related period. The GoR has included the sector goals in the draft NST1, which is an indication that the GoR would focus on raising funds to meet sector goals. • Decrease in system quality and reliability. The reliability of the electricity system is affected by a combination of sources of electricity, age and maintenance status of the transmission and distribution network, and quality of network operation. The GoR is aware of the risks and consequences of low reliability and has initiated several projects to strengthen the network. • Affordability related to high costs of electricity services. The cost of supply of electricity service in Rwanda remains high, at approximately US¢30 per kWh; however, the GoR is focused on addressing this problem in the medium to long term. • Sector financial sustainability and institutional capacity to plan and manage the utility prudently. The Government is aware of the financial and fiscal risks and has initiated a reform program, which is supported by a three-year World Bank Development Policy Operation during FY2018– 2020 (US$125 million each). The reform program includes a comprehensive set of measures to reduce the cost of service, including a transition Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) to fully competitive procurement, least-cost planning of power generation investments, least-cost planning of electrification investments, and utility reforms. 8. Assessment of Bank Performance a. Quality-at-Entry The project design reflected lessons from both the World Bank engagement in Rwanda and the World Bank’s experience in national electrification programs and SWAp framework development. The SWAp design, which led to an MOU between the GoR and leading development partners in the energy sector and setting up of the GoR Program, was instrumental in consolidating the design of the project. The Bank team acknowledged the capacity challenges in the areas of finance, technical, planning, procurement, communication, and stakeholder management that were identified in the IP-2009, and as clarified by the project team, considered those moderate relying on the capacity of the Project Management Directorate with experience in WB project implementation. The project kickoff was delayed due to the low implementation readiness. No substantial or high risks were identified at appraisal (PAD, Table E, pages 17-18), albeit during the project implementation, slow PIU decision-making and procurement (pages 34-35 and 38-39), and related delays resulted in the cancellation of a whole component and the necessity to expend the closing date four times (page 38). The ICR's account on Component 2 points to design shortcomings. As indicated in para 126 of the ICR, "The CFL and SWH activities under Component 2 may not have been properly analyzed at project preparation, leading to failure of the activities to be funded by the project." The utility already had a similar ongoing CFL project. Para 126 of the ICR also notes that it was also not clear from the existing documentation whether adequate analysis was carried out to support the design of the SWH activity involving private sector and financial institutions. These activities were subsequently dropped from the project implementation. In a follow-up discussion with IEG, the ICR team reiterated that the utility had adequate stocks of CFLs and was able to successfully distribute 800,000 CFLs, of which 600,000 were distributed during the project's lifetime. Moreover, GoR adopted a policy of banning imports of all non- CFL lighting accessories, which enhanced the success of 100 percent CFL use in Rwanda. Regarding SWH provision, GoR's decided to finance all SWH subsidies under the Nordic Fund grant financing, while reallocating IDA funds to access-related activities, hence SWH activities were dropped from the project. However, the PDO was still achieved, as follows: (i) SWHs were distributed to 3,459 households and public institutions; (ii) 34 local technicians were trained in the design, installation and maintenance of SWH systems. The program is also supporting IPRC North, a technical school, to promote SWH manufacturing. The ICR team also added that the project was designed to buy down the connection cost for poorer households (e.g., thatched roof dwelling that are not suitable for a connection) by reducing the need for household writing through the provision of ready boards, leading to the procurement and installation of 60,000 ready boards for the project's access activities. Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) Quality-at-Entry Rating Satisfactory b. Quality of supervision The project experienced a delayed start owing to the low implementation readiness and inadequate structure of the PIU. The project was approved in October 2009 and became effective eight months later. The first disbursement was four months after effectiveness, delaying the project start by one year. During project implementation, as reported by the ICR, the World Bank task team continuously monitored the project progress against the targets. The project was restructured four times to change the design (drop a component) due to implementation delays and the progress on these activities by the government. Three of the four task team leaders were resident in the country and accessible to the client to provide necessary hands-on support. With the position of co-chair, the World Bank was also tasked with coordination of the donors to the energy sector and leading sector dialogue on behalf of the donor community. It was also the de facto co- chair of the Sector Working Group (SWG), which also entailed overseeing the activities of the eSWAP secretariat. The SWG was responsible for approval of policy documents in the energy sector and reviewing sector reports. The World Bank task team oversaw the quality of information transmitted to the SWG for review and approval, including information generated from the M&E exercise of the project that reported both project and program results. With four project restructurings, the supervision team could have addressed weaknesses in the project's results framework and M&E design, in particular as Component 3 supported sector unbundling reforms and institutional strengthening that amounted to a total of US$26 million at project closure. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) There were four outcome indicators linked with the project objectives. These were defined to measure the increase in access for households and public institutions (disaggregated by type); improvement in reliability (reduction in monthly interruptions per km of MV measured for the whole network); and cost- effectiveness of electricity services (average cost per connection) (PAD, Annex 3). Under the access indicator, two sub-indicators were monitored: (a) households connected by the Program and (b) households connected under the project; for public institutions, there was a program-level results indicator. The monitoring and evaluation (M&E) system for the project was to be in line with the proposed SWAp framework and M&E system within the National Electricity Access Program (PAD, page 57). There were, however, significant shortcomings in the project's results framework and M&E design, i.e., : • No outcome indicators were defined to capture the institutional building and capacity strengthening activities, particularly for a US$10 million commitment under the related component. • No indicators were defined for various planned activities (efficient lightning, energy efficiency, solutions adapted for low-income households, and productive users) under Component 2 at the outcome level . • All outcome indicators were limited to the results from physical investments into the national grid rollout under Component 1. b. M&E Implementation The project was managed under the SWAp concept. The GoR set up a reporting mechanism through the backward looking joint sector reports and forward looking joint sector reports (FLJSRs) that were presented to the SWG and approved by the GoR. The SWAp setup ensured the participation of all stakeholders in the energy sector by ensuring approval of major policy documentation through the SWGs. An impact evaluation was carried out during the project implementation. The project was part of the IDA16 impact evaluation exercise, where the baseline (2014) and follow-up (2016) exercises were completed. The ICR (page 36) reports that the data on the PDO indicators were collected and reported in a methodologically sound manner, however the overall process of data collection was slow. No revisions were made to improve the project M&E design, in particular with respect to reliability and considering that the cost under the institutional strengthening component supporting sector reforms more than doubled to US$26 million, thus with no accountability for results. c. M&E Utilization Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) The M&E data were used in sector annual forecasts related to electrification; project restructuring, and decision-making on reaching connections further downstream. The ICR reports that the results were also used to inform the setting of targets for the GoR’s Forward Looking Joint Sector Review, through which annual sector targets were set. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was assigned the Environmental Assessment Category B in accordance with the World Bank Operational Policy on Safeguards, due to limited expected adverse environmental and social impacts. The project triggered two safeguards policies on Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12). An Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF) were prepared and disclosed both in country and at the World Bank InfoShop in February 2009. During the preparation of the AF, both the ESMF and RPF were updated and disclosed in Rwanda in November 2012 and at the World Bank InfoShop on November 20, 2012. The framework approach (ESMF and RPF) was adopted because the precise locations of project implementation sites were not known at the time of project preparation. The ICR reports (page 33) that the project was always in compliance during the implementation phase. According to the ICR (page 38), the PIU prepared a report covering the consolidated implementation progress of the eight subproject ESMPs throughout the project duration. This report points out the mitigation measures implemented in response to the negative environmental and social impacts encountered during project implementation. The report also shows that all anticipated negative impacts were short term, site specific, and managed through the implementation of the respective ESMPs. Involuntary Resettlement. One subproject (Rukarara substation funded by OPEC FUND) involved land acquisition and physical relocation. The other subprojects that were distribution lines had no land acquisition implications and thus involved only compensation for trees and crops. All compensation payments were completed by end December 2017. In total, 3,678 households were affected, including 75 households that lost land and 30 households that were physically relocated. The key challenge in the implementation of the RAPs was the delays in compensation because of unavailability of the required cash on time. A Resettlement Completion Report was prepared and shared with the World Bank. The report shows that the project was always in full compliance with rsettleme guidleniues during implementation (ICR, page 38). Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) b. Fiduciary Compliance Procurement. Project procurement activity at project start was performing well and rated ‘satisfactory’; however, from October 2015, the rate was downgraded to ‘Moderately Satisfactory’. The major reasons for the downgrading were (a) the slow movement of the procurement plan owing to a slow decision-making process by the utility top management and (b) high procurement staff turnover. Restructuring of the implementing agency in the middle of the project implementation and subsequent move of the project to a newly formed agency also contributed to delays. A few contracts were not delivered by the time of project closure, and these were moved to under another project (ICR, page 39). The ICR does not discuss it but according to the project restructuring paper, in March 2017, the Project Implementation Unit terminated two contracts of an engineering, procurement, and construction (EPC) company that had declared insolvency (PRP, Nov 2017). Financial management. As reported by the ICR (page 39), the FM performance was rated moderately satisfactory during project implementation mainly because of (a) qualified audit opinions and (b) delayed adequate FM staffing, which affected the capacity to effectively monitor FM-related issues on time. The ICR does not discuss if the issues were adequately addressed. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- There were shortcomings in Quality of M&E Substantial Modest the M&E design that were not fully addressed. Quality of ICR Substantial --- 12. Lessons Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) IEG selected three lessons from the ICR, with some modification: • A combination of grid and off-grid solutions based on geographic information system mapping techniques helps accelerate electricity access expansion. Rwanda was one of the first countries to prepare and implement a nationwide electrification program combining grid and off-grid means, based on a systematic and least-cost plan aided by geographic information system mapping techniques that combine technical, economic, demographic, and demand and supply data. The objective of this prioritization was to ensure that the program maximizes benefits of electrification, while minimizing program costs. The prioritization took advantage of Rwanda being a small country with high population density, making potential rollout faster than larger low densely populated countries. • Strong Government ownership of projects, and projects that are aligned with national priorities are in a strong position to succeed from the onset. When governments own the access agenda and incorporate it in their growth strategies, the access expansion agenda is most likely to be successful. The GoR participated in the design and preparation of the national program and geospatial plan, drew up clear results frameworks, and established M&E accountability under the overall institutional setup. • Well coordinated donor support can greatly enhance results from an inclusive sector effort, and is of particular value in a post-conflict situation. This energy sector project is an example of how development partners can come together in a constructive manner in a post-conflict country. The SWAp structure in Rwanda was built upon existing institutions and assigns clear responsibilities to the different parties in the sector working group, including government institutions, development partners, private sector, and other energy sector stakeholders. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is outcome-oriented and written in line with the guidelines. It provides an extensive narrative on Rwanda's electrification program and the project's implementation experience and results. Lessons are Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Rwanda Electricity Access Scale-up Proj. (P111567) experience-based. The ICR would have benefitted from streamlining the text and being less defensive in its narrative. A lot of issues were attributed to factors beyond the project control. While the efficacy section focuses on outcomes, the ICR should be still clear what outputs were delivered to achieve those outcomes. Not much is said regarding the institutional strengthening activities and related results, despite the US$26 million spent under the capacity building and implementation support component. More explanation is needed with regard to procurement issues and unclean audits. The ICR (Annex 4, page 63) factors calculations into the ex-post ERR results estimating savings derived from the use of (a) efficient CFLs (131 kWh per household per year) and (b) SWHs (3,240 kWh per building per year). However, the ICR does not report on the progress and deliverables under these activities before the component was discontinued in 2016. Editing mistakes include: (a) the ICR refers to CAS that should be CPS; (b) discrepancies in numbers, i.e., Table 1 on page 11 indicates actual cost for component B as US$3.6 million, while Annex 3 shows it as US$0.028 million. (c) approval date of AF inconsistently shows December 2012 on page 13 and February 2013 on page 1. (d) Results chain mixes up the outputs and outcomes, i.e, under outputs on page 11 it lists a number of households/institutions with access, average cost of connections, etc; and under intermediate outcomes, it lists construction of MV distribution network, rehabilitation, etc. Overall, the ICR Quality is rated substantial albeit on a margin. a. Quality of ICR Rating Substantial Page 21 of 21