Policy Research Working Paper 9842 Impacts of Energy Efficiency Projects in Developing Countries Evidence from a Spatial Difference-in-Differences Analysis in Malawi Dominik Naeher Raghavan Narayanan Virginia Ziulu Independent Evaluation Group November 2021 Policy Research Working Paper 9842 Abstract Spatial difference-in-differences analysis is used to study the and a switch from traditional fuels to electricity as the main impacts of a large-scale development intervention aimed source of energy for lighting (but not for cooking). At the at improving energy efficiency in Malawi. The estimation same time, there is no evidence that the intervention caused strategy takes advantage of the geographical variation in households to pay more for electricity. The results are con- the implementation of different project components and sistent with an emerging view in the literature that there is based on a combination of remote-sensing (satellite) data are synergies between energy efficiency and energy access, and national household survey data. The results suggest that especially in places where the bottleneck to wider electricity a combination of demand-side and supply-side interven- access is limited electricity generation capacity rather than tions was associated with a statistically significant increase the cost of connecting more clients to the grid. in electricity access, a decrease in the frequency of blackouts, This paper is a product of the Independent Evaluation Group. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at rnarayanan1@ifc.org, dnaeher@worldbank.org, and vziulu@ifc.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Impacts of Energy Efficiency Projects in Developing Countries: Evidence from a Spatial Difference-in-Differences Analysis in Malawi Dominik Naeher Raghavan Narayanan+ Virginia Ziulu‡ Keywords: Energy efficiency, Demand-side management, Electricity access, Developing countries, Malawi JEL Codes: O13, O22, O33, Q41, Q54, Q55  University of Goettingen, Goettingen, Germany (e-mail: dnaeher@uni-goettingen.de). + Independent Evaluation Group, World Bank Group, Washington, DC, United States (e-mail: rnarayanan1@ifc.org). ‡ Independent Evaluation Group, World Bank Group, Washington, DC, United States (e-mail: vziulu@ifc.org). A previous version of this paper was prepared as a background paper for the World Bank’s Independent Evaluation Group’s evaluation study World Bank Group Support to Energy Efficiency: An Independent Evaluation of Demand-side Approaches. We thank Željko Bogetić, Guillermo Hernández González, Giuseppe Iarossi, Jevgenijs Steinbuks, Jos Vaessen, members of the evaluation team, and several World Bank internal referees for valuable comments and suggestions. The views expressed in this paper are solely those of the authors and do not necessarily reflect the views of the World Bank Group. I. Introduction According to recent estimates by the International Energy Agency, global energy demand will rebound to its pre-Covid-19 level sometime between 2023 and 2025, and then grow by 9 percent on average each year until 2030 (IEA 2020). Since the demand in most advanced economies is on a declining trend, almost all this increase will come from developing and emerging market economies, driven by factors such as industrial development, demographic change, and changes in consumer behavior. Given limited resources and the costs associated with increasing energy generation capacity, energy efficiency (EE)1 projects are increasingly seen by policy makers around the world as a critical tool to help meet the rapid growth in energy demand.2 In addition, improvements in EE are often argued to be a key channel to reduce greenhouse gas emissions and achieve the ambitious 2°C stabilization target in the Copenhagen Consensus (World Bank, 2010). In fact, improving EE has been called ``the cheapest, fastest, and most environmentally friendly way to meet a significant portion of the world’s energy needs’’ (Kaygusuz 2012). Moreover, EE projects are generally believed to provide important socio-economic benefits, including reducing countries’ dependence on fossil fuels (thus enhancing energy security), easing infrastructure bottlenecks and impacts of temporary power shortfalls, improving industrial and commercial competitiveness through reduced operating costs (Sarkar and Singh 2010), and facilitating the expansion of energy access (Can et al. 2018). Despite these benefits, achieving significant and sustained gains in EE continues to be a challenge across countries, especially in many developing economies. An important factor in this context is the scarce empirical evidence of the impacts of EE projects in developing economies, whose energy systems and associated challenges often differ fundamentally from those in advanced economies (Urban et al 2007; Ouedraogo 2017). Recent studies on low and middle income countries (LMIC) find preliminary evidence that investments focused on more efficient lighting can be cost-efficient (Iimi et al. 2019; Carranza and Meeks 2021). For example, transitioning from kerosene lighting to solar lighting reduces energy expenditures by 42 percent in rural households in Kenya (Rom and Günther 2019). There is, however, at best mixed evidence in the literature as to whether these interventions cause energy savings,3 and most of the existing evidence is based on small-scale interventions with unknown external validity and potential for scaling up. As noted by Fowlie and Meeks (2021), there is “tremendous value in ex post evaluations of these interventions. Empirical research that objectively evaluates the impacts of these and other programs can inform the course of future policy initiatives aimed at improving energy efficiency.” This paper contributes to addressing this gap in the literature by providing quasi-experimental evidence on the impacts of a large-scale EE project in Malawi which was supported by the World Bank between 2015 and 2018. Our estimation strategy is based on a difference-in-differences (DiD) approach using a combination of remote-sensing data from satellite images and data from national household surveys. The estimation strategy takes advantage of the geographical variation in the implementation of different project components, including variation across districts, subdistricts (cities), and individual households. Various project components were implemented in different areas across the country, 1 In this paper, EE refers to reductions in the amount of energy required to provide the same output or level of service. For example, such reduction can result from the adoption of improved technologies or practices that help to save energy or reduce energy losses. 2 For example, the IEA (2017) estimates that, without the improvements in EE achieved since 2000, the world would have needed to generate 12% more energy in 2016. 3 On the one hand, studies have found that replacing traditional cooking stoves in Senegal with more fuel-efficient versions causes a decrease in firewood consumption by 30 percent (Bensch and Peters 2015), and that upgrading to more efficient cookstoves in Kenya delivered, on average, 40 percent decrease in charcoal expenditures (Berkouwer and Dean 2020). On the other hand, energy savings have been found to be either negligible or negative on investments in LMIC replacing refrigerators, air conditioners, and building insulation with more energy efficient alternatives (Davis et al. 2014; Davis et al. 2018; Ryan 2018). One factor that could help explain these findings is an energy rebound effect; that is, an efficiency-induced reduction in the cost of this service triggering an increase in energy consumption. The existing literature on energy rebound effects focuses mostly on high income countries, but there is incipient research that suggests that rebound effects could be higher in LMIC than in high income countries (Ouyang et al., 2010; Davis et al., 2014). 1 benefitting households living in (or nearby) project areas relatively more than households residing in non- project areas. Given that the allocation of project locations was not determined randomly, a simple (ordinary least squares) regression of household outcomes on the corresponding treatment indicator would clearly suffer from endogeneity, since the treatment indicator would be correlated with (unobservable) factors that also tend to affect the considered outcomes. The DiD approach helps to circumvent this issue, based on additional assumptions which we discuss in detail below.4 The analysis shows that a combination of demand-side and supply-side interventions was associated with a statistically significant increase in electricity access, a decrease in the reported frequency of blackouts, and a switch from traditional fuels to electricity as the main source of energy for lighting (albeit not for cooking).5 These results are based both on evidence obtained from national household survey data and from the fact that nighttime radiance (as measured by satellite images) increased relatively more in grid cells belonging to areas with project components than in non-project areas. There is no evidence that the project caused households to pay more for electricity (e.g., due to excessive rebound effects). The paper contributes to several strands of literature. First, we add to the literature on the impacts of development interventions aimed at improving EE and facilitating the adoption of modern technologies to mitigate climate change. Due to limited data availability on EE outcomes, especially at a nationally representative level, existing EE project evaluations are often based on information collected during the project’s implementation phase or on expert ratings and stakeholder analysis at the end of the project (e.g., SENER and World Bank 2015; Agyarko et al. 2020). The methodology and data sources used in this study offer several benefits over traditional approaches to project evaluation. Specifically, our evidence is obtained from data that are (a) nationally representative, (b) collected independently of the studied project, and (c) able to control for improvements in the considered outcomes that might have occurred even without the project. The second aspect (b) matters, because using data that are unrelated to the evaluated projects minimizes the risk of researcher demand bias (e.g., the risk that beneficiaries of a project provide more favorable responses when they are interviewed by someone who is associated with the project). In addition, the employed DiD method can separate observed changes in the outcomes of beneficiaries that were due to the project from changes that would have also occurred in the absence of the project. Moreover, existing research on EE interventions tends to focus on supply-side solutions (Creutzig et al. 2018) whereas the project studied here represents a combination of demand- and supply-side measures. Our findings also relate to studies that argue that there exists a link between EE and energy access (Can et al. 2018; Dagnachew et al. 2018). Specifically, these studies argue that improvements in EE can act like an increase in energy generation capacity which helps to facilitate the expansion of electricity access, especially in areas where the bottleneck to wider electricity access is not the cost of connecting more clients to the grid, but limited electricity generation capacity. The results we obtain from the project in Malawi are consistent with this view and the existence of synergies between EE and energy access. Finally, the paper contributes to the growing literature that uses remote-sensing data from satellites to measure economic and energy-related outcomes (Henderson et al 2012; Shi et al. 2018; Falcheta et al 2019). Such data have been shown to give rise to valid indicators of energy production, energy consumption, and carbon emissions (see Jasiński 2019 and the survey in Zhu et al. 2019). Our findings are consistent with the view that remote-sensing data may also be useful as a proxy for electricity access and, to some extent, even the reliability of electricity (i.e., a reduction in the frequency of blackouts) in developing countries such as Malawi. 4 The DiD estimator also requires that (a) the intervention’s design permits the identification of both a treatment and a contr ol group (i.e., households that benefited from the program and those that did not benefit), and (b) there is sufficient data available on the outcomes for both groups, and both before and after the implementation of the project. For the project studied in this paper, (a) is fulfilled based on the variation in the geographical locations of project components, and (b) is achieved by using existing, nationally representative household surveys as well as data from satellite images. Both types of data are available for years before and after the implementation of the project. 5 Our data only allow us to study the impact of the project as a whole. Thus, we are unable to disentangle the effects of individual project components (including of components focused on the supply side and those focused on the demand side). 2 The rest of the paper is structured as follows. Section II provides more background on the evaluated project and country context. Section III describes the data. Section IV explains the empirical strategy and different DiD regression models that we estimate. Section V presents the results. Section VI concludes. II. Background on the Project and Country Context6 Malawi’s electricity access rate was below 10 percent of the population prior to the project (in 2011), and mostly concentrated in urban centers. For the 80 percent of the people living in rural areas, access to electricity was less than 1 percent. Almost all (98 percent) of Malawi’s grid-supplied electrical power is generated by six run-of-the-river hydropower projects on the Shire River. The Electricity Supply Corporation of Malawi (ESCOM), a government-owned electric utility, is the main electricity service provider. ESCOM owns and operates all the formal generation capacity in the country and operates the national electricity grid. In 2011, the transmission network comprised 1,250 km of wood pole lines and 815 km of steel tower lines as well as 70 transformers located at 39 substations across the country. ESCOM’s transmission network had suffered from many years of under-investment. Malawi’s power grid was characterized by heavily loaded transmission lines and transformers, resulting in frequent failures especially during the rainy season, and generally poor quality and unreliable supply. Demand for electricity had risen over the years without corresponding investments in systems, causing the system to be greatly strained with overloading, bottlenecks and load shedding. Overall, the existing system was greatly strained and the frequency of both scheduled and unscheduled blackouts or brownouts was increasing, constraining both industrial production and reliable provision of electricity to households. The Malawi Energy Sector Support Project, on which our study focuses, aimed at improving EE in Malawi through a combination of interventions targeted at the supply side and the demand side of electricity. The project encompassed several components, including (i) electricity network strengthening and expansion, (ii) demand-side management and EE measures, and (iii) capacity building and technical assistance. The first component included the construction of new substations and transmission lines, uprating of existing substations, rehabilitation of underground cables, extension of peri-urban networks, and purchase of spare parts for generation (the disbursed amount for this component was USD 49 million). Component (ii) included time-of-use meters and sensitization campaigns, derating of hot water geyser (HWG) element ratings, HWG management system with insulation, installation of solar water heaters, radio control to switch off water heaters to reduce demand at peak times, SMS messages to manage peak load demand, and media campaigns (USD 2 million). Component (iii) provided institutional strengthening and technical assistance to ESCOM and the Ministry of Natural Resources, Energy, and Mining (USD 8 million). Another project component supported feasibility studies for the development of additional generation capacity and transmission lines (USD 10 million). The project was implemented in the period 2015-2018 and comprised 26 project sites located in 20 subdistricts (mainly cities) across 12 of Malawi’s 28 districts.7 Figure 1 illustrates the geographic coverage of the project. Many of the project components focused on urban areas with existing (but deteriorating) power grid infrastructure and were designed to both improve the quality of electricity supply and support higher EE on the demand side. The project’s total amount of USD 69 million was disbursed between the years 2015 and 2018. Since some first effects of the project were expected to take place already during this period, we consider the year 2014 as the last year of the pre-program period (and the years starting with 2015 as the post-program period). 6 The information and numbers in this section are mostly based on the Project Appraisal Document (PAD) and Implementation Completion and Results Report (ICR) (see World Bank 2011; 2019), which are publicly available from the World Bank’s website. 7 The districts with project sites are: Balaka, Blantyre, Karonga, Kasungu, Lilongwe, Mwanza, Mzimba, Neno, Nkhata Bay, Nkhotakota, Ntcheu, and Zomba. 3 Figure 1. Geographic coverage and urban extents of the project locations Notes: The map on the left shows the geographic coverage of the project (red boxes) overlaid against Malawi’s sub-district/city boundaries (administrative level 2). The map in the middle shows the geographic coverage of the project (red boxes) overlaid against Malawi’s district boundaries (administrative level 1). The map on the right shows the geographic coverage of the project (green boxes) overlaid against Malawi’s urban extents (red areas). Urban extents represent the shape and area of urbanized places (defined as places with 5,000 or more inhabitants that are delineated by stable nighttime lights). Source: Base map (including urban extents) was developed by the Center for International Earth Science Information Network (2009). The top layer was added by the authors. III. Data The data used in this study are publicly available from the following sources. Data on the design and geographical locations of the studied projects are obtained from the World Bank’s project documentation, including the Project Appraisal Document (World Bank 2011) and Implementation Completion and Result Reports (World Bank 2019) available from the website of the World Bank Group. In addition, we use household-level data from existing national household surveys and geospatial (grid-level) data from nighttime satellite images from the sources described below. Data on household outcomes and characteristics come from the Living Standards Measurement Study (LSMS). The LSMS is a nationally representative household survey that tracks households over time. The closest years before and after the implementation period of the project (2015 to 2018) for which panel data are available in the LSMS are the years 2013 and 2019. Our DiD analysis thus uses the survey from 2013 as the pre-program baseline and the one from 2019 as the post-program endline. Starting with the full sample of households, we exclude households that moved or split off during this period. This leaves us with a balanced panel of 1,480 households (2,960 observations across 2013 and 2019). Figure 2 illustrated the geographical locations and corresponding treatment status of the households. Several household-level outcomes are considered. The outcome variable Access to Electricity is a dummy that equals 1 if the household reported to have electricity working in their dwelling, and 0 otherwise. The variable Amount Paid for Electricity is the reported amount paid by the household per month for electricity (converted into US dollars using yearly average exchange rates from the World Development Indicator database). The variable Regular Blackouts is a dummy that equals 1 if the household reported to 4 have experienced blackouts at least several times a month over the last 12 months, and 0 otherwise. It has been argued that EE improvements in developing countries may involve switching away from (or reducing the use of) traditional fuels such as kerosene and charcoal, yielding sizeable co-benefits in the form of health improvements. To capture such potential co-benefits, we consider two more outcome variables. The variable Use Electricity for Lighting is a dummy that equals 1 if the household’s main source of lighting fuel was electricity (rather than firewood, grass, paraffin, gas, or others). Analogously, the variable Use Electricity for Cooking is a dummy that equals 1 if the household reported to use electricity as the main source of cooking fuel. Figure 2. Geographic location of treatment households (blue dots) and control households (red dots) in Malawi Notes: The map on the left shows Malawi’s sub-district/city boundaries (administrative level 2) and households’ treatment status according to the indicator Treat (Admin. 2). The map on the right shows Malawi’s district boundaries (administrative level 1) and households’ treatment status according to the indicator Treat (Admin. 1). Source: Authors’ illustration using data from the Living Standards Measurement Study (LSMS). In addition, we make use of the detailed information on household characteristics in the LSMS datasets. The control variables at the household level include gender, age, and educational attainment of the household head, and household size. We also observe characteristics of the household’s dwelling, including the type of construction material used for the dwelling (permanent, semi-permanent or traditional), whether the roof was made of grass, whether the dwelling is owned by the household, and the distance from the dwelling to the nearest road. Data on nighttime average radiance in Malawi were obtained from nightly day/night band (DNB) low light imaging data collected by the NASA8/NOAA9 Visible Infrared Imaging Radiometer Suite (VIIRS). More specifically, we rely on the V.2 annual composite product developed by the Earth Observation Group (EOG). This product is the result of processing nightly observations for each year and applying an initial filter to remove cloudy, sunlit, and moonlit, followed by a subsequent filter to remove extraneous features (such as 8 National Aeronautics and Space Administration. 9 National Oceanic and Atmospheric Administration. 5 biomass burning and aurora). The result is a stable measure of brightness as seen from space (the unit of measurement is nW/cm2/sr). The main advantage of this product lies in the consistent processing of the data for the period 2012-2020, which makes it well suited for comparisons and change detection analysis across multiple years. Figure 3 illustrates the changes in average radiance between 2013 and 2019 in Malawi. These data are observed at the grid level, where the size of each grid cell is 100 square kilometers. A critical issue regarding the use of nighttime lights data is whether this data can be reliably used as proxy for socioeconomic indicators. This is a well-studied issue in the academic literature, where different causal-effect inference methods have been applied to estimate the correlation between nighttime lights data and various socioeconomic indicators such as energy consumption, GDP, electrification, urban extent, or population (Elvidge et al. 1997; Shi et al., 2014; Dugoua et al., 2018). More specifically, studies have shown that nighttime satellite imagery can be used to estimate (i) energy consumption at the subnational level (Letu et al. 2009; Falchetta et al. 2019), (ii) electric power consumption in developing countries (Zhu et al., 2019), and (iii) household electrification at the village level (Dugoua et al., 2018). Figure 3. Nighttime light in Malawi in 2013 (left), 2019 (middle), and change over time (right) Notes: The map on the right shows the difference in radiance between 2013 and 2019 where blue represents more-light, magenta represents less-light, and gray means no change. Source: Authors’ analysis using VIIRS data. IV. Empirical Strategy The strategy for estimating the impacts of the EE intervention on the considered outcome variables relies on the geographical variation in the implementation of different project components. Ideally, we would like to compare the outcomes across two identical groups of households that differ only with respect to whether they benefited from the project or not. Since it is impossible to observe the same household (at the same point in time) both with and without a treatment, counterfactual analysis seeks to identify a suitable comparison (the “control group”) for the beneficiaries (“treatment group”). In our setting, this identification is based on the geographical locations and associated benefits of project components which were distributed across different areas of the country, benefitting primarily those households living in these areas. Specifically, the DiD approach estimates the impact of a project by comparing the difference between 6 before and after the start of the project for households who were able to benefit from the project as compared to households who were unable (or systematically less able) to benefit. The regression equation of the household-level DiD estimator in Malawi can be written as: = 0 + 1 + 2 + 3 ( × ) + + , (1) where is the outcome of household in district at time . is a dummy time variable that equals 1 for the post-program year (2019) and 0 for the pre-program year (2013). is a dummy treatment variable that equals 1 for households residing in districts with project components, and 0 otherwise. The variables and are interacted to estimate the coefficient 3 , which is the main coefficient of interest. is a vector of control variables that are included to account for potential imbalances in characteristics between treated and untreated households that might be correlated with the outcome. Some specifications also include district fixed effects (FE). The estimated coefficients from the model in equation (1) can be interpreted in the following way. The coefficient 1 captures the change in outcome between the pre- and post-program year for households residing outside in areas without project components (the control group). The coefficient 2 captures the difference between households benefitting from the project and households in the control group before the start of the project (pre-program difference). The coefficient of interest is 3 which captures the difference in the change in for the benefitting households and those in the control group. If the assumptions underlying the DiD approach (see below) hold, then 3 can be interpreted as the change in that the households in the treatment group experienced because they benefitted from the project, i.e., the project’s impact on over the considered time period. The DiD strategy relies on the assumption that any pre-existing difference between the households benefitting from the project and those forming the control group would be constant over time in the absence of the project (“parallel trends assumption”). One of the advantages of DiD is that it can account for systematic differences between the treatment and control group that prevailed before the start of the intervention. The existence of such differences is to be expected in the context we study, because the allocation of project locations was not determined randomly. The estimation of the program effect 3 in equation (1) will be robust as long as these differences are constant over time in the absence of the project (i.e., in the absence of the project the households residing in project areas would have featured the same time trend as the control group). There is no statistical test to verify (or reject) this assumption. However, a notion in the literature is to use data on the time before the start of the intervention to visually verify whether trends appear to be parallel. Since the LSMS data are available for Malawi in multiple years (2010, 2013, 2016 and 2019), we are able to follow this approach (see Section V). In addition, it is possible that the parallel trends assumption holds only after conditioning on relevant covariates. Since our data allow us to control for a rich set of household characteristics, it will be sufficient for our identification strategy if trends are parallel when conditioning on these covariates. In addition to estimating equation (1) based on household survey data for two years (pre and post program implementation), we use grid-level data from satellites (which are available annually for the period 2012-2020) to estimate a more general spatial DiD specification of the form: = 0 + ∑ + ∑ + 3 ( × ) + , (2) where is the nighttime radiance in grid cell at time , is a set of dummy variables for each year (i.e., year fixed effects), is a set of dummy variables for each cell (i.e., grid cell fixed effects), and is a treatment indicator that equals 1 for grid cells corresponding to project areas and 0 otherwise. is defined analogous to equation (1) as a dummy time variable that equals 1 for the years after the start of project implementation (2015 in Malawi), and 0 for all earlier years. The variables and 7 are interacted to estimate the coefficient 3 , which is the main coefficient of interest capturing the impact of the project on nighttime radiance at the grid level. Two alternative treatment indicators are considered in the model in equation (2) to capture potential spillover effects of project components on the areas surrounding each project location. Many of the (supply side) project components may have effects on wider areas around the subdistrict (city) where the component was located (e.g., upgrading a power plant or part of a distribution network in a particular city may also affect the quality of electricity supply in the surrounding area outside the city). The most disaggregated level of information we have about project locations is the subdistrict level (administrative level 2). To capture potential spillover effects, we consider two alternative approaches to defining the treatment indicator in the model in equation (2). In the first approach, only those grid cells are coded as "1" for the treatment indicator that correspond to project areas at the most disaggregated level of information about project locations (administrative level 2). We refer to this treatment indicator as Treat(Admin. 2). In the second approach, denoted Treat(Admin. 1), also those grid cells are coded as one for the treatment indicator that fall into the next higher level of administrative division (administrative level 1, i.e., districts) of each project location. For example, if a project component was implemented in city A of district D1, then Treat(Admin. 2) will equal one for the grid cells belonging to city A (and zero otherwise), whereas Treat(Admin. 1) will equal one for all grid cells belonging to district D1 (including the cells belonging to city A). V. Results Figure 4 shows the pre-program trends of the five considered outcome variables in project areas (treatment) and other areas (control). From eyeballing these graphs, trends appear to be approximately parallel for most of the variables prior to the start of the project (in 2015). At the same time, there are large differences between treatment and control group in the level of most of these variables. Table 1 quantifies these pre-program differences, including for other household characteristics, and also reports test statistics for the differences. Columns (1) and (2) in Table 1 show the means of each variable, measured prior to the intervention in 2013, for the households residing in project areas (the treatment group) and those residing in areas without project locations (control group). The last two columns report the t-statistics and p-values for tests on the equality of the means in columns (1) and (2). Unsurprisingly, most of these variables show statistically significant differences between the two groups.10 As discussed in Section IV, these differences are to be expected in the context we study, because the allocation of project locations was not determined randomly. This also suggests that simple ordinary least squares (OLS) regressions of household outcomes on the treatment indicator would suffer from endogeneity, since the treatment indicator would be correlated with factors that also tend to affect the considered outcomes. Therefore, a method such as DiD that can account for the differences in levels (subject to the parallel trends assumption discussed above) is to be preferred. In addition, the regression analysis below will include observable household characteristics as control variables to account for the fact that the parallel trends assumption might only hold after conditioning on relevant covariates.11 10 For example, Table 1 shows that project areas were much more often urban (41.8 compared to 0.5 percent) and featured a larger share of households with access to electricity (18.6 compared to 3.2 percent) and stronger presence of ESCOM (38.7 compared to 21.8 percent) than areas outside the scope of the project. These results are consistent with the project’s stated goals (see Section II) to support the rehabilitation, upgrade, and expansion of priority parts of the existing distribution and transmission system, which were located predominantly in urban areas. 11 If trends are still not parallel when conditioning on covariates, then the expected scenario in the absence of the project according to the project’s PAD (see Section II) will likely work against us. Specifically, the PAD argues that households residing in project areas would have experienced a deterioration in electricity access and quality in the absence of the project (due to the lack of maintenance of the existing infrastructure), while the corresponding trends for the control group would have been mostly flat. If this is the case, then our DiD framework will tend to underestimate the impact of the project. Thus, to the extent that we still find positive and significant results, differences in characteristics between treatment and control group are less of a concern. 8 Figure 4: Pre-program trends in project areas (treatment) and other areas (control) Access to electricity Amount paid for electricity Regular blackouts Use electricity for lighting Use electricity for cooking Notes: The vertical (red) line indicates the start of the program (2015). The values of “Amount paid for electricity”, and “Frequent blackouts” are based on the subset of households that have electricity in their dwelling. Source: Authors’ analysis using data from the Living Standards Measurement Study (LSMS). 9 Table 1. Pre-program differences between households residing in project areas and others, 2013 Mean Difference in Means Variable Control Treatment T-Statistic P-value (1) (2) (3) (4) Access to electricity 0.032 0.186 -9.417 0.000 ESCOM present in village 0.218 0.387 -6.738 0.000 Amount paid for electricity 10.749 14.964 -0.981 0.328 Regular blackouts 0.833 0.945 -1.797 0.074 Use electricity for lighting 0.030 0.186 -9.541 0.000 Use electricity for cooking 0.000 0.043 -5.409 0.000 Urban 0.005 0.418 -21.350 0.000 Distance to road 10.252 5.860 9.093 0.000 Non-permanent dwelling 0.503 0.271 9.419 0.000 Grass roof 0.725 0.454 10.888 0.000 Dwelling owned by household 0.892 0.740 7.526 0.000 Household size 5.163 5.254 -0.760 0.447 Male head 0.723 0.777 -2.383 0.017 Secondary education (head) 0.150 0.328 -8.016 0.000 Age (head) 44.065 43.681 0.468 0.640 Observations (households) 658 822 Notes: Numbers in columns (1) and (2) are raw means. The means in the rows “Amount paid for electricity”, and “Regular blackouts” are based on households that have electricity in their dwelling. Source: Authors’ analysis using data from the Living Standards Meas urement Study (LSMS). Table 2 presents the results of the DiD regressions specified in equation (1) for three different outcome variables capturing access to electricity, amount paid for electricity, and regular blackouts. In columns (1), (4), and (7), the respective outcome variable is regressed on the time indicator (pre or post program), treatment indicator, and the interaction term between the time and treatment indicators. The specifications reported in columns (2), (5), and (8) include additional control variables. The regressions in columns (3), (6), and (9) further add district fixed effects (FE). The interaction term (Post × Treat) is statistically significant in most of the specifications in columns (1) to (3) and (7) to (9) in Table 2, indicating that the project was associated with an increase in electricity access and a decrease in the reported occurrence of regular blackouts. At the same time, the insignificant coefficients of the interaction term in columns (4) to (6) suggest that the project did not cause households who were living in project areas to pay more for their electricity. According to the results in columns (1) to (3), the change in the share of households with access to electricity increased by 5.7 to 5.9 percentage points more in areas with project components than in areas without project components. The coefficient (0.057) of the interaction term (Post × Treat) in column (1) is statistically significant at the 1-percent significance level. Once the control variables are added in column (2), the coefficient increases to 0.059 and remains highly significant. The result is also robust to the inclusion of district fixed effects (as reported in column 3) and to clustering standard errors at the district or stratum level (not reported here).12 12 In the LSMS dataset, stratum is defined based on region and a dummy for rural areas. There are three regions in Malawi (North, Center, South) so that stratum takes six different values. 10 Table 2. Difference-in-difference estimation (household level) between 2013 and 2019 Access to Electricity Amount Paid for Electricity Regular Blackouts (US$) (1) (2) (3) (4) (5) (6) (7) (8) (9) Post × Treat 0.057*** 0.059*** 0.057*** 0.019 1.618 0.779 -0.179 -0.216* -0.193 (0.000) (0.000) (0.000) (0.996) (0.723) (0.884) (0.141) (0.080) (0.184) Post 0.016** 0.008 0.009 -4.177 -6.223 -5.215 0.029 0.040 0.008 (0.010) (0.323) (0.241) (0.295) (0.138) (0.304) (0.805) (0.738) (0.954) Treat 0.152*** -0.017 -0.062 4.215 6.735 -0.470 0.112 0.145 0.003 (0.000) (0.143) (0.117) (0.320) (0.190) (0.957) (0.216) (0.148) (0.983) Urban 0.291*** 0.257*** -2.600 -0.783 -0.039 0.017 (0.000) (0.000) (0.380) (0.790) (0.561) (0.814) Distance to -0.001 -0.002*** 0.302 0.504* -0.003 0.005 Road (0.361) (0.009) (0.135) (0.071) (0.351) (0.135) Grass roof 0.010 0.018 -5.818*** -3.727 0.048 -0.028 (0.457) (0.191) (0.008) (0.201) (0.722) (0.849) Dwelling -0.048** -0.045** 0.808 0.829 0.014 0.022 Owned (0.013) (0.022) (0.576) (0.588) (0.692) (0.562) Male head 0.004 0.004 0.056 0.087 -0.022 0.003 (0.703) (0.763) (0.978) (0.963) (0.609) (0.952) Age (head) 0.000 0.000 0.060 0.043 0.001 0.001 (0.200) (0.143) (0.294) (0.464) (0.670) (0.608) Secondary 0.151*** 0.146*** 2.523 2.610 0.015 -0.011 educ. (head) (0.000) (0.000) (0.102) (0.137) (0.711) (0.800) HH size dummies yes yes yes yes yes yes Material dummies yes yes yes yes yes yes District FE yes yes yes Observations 2,960 2,943 2,943 364 363 363 388 387 387 R-squared (within) 0.038 0.031 0.032 0.026 0.043 0.037 0.093 0.128 0.142 R-squared (overall) 0.076 0.368 0.387 0.034 0.092 0.175 0.040 0.098 0.197 Notes: Difference-in-difference regressions with robust standard errors. * p < 0.10, ** p < 0.05, *** p < 0.01. p-values in parentheses. Full sets of control dummies are included for household size and the type of construction material used for the dwelling (permanent, semi- permanent or traditional). The dependent variable in columns (7) to (9) is a dummy that equals one for households that report to have experienced blackouts at least several times a month over the last 12 months. Source: Authors’ analysis using data from the Living Standards Measurement Study (LSMS). According to the point estimates of the interaction term in columns (7) to (9) of Table 2, the project was associated with a decrease in the share of households reporting regular blackouts of 17.9 to 21.6 percentage points (from the baseline of 94.5 percent in 2013 in treated areas; see Table 1). However, only the coefficient (0.216) in column (8) is statistically significant at the 10-percent significance level (the same holds when standard errors are clustering at the stratum level). One reason for the low (or lack of) significance in the regressions in columns (7) to (9) might be the relatively small number of observations, which is due to the small share of households with access to electricity in our sample (the survey only asked households with electricity in their homes about the frequency of blackouts). Table 3 presents the results of estimating equation (1) for the two outcome variables capturing whether households relied on electricity as the main source of energy for lighting and cooking or not, 11 respectively. According to the results, the number of households that used electricity as the main source of energy for lighting increased by around 4.7 percentage points more in project areas than in areas without project components. The coefficients in columns (1) to (3) are statistically significant at the 1-percent significance level (the coefficients in columns (2) to (3) remain significant at the 10-percent level when standard errors are clustered at the stratum level). This suggest that there may have been co-benefits to the program, including the above-mentioned link between health improvements and a reduction in the use of traditional fuels for lighting. At the same time, the results in columns (4) to (6) indicate that the association between project locations and the use of electricity for cooking was negative. It thus remains unclear whether or to what extent the program contributed to an overall reduction in the use of traditional fuels and switch to electricity in Malawi. Table 3. Difference-in-difference estimation (household level) between 2013 and 2019 (continued) Use Electricity for Lighting Use Electricity for Cooking (1) (2) (3) (4) (5) (6) Post × Treat 0.048*** 0.049*** 0.047*** -0.019** -0.021*** -0.022*** (0.001) (0.001) (0.001) (0.016) (0.009) (0.008) Post 0.023*** 0.017* 0.019** 0.005* 0.007** 0.008** (0.003) (0.061) (0.039) (0.083) (0.027) (0.017) Treat 0.156*** -0.024** -0.085** 0.043*** 0.009* 0.021* (0.000) (0.046) (0.031) (0.000) (0.055) (0.092) Urban 0.299*** 0.266*** 0.059*** 0.056*** (0.000) (0.000) (0.000) (0.000) Distance to -0.001 -0.002** 0.000 -0.000 Road (0.310) (0.011) (0.724) (0.953) Grass roof 0.017 0.025* 0.022*** 0.028*** (0.238) (0.084) (0.001) (0.000) Dwelling -0.055*** -0.051** -0.011 -0.009 Owned (0.005) (0.010) (0.253) (0.374) Male head 0.004 0.003 -0.001 -0.001 (0.731) (0.805) (0.844) (0.800) Age (head) 0.000 0.000 0.000 0.000 (0.265) (0.205) (0.476) (0.448) Secondary 0.152*** 0.147*** 0.046*** 0.046*** educ. (head) (0.000) (0.000) (0.000) (0.000) HH size dummies yes Yes yes Yes Material dummies Yes Yes Yes Yes District FE yes yes Observations 2,960 2,943 2,943 2,959 2,942 2,942 R-squared (within) 0.037 0.027 0.028 0.005 0.005 0.005 R-squared (overall) 0.072 0.375 0.394 0.015 0.094 0.107 Notes: Difference-in-difference regressions with robust standard errors. * p < 0.10, ** p < 0.05, *** p < 0.01. p-values in parentheses. Full sets of control dummies are included for household size and the type of construction material used for the dwelling (permanent, semi-permanent or traditional). Source: Authors’ analysis using data from the Living Standards Measurement Study (LSMS). 12 Table 4 reports the results of the spatial DiD regression model specified in equation (2). The dependent variable in all columns is the nighttime radiance in each grid cell measured using data from satellite images. In columns (1) and (2), a simple DiD specification based on only two years (pre and post program) is reported (where the years correspond to those of the household-survey data used for estimating the results in Tables 2 and 3). In columns (3) and (4) of Table 4, the regression model with all available years (2012 to 2020) as specified in equation (2) is reported. For each model, results for two different treatment indicators are reported. The treatment indicator in columns (1) and (3) equals one for the grid cells corresponding to project areas at the most disaggregated level of information about project locations, which is the subdistrict/city level (administrative level 3). The treatment indicator in columns (2) and (4) equals one for all grid cells belonging to the next higher level of administrative division (administrative level 2, i.e. district) of each project location, to capture potential spillover effects on the area surrounding each project location. According to the results in column (1) in Table 4, project areas (defined at the administrative level 3) experienced an increase in nighttime radiance between the years 2013 and 2019 that was 0.21 units larger than the increase in non-project areas (from the baseline of 1.64 in 2013 in treated areas). When all years between 2012 and 2020 are considered in column (3), the coefficient decreases to 0.048 but remains statistically significant at the 10-percent significance level. In addition, the significant results in columns (2) and (4) suggest that the project also had positive effects on radiance in areas around the cities (or subdivisions) in which the project components were located. Various studies have shown that remote-sensing data can give rise to useful proxies of energy-related outcomes such as energy production, energy consumption, and carbon emissions (see Section I). The fact that our results from the spatial DiD analysis are consistent with the results obtained from national household survey data suggests that higher nighttime radiance might also serve as a proxy for higher electricity access and, to some extent, even the reliability of electricity (i.e., a reduction in the frequency of blackouts), at least in countries at a similar stage of development as Malawi. Overall, the results are consistent with the view that the project contributed to improvements in EE on both the supply side and the demand side of electricity in Malawi. The literature has identified several ways in which higher electricity access and quality are linked to improvements in EE. On the supply side, a reduction in blackouts is generally associated with a reduction of electricity losses in the grid (the project’s ICR estimates that these losses were reduced from 25% at baseline to 17% at the end of the project). A similar mechanism applies to the demand side (e.g., a fridge or water boiler interrupted by a blackout must restart its cooling or boiling once electricity has been restored, so that part of the generated energy is lost). In addition, increased access to electricity can help to improve demand-side EE given that households without electricity in their dwelling must rely on alternative, less efficient sources of energy for things like lighting and heating, that could otherwise be performed by using electricity. The results are also consistent with existing evidence of an (often overlooked) synergy between EE and energy access, according to which improvements in EE constitute a channel to also expand access to electricity (Can et al. 2018; Dagnachew et al. 2018). These studies argue that improvements in EE can help to facilitate the expansion of electricity access. This applies especially to areas where the bottleneck to wider access to electricity is not the cost of connecting more clients to the grid, but limited electricity generation capacity. According to this view, efforts that increase EE among existing clients essentially serve as an energy generation system by increasing the amount of energy that is available for new clients (or offsetting the additional generation capacity that would have otherwise been needed to match increases in the demand of existing clients if these efforts were not implemented). The fact that our findings in Malawi show that a combination of supply-side interventions (including grid expansion) and demand-side measures targeted at increasing EE was associated with a statistically significant increase in electricity access seems to support the existing evidence in this context. 13 Table 4. Spatial difference-in-difference estimation—Dependent variable: Nighttime radiance Pre-Post (2013, 2019) All Years (2012–2020) (1) (2) (3) (4) Post × Treat (Admin. 2) 0.213*** 0.048* (0.000) (0.061) Post × Treat (Admin. 1) 0.110*** 0.025* (0.000) (0.052) Post 0.018*** 0.016*** (0.000) (0.000) Treat (Admin. 2) 1.592*** (0.000) Treat (Admin. 1) 0.800*** (0.000) Year FE yes yes Grid Cell FE yes yes Observations 1,560 1,560 7,020 7,020 R-squared (within) 0.150 0.089 0.073 0.072 R-squared (overall) 0.313 0.103 0.030 0.007 Notes: Difference-in-difference regressions at the grid-cell level with robust standard errors. The dependent variable is average masked nighttime radiance in each cell (where the size of each grid cell is 100 square kilometers). * p < 0.10, ** p < 0.05, *** p < 0.01. p-values in parentheses. The administrative level 1 corresponds to districts; the administrative level 2 corresponds to subdistricts or cities. Source: Authors’ analysis using data from Global VIIRS (Visible Infrared Imaging Radiometer Suite) Nighttime Lights, Annual VNL 2 Composite. VI. Conclusion This paper studies the impacts of a large-scale development intervention aimed at improving energy efficiency in Malawi, using a DiD approach based on a combination of remote-sensing (satellite) data and data from national household surveys. The estimation strategy takes advantage of the geographical variation in the implementation of different project components, which allows us to identify treatment and control groups to construct a counterfactual. We find that a combination of demand-side and supply-side interventions was associated with a statistically significant increase in electricity access, a decrease in the reported frequency of blackouts, and a switch from traditional fuels to electricity as the main source of energy for lighting (albeit not for cooking). These results are based both on evidence obtained from national household survey data and from the fact that nighttime radiance (as measured by satellite images) increased relatively more in grid cells belonging to areas with project components than in non-project areas. The results are consistent with existing evidence in the economic literature of an (often overlooked) synergy between EE and energy access, according to which improvements in efficiency constitute a channel to also expand access to electricity. Overall, our results support the view that energy efficiency projects in developing countries can have significant impacts on relevant outcomes that are measurable both from national household survey data and from remote-sensing data from space. At the same time, we stress that the analysis presented in this paper focuses on a specific project where the external validity remains generally unclear. More empirical research to investigate the impacts of energy efficiency projects, and guide policy decisions, is clearly warranted. 14 References Agyarko, K. A., Opoku, R., and Van Buskirk, R. (2020). Removing barriers and promoting demand-side energy efficiency in households in Sub-Saharan Africa: A case study in Ghana. Energy Policy, 137(2020), 111-149. Bensch, G., and J. Peters (2015). The intensive margin of technology adoption: Experimental evidence on improved cooking stoves in rural Senegal. Journal of Health Economics, 42, 44-63. Berkouwer, S., and J. Dean (2020). Credit and attention in the adoption of profitable energy efficient technologies in Kenya. Energy Institute Working Paper 303, Energy Institute at Haas, University of California, Berkeley. Can, S., Pudleiner, D., and Pielli, K. (2018). Energy efficiency as a means to expand energy access: A Uganda roadmap. Energy Policy, 120(2018), 354-364. Carranza, E., and R. Meeks (2021). Energy efficiency and electricity reliability. Review of Economics and Statistics 103(3), 1-15. Center for International Earth Science Information Network (2009). Malawi: Urban Extents. Available at https://sedac.ciesin.columbia.edu/data/collection/grump-v1/maps/gallery/search/3?facets=region:africa (last accessed June 22, 2021). Creutzig, F., Roy, J., Lamb, W. F., et al. (2018). Towards demand-side solutions for mitigating climate change. Nature Climate Change, 8(4), 260-263. Dagnachew, A.G., Lucas, P.L., Hof, A.F., and van Vuuren, D.P. (2018). Trade-offs and synergies between universal electricity access and climate change mitigation in Sub-Saharan Africa. Energy Policy, 114(2017), 355-366. Davis, L.W., A. Fuchs, and P. Gertler (2014). Cash for coolers: Evaluating a large-scale appliance replacement program in Mexico. American Economic Journal: Economic Policy, 6(4), 207-38. Davis, L.W., S. Martinez, and B. Taboada (2018). How effective is energy-efficient housing? Evidence from a field experiment in Mexico. NBER Working Paper 24581, National Bureau of Economic Research, Cambridge, MA. Do, Q. T., Shapiro, J. N., Elvidge, C. D., et al. (2018). Terrorism, geopolitics, and oil security: Using remote sensing to estimate oil production of the Islamic State. Energy Research & Social Science, 44, 411-418. Dugoua, E., Kennedy, R., and Urpelainen, J. (2018). Satellite data for the social sciences: measuring rural electrification with night-time lights. International journal of remote sensing , 39(9), 2690-2701. Elvidge, C. D., Baugh, K. E., Kihn, E. A., et al. (1997). Relation between satellite observed visible-near infrared emissions, population, economic activity and electric power consumption. International Journal of Remote Sensing , 18(6), 1373-1379. Elvidge, C.D., Zhizhin, M., Ghosh T., et al. (2021). Annual time series of global VIIRS nighttime lights derived from monthly averages: 2012 to 2019. Remote Sensing, 13(5): 922. Falcheta, G, Pachauri, S, Parkinson, S., and Byers, E. (2019). A high-resolution gridded dataset to assess electrification in sub-Saharan Africa. Scientific Data, 6:110. Fowlie, M., and Meeks, R. (2021). The Economics of Energy Efficiency in Developing Countries. Review of Environmental Economics and Policy, 15(2), 238-260. He, C., Ma, Q., Li, T., et al. (2012). Spatiotemporal dynamics of electric power consumption in Chinese Mainland from 1995 to 2008 modeled using DMSP/OLS stable nighttime lights data. Journal of Geographical Sciences, 22(1), 125-136. Henderson, J. V., Storeygard, A., and Weil, D. N. (2012). Measuring economic growth from outer space. American Economic Review, 102(2), 994-1028. IEA (2017). Market Report Series: Energy Efficiency 2017, pp. 1–143. IEA (2020). World Energy Outlook 2020. Paris: International Energy Agency. IEG (2021). World Bank Group Support to Energy Efficiency: An Independent Evaluation of Demand-side Approaches - Approach Paper. Washington, D.C.: World Bank. Iimi, A., R. Elahi, R. Kitchlu, and P. Costolanski (2019). Energy saving effects of progressive pricing and free CFL bulb distribution program: Evidence from Ethiopia. World Bank Economic Review, 33(2), 461–78. 15 Jasiński, T. (2019). Modeling electricity consumption using nighttime light images and artificial neural networks. Energy, 179, 831-842. Kaygusuz, K. (2012). Energy for sustainable development: A case of developing countries. Renewable and Sustainable Energy Reviews, 16(2), 1116-1126. Letu, H., Hara, M., Yagi, H., Tana, G., and Nishio, F. (2009). Estimating the energy consumption with nighttime city light from the DMSP/OLS imagery. International Journal of Remote Sensing, 1-7. Ouedraogo, N.S. (2017). Africa energy future: alternative scenarios and their implications for sustainable development strategies. Energy Policy, 106(2017), 457-471. Ouyang, J., E. Long, and K. Hokao (2010). Rebound effect in Chinese household energy efficiency and solution for mitigating it. Energy, 35, 5269-76. Rom, A., and I. Günther (2019). Decreasing emissions by increasing energy access? Evidence from a randomized field experiment on off-grid solar. Unpublished working paper. Ryan, N. (2018). Energy productivity and energy demand: Experimental evidence from Indian manufacturing plants. NBER Working Paper 24619, National Bureau of Economic Research, Cambridge, MA. Sarkar, A., and Singh, J. (2010). Financing energy efficiency in developing countries: Lessons learned and remaining challenges. Energy Policy, 38(2010), 5560-71. SENER and World Bank (2015). Project Evaluation of Component 1 (Sustainable Light) and Component 2 (Replacement of Appliances) (in Spanish). Mexico City: Secretariat of Energy, Washington, D.C.: World Bank. Shi, K., Yu, B., Huang, Y., et al. (2014). Evaluating the ability of NPP-VIIRS nighttime light data to estimate the gross domestic product and the electric power consumption of China at multiple scales: A comparison with DMSP-OLS data. Remote Sensing, 6(2), 1705-1724. Shi, K., Yu, B., Huang, C., Wu, J., and Sun, X. (2018). Exploring spatiotemporal patterns of electric power consumption in countries along the Belt and Road. Energy, 150, 847-859. Townsend, A. C., and Bruce, D. A. (2010). The use of night-time lights satellite imagery as a measure of Australia's regional electricity consumption and population distribution. International Journal of Remote Sensing , 31(16), 4459-80. Urban, F., Benders, R. M. J., and Moll, H. C. (2007). Modelling energy systems for developing countries. Energy policy, 35(2007), 3473-82. World Bank (2010). World Development Report 2010: Development and Climate Change . Washington, D.C.: World Bank. World Bank (2011). Project Appraisal Document for Energy Sector Support Project (P099626) in Malawi. Washington, D.C.: World Bank. World Bank (2016). Implementation Completion and Results Report for Efficient Lighting and Appliances Project (P120654) in Mexico. Washington, D.C.: World Bank. World Bank (2019). Implementation Completion and Results Report for Energy Sector Support Project (P099626) in Malawi. Washington, D.C.: World Bank. Xiao, H., Ma, Z., Mi, Z., et al. (2018). Spatio-temporal simulation of energy consumption in China's provinces based on satellite night-time light data. Applied Energy, 231, 1070-78. Xie, Y. and Weng, Q. (2016). Detecting urban-scale dynamics of electricity consumption at Chinese cities using time- series DMSP-OLS (Defense Meteorological Satellite Program-Operational Linescan System) nighttime light imageries. Energy, 100, 177-189. Zhu, Y., Xu, D., Ali, S. H., et al. (2019). Can Nighttime Light Data Be Used to Estimate Electric Power Consumption? New Evidence from Causal-Effect Inference. Energies, 12(16), 3154. 16