                                                    WPS7891


Policy Research Working Paper                      7891




                           Shedding Light
 Understanding Energy Efficiency and Electricity Reliability

                                 Eliana Carranza
                                   Robyn Meeks




Poverty and Equity Global Practice Group
November 2016
Policy Research Working Paper 7891


  Abstract
  Overloaded electrical systems are a major source of unreliable                     of CFLs within a transformer leads to aggregate reliability
  power (outages) in developing countries. Using a random-                           impacts of two fewer days per month without electricity due
  ized saturation design, we estimate the impact of energy                           to unplanned outages relative to pure controls. Increased
  efficient lightbulbs on household electricity consumption                          electricity reliability permits households to consume more
  and local electricity reliability in the Kyrgyz Republic.                          electricity services, suggesting that CFL treatment results
  Receiving compact fluorescent lamps (CFLs) significantly                           in technological externalities. The spillovers in take-up
  reduced household electricity consumption. Estimates not                           and technological externalities may provide an additional
  controlling for spillovers in take-up underestimate the                            explanation for the gap between empirical and engineering
  impacts of the CFLs, as control households near the treated                        estimates of the impacts of energy efficient technologies.
  are likely to take-up CFLs themselves. Greater saturation




  This paper is a product of the Poverty and Equity Global Practice Group. It is part of a larger effort by the World Bank to
  provide open access to its research and make a contribution to development policy discussions around the world. Policy
  Research Working Papers are also posted on the Web at http://econ.worldbank.org. The authors may be contacted at
  ecarranza@worldbank.org and meeks@umich.edu.




          The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development
          issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the
          names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those
          of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and
          its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent.


                                                        Produced by the Research Support Team
  Shedding Light: Understanding Energy Eﬃciency and
                  Electricity Reliability∗
                          Eliana Carranza†                 Robyn Meeks‡




Keywords: Energy Eﬃciency, Externalities, Outages
JEL: D01, D62, O13
  ∗
      We thank Arstan Omuraliev and Ruslan Isaev for invaluable input. We are grateful to Sendhil Mul-
lainathan, Rohini Pande, Rema Hanna, Claudia Goldin, Kelsey Jack, Sebastian Bauhoﬀ, Jeremiah Johnson,
Hunt Allcott, Raymond Guiteras, Shaun McRae, Mushﬁq Mobarak, Ach Adhvaryu, Michael Moore, Catie
Hausman, Judd Boomhower, Paul Ferraro, as well as many seminar participants for helpful discussions and
comments. Cholpon Chargynova, Saikal Nogoibaeva, Merim Japarova, Wincy Poon, Ryoko Sato, and Daniel
Velez-Lopez provided excellent research assistance. Generous funding was provided by the Harvard Sustain-
ability Science Program, Harvard Graduate Student Grant, the University of Michigan, the Weiser Center,
and the National Science Foundation (Award #1061989). All errors are our own.
    †
      World Bank. Email: ecarranza@worldbank.org
    ‡
      University of Michigan. Email: meeks@umich.edu (corresponding author)
1       Introduction
Overloaded systems are a substantial source of unreliable power in developing countries.
Overloads occur when the grid is asked to deliver more power than its capacity allows, caus-
ing components of the grid to break. Such breakage results in electricity outages, a major
problem for electricity service provision in developing countries. This problem is likely to
worsen in the future for at least two reasons. First, residential electricity demand is expected
to increase with pro-poor development, as households buy their ﬁrst appliances (Wolfram et
al., 2012), putting further pressure on the existing electrical grid. Additionally, in certain
settings, subsidized electricity prices provide little incentive for service providers to upgrade
infrastructure, resulting in persistent low quality infrastructure (McRae, 2015).

With this in mind, programs distributing energy eﬃcient technologies in developing coun-
tries1 potentially provide an additional beneﬁt beyond reducing emissions and decreasing the
cost of energy services to technology adopters. CFLs are frequently deployed en masse in
developing countries via programs with the speciﬁc goals of reducing electricity outages and
increasing reliability of electricity services.2 Compact ﬂuorescent lightbulbs (CFLs), which
consume 25% of the electricity used by traditional incandescent bulbs (per lumin), can re-
duce overall household electricity consumption without requiring a reduction in the hours
of lighting services consumed (DOE, 2009). They may also provide important aggregate
impacts in the form of reduced peak electricity load and improved reliability, permitting
households to consume more electricity services (World Bank, 2006).3 Such technological
externalities, through which the returns to an individual adopting CFLs are increasing in
the fraction of the population utilizing the technology, can be a substantial beneﬁt from
adoption of energy eﬃcient technologies.4
    1
     There are many such programs. Just between 1990 and the mid-2000s (World Bank, 2006), the World
Bank alone committed more than US$11 billion to energy eﬃciency in developing countries.
   2
     For example, 600,000 CFLs were distributed in Uganda to reduce peak load by 25 MW. In Rwanda,
400,000 CFLs were distributed to reduce peak load by 16 MW and oﬀset the need for diesel-based power
generation. In Ethiopia, 200,000 CFLs were provided to reduce peak load by 6.8 MW to increase reliability.
   3
     At an aggregate level, energy eﬃcient lightbulbs can help address electric power shortages, permit utilities
to reach a greater number of customers with existing supplies, reduce need for investment in capacity and
distribution, accommodate growth in economic activity, and reduce environmental impact.
   4
     We follow the deﬁnition of technological externality employed by Foster and Rosenzweig (2010). For a
particular technology, adoption of the technology by others is said to generate a positive (negative) tech-
nological externality if it increases (decreases) an individual’s returns to the adoption of such technology.
Miguel and Kremer (2007) discuss such eﬀects in the context of de-worming medication. By reducing the
exposure to individuals infected with worms, the net beneﬁt of the deworming pill to an individual is small
(as long as the fraction of treated individuals is reasonably high). Similarly, Dupas (2014) cites medical


                                                       1
The potential gains from distribution of energy eﬃcient technologies in developing countries
are highly promising; however, there is relatively little evidence to date on the empirical
impact of energy eﬃciency on individual household electricity consumption.5 Perhaps more
notably, there is a dearth of evidence on the technological externalities that may result
from the aggregate consumption impacts of energy eﬃciency. This is likely due to several
reasons. First, spillovers in the adoption of energy eﬃcient technologies confound the es-
timated impact on individual household energy consumption. Second, data on outcomes
related to infrastructure quality (such as actual outages) tend to be diﬃcult to acquire for
developing countries (Klytchnikova and Lokshin, 2009). And third, levels of energy eﬃciency
take-up and reliability of electricity services within a community are mutually endogenous6
and jointly explained by community characteristics7 , making the causal aggregate impact of
energy eﬃciency challenging to estimate.

To overcome these empirical challenges and assess the impacts of energy eﬃcient technologies,
we implemented an experimental distribution of CFLs employing a randomized saturation
design, which allows us to account for spillovers when estimating the impact of treatment
on individual household electricity consumption.8 In a novel application of the randomized
saturation design, we experimentally vary the treatment saturation at a technologically-
meaningful and policy-relevant scale. Speciﬁcally, we randomize at the level of the typical
infrastructure failure, the distribution transformer.9 While the direct aggregate impacts of
CFL treatment, in the form of reduced transformer-level electricity loads, may be propor-
tional to CFL saturation, the externality beneﬁts are likely to vary non-linearly with CFL
technology saturation. By randomizing saturation intensity at the technologically-relevant
scale, we are able to test for evidence of technological externalities in the form of increased
evidence suggesting that the population of infected mosquitoes decreases as the fraction of households using
a high quality bednet increases, thus reducing the individual beneﬁts from bednet use.
    5
      The notable exception is a quasi-experimental evaluation of a Mexican appliance (air conditioners and
refrigerators) replacement program (Davis, Fuchs and Gertler, 2014).
    6
      Greater consumption of CFLs within a community increases electricity availability. Greater electricity
availability increases the gains from adoption of CFLs, potentially leading households to consume more
CFLs. These eﬀects reinforce each other.
    7
      For instance, CFL consumption and electricity availability may be greater in communities with higher
education levels, higher incomes, etc.
    8
      Baird et al. (2014) demonstrate the use of a randomized saturation design to account for spillovers.
    9
      Distribution transformers, which convert higher-voltage electricity from the distribution system to low-
voltage electricity for household use, are a crucial part of the electrical grid and their failure is a common
source of electricity outages (Glover, Sarma and Overbye, 2011).



                                                      2
electricity service reliability (reduced blackouts) and increased household electricity con-
sumption in higher saturation transformers.

We implement this experiment in the Kyrgyz Republic, a lower-middle-income developing
country that suﬀers from frequent electricity outages and therefore oﬀers suitable condi-
tions for research on electricity reliability. Although CFLs were available in some stores at
the project onset, they were not commonly used outside of Bishkek, the capital city. Im-
plemented in a district adjacent to Bishkek, the randomized saturation design employs an
electricity utility’s data on its residential customers and the infrastructure through which
these consumers are served.

Treatment status was randomly assigned in two stages. First, we randomize electricity
transformers to diﬀerent treatment saturations (i.e. diﬀerent proportions of households are
assigned to treatment within a transformer). Second, we randomize households individually
to treatment and control groups, according to the transformer saturation intensity assigned
in the ﬁrst stage. After completing a baseline survey, the treated households receive up to
four CFLs at a highly subsidized price. We follow household monthly electricity consumption
via the electricity utility’s records for 18 months post-intervention to estimate the impacts
on electricity consumption. To measure CFL use and spillovers in adoption, we return for a
follow-up survey one year after CFL distribution.

We measure the impacts of CFLs on residential electricity consumption, both with and with-
out controls for any potential spillovers in CFL take-up. The randomized saturation design
in the initial CFL distribution allows us to test for such spillovers. In our sample, control
households close to treated households are more likely to have CFLs at follow-up than pure
control households in located control transformers. Not accounting for spillovers in take-up
among the control group leads to estimates of electricity consumption reductions that are
downward biased (in other words, they understate the savings from energy eﬃciency). When
accounting for these spillovers, we ﬁnd that CFLs lead to a signiﬁcant reduction in monthly
electricity consumption that falls within the expected range for the technology based on
engineering performance estimates.

Using the transformer-level randomization of treatment intensity, we then test for external-
ity eﬀects resulting from the energy eﬃciency, in the form of improved electricity reliability.


                                              3
We ﬁnd that households in transformers with a higher intensity of treatment (i.e., a higher
saturation of households within a transformer receive CFLs) have fewer days reported with-
out electricity due to unplanned outages.10 This result is robust to including a number of
controls, such as the baseline number of reported outages, and is supported by calculations
of expected peak load reduction. We also ﬁnd evidence suggesting that households in trans-
formers with improved reliability consume more hours of electricity services due to fewer
outages, and that the household energy savings are larger after accounting for this tech-
nological externality. Lastly, using these results we perform a simple cost-beneﬁt analysis,
which illustrates the importance of accounting for these spillovers and technological exter-
nalities when estimating the welfare impacts of such a program.

Our experiment contributes to the literature on energy eﬃciency in several ways, and oﬀers
a novel experimental design for identifying externalities at various treatment levels.

First, it contributes to the debate regarding the impacts of energy eﬃcient technologies and
their potential to reduce energy demand. Energy eﬃciency depends not only on the de-
velopment of relevant technologies, but also on the interaction of end-user behavior with
such technologies (Allcott and Mullainathan, 2010). Prior to ours, no experimental or quasi-
experimental studies existed measuring the impacts of energy eﬃcient lighting. The one
previous study of a large-scale CFL distribution program found that approximately 20% of
initial electricity savings dissipated over time (Costolanski et al., 2013), thereby contributing
to skepticism regarding the gains from eﬃcient lighting technologies and from energy eﬃ-
ciency more generally. Our experimental ﬁnding that CFL treatment results in a signiﬁcant
electricity consumption reduction very close in size to the expected technologically feasible
reductions contributes to balance the evidence from such existing misidentiﬁed studies. Our
ﬁnding is not only relevant for developing countries, but is also consistent with results from
the United States, where lighting is a relatively smaller share of energy consumption.11 This
new evidence suggests that energy eﬃcient lighting, and perhaps other energy eﬃcient tech-
nologies as well, should remain on the menu of potential policy options.

Second, to our knowledge, our experiment is the ﬁrst to use a randomized saturation design
  10
     We collect data on reported days without electricity due to unplanned outages during both the baseline
and follow-up surveys.
  11
     Burlig et al. (2016) are performing a machine-learning analysis of energy eﬃciency in California schools.
Their preliminary results indicate substantial savings from the energy eﬃcient lighting interventions.



                                                      4
to provide evidence of technological externalities at multiple policy-relevant levels. Given
the scale at which technological externalities typically occur, designing an experiment that
can identify such externalities is challenging; as a result, there is relatively little causal em-
pirical evidence on them.12 This study designs such experiment by taking into account the
constraint within the electricity distribution system that most frequently causes electricity
outages and randomly varies the saturation intensity of CFLs at that level. By doing so, we
are able to provide evidence of technological externality eﬀects on both electricity reliability
at the transformer level and on household electricity consumption. We gauge how the exter-
nality varies with the number of treated households within a transformer to decompose the
technology’s overall eﬀect on household energy consumption into its direct and externality
components. Our results suggest that energy eﬃciency programs can improve electricity
reliability, not only in the Kyrgyz Republic but also in other contexts, if they reach suﬃcient
levels of saturation. More generally, they speak to the extent to which energy eﬃciency
programs can accomplish various, and sometimes opposing, goals using a technology-based
solution.13

Third, our experiment contributes to the discussion regarding the energy eﬃciency gap. In
spite of their promise as potentially welfare-improving, many believe households are not using
energy eﬃcient technologies when they should. Building upon Hausman (1979), cumulative
research suggests an energy eﬃciency gap due to individuals not maximizing the net present
value of their energy spending when making energy purchase decisions.14 There is a grow-
ing body of work investigating the factors impacting purchase decisions of energy eﬃcient
technologies, thereby causing these investment ineﬃciencies (Jaﬀe and Stavins, 1994; Allcott
and Greenstone, 2012; Gillingham and Palmer, 2014). Most existing economic research on
the take-up of energy eﬃcient technologies largely focuses on private adoption decisions and
  12
     A notable exception is Miguel and Kremer (2004), who provide experimental evidence of positive cross-
school externalities from deworming medicines in Kenya, but have to rely on more tentative non-experimental
methods to decompose the overall eﬀect on treated schools into a direct eﬀect and within-school externality
eﬀect. As acknowledged by them: “When local treatment externalities are expected, ﬁeld experiments can
be purposefully designed to estimate externalities by randomizing treatment at various levels. [...] However,
this multi-level design may not be practical in all contexts: for example, in our context it was not possible
to randomize treatment within schools. Randomization at the level of clusters of schools also dramatically
increases the sample size needed for adequate statistical power, raising project cost.”
  13
     For example, energy eﬃciency programs may be designed to reduce overall consumption and pollution
or to improve service reliability in a way that could result in increases in electricity consumption.
  14
     Energy eﬃcient products often require a larger upfront cost than the standard products, but exhibit
lower operating costs. Consumers’ decision to invest in energy-saving devices relies on this trade-oﬀ between
initial investment and operating costs.



                                                     5
the returns to individual adopters.15 In contrast, very little attention has been paid to the
role of spillovers in adoption decisions and technological externalities. In their presence, the
level of private investment in some energy eﬃcient technologies may be even less optimal
than previously thought.

Finally, it connects with the literature on infrastructure and development. Research indicates
that electriﬁcation is important for development (Dinkelman, 2011; Lipscomb, Mobarak and
Barnham, 2013; Rud, 2012; Van de Walle et al., 2013), and residential access to modern en-
ergy and lighting can improve living standards and productivity (World Bank, 2006). And in
many developing countries, lighting is a major component of residential electricity consump-
tion. Yet access to electricity infrastructure does not guarantee reliable service (Klytchnikova
and Lokshin, 2009). Electricity outages can impact both households (Chakravorty, Pelli and
Marchand, 2013) and ﬁrms (Allcott, Collard-Wexler and O’Connell, 2015; Alam, 2013), yet
low-quality electricity infrastructure can be persistent (McRae, 2015).

The remainder of the paper is as follows: Section 2 provides information on electricity use
in the Kyrgyz Republic and the potential impacts of energy eﬃciency; Section 3 details the
sampling process and randomized design; Section 4 addresses the data collected and results
of the randomization and compliance checks; Section 5 estimates impacts of CFLs at the
household level, both with and without spillovers; Section 6 estimates the impacts of energy
eﬃcient technology at an aggregate level; Section 7 provides additional supporting evidence
of a technological externality; and Section 8 concludes.


2      Institutional setting and energy eﬃciency
2.1     Electricity services and infrastructure in the Kyrgyz Republic
The Kyrgyz Republic provides a suitable context in which to study energy eﬃciency and
electricity reliability in a developing country setting.16 Due to its history as part of the
former Soviet Union, the country is highly electriﬁed, with nearly 100 percent of households
covered by formal electricity connections (Gassmann, 2014). Residential electricity demand
  15
     Interventions to understand or increase uptake of energy eﬃcient technologies have focused on energy
labeling (Newell and Siikamaki, 2015), social norms (Herberich, List, and Price, 2011), information on energy
costs (Allcott and Taubinsky, 2015), and subsidies (Allcott and Sweeney, 2015).
  16
     The country ranks 147th out of 187 countries for GDP (PPP) per capita (IMF, 2012).



                                                     6
has increased since the country’s independence in 1992. Over the past two decades, the
proportion of total electricity consumption comprised by the residential sector steadily in-
creased, with 63% of the country’s current electricity supply consumed by the residential
sector (Obozov et al., 2013).

Similar to many other developing countries, the electricity infrastructure is insuﬃcient to
meet current and growing electricity demand. In the Kyrgyz Republic, much of the existing
electricity infrastructure dates back to the Soviet Union including all 16 of its power plants
(Zozulinsky, 2007). Ninety percent of electricity generation within the country is hydro-
electric, so supply ﬂuctuates with annual variability in reservoir water levels.17 Technically,
the capacity of both generation and transmission infrastructure could constrain household
electricity services and result in unreliable electricity services (frequent electricity outages);
however, during the study period, distribution constraints are the primary source of unreli-
able service.

Due to very low residential electricity prices ($0.02 per kWh throughout this study), many
households heat with electricity in winter. In spite of low prices, concerns regarding electric-
ity bills and household energy expenditures are common. Household energy expenditures
comprise an estimated 7.1 percent of total household expenditures in the Kyrgyz Republic
(Gassmann, 2014), much of which is due to winter heating. Electric heating leads to large sea-
sonal variations in electricity consumption, with average winter consumption approximately
three times that of summer. The country’s utilities face growing electricity consumption
while constrained by a distribution system designed for substantially lower demand. As a
result, the country is plagued by frequent unplanned outages particularly in winter.18

Transmission and distribution systems are consistently overloaded, acting as a binding con-
straint and source of unplanned outages. To put this in perspective, most of the transformers
have a load factor of 0.9 - 1.2, with 0.7 being the optimal load (Amankulova, 2006).19 These
constraints are a concern for both consumers and the electricity utility. Unplanned outages
typically occur when local distribution systems experience an overload. The distribution
  17
     In years of low water availability, the country has instituted planned, rolling blackouts during the winter.
This did not occur during our study period, according to the electricity utility.
  18
     For example, in 2010 the country had 12,578 unplanned power outages (approximately 34 outages per
day), which is considered unreliable service by international standards (USAID, 2011).
  19
     This is for the 35/220 kV transformers, which is the last step in delivering electricity to homes.



                                                       7
network was constructed for peak electricity consumption associated with households own-
ing only a few electricity-using durables. However, peak household electricity demand has
increased as households have bought more appliances, which is consistent with predictions
made for pro-poor economic growth in developing countries, more broadly (Wolfram, Gertler
and Shelef, 2012).

Transformers are a critical part of the electricity distribution infrastructure. A distribution
transformer on the electrical grid converts high-voltage electricity to usable, low-voltage elec-
tricity for household consumption (Glover, Sarma and Overbye, 2011). There is a maximum
electricity load that a transformer can transfer at any given time and exceeding that may
lead to unplanned outages. During the study, transformer overloads were the primary source
of unplanned outages. Quality of electricity services is correlated within a transformer, as
households thus are exposed to the same electricity-related shocks. A transformer-level out-
age aﬀects all households sharing the transformer.20


2.2     Energy eﬃcient lighting
Energy eﬃcient technologies have the potential to decrease individual residential electricity
consumption and therefore costs. Using approximately 75% less electricity than incandes-
cent lightbulbs, CFLs are one tool for meeting lighting needs while reducing electricity con-
sumption. If take-up rates are suﬃciently high, energy eﬃcient technologies could impact
electricity load at an aggregate level. For example, CFLs could decrease aggregate electricity
demand within a transformer reducing the probability of a transformer overload. By alle-
viating the stress on the infrastructure, energy eﬃciency could reduce unplanned electricity
outages and increase service reliability.

Prior to this study, CFLs were available for purchase in large home repair stores and markets
located in the capital city, but not in villages. CFLs cost between 100 and 170 Kyrgyz soms,
depending on the quality.21 In contrast, incandescent lightbulbs were available to purchase
in both rural and urban markets for approximately 15 to 20 Kyrgyz soms. Even with low
electricity prices, the payback period for the CFLs was between 1 and 2 years.22 This study
  20
     In our setting, 54 households on average receive their electricity via a single transformer.
  21
     In March 2013, the exchange rate was approximately 1 USD = 46 Kyrgyz soms.
  22
     Payback period calculations (not shown) were based on typical lightbulb use in our sample and electricity
and CFL prices within the region. We show cost-beneﬁt calculations for the ﬁrst year post-distribution in
Appendix Calculation 4.


                                                      8
was implemented in a district adjacent to the capital, Bishkek. Given residents of this district
frequently travel to the city, they could have purchased CFLs pre-intervention; however, very
few households outside of the capital used CFLs at baseline.


2.3     Potential impacts of energy eﬃciency
Based on constraints in the electricity distribution system and the potential for energy eﬃ-
cient technologies to decrease demand both at the household and aggregate-level, we might
expect the following impacts on electricity consumption following installation of CFLs:

   1. Impacts on household electricity consumption: The eﬀect of energy eﬃcient lightbulbs
      within one’s home depends on the net sum of both engineering and human factors.
      Based on engineering calculations, the technologically feasible beneﬁt would be a re-
      duction in electricity consumption, and thus in the electricity bill.23 There are, how-
      ever, multiple reasons a household’s electricity consumption might not change, might
      decrease by less than expected, or might even increase following the installation of
      CFLs. These vary from CFLs not meeting their technological promise (poor quality,
      failure), human interactions with the technology (incorrect use), human behavioral
      responses (rebound eﬀect)24 or some combination of these.25

   2. Spillovers: Installation of energy eﬃcient technologies could inﬂuence others’ adoption
      decisions via channels such as learning, imitation, etc. These spillovers in take-up could
      be experienced by households with close linkages to the original adopters, and could be
      positive, negative, or zero. If present, spillovers in take-up can bias empirical analyses
      that do not account for them. For instance, when estimating the impacts of CFLs
      on household electricity consumption, if the comparison group ﬁnds alternative means
      by which to access the technology, they could also experience reductions in electricity
  23
      With some basic information (for example, information on the wattage of both the original bulb and
the energy eﬃcient bulb to which one is switching, hours of lightbulb use, etc.), one can perform simple
calculations to predict reductions in electricity consumption. We perform such calculations and discuss
them in Section 5.
   24
      By cutting down the energy requirements and monetary costs of lighting, CFLs may promote additional
electricity consumption or generate a “rebound” that oﬀsets the technologically feasible savings. As a result,
the kWh consumed by a household may decrease, remain unchanged or even increase. For theoretical
discussion of the rebound eﬀect, see Borenstein (2015) and Chan and Gillingham (2015). Although the
existing literature appears to agree on the existence of rebound eﬀects, there is much debate over the
magnitude of such an eﬀect (Gillingham, Rapson and Wagner, forthcoming).
   25
      For example, by being more eﬃcient, CFLs will result in less waste heat lost than incandescent bulbs.
This may result in households compensating through other practices, such as winter heating.


                                                      9
       consumption, leading to underestimation of true impacts. A reduction in electricity
       consumption may occur, but it may not be detected (or muted) due to contamination
       of the comparison group.

    3. Aggregate eﬀect: Energy eﬃcient technologies could have an eﬀect on aggregate elec-
       tricity consumption within the local distribution system. The sign and magnitude of
       this aggregate eﬀect will depend on the proportion of households that use CFLs, as
       well as on the sign and magnitude of the impacts they individually experience. If a
       large enough proportion of households within a transformer adopt CFLs and their indi-
       vidual impacts combine into aggregate electricity savings, the probability of an outage
       occurring due to a transformer overload would be reduced, thus increasing reliability
       of electricity services.

    4. Technological externality: The distribution of CFLs could generate a positive (negative)
       technological externality if the returns to a household from adopting such technology
       increase (decrease) in the fraction of the population utilizing the energy eﬃcient tech-
       nology. In the scenario of a positive technological externality, when consumers install
       energy eﬃcient devices at a high saturation and aggregate demand decreases such that
       reliability improves, households are able to consume more hours of electricity services.
       This eﬀect is experienced by all households within a transformer, regardless of the indi-
       vidual household’s own adoption. The returns from CFL adoption increase because the
       potential for electricity consumption increases and therefore the potential for energy
       savings through CFL use does as well.26

This ﬁeld experiment employs a randomized saturation design to address these potential
impacts of energy eﬃciency.


3      Randomized experiment with energy eﬃciency
3.1     Sampling process
In order to identify transformers and households within the transformers, we used data
from electricity utility records on over 40,000 residential customers in a district adjacent to
  26
    If electricity is limited, the beneﬁts from CFL use are positive but small–zero if there is no electricity.
As electricity becomes more available the beneﬁts increase, possibly in a non-linear way, at a decreasing rate
and/or discontinuously.



                                                      10
Bishkek, the country’s capital.27 These records contain crucial information for the design
of our sampling procedure. The data identify the address of each of the households in the
district as well as the transformer through which each of them is served.

Within the district for which we have utility data, seven villages (comprising 248 eligi-
ble transformers)28 were chosen due to their accessibility from Bishkek during the winter
months.29 Given that the area surrounding the capital tends to be better oﬀ than those in
the country’s other regions, the district in which the study occurs is not representative of the
entire country. In order to improve external validity, the study sample was restricted to the
124 eligible transformers with below median monthly household electricity consumption in
the year prior to the intervention. Such households and transformers are more representative
of the country as a whole. To complete the sampling process, 20 percent of the households
from each transformer were randomly selected to participate in the study.30 Because the
number of households per transformer is heterogeneous, the exact number sampled in each
transformer also varies.


3.2     Experimental saturation design
The randomized saturation design varies household exposure to the CFL technology to test
for evidence of a household energy eﬃciency eﬀect, spillovers in take-up, and a technological
externality due to aggregate impacts. Transformers are ﬁrst randomized to diﬀering treat-
ment saturations and then households within those transformers are randomized either to
receive CFLs or to control status, according to the saturation previously assigned.

Figure 1 depicts how treatment status was randomly assigned in two stages. In the ﬁrst
stage, the 124 sampled eligible transformers were randomized into three groups: control,
lower treatment saturation, and higher treatment saturation. Due to funding constraints,
households in 14 control transformers were not surveyed. This resulted in only 110 eligible
transformers being ﬁnally included in the study, with 25, 45, and 40 transformers in control,
  27
     The data were provided by the electricity utility through a Data Use Agreement. The utility, however,
was not informed as to which transformers or households were being treated through the experiment.
  28
     Transformers providing at least 5 households with electricity were eligible. According to the utility,
transformers serving fewer households likely also serve industrial consumers.
  29
     This was to ensure that survey enumerators could reach households in March, when weather conditions
can make transportation challenging.
  30
     Due to funding constraints, households in only 25 of the 39 control transformers were surveyed, resulting
in households in 110 transformers being surveyed.


                                                     11
lower and higher saturation groups respectively. In control transformers, no study house-
holds were treated. In lower-saturation transformers, 60% of study households were treated.
In higher-saturation transformers, 80% of study households were treated. As mentioned in
the sampling discussion above, 20 percent of the households in each transformer were sam-
pled into the study. Thus, the ﬁrst stage randomization results in approximately 12% to
16% of all households within treated transformers being assigned to treatment.

In the second stage, a total of 1,000 study households within these 110 transformers were
randomized into either control or treatment status. This resulted in 457 and 543 households
in each group respectively. By design, as depicted in Figure 2, treated households are found
only in treated transformers, in the proportions set by each transformer’s treatment intensity.
Control households, however, can be in either control or treated transformers. The two stages
of randomization also induced spatial heterogeneity in the location of the treated households,
leading to variation in the proximity to and number of treated neighbors.


3.3     Intervention and household survey
In the spring of 2013, all study households were visited and invited to participate in a base-
line survey regarding electricity use. Their treatment status assignment was not disclosed at
the time of the baseline survey. All households were given 150 Kyrgyz soms (approximately
3.26 USD) to compensate them for their time, after ﬁnishing the survey.31

After the baseline survey, baseline interaction with the control households was complete. In
contrast, households randomly assigned to the treatment group were oﬀered the opportunity
to purchase up to 4 CFLs at a highly subsidized, randomly-drawn price, via a willingness to
pay experiment.32 The set of possible prices was {0, 5, 10, 15, 20}. The market price for
CFLs was a minimum of 100 KGS, so treated households were paying a maximum of 20%
of market price. The market price for incandescent lightbulbs was between 15 and 20 KGS.
On average, treated households received 3.2 CFLs from the study intervention. Take-up of
CFL treatment was not perfect, as some treated households opted to stop after the survey.
  31
      As of 2011, the average monthly nominal employee wage was 9,352 KGS per month (or an estimated
467 KGS per day of work) (National Statistical Committee of the Kyrgyz Republic, 2012).
   32
      We provided up to four CFLs as pilot surveys indicated that, on average, households had ﬁve to six
lightbulbs at baseline. We sought to replace most of their incandescent bulbs with CFLs. The experiment,
which utilizes the Becker-de Groot-Marschak methodology to measure demand for CFLs, is explained in
Meeks (2016).



                                                  12
The rate of non-compliance that resulted was 12%. In those cases, the household received
zero CFLs.33

All study households were visited again in the spring of 2014, one year following the in-
tervention, for a follow-up survey. Of the original 1,000 study households, 101 addresses
were identiﬁed as having new tenants in the year since the intervention. We interviewed all
households currently living at the original addresses, as it was diﬃcult to know exactly when
residents moved or if the CFLs had moved with them.34 After completion of the follow-up
survey, survey respondents were again oﬀered 150 KGS to compensate for their time.


4      Study sample
4.1     Data
We use several datasets, including electricity utility data, baseline and follow-up household
survey data, and GIS spatial data, which are matched at the household-level. All of these
data are then matched to heating degree day data from a nearby weather station. A descrip-
tion is provided in the Appendix.

Electricity utility data: We utilize electricity utility records identifying both transformers
and households served by them, as well as the monthly household electricity bills. These
data start in October 2010 and continue through September 2014. This provides observa-
tions 30 months prior to and 18 months following the intervention, held in March 2013. One
important feature of this time period is that electricity prices remained constant at 0.02 USD
per kWh.35 The utility, however, was not collecting transformer-speciﬁc outage data at the
time of the intervention.36

Household survey data: As indicated in the previous section, we conducted two rounds
of household surveys, one immediately prior to the intervention (baseline) and one a year
  33
     For intent-to-treat estimates, these households are considered treated. Compliance is discussed below.
  34
     Survey enumerators made at least four attempts to survey the household. If enumerators were informed
that the previous respondents had moved, then the new residents were surveyed. A total of 835 original
respondents were re-interviewed for the follow-up survey.
  35
     A tariﬀ reform was introduced in late 2014 (see McRae and Meeks (2016) for details of reform) and
therefore we end our analysis in September to avoid conﬂating the CFL intervention with the tariﬀ change.
  36
     Because electricity consumption data are monthly, we cannot use them to calculate timing of outages.



                                                    13
following the intervention (follow-up). The baseline survey collected information on appli-
ance ownership and use, lightbulb ownership (type, wattage, etc) and use (room of use,
hours used in a typical day, etc.), electricity-related behaviors, and various household de-
mographics. Importantly, we asked households to report the number of days in the past
month that they did not have electricity due to outages.37 Implementation of both survey
rounds began in the month of March, speciﬁcally so that, when asking about the outages in
the month prior, we were capturing winter months.38 The number of CFLs distributed to
each treatment household was tracked. The follow-up survey repeated questions from the
baseline and also asked new questions on perception and understanding of CFLs.

Spatial data: GPS data were collected on the location of each participating house during
the baseline survey. These data points permit various calculations relevant to potential
spillovers in adoption, including the distance to nearest treated household and the number
of treated households within certain radii. In addition, we use GIS data from OpenStreetMap
on locations of all buildings in the study villages. These data permit calculations of the total
number of households located within various radii (e.g., 100 meters, 200 meters, and so on)
of each participating household.39


4.2     Household characteristics
Households in the study sample are small, with an average of just under 4 members. House-
hold monthly income per capita is on average 76 USD per month (2.45 USD per person per
day) and household heads are educated, with 84 percent having ﬁnished secondary school.
Most houses (91 percent) are owner-occupied, the majority of which were constructed during
the period of the Soviet Union. They comprise an average of 4.3 rooms, and are typically con-
structed of brick (54 percent) or a hay/adobe mix (38 percent). Houses are served by formal
connections to the electrical grid and metered individually (i.e., houses do not share meters).

Households receive a monthly electricity bill based on the meter readings of their consump-
tion. At baseline, they use an average of 232 kWh per month in the summer (June to
  37
     These questions were piloted extensively prior to the baseline survey. We asked about the number of days
due to heterogeneity in outage length. The transformer repair required and availability of replacement parts
determines the outage length after a transformer overload. According to the utility, transformer outages last
between a few hours and a few days.
  38
     Winter is when electricity demand is the greatest, stressing the transformers. For this reason, outages
are most prevalent in the winter months.
  39
     The calculations of the total number of buildings are further described in Appendix Calculation 1.


                                                     14
September) and 633 kWh per month – more than double – in the winter (November to
February). On average, households have 8 electricity-using durables40 and many households
(39 percent) report heating with electricity.41 A small proportion have an electric hot water
heater (14 percent), and almost none (2 percent) have air conditioners.

Households largely indicate that they both frequently worry about saving electricity (95
percent) and take measures to save electricity (86 percent). More than half the households
report knowing about energy eﬃcient lightbulbs (56 percent). However, CFLs use was
low. Only a few households had them, and in small numbers (resulting in 0.17 CFLs per
household, on average). The majority of households did not know or believe that CFLs
consume less electricity (70 percent), did not expect savings in their electricity bill from
replace incandescent bulbs with CFLs (72 percent), and did not believe the electricity savings
would pay back the upfront costs of the CFLs (69 percent).42


4.3     Randomization balance
Baseline characteristics were compared both at the transformer and household levels to un-
derstand the outcome of randomization. Comparisons use both household survey data and
data on transformer characteristics, provided by the electricity utility.

Transformer-level balance test results are shown in Appendix Table 1. There are no sta-
tistically signiﬁcant diﬀerences between the transformers treated with a low saturation and
the control transformers, nor are there any statistically signiﬁcant diﬀerences between the
high saturation transformers and the control transformers. The high and low saturation
transformers do have one signiﬁcant diﬀerence from each other. They diﬀer in the number
of households within the transformers. Following Bruhn and McKenzie (2008), we chose not
to re-randomize and instead control for this characteristic in related regressions and perform
additional robustness checks.

Results from the household-level balance tests are shown in Appendix Table 2. The two
  40
      Almost all homes have a television and refrigerator. Approximately three-quarters of households have
electric stoves, an iron, and a clothes washing machine.
   41
      Households conserve on heating. Most households report heating their houses at least sometimes with
coal (80%), and on average they heat 3/4 of all their rooms during winter.
   42
      Households were also unaware of diﬀerences in quality (88%) and in potential electricity savings (80%)
between diﬀerent types and brands of CFL.



                                                    15
household-level treatment groups are statistically identical along most dimensions. Impor-
tantly, most households in all groups have their own individual electricity meters. There are,
however, two slight diﬀerences. Control households are slightly more likely than treatment
households to have a household head that has completed secondary school education, and
to be in homes that are single family buildings (in comparison to the multi-unit apartment
buildings). This diﬀerence is about what could be expected by chance. If anything, these
diﬀerences would downward bias our results.

A graph of pre-intervention electricity consumption over time, in Appendix Figure 1, high-
lights important heterogeneities in electricity consumption over a year and shows the same
seasonal variation for both treatment and control households. The large spike in winter elec-
tricity consumption is due in large part to electric heating and is the reason that electricity
outages are most frequent in that season. Longer hours of lighting due to shorter days play
a lesser role. The winter peak is somewhat greater among the treated households than the
control in both pre-intervention winters (winter 2011 and winter 2012). To account for these
patterns, our analysis will include household ﬁxed eﬀects and month-by-year ﬁxed eﬀects.


4.4    Compliance and attrition
There are two aspects of the intervention take-up and survey data collection, described in
Section 3.3, with which we might be concerned: (i) whether or not treated households took
the CFLs that we distributed as the treatment, and (ii) whether study households were still
inhabiting the home at time of follow-up survey. We address these issues in the following
ways. First, in our intent-to-treat estimates, treated households that did not comply with
the treatment are considered treated. Second, we obtain estimates including all households
(movers and non-movers) and excluding those houses with new tenants (just non-movers).
Results are consistent across these analyses.




                                              16
5      The impacts of energy eﬃcient lighting on household
       electricity consumption
5.1     Bench-marking and previewing impacts
Prior to estimating the intervention’s impacts, we ﬁrst calculate the technologically feasible
expected electricity reductions for the winter, spring/fall, and summer seasons.43 Back-of-
the-envelope calculations for a typical household in our sample, in Appendix Calculation 2
and Appendix Table 3, suggest that replacing 3.2 incandescent lightbulbs of 100 watts each
by CFLs of 21 watts each could potentially decrease electricity consumption by between 26
and 42 kWh per month.44

These technologically feasible savings do not account for human interaction or behavioral
responses to the technology. Estimates of the eﬀect of CFLs in actual use conditions could
be smaller or larger. They can diﬀer from the engineering calculations for reasons such as a
rebound in electricity consumption, a reduction in electricity outages permitting more elec-
tricity services, or spillovers in technology take-up that bias the results. The randomized
saturation design helps us understand the impacts of energy eﬃcient lighting and untangle
the channels through which they operate.

To motivate the regression analysis, we begin with an event study-style graph of electricity
consumption month-by-month (Figure 3). We plot our calculations of the technologically
feasible electricity reductions alongside the estimated impacts of CFL treatment (the re-
sult of comparing treatment relative to control households in control transformers) month-
by-month.45 A few patterns are noticeable. First, the ﬁgure shows a downwards shift in
electricity consumption for treatment households post-intervention (with CFL distribution
starting in late March 2013). Second, the impacts are quite noisy, particularly in the winter
months.46 Although they indeed have reductions in electricity consumption, the noisiness
  43
     Through discussions with the electricity utility in fall 2012, we learned that winter (also known as the
“heating season”) is the time of highest electricity demand and greatest strain on transformers.
  44
     During early survey piloting, we learned that households typically used 100 watt bulbs in their homes.
We provided 21 watt CFLs, because they were advertised to be equivalent to 100 watt incandescent bulbs.
These and other underlying assumptions are further described in the Appendix Calculation 2.
  45
     Because in this ﬁgure we calculate the treatment impact separately for each month, we cannot include
household ﬁxed eﬀects.
  46
     This noisiness is likely due to some households heating with electricity in the winter (which would cause
a large spike in their electricity consumption), while others heat with coal.


                                                     17
might make them less salient for treated households. Third, there is much heterogeneity in
the impacts across seasons. Electricity savings start in the months following the distribution
of CFLs in the spring of 2013, and continue over the summer and fall. In the winter, impacts
diverge greatly from the engineering predictions, but in the spring of 2014 the electricity
savings return and are similar to those predicted. The regression analysis that follows looks
more closely into these impacts of the CFL treatment.


5.2     Basic estimates
To assess the impacts on household electricity consumption, we ﬁrst estimate a simple
diﬀerence-in-diﬀerences model using the electricity utility records:

                 qit = τ T reati ∗ P ostt + βP ostt + δT reati + αXit + γt + λi +         i              (1)

where qit is household i’s electricity consumption (kWh) in month t, T reati is an indicator
of treatment status, and Xit is a vector of household-level variables.47 Month-by-year ﬁxed
eﬀects, γt , and household ﬁxed eﬀects, λi , address the large seasonal variations in consump-
tion and potential heterogeneities across households in baseline electricity consumption, as
identiﬁed in the event-study graph.

The T reati ∗ P ostt interaction denotes the household’s assignment to treatment and if that
treatment occurred in that month or prior months. The coeﬃcient on this interaction term,
τ , is therefore an estimate of the average change in household monthly electricity consump-
tion (in kWh) that resulted from random assignment to treatment.

Table 1 reports intent-to-treat estimates of τ . In Table 1, Column 1, the estimated impact
is identiﬁed according to the above basic regression, from variation within households over
time, controlling for month-by-year shocks. This basic estimate indicates that the CFL
treatment reduces household electricity consumption by 16 kWh per month. The magnitude
of this impact is only half the expected size based on engineering calculations. However, in
Column 1, the omitted group is comprised of all control households, regardless of whether
  47
     These include controls for number of days in monthly billing period and whether the household uses elec-
tricity for heating. We also control for heating degree days; however, we only have variation in temperature
over time. The 7 villages in the study sample are all covered by one weather station and data are reported
at that level. Nevertheless, we do not expect for there to be much spatial variation in temperatures across
villages included in the study given their size and proximity.



                                                     18
they are in a treated or control transformer. If there are any spillovers in adoption among
control households close to treatment households, or any technological externality aﬀecting
all households within a treated transformer, the omitted group will be contaminated and the
basic impact may be mis-estimated.


5.3     Accounting for contamination
We re-estimate the impacts of CFL treatment on household electricity consumption, account-
ing for potential contamination within the transformer. We employ the following diﬀerence-
in-diﬀerences speciﬁcation:48

 qigt = τ T reatig ∗ P ostt + βP ostt + δT reatig + θCig ∗ P ostt + φCig + αXig + γt + λig +         igt   (2)

where qigt is our outcome of interest for household i located in transformer group g ; T reatig
indicates the treatment status of household i in transformer group g ; Cig indicates if the
household i is a control household in a treated transformer group; and Xig is a vector of
variables, additionally including the number of houses served by a particular transformer.

As before, τ is an estimate of the average impact of the CFL treatment on household monthly
electricity consumption; and the new coeﬃcient θ estimates the average impact on control
households in treated transformers. Importantly, the omitted group in these regressions is
now a “pure control”, in that it only includes control households that are located in control
transformers.

The results presented in Table 1, Columns 2 and 3, indicate that the CFL treatment led
to a reduction in household electricity consumption of 30 kWh per month. The coeﬃcients
are signiﬁcant regardless of the level at which standard errors are clustered. Accounting
for potential contamination within transformer, the estimated impacts are statistically and
substantially larger than basic estimates in Column 1, and comparable to the expected en-
gineering reductions.

We replicate the previous analysis, but this time separately for each season.49 Appendix
  48
     We account for the potential within-transformer contamination in a fashion similar to Gine and Mansuri
(2011) and Banerjee et al. (2014).
  49
     We divide the months according to the electricity utility’s seasonal deﬁnitions, with the winter heating
months including November, December, January, and February.


                                                     19
Table 4 shows that the only season in which the impacts of treatment are of the magnitude
expected and statistically signiﬁcant is in the spring/fall. Impacts in the winter and summer
seasons are far smaller than expected and insigniﬁcant.50 These ﬁndings suggest a techno-
logical externality, by which households in higher intensity treatment transformers consume
more electricity due to fewer electricity outages in the winter. Winter is when unplanned
outages most frequently occur, as peak electricity demand is quite high. CFLs have the po-
tential to reduce electricity consumption, but they also have the potential to reduce outages
in winter (more than any other season). If households have more reliable electricity service
(fewer outages) in the winter, then they will be able to consume more electricity services and
we would not see the expected reduction in electricity consumption. Potential technological
externalities could indeed account for heterogeneity in impacts across seasons.

In summary, contamination of control households may be biasing downwards the basic im-
pact estimates. That we see a signiﬁcant reduction in electricity consumption among control
households in treated transformers suggests that spillovers in CFL take-up may be biasing
downwards the basic impact estimates. Potential technological externalities due to an ag-
gregate eﬀect on electricity reliability at the transformer level may also contribute to the
miss-estimation of impacts. We look more into these and other possible mechanisms in
Section 7. Next, we analyze the impacts of CFL treatment at the transformer level.


6     The impacts of energy eﬃcient lighting on aggregate
      electricity loads
6.1     Bench-marking and previewing aggregate impacts
As in many developing countries, in the Kyrgyz Republic outages are most frequently the
result of overloads within the distribution system. Overloads typically occur during peak
times and are more common in the winter, particularly in the winter evenings, when house-
hold energy demand is greatest.

We perform a back-of-the-envelope calculation of the peak load reduction induced by the
CFL treatment. Switching 4 incandescent lightbulbs to CFLs could save 60 kWh per month,
  50
     Based on our ex-ante calculations of the expected electricity savings, we believed that electricity con-
sumption reductions would be greatest in the winter (followed by fall/spring and then summer).



                                                     20
which is 10% of a 600 kWh household monthly bill. However, lighting is disproportionately
used “on peak”. Therefore, the resulting reduction is actually 21% of the household peak
load demand.51 For a transformer with about 20% of all households treated, this would equal
approximately a 4% reduction in transformer peak load.52

Using household survey data on the number of days without electricity (due to outage) during
the month prior, we provide suggestive evidence of an aggregate impact of transformer-level
treatment on unplanned outages.53 In Figure 4, we plot the reported outages at follow-up
by treatment group. As shown, the distribution of reported outages among households in
treated transformers is shifted leftward (towards zero outages) in comparison to the graphed
responses of households in the control transformers. This ﬁgure motivates the regression
analysis that follows.


6.2     Estimating aggregate impacts
We use the transformer-level randomization to estimate the impact on reported unplanned
electricity outages using the following equation:

                            Oig = πHig + ρLig + βT reatig + ηXig +          ig                         (3)

where Oig is the number of days without electricity due to outages in the month prior to
the follow-up survey as reported by household i in transformer g ; Hig indicates if household
i is in a higher-saturation transformer, with between 15 to 18 percent of all households as-
signed to treatment; Lig indicates whether household i is in a lower-saturation transformer,
with between 10 to 14 percent of households assigned to treatment;54 T reatig remains the
household’s own treatment status; and Xg is a vector of transformer-level controls. Standard
  51
     According to the utility, times of peak demand are between 6 to 9 am and 6 to 10 pm. The calculation
assumes a reduction of peak demand of 1.5 kW peak, and switching from 100 W incandescent bulbs to 21
W CFLs (Appendix Calculation 3).
  52
     We “ground-truth” these results by discussing them with our utility collaborators. Engineers indicated
that this would be a substantial reduction in peak demand and would very likely reduce transformer outages.
  53
     The follow-up survey occurred in March and April 2014, so the months in which we are measuring days
without electricity due to outage include February and March.
  54
     In higher (lower) saturation transformers, 80% (60%) of study households were assigned to treatment.
There is heterogeneity in the number of households across transformers, and we sampled 20% of all house-
holds within each transformer to include them in the study. Therefore, the randomized study saturation
proportions result in between 15 to 18 (10 to 14) percent of all households in high (low) saturation trans-
formers actually receiving the CFL treatment



                                                    21
errors are clustered at the transformer level.55 We also perform the standard error correction
for multiple hypothesis testing developed by List, Shaikh and Xu (2016).

Results in Table 2 indicate that the energy eﬃcient technology led to an aggregate impact
in the form of improved reliability. Table 2, Column 1, shows the basic results at the trans-
former level, given by π and ρ in the equation above. We see between one and two fewer days
without electricity in both the low and high saturation transformers, respectively. However,
the reduction in days without electricity due to outages is only statistically signiﬁcant for
households in the high saturation treatment transformers. They report approximately half
as many days of outages (two in comparison to four) as households in control transformers.
The estimates are robust to including a suite of diﬀerent transformer-level controls, most
important of which are the number of households within the transformer and the baseline
number of outages reported. The diﬀerence in outage-days reductions between households
in high and low saturation transformers (two versus one) is also statistically signiﬁcant.

Table 2, Column 2, controls for the reporting household’s own treatment status. High
saturation transformers have more treatment households (by deﬁnition), so we may worry
that those households have an incentive to report fewer outages. Accounting for individual
household treatment status shows that such possibility is not a concern. The estimates
remain the same as the basic impacts, suggesting that diﬀerences in reporting between
treated and control households are not driving results.


6.3     Evidence of technological externality
We re-estimate the impacts of CFL treatment on household electricity consumption, this
time breaking up the analysis by whether the household is in a higher or lower treatment
saturation transformer. Importantly, these estimations still use variation induced by the
randomized saturation process. Results shown in Table 3 indicate a diﬀerence between the
transformers with lower and higher intensities of treatment in the extent to which a tech-
nological externality is created. These results are consistent with our working deﬁnition of
a technological externality, in that the more individuals that take up the technology, the
  55
    We can and have also run these regressions collapsing to the transformer level and using the average
reported outages per transformer. In doing so, we get similar results; however, in using the transformer level
average, we lose our ability to control for the respondent household’s own treatment status.




                                                     22
greater are the returns to the technology.56

The intuition is simple. In higher saturation transformers, the households that take up CFLs
generate a reduction in aggregate demand load substantial enough to signiﬁcantly decrease
the number of days without electricity due to outages. Therefore, treated households at-
tain smaller reductions in electricity consumption. Although they use fewer Kw per hour
of electricity consumed (due to the greater energy eﬃciency of CFL technologies), they can
consume more hours of electricity (due to the greater electricity reliability induced by the
take-up of CFLs). In contrast, in the lower saturation transformers, treated households at-
tain much larger overall electricity savings. The households’ take-up of the CFLs reduces to
some extent the aggregate load, but its impact in the number of outage-days is insigniﬁcant.

In summary, when a large enough proportion of households take up the CFLs, an aggregate
eﬀect is possible and, if substantial enough, it may induce a technological externality in the
form of a reduction in electricity outages. This increased electricity reliability is experienced
by all households served by the impacted transformer, regardless of whether they themselves
are treated or control households. When controlling for this potential externality, the CFLs
exert a signiﬁcant reduction in household electricity consumption that is of a similar magni-
tude to that which is technologically feasible. This is the case of the coeﬃcient on the treated
households in low-saturation transformers, which do not experience changes in outage-days.


7      Additional checks and supporting evidence
7.1     Evidence of spillovers in CFL take-up
As discussed in Section 5.3, control households located close to a treated household were likely
to experience spillovers, and this contamination may have biased the measurement of the CFL
treatment impacts on electricity consumption.57 To test whether there are indeed spillovers
in CFL take-up following the intervention, we estimate the impact of our intervention on
  56
      We think of this as an analogy to children within a school being vaccinated against a disease, such as
measles. Vaccinated children provide a positive externality to non-vaccinated children. With each additional
child vaccinated, the probability of a measles outbreak within the school is reduced; however, the propor-
tion of children within the school that are vaccinated must surpass some threshold in order for the school
community to achieve herd immunity.
   57
      Our deﬁnition of closeness between households is a distance measure of geographical proximity. As
discussed in Breza (2015), a number of studies have used geographical proximity to measure spillovers,
including Dupas (2014), Godlonton and Thornton (2012), and Cohen, Dupas and Schaner (2015).


                                                    23
the number of CFLs in a house at the follow-up survey, controlling for the number of CFLs
distributed to the households at baseline. We utilize the spatial variation in exposure of
control households to close treated neighbors induced by the two-stage randomization to
account for potential peer eﬀects in technology adoption as follows:

                              Yig = βT reatig + θCid + ρNd + θXig +           ig                           (4)

where the new term Cig is an indicator for whether the household i is a control household
with at least one treated household within 100 meters from the house; Nd is the total number
of households within 100 meters from the house;58 and Xig is a vector of controls, including
the number of CFLs the household received from our project at baseline. Standard errors
are clustered at the transformer level.

Table 4 reports the estimated impacts on CFL adoption at the follow-up. Column 1 uses the
basic speciﬁcation from Eq. 1. As before, these results are expected to be biased because the
omitted group comprises all control households and some of them are potentially contam-
inated by spillovers. Column 2 shows results from Eq. 4, which accounts for spillovers by
controlling for whether a control household is within 100 meters from any treated household;
and Column 3 mirrors the alternative speciﬁcation given by Eq. 2, which corrects for con-
tamination by including a dummy variable for control households in treated transformers.
Their corresponding omitted groups are detailed at the bottom of each column.

Results from Columns 2 and 3 are similar, as they should be, given that households within
the same transformer are in close proximity spatially.59 Both treated and control households
have an average of half an additional CFL at follow-up, controlling for the CFLs distributed
by our intervention. This result contrasts with the basic speciﬁcation’s much smaller and
insigniﬁcant estimated impact, making clear that spillovers also bias the basic estimates
of the eﬀect on later take-up. Interestingly, the correct estimated impacts suggest that
treatment households likely purchased CFLs themselves between baseline and follow-up, in
addition to those they had received from our project. The correct estimates also indicate
that, indeed, there is a positive and statistically signiﬁcant relationship between being close
  58
     We consider a control household to be “close” if the entrance is within 100 meters of a treated household.
We choose 100 meters as the cutoﬀ, as some prior analyses of peer eﬀects on a number of outcomes variables
had suggested that this distance was an important cutoﬀ, past which the eﬀect dissipates.
  59
     This reassures that the impacts on household electricity consumption reported in Table 1, Columns 2
and 3, are well estimated.


                                                      24
to a treated household and the number of CFLs that household has at follow-up.


7.2     Checks of aggregate reliability eﬀects
To ensure that in Section 6.2 we are indeed measuring a local aggregate reliability eﬀect, we
perform two checks.

First, we calculate the intra-cluster correlation in the number of reported outages at follow-
up. We have argued that intense saturation of treatment households within a transformer is
causing a reduction in outages at the transformer-level. Then, household reported outages
should be correlated with other household responses within the transformer. Our calculation
indicates a intra-class correlation of 0.56, which means that responses within transformers
are indeed highly correlated.60

Second, we consider the estimated reduction in outages in conjunction with the diﬀerence
between the expected and the estimated reductions in household electricity consumption to
conﬁrm that our series of results tell a consistent story. We make this comparison for the
winter months, as that is the season for which we collected outage data.61 In Section 5, we
expected a reduction in households’ winter electricity consumption of approximately 42 kWh
per month (Appendix Table 3). However, what we actually estimated was a winter reduction
in electricity consumption of approximately 26 kWh per month (Appendix Table 4). This is a
diﬀerence of 16 kWh per month, which is equivalent to approximately 14.4 hours of additional
electricity consumption. In Section 6, we found the aggregate impacts of these electricity
savings to be two fewer days without electricity per month due to unplanned outages. In
conjunction with the previous result, this would mean two outages of approximately seven
hours in duration each. This is very reasonable and assures us of the consistency across
results.62
  60
     Responses won’t be perfectly correlated with one another, as households in a transformer were surveyed
on diﬀerent dates and therefore the reference point of “past month” diﬀers across households within a
transformer.
  61
     Winter is when outages are most prevalent and therefore when we would expect to see an aggregate
reliability impact, if one occurs.
  62
     Anecdotal evidence from both consumers and the utility indicate that outages can last between a few
hours to a few days, depending on the repair required following the transformer overload.




                                                    25
7.3     Evidence on alternative mechanisms
Although we cannot perform a direct test to rule out the possibility of a rebound, a number
of analyses are inconsistent with a direct or indirect rebound in electricity consumption. For
instance, in Figure 3 we provided an event-study graph of the impacts on electricity consump-
tion by season. We found that following a winter spike, savings in electricity consumption
returned to the expected amount in the spring.63 If there were a rebound, a persistent change
in behavior would be expected, and a reversal in the spring would have been unlikely. Fur-
thermore, using our detailed baseline and follow-up survey data, we can test for impacts of
treatment on appliance ownership and average monthly and daily use (results not shown).64
We found no eﬀects of treatment on lightbulb use (which would be the measure of a direct
rebound) and only one signiﬁcant eﬀect of treatment on use of any household appliances
(which would be a measure of an indirect rebound). In fact, treatment households were
signiﬁcantly less likely to report using electric heaters at follow-up. If anything, this ﬁnding
is evidence counter to a rebound eﬀect.


7.4     External validity
We implemented a randomized saturation design to study the impact of energy eﬃcient
lightbulbs on household electricity consumption and local electricity reliability in a lower-
middle-income developing country. A policy-relevant question that guides our thinking about
external validity is to what extent our results are location-speciﬁc or technology-speciﬁc.

A feature of this study’s setting, the Kyrgyz Republic, is that essentially all households are
electriﬁed, formally connected to the network and metered individually. We believe that
research in this context does not harm the external validity of our study, rather it just helps
with the internal validity. It has the additional beneﬁt of ensuring that we are not just
studying the responses of the richest households. The Kyrgyz Republic also has an extensive
electricity generation and distribution infrastructure. However, similar to other developing
countries, the Kyrgyz Republic has a problem of insuﬃcient infrastructure capacity relative
  63
    This is also supported by the regression results by season in Appendix Table 4.
  64
    In both survey rounds we ask a number questions regarding the household use of appliances, including
(1) “On average, how many days per month do you use [the appliance]?” (2) “On average, how many hours
per day does your HH use [the appliance]?” Important to note, these speciﬁc survey questions are asking
about average use in the a month and in a day. In contrast, our survey question on outages asks about “the
past month.” For this reason, that we ﬁnd no impacts on the use of lightbulbs is not inconsistent with our
aggregate reliability results.



                                                   26
to demand. Constraints in the distribution system, more frequently than other factors lim-
iting generation capacity, are a source of unplanned outages in developing countries. Thus,
electricity reliability is largely driven by congestion within the electricity distribution net-
work. Aggregate electricity demand relative to infrastructure capacity, not infrastructure
alone, determines the probability of system overload, which results in unplanned outages.
As such, we believe this work is relevant to many developing countries in which demand for
electricity is either currently pushing the capacity of the current infrastructure or is rapidly
increasing such that we expect the constraint to be binding in the near future.

Our study setting is also unique in that it allows us to generate agregate reliability eﬀects
and observe technological externalities with relatively low treatment saturation. However,
the point we are making is not that at some particular saturation of energy eﬃciency, tech-
nological externalities can be induced. Rather, we illustrate that these externalities are
possible to generate and that they are crucial in understanding the welfare implications of
such energy eﬃciency programs. Although similarly low saturation levels in other settings
may not induce such eﬀects, their existence should not be ruled out. Instead, we should ask
how high the saturation ought to be to generate such aggregate eﬀects and externalities. The
answer will depend on many factors, including the expected technologically feasible impact
of the technology distributed, the type of aggregate impact a program should induce, the
capacity of the electricity infrastructure, the number of consumers the infrastructure serves,
etc. Given a variety of factors, a program might need a higher or lower saturation to achieve
agregate eﬀects and technological externalities. They may not be limited to a reduction in
outages, but may take other forms such as reductions in prices due to lower peak loads.

Although deployment of CFL technologies in higher income countries may not have similar
impacts, we believe our ﬁndings are not just speciﬁc to energy eﬃcient lighting or developing
countries. In developing countries, lighting comprises a substantial portion of electricity
consumption for many households. For this reason, an energy eﬃcient lighting intervention
can reduce a household’s electricity bill by a substantial proportion. In wealthier settings,
such as the United States and other developed countries, where households own a greater
number of electricity-using durables, introducing energy eﬃcient lighting would likely result
in a smaller reduction in the electricity bill proportional to the total. In such settings, a
program wishing to induce an aggregate impact may need to focus on a technology that
accounts for a larger proportion of the electricity bill.


                                              27
8     Conclusions
Through an experiment with a randomized saturation design, we provide several substantial
contributions to the literatures on energy eﬃciency, electriﬁcation, and electricity reliability.
We show that energy eﬃcient lightbulbs can indeed lead to signiﬁcant reductions in elec-
tricity consumption. We ﬁnd that controlling for spillovers is critical, suggesting that some
estimates of energy eﬃciency that do not account for such spillovers may be downward biased.

In addition, we show that the energy eﬃcient technology, when taken up at a high enough
saturation levels, can have a local aggregate reliability eﬀect, in the form of fewer days
without electricity due to outages at the transformer level. By improving electricity service
reliability, the energy eﬃcient technology becomes more valuable; households can use the
CFL for more hours when the electricity service is more reliable (i.e. providing electricity
for more hours per month at a lower cost than traditional lightbulbs). This is a classic
example of a technological externality, through which the returns to a particular technology
are increasing with the number of other adopters. The results in both Table 1 and those
in Appendix Table 4 highlight ways in which estimates of the eﬀects of energy eﬃciency
could be biased, if we do not account for spillovers in take-up, potential externalities, and
heterogeneity in impacts across seasons.

Our study also highlights that an increase in consumption following the introduction of CFLs
is a welfare improvement, not a sign of ineﬀective technology. Other technologies inducing
positive externalities may create incentives for households to free-ride on the adoption by
others. We show that in this case, in which the aggregate eﬀect increases the returns to the
technology, the externality may ameliorate (or even oﬀset) the incentive to free-ride. Think-
ing about this interaction between technological externalities and incentivizing adoption is
important for both policy design and the development of new technologies.

As a result of this analysis, we can perform several variations of cost-beneﬁt analyses both
with and without accounting for the aggregate beneﬁts of improved electricity service reli-
ability. These calculations, shown in Appendix Calculation 4, demonstrate that accounting
for externalities in the welfare calculations is crucial. Beneﬁt calculations that include both
reductions in electricity consumption and increased electricity services are more than double
the estimated beneﬁts from electricity savings alone in the ﬁrst year post-adoption (approx-
imately $15 in estimated beneﬁts instead of $7). The beneﬁts in year 1 of the program are

                                               28
substantially larger than the upfront cost of purchasing and distributing the CFLs (approx-
imately $9 per household). These simple calculations provide a lower bound estimate of
beneﬁts from such energy eﬃciency distribution programs, given they do not account for
other beneﬁts, such as pollution reductions. An energy eﬃciency distribution program, such
as ours, looks much more favorable after making this correction.

Finally, the paper provides a novel application of a randomized saturation design at a policy-
relevant and technologically meaningful scale. In doing so, we demonstrate the usefulness of
such a design to inform our understanding of aggregate impacts and technological external-
ities from various interventions. This methodology can be applied to study other topics for
which decomposing private returns and technological externalities is important in measuring
the impacts of technology adoption and choosing between various policy options or program
subsidy levels.




                                             29
References
Alam, M. 2013. “Coping with Blackouts: Power Outages and Firm Choices.” Working
 paper.
Allcott, H., A. Collard-Wexler, and S. O’Connell. 2015. “How Do Electricity Short-
 ages Aﬀect Industry? Evidence from India.” American Economic Review, forthcoming.
Allcott, H., and D. Taubinsky. 2015. “Evaluating Behaviorally-Motivated Policy: Ex-
 perimental Evidence from the Lightbulb Market.” American Economic Review, 105(8).
Allcott, H., and M. Greenstone. 2012. “Is there an Energy Eﬃciency Gap?” Journal of
 Economic Perspectives, 26(1).
Allcott, H., and R. Sweeney. 2015. “The Role of Sales Agents in Information Disclosure?
 Evidence from a Field Experiment.” Management Science, forthcoming.
Allcott, H., and S. Mullainathan. 2010. “Behavior and Energy Policy.” Science,
 327(5970).
Amankulova, K. 2006. “Transmission and Congestion Management in the Kyrgyz Re-
 public.” National Agency for Anti-Monopoly Policy and Development of Competition
 (NAAPDC), Bishkek, Kyrgyzstan.
Baird, S., A. Bohren, C. McIntosh, and B. Ozler. 2014. “Designing Experiments to
 Measure Spillover Eﬀects.” World Bank Policy Research Working Paper, 6824.
Banerjee, A., R. Chattopadhyay, E. Duﬂo, D. Keniston, and N. Singh. 2014.
 “Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives,
 Managerial Auronomy, and Training.” Working Paper.
Borenstein, S. 2015. “A Microeconomic Framework for Evaluating Energy Eﬃciency Re-
 bound And Some Implications.” Energy Journal, 36(1).
Breza, E. 2015. “Field Experiments, Social Networks, and Development.” Working paper.
Bruhn, M., and D. McKenzie. 2008. “In Pursuit of Balance: Randomization in Practice
 in Development Field Experiments.” World Bank Policy Research Working Paper, 4752.
Burlig, F., C. Knittel, D. Rapson, M. Reguant, and C. Wolfram. 2016. “Learning
 from Schools about Energy Eﬃciency.” Working Paper.
Chakravorty, U., M. Pelli, and B. Marchand. 2013. “Does the Quality of Electricity
 Matter? Evidence from Rural India.” Working Paper.
Chan, N., and K. Gillingham. 2015. “The Microeconomic Theory of the Rebound Eﬀect
 and its Welfare Implications.” Journal of the Association of Environmental & Resource
 Economists, 2(1).

                                          30
Cohen, J., P. Dupas, and S. Schaner. 2015. “Price Subsidies, Diagnostic Tests, and
 Targeting of Malaria Treatment: Evidence from a Randomized Controlled Trial.” The
 American Economic Review, 105(1).

Costolanski, P., R. Elahi, A. Iimi, and R. Kitchlu. 2013. “Impact of Evaluating a
 Free-of-Charge CFL Bulb Distribution in Ethiopia.” World Bank Working Paper, 6383.

Davis, L., A. Fuchs, and P. Gertler. 2014. “Cash for Coolers: Evaluating a Large-Scale
 Appliance Replacement Program in Mexico.” American Economic Journal: Economic
 Policy, 6(4).

Dinkelman, T. 2011. “The Eﬀects of Rural Electriﬁcation on Employment: New Evidence
 from South Africa.” American Economic Review, 101(7).

DOE. 2009. “How Compact Fluorescents Compare with Incandescents.” U.S. Department
 of Energy, Washington, D.C.

Dupas, P. 2014. “Short-Run Subsidies and Long-Run Adoption of New Health Products:
 Evidence from a Field Experiment.” Econometrica, 82.

Foster, A., and M. Rosenzweig. 2010. “Microeconomics of Technology Adoption.” An-
  nual Review of Economics, 2010(1).

Gassmann, F. 2014. “Switching the Lights Oﬀ: The Impact of Energy Tariﬀ increases on
 Households in the Kyrgyz Republic.” UNU-MERIT Working Paper Series, 2012(066).

Gillingham, K., and K. Palmer. 2014. “Bridging the Energy Eﬃciency Gap: Policy
 Insights from Economic Theory and Empirical Analysis.” Review of Environmental Eco-
 nomics & Policy, 8(1).

Gillingham, K., D. Rapson, and G. Wagner. forthcoming. “The Rebound Eﬀect and
 Energy Eﬃciency Policy.” Review of Environmental Economics & Policy, forthcoming.

Gine, X., and G. Mansuri. 2011. “Together We Will: Experimental Eidence on Demale
 Voting Behavior in Pakistan.” World Bank Policy Research Working Paper, 5692.

Glover, J.D., M.S. Sarma, and T. Overbye. 2011. Power Systems Analysis and Design.
 . 5th ed., Stamford, CT:Cengage Learning.

Godlonton, S., and R. Thornton. 2012. “Peer Eﬀects in Learning HIV Status.” Journal
 of Development Economics, 97.

Hausman, J.A. 1979. “Individual Discount Rates and the Purchase and Utilization of
 Energy-Using Durables.” The Bell Journal of Economics, 10(1).

Jaﬀe, A., and R. Stavins. 1994. “The Energy Eﬃciency Gap.” Energy Policy.



                                         31
Klytchnikova, I., and M. Lokshin. 2009. “Measuring Welfare Gains from Better Quality
 Infrastructure.” Journal of Infrastructure Development, 1(2).

Lipscomb, M., A.M. Mobarak, and T. Barnham. 2013. “Development Eﬀects of Elec-
  triﬁcation: Evidence from the Geological Placement of Hydropower Plants in Brazil.”
  American Economic Review: Applied Economics.

List, J., A. Shaikh, and Y. Xu. 2016. “Multiple Hypothesis Testing in Experimental
  Economics.” NBER Working Paper, 21875.

McRae, S. 2015. “Infrastructure Quality and the Subsidy Trap.” American Economic Re-
 view, 105(1).

McRae, S., and R. Meeks. 2016. “Price Perception and Electricity Demand with Non-
 linear Prices.” Working Paper.

Meeks, R. 2016. “Demand for Energy Eﬃciency in Developing Countries.” Working Paper.

Miguel, E., and M. Kremer. 2004. “Worms: Identifying Impacts on Education and
 Health in the Presence of Treatment Externalities.” Econometrica, 72(1).

Newell, N., and J.V. Siikamaki. 2015. “Individual Time Preferences and Energy Eﬃ-
 ciency.” The American Economic Review, 105(5).

Obozov, A., R. Isaev, V. Valiyev, Y. Hasanov, I. Mirzaliyev, and F.
 Imamverdiyev. 2013. “Prospects of use of renewable energy resources and energy-eﬃcient
 technologies in Azerbaijan and Kyrgyzstan.” EcoMod Network Report, Baku, Azerbaijan.

Rud, J.P. 2012. “Electricity Provision and Industrial Development: Evidence from India.”
 Journal of Development Economics, 97(2).

Van de Walle, D, M Ravallion, V Mendiratta, and G Koolwal. 2013. “Long-term
 Impacts of Household Electriﬁcation in Rural India.” World Bank Working Paper.

Wolfram, C., P. Gertler, and O. Shelef. 2012. “How Will Energy Demand Develop in
 the Developing World?” Journal of Economic Perspectives, 26(Winter).

World Bank. 2006. “Improving Lives: Program on Renewable Energy and Energy Eﬃ-
 ciency.” The World Bank Group, Washington, D.C.

Zozulinsky, A. 2007. “Kyrgyzstan: Power Generation and Transmission.” BISNIS, Wash-
  ington, D.C.




                                          32
Figure 1: Randomized saturation process




                  33
    Figure 2: Stylized example of randomized saturation design




                                                                  

 




                               34
Figure 3: Predicted and actual electricity reductions (kWh per month)




 Notes: Distribution of CFLs began in March 2013, so by design the expected impacts are zero up until that time.




                                               35
      Table 1: Impacts of CFL treatment on household electricity consumption



               Dependent Variable: Monthly Household Electricity Consumption (kWh)
                                            (1)                          (2)                          (3)

Treated*post                            -16.269*                    -29.949***                   -29.949***
                                         (8.803)                      (11.399)                     (10.434)
Control in treated TG*post                                           -25.086**                    -25.086**
                                                                      (12.579)                     (12.384)

Omitted group                     All control houses         Houses in Control TGs Houses in Control TGs


Std error cluster level                Household                    Household                   Transformer
Households                               899                          899                           899
Observations                            31,143                       31,143                        31,143
Notes: Results are intent-to-treat. All regressions include month-by-year fixed effects, household fixed effects, and
controls for heating degree days, number of days in monthly billing period, whether the household uses electricity for
heating, and dummy variables for Treated and Post. Columns 2 and 3 also include a dummy variable for being a
control household in a treated transformer. Each month drops the top 1% of observations with respect to electricity
use. All regressions drop households that moved during period between intervention and follow-up survey (101
households). "TG" is the abbreviation for transformer group. Standard errors are in parentheses, with * significant
at 10% level; ** significant at 5% level; and *** significant at 1% level.




                                                     36
Figure 4: Number of days without electricity, by transformer-level treatment status




                                     Outages by transformer treatment status
                                     Control transformers                     Treated transformers
                            50
                            40




                                 .
      Percent of reponses
                            30
                            20
                            10
                            0




                                 0              5 to 9              15+   0              5 to 9              15+
                                       1 to 4            10 to 14               1 to 4            10 to 14
                                         Number days without electricity (in past month)




                                                                    37
       Table 2: Technological externalities: improved electricity reliability



          Dependent Variable: Number of Days Without Electricity (in past month)
                                                               (1)                           (2)
TG low saturation                                            -1.321                         -1.164
                                                            (0.851)                        (0.868)
                                                            [0.433]                        [0.434]

TG high saturation                                         -1.866**                      -2.162***
                                                            (0.812)                        (0.822)
                                                            [0.000]                        [0.000]

Household treatment status controls                           No                            Yes
Constant                                                   3.810***                      3.811***
                                                            (0.836)                       (0.838)

p-value: TG low = TG high                                     0.228                         0.047

Observations                                                   838                           838
R-squared                                                     0.051                         0.053
Notes: "TG low" is a dummy variable that equals 1 if 10 to 14 percent of households in a transformer were assigned to
treatment. "TG high" is a dummy variable that equals 1 if 15 to 18 percent of households in a transformer were assigned
to treatment. Omitted group is comprised of households in control TGs. All columns control for the number of
households in the transformer. The "Household treatment status controls" are separate binary indicators that equal one for
treated households. Standard errors are clustered at the transformer level and shown in parantheses, with * significant at
10% level; ** significant at 5% level; and *** significant at 1% level. P-values accounting for multiple-hypothesis
testing (List, Shaikh, and Xu, 2015) are shown in brackets.




                                                      38
Table 3: Technological externalities: heterogeneous impacts by transformer treatment
         saturation




               Dependent Variable: Monthly Household Electricity Consumption (kWh)
                                                                             (1)                        (2)


    Treated household in TG low * Post                                  -42.798***                 -41.392***
                                                                         (12.025)                    (13.320)
    Treated household in TG high * Post                                  -22.269*                    -20.337
                                                                         (11.911)                    (13.752)

    Control household in TG low * Post                                  -42.986***                 -50.243***
                                                                         (13.713)                    (13.769)
    Control household in TG high * Post                                   13.014                      21.283
                                                                         (16.355)                    (17.893)

    Controls for reported outages                                         No                         Yes
    Omitted group                                                   Houses in Control          Houses in Control
                                                                     Transformers               Transformers
    Observations: households                                               31,143                    26,043
    Notes: The omitted group is comprised of houses in control transformers. "TG low" is a dummy variable that
    equals 1 if 10 to 14 percent of households in a transformer were assigned to treatment. "TG high" is a dummy
    variable that equals 1 if 15 to 18 percent of households in a transformer were assigned to treatment. All
    regressions include time fixed effects, household fixed effects, and controls for heating degree days, number of
    days in monthly billing period, and the use of electric heater. Each month drops the top 1% of observations with
    respect to electricity use. All regressions drop households that moved during period between intervention and
    follow-up survey (101 households). Standard errors are clustered at the transformer level, with * significant at
    10% level; ** significant at 5% level; and *** significant at 1% level.




                                                       39
                       Table 4: Spillovers: CFL stock at follow-up



              Dependent Variable: Number of CFLs at Follow-up
                                    (1)          (2)                                  (3)


Treat                                       0.185              0.442**             0.479**
                                           (0.174)             (0.212)             (0.219)
C close to Treat                                               0.458**
                                                               (0.207)
Control in treated TG                                                              0.502**
                                                                                   (0.205)

Constant                                  0.617***             0.365*              0.332*
                                           (0.187)             (0.204)             (0.199)

R-squared                                   0.012               0.013               0.012
Omitted group                           All controls        Controls far        Controls in
                                                              from T            control TGs

Observations                                 834                 834                 834
Notes: All specifications control for the total number of households in the transformer and
the number of CFLs received at baseline through the project. A "C close to Treat" is an
indicator for a control household located < 100 meters from treated household. "Controls far
from T" are control households location > 100 meters from a treated household. Standard
errors are clustered at the transformer level, with * significant at 10% level; ** significant at
5% level; and *** significant at 1% level.




                                                40
Appendix: For on-line publication




                          41
                                               Appendix Table 1: Transformer-level randomization check


                                                                                                                                   Joint F tests (p-value)
                                                                                              Lower            Higher
                                                       Number of           Control                                       Control = Control =               Low =
                                                                                           Transformer       Transformer
                                                      Transformers       Transformer                                       Low       High                   High
                                                                                            (<=14%)            (>14%)
                                                            (1)               (2)               (3)               (4)             (5)          (6)           (7)

     Panel A: Household-level
     HH head completed seconday school                      111               0.8              0.88             0.74            0.415         0.572        0.111
     Household income (Kyrgyz soms)                         111            13107.08          12647.62         13285.47          0.842         0.938        0.748
     Own house                                              111              0.88              0.95             0.84            0.380         0.602        0.105
     Private house                                          111              0.88              0.83             0.74            0.649         0.185        0.312
     Number of rooms                                        111              4.32              4.50             4.14            0.584         0.582        0.203
     Lightbulbs in house                                    111              6.36              6.21             6.23            0.831         0.851        0.975

     Panel B: Transformer-level




42
     Years since last maintenance                           102               3.38              3.03              3.90          0.568         0.385        0.101
     Total # of households                                  110              51.80             63.12             47.58          0.126         0.565        0.015
     Total # of households with 3 phase meter               110              12.92             15.36             15.81          0.324         0.240        0.829
     Proportion of HH with 3 phase meter                    110               0.28             0.28              0.34           0.951         0.102        0.068

     Notes: Panel A calculated using responses from the baseline household survey. 20% of all households in a transformer were surveyed at baseline. These
     household responses were then used to create transformer-level averages. Panel B is calculated using transformer-specific data provided by the electricity
     utility. Exchange rate in March 2013 was 1 USD = 48 KGS.
                  Appendix Table 2: Household-level randomization check



                                                                                                           Joint F tests
                                                                    All        Control      Treatment
                                                                                                            (p-value)
                                                                    (1)           (2)           (3)             (4)

General characteristics
Household head completed secondary school                          0.840         0.867         0.818           0.090
Household income past month (KGS)                                  10900         11463         10427           0.138
Household income past month per capita (KGS/person)                 3668          3740          3608           0.603
Owner-occupied house                                               0.912         0.919         0.906           0.506
Number of people living in the home                                  3.6           3.7           3.5           0.218
Time at address (months)                                            203           201         204.137          0.789

Housing characteristics
Single-family dwelling                                             0.793         0.829         0.762           0.053
Number of rooms                                                    4.302         4.245          4.35           0.409
Home made from brick                                               0.535         0.569         0.507           0.100
Floors that are wood                                               0.877         0.864         0.887           0.388
Age of dwelling (years)                                            41.29         41.27         41.30           0.987
Electricity meter for single house                                 0.991         0.993         0.989           0.546

Electricity consumption practices
Total number of appliances                                         8.393         8.578         8.238           0.210
Lighting hours per day                                              17.5          17.9          17.2           0.643
Think about saving electricity                                     0.946         0.934         0.955           0.500
Do something to save electricity                                    0.86         0.829         0.885           0.185
Total light bulbs in house                                          6.2           6.5           6.0            0.128
Total incandescent bulb in house                                    6.1           6.3           5.8            0.177
Believe CFL use less energy                                        0.305         0.319         0.292           0.436
Rooms heated in winter                                              3.14          3.12          3.15           0.764
Days without electricity in the past month                          1.66          1.58          1.75           0.338

Number of households                                               1000           457           543
Note: In March 2013, the exchange was 1USD = 48 KGS. For these calculations, the winter months include November through
February and summer months include May through August. Baseline surveys were implemented in the late spring and early
summer.




                                                      43
 Appendix Figure 1: Seasonality of electricity consumption pre-treatment




                    Baseline Monthly Electricity Consumption
     700
     600
      500
   kWh
400  300
     200
            Sep11




                     Dec11




                                                Jun12




                                                        Sep12




                                                                 Dec12
                              Mar12




                                                                           Mar13
                                      Control           Treat




                                      44
Appendix Calculation 1: Number of residential buildings
To control for the total number of households within various radii of distances from project
households, we use spatial data on residential building locations available through Open-
StreetMap.org. The ArcGIS building layer for the Kyrgyz Republic was downloaded from
OpenStreetMap in March 2015. We calculate the total number of residential buildings within
radii of project households. Radii were of the following sizes: 100 meters, 200 meters, 300
meters, and 400 meters.

The following building types are considered residential for the calculation: residential, house,
farm, dormitory, and apartment. Industrial and commercial buildings were omitted from the
residential building count. ArcGIS aerial photographs were used to cross-check the building
counts and also to manually add building polygons in areas not completely covered in data
from OpenStreetMap.

Calculations needed to account for multi-family buildings, which are prevalent in the study
district. Buildings were deﬁned to be multi-family residential buildings if (1) the building
polygon has an area of 369 square meters or more or (2) at least one side of the building
is longer than 19.4 meters. These thresholds were made based on a random sampling of
buildings and visual interpretation of the images. Fortunately for variable construction,
essentially all multi-family residential buildings in this region were constructed during the
Soviet Union and were built according to very standardized speciﬁcations. This permits us
to make several assumptions regarding the number of households per building. We assume
these multi-family buildings have 5 ﬂoors each and that each stairwell has 3 units per ﬂoor.
The number of stairwells assumed depends on the size of the building.




                                              45
Appendix Calculation 2: Technologically feasible electricity savings
from CFLs
The expected technologically feasible electricity savings from the treatment can be calculated
through the following equation:

Expected electricity savings/month = # Incandescent bulbs replaced with CFLs * (CFL
wattage - Incandescent wattage) * (# hours Incandescent bulbs used per day) * (days in a
monthly billing period)

We calculate the technological feasible electricity savings using data from the project pilot in
Fall 2012, the baseline survey in Spring 2013, and the intervention through which CFLs were
distributed. On average, treatment households received 3.2 CFLs through the intervention.

We know from piloting exercises and the baseline survey data that 100 watt incandescent
bulbs were most common in households prior to the intervention. We selected 21 watt CFL
bulbs as replacements because they were rated to be 100 watt equivalent bulbs. Therefore
we know that typical household in our treated group is shifting from 100 watt to 21 watt
bulbs.

The calculations on lightbulb usage employ data on the self-reported hours of lighting use at
baseline. Estimates of hours of lighting use throughout the year are extrapolated using data
on the timing of sunrise and sunset in the region. These predictions assume behavior with
respect to lighting and other electricity uses remain constant after the intervention, which is
consistent with our results comparing various behaviors at baseline and follow-up.

These calculations of the expected impacts on electricity consumption are shown in the table
below. Based on these data, electricity consumption is estimated to decrease by between
26 kWh per month (summer) and 42 kWh per month (winter). The percent by which the
electricity bill is expected to decrease in each season is also calculated.




                                              46
               Appendix Table 3: Expected impacts on electricity consumption



Assumptions                                             Winter scenario              Spring/Fall             Summer scenario
Average number of light bulbs replaced                       3.2                         3.2                      3.2
Incandescent wattage                                         100                         100                     100
CFL wattage                                                   21                          21                      21

Average hours per use per day                                   5.5                        4.5                       3.5
Average monthly bill (kWh)                                      586                        340                       245
Prop bill in lighting baseline                                 0.090                      0.127                     0.137

Expected CFL savings (kWh)                                    41.712                     34.128                     26.544
Expected reduction in bill (no rebound)                        7.1%                      10.0%                      10.8%
Notes: These calculations are for an estimated scenario. For these calculations, the winter months include November through
February; spring/fall months include March, April, September, and October; and summer months include May through August.
Average number of light bulbs replaced is based on the actual numbers of CFLs distributed on average through the intervention.
Average hours of use per day are calculated using the baseline survey data and data on sunrise and sunset to estimate for the rest
of the year. CFL wattage is the actual wattage for the light bulbs that were distributed. Incandescent wattage is the typical wattage
found in households at the time of the baseline survey. Calculations are assuming an average of 30 days per month. Average
monthly electricity bill is calculated using baseline electricity use amongst households in our sample during the year prior to the
intervention.




                                                            47
Appendix Table 4: Impacts of treatment on household electricity consumption by season



                                              (1)                          (2)                        (3)
                                            Winter                     Spring/fall                  Summer
                                           Electricity                 Electricity                 Electricity
                                       consumption (kWh)           consumption (kWh)           consumption (kWh)
Treated*post                                   -25.538                  -32.375***                     -8.835
                                              (32.171)                    (10.985)                    (6.672)
Control in treated TG*post                     -26.314                     -8.373                     -12.114
                                              (35.203)                    (13.906)                    (8.041)

Households                                      899                         899                         899
Observations                                   10680                        9784                       10679
Notes: The omitted group is comprised of households in control transformers. All regressions include time fixed effects,
household fixed effects, and controls for heating degree days, number of days in billing period, and dummy variables for
Treated and Post. Columns 2 and 3 also include a dummy variable for being a control household in a treated transformer.
Each month drops the top 1% of observations with respect to electricity use. All regressions drop households that moved
during period between intervention and follow-up survey (101 households). Standard errors are clustered at the
transformer level, with * significant at 10% level; ** significant at 5% level; and *** significant at 1% level.




                                                      48
Appendix Calculation 3: Back-of-the-envelope expected peak load
reduction
Times of peak demand are in the early morning and in the evening. Lighting is dispropor-
tionately “on peak.” To better understand the impact that switching from incandescent
bulbs to CFLs could have on peak load, we perform a back-of-the-envelope calculation based
on data from our sample and some informed assumptions. The calculation is as follows:

   • We assume a household’s winter monthly electricity demand is 630 kWh per month,
     which is based on data from our sample.

   • Dividing this by the number of days per month and number of hours per day, we
     estimate an average hourly winter electricity demand of .875 kW.

   • We assume that peak load is approximately 70% more than average load, which is
     in line with the U.S. Energy Information Administration’s calculations for peak-to-
     average electricity demand ratios. Therefore, household peak is 1.49 kW.

   • We assume the household has 4 incandescent bulbs (100 W each) that are replaced by
     CFLs (21 W) used in our project. This change would reduce peak load by 0.32 kW.

   • Given our estimated peak demand of 1.49 kW, reducing peak load by 0.32 kW repre-
     sents a 21% reduction in peak demand for a household.

   • For a transformer with 20% of households making this shift to CFLs, this would mean
     a 4% reduction in peak load for the transformer.




                                           49
Appendix Calculation 4: Cost-beneﬁt analysis of the CFL program
To understand the welfare implications of such a CFL distribution program, we perform
some simple cost-beneﬁt calculations. By using the estimated impacts of CFLs on electric-
ity savings and the aggregated impacts on reliability of electricity services, we are able to
demonstrate the implications of performing these welfare calculations both with and without
accounting for reliability improvements.

To simplify these calculations, we perform the cost-beneﬁt analysis for the ﬁrst year of the
CFL program. The one-year analysis is suﬃcient to demonstrate the importance of the re-
liability impacts for welfare calculations. In addition, this simpliﬁcation is useful for several
reasons, in that we: can use the estimated coming from our experiment, which measures
impacts over the course of 18 months following the CFL distribution; avoid having to make
assumptions about the life span of the CFLs; do not have to worry about multi-year equilib-
rium adjustments in consumption; and, ﬁnally, we need not make any assumption regarding
the discount rates.

Cost calculations
We perform program cost calculations from the perspective of a government entity imple-
menting an energy eﬃciency program through a door-to-door campaign. These calculations
are made based on a CFL distribution program with the design of our experiment: in which
CFLs are distributed through individual house visits, at which time information on the ben-
eﬁts of CFL adoption are provided to households. Incandescent bulbs currently in-use at
the households are not taken from the households. To encourage households to install the
CFLs quickly, the entity distributing the CFLs can remove the packaging at the time of dis-
tribution. Although such door-to-door campaigns may be eﬀective at inducing technology
take-up, this is one of the more expensive distribution options available. Cheaper distri-
bution programs include ones that distribute coupons at stores or through mailings, which
permit households to receive the technology for free or a subsidized price.

In the calculations, we divide costs into two components: the cost of CFL purchase and
the cost of distributing the technology through the door-to-door campaign. Calculations are
shown in Appendix Table 5. We base these calculations on details from our own experiment,
such as the price per CFL, the number of households served by the program, the average
number of CFLs distributed per household, etc. These calculations do diﬀer from our ex-


                                               50
periment in that here we assume the government bears 100% of the program costs. This
need not be the case given that we ﬁnd households are willing to pay a positive price for the
CFLs (Meeks, 2016). We can shift the assumptions as to the number of households such a
door-to-door campaign can reach per day, but such shifts do not alter the costs substantially.

Beneﬁt calculations
We perform three versions of beneﬁt calculations for such a CFL distribution program, as
shown in Appendix Table 6. Important to note, these calculations do not include the value
of any reductions in pollution resulting from the CFL adoption.

Version A is our most simpliﬁed calculation of average beneﬁts for households in all trans-
formers. This is based on the estimate of electricity consumption impacts from Table 1,
Column 2. This estimate does not account for any aggregate impacts in reliability of elec-
tricity services and is therefore an underestimate of the beneﬁts. Even so, the beneﬁts per
household in the ﬁrst year are approximately $1.20 less than the costs per household.

Version B estimates the beneﬁts from the CFLs among households that do not have any
changes in reliability of electricity services. These calculations use estimated electricity sav-
ings amongst treated households in transformers not experiencing any reliability improve-
ments (see Table 4, Column 1). Here the beneﬁts per treated household in year 1 are greater
than the costs per household.

Finally, Version C of the calculation includes the beneﬁts from the CFLs amongst households
that experience improvements in the reliability of electricity services. These calculations use
the reduction in electricity consumption amongst treated households experiencing reliability
improvements (see Table 4, Column 1). Part 2 of these calculations are still likely an under-
estimate of household beneﬁts given that electricity prices were very low. In this calculation,
the beneﬁts per treated household in year 1 are nearing double the costs per household.




                                               51
                  Appendix Table 5: Costs of CFL Distribution Program


          COSTS FOR CFL DISTRIBUTION PROGRAM

Part 1:   CFL purchase cost
          Average # CFLs distributed                      3.2     per household
          Cost per CFL                                    120     KGS
          Cost per household                              384     KGS
          Number of households                            543
          Total CFL purchase cost                       208512    KGS

Part 2:   CFL Distribution cost
          Number of households                           543
          Households visited per day                      12
          Time to distribute CFLs                         45      Days
          Cost per workday                               467      KGS
          Total distribution cost                       21132     KGS

Total Program Cost (Purchase+ Distribution):
          Costs                                     229643.75 KGS
          Exchange rate                               48.00   KGS = 1 USD
          Costs                                      4784.24 USD
Cost of Program Per Household                       $      8.81




                                               52
                                                      Appendix Table 6: Beneﬁts of CFL Distribution Program


                                                 BENEFITS OF CFL DISTRIBUTION PROGRAM IN YEAR 1
     VERSION A:                                             VERSION B:                                                 VERSION C:
     Basic calculations, households in all transformers     Benefits from CFLs, with no reliability change             Benefits from CFLs, with improved reliability
     Uses estimate of kWh reduction from Table 1, Col 2     Uses estimate for Treated households in low saturation     Uses estimate for Treated households in high saturation
                                                            transformers from Table 4, Col 1                           transformers from Table 4, Col 1

     Part 1: electricity savings due to CFLs                Part 1: electricity savings due to CFLs                    Part 1: electricity savings due to CFLs
     Average monthly savings              -30   kWh/month   Average monthly savings (kWh/m -42             kWh/month   Average monthly savings              -42    kWh/month
     Savings in a year                   -360   kWh/year    Savings in a year (kWh/ year)       -504       kWh/year    Savings in a year                   -504    kWh/year
     Price per kWh                       0.02   USD         Price per kWh                       0.02       USD         Price per kWh                       0.02    USD
     Value of savings                    -7.2   USD         Value of savings                   -10.08      USD         Value of savings                   -10.08   USD
     Absolute value of benefits         7.20    USD         Absolute value of benefits            10.08    USD         Absolute value of benefits         10.08    USD

                                                                                                                       Part 2: additional electricity consumed
                                                                                                                       Additional electricity consumptio 20        kWh/month
                                                                                                                       Additional electricity consumptio 240       kWh/year
                                                                                             .                         Price per kWh                       0.02    USD




53
                                                                                                                       Value of additional consumptio      4.8     USD

     BENEFITS / HOUSEHOLD             $ 7.20                BENEFITS / HOUSEHOLD                 $ 10.08               BENEFITS / HOUSEHOLD              $ 14.88