Lorebgmipllsumdolyorsltzmhhabc Land Governance POLICY BRIEF WORLD BANK ✦ DEVELOPMENT RESEARCH GROUP ISSUE 4 ✦ AUGUST 2018 Using Satellite Imagery to Assess Impacts of Soil and Water Conservation Measures Evidence from Ethiopia’s Tana-Beles Watershed ✳✳ Daniel Ayalew Ali, Klaus Deininger, and Daniel Monchuk Although efforts at Soil and Water Conser- vation (SWC) are viewed as instrumental in reducing vulnerability to climate change, their impact has rarely been quantified. Combining data on the timing and intensity of SWC inter- ventions in select Ethiopian watersheds from 2009 to 2016 with a pixel-level panel of vegeta- tive cover and soil moisture data derived from satellite imagery, makes it possible to assess the biophysical impacts of such measures using a difference-in-differences specifica- tion. Results point towards significant effects overall that vary by season. Tree planting and other SWC activities are more effective on degraded than on cultivated land. The results are consistent with before-after regressions for daily sediment load and stream flows in a subset of micro-watersheds. It thus appears that satellite imagery can improve the design Watershed Activities for Slope Stabilization and Productivity Improvement. Taken by: Water Resources Development Project. and near-real-time monitoring of sustainable Coordination Unit. land management interventions for both watersheds and landscape. (SWC) activities and the associated micro- even thousands of households depending economic costs and benefits, there is as on the basin under consideration and the yet scant evidence on how much impact water discharge point. Introduction such investments have on watersheds or landscapes—the units at which these in- Poor management of steep, erosion-prone, terventions are usually conducted. That Opportunity of new data and often fragile tropical soils increases the makes it difficult to assess whether or how and methodologies exposure and vulnerability of poor people effectively these programs achieve their to climate change; by contributing to loss short- and longer-term objectives and to Soil erosion and changes in water content of vegetation and soil fertility, it may also identify corrective actions in case they fail are difficult to observe, making traditional heighten the severity of climate-induced to do so. methods of information collection, such as shocks. Measures such as terracing, bund- household surveys, ineffective. Moreover, ing, micro-dam construction, tree planting, One critical issue is that SLM activities differ as interventions are applied to watersheds, and establishing agro-forestry systems have from the household-based interventions, the effects will be dispersed over large thus gained popularity and attracted major for which impact evaluation is well estab- areas affected by significant inter-tempo- resources. In Africa alone, since 2006, the lished. Its main goal is to make better use ral fluctuations of weather. Outcomes for World Bank has invested more than US$10 of vegetative cover to enhance the soil’s any period can be properly measured only billion in sustainable land management capacity to capture and store water, reduc- if those climate variables are considered (SLM) to reduce soil loss and increase soil ing run-off and arresting soil erosion, rather and require high frequency data in order moisture and vegetative cover. than improving livelihoods directly. The to establish causality by separating signal relevant unit of treatment is a watershed from noise. Finally, often no good base- Yet while numerous studies have explored or a landscape. Their geographical scale line or pre-intervention data are available the biophysical variables that may be af- is considerably larger, encompassing a for intervention areas. In principle, real- fected by SLM or soil and water conservation number of communities and hundreds or time feedback based on remotely sensed imagery could help make it possible not The main objective of TBWIRD was to the regressions as a proxy for treatment only to assess the extent to which desired improve vegetative cover in order to reduce intensity. impacts are realized, but also to adjust inter- water and sediment run-off so as to contrib- ventions in an iterative evidence-driven ute to the long-term economic well-being Multispectral Landsat 7 imagery can then process. Yet, although there are numerous of farmers, and increase their resilience to be used to create time series of vegetative studies of deforestation as an outcome vari- future climate change and severe weather. conditions on the ground before and after able, there has been little use of such data Table 1 illustrates that by the end of the proj- the project by computing indices that are at finer spatial and temporal resolution and ect in 2015, some 63 percent of cultivated routinely used, namely the Normalized Dif- applied to SLM, even though the potential land and 22 percent of degraded land had ference Vegetation Index (NDVI) and the to measure vegetative cover has been avail- been covered by SWC measures and 8 per- Soil Adjusted Vegetation Index (SAVI), to able for some time. cent of the project area with tree plantings. measure vegetation cover and photosyn- thetic activity and the Land Surface Water Recent advances in computing platforms Index (LSWI) as a measure of plant and soil and access to satellite data have greatly Table 1. Summary Statistics: water holding content. To assess the impact improved the ability to evaluate these Characteristics and Treatment Variables for Program of TBIWRD, we compute seasonal averages SLM programs. Free access to archived Watersheds for each index for 30x30 m pixels for the medium-resolution imagery and process- Variable Total 5-year period before the project started and ing power on cloud computing platforms, the 7 years of its activity. Basic characteristics such as Google Earth Engine (GEE), allows anyone to conduct analysis that a decade Total area watersheds (ha) 85,702 Rainfall data used in this study are based on ago would have been costly and time-con- Total treated area (ha), 2015 the Climate Hazards Group InfraRed Precip- suming even for experts. itation with Station data (CHIRPS), with daily Soil & water cons. (SWC) 72,634 rainfall measured at the five hydro-stations Accessing and processing a large amount Percent of total SWC on cultivated land 0.63 in the area. With respect to measurement of readily available remotely-sensed satellite Percent of total SWC on degraded land 0.22 of intensity of run-off and the amount of images via GEE, and combining them with Percent of total Area planted w. trees 0.08 sediment dissolved as measures of soil information on the timing and location of erosion, community members had been Source: Project data. interventions makes it possible to rigorously trained by TBIWRD to use these techniques evaluate the impact of SWC activities on soil and were regularly supervised—part of a water content and photosynthetic activity Evaluation design and data project initiative to build local capacity for at the pixel level. To demonstrate how such better understanding factors leading to soil data can provide evidence that can serve as Adoption of SLM practices can be affected erosion and downstream impacts of SWC a basis for informed decision-making, we by many factors including tenure security, measures. They took daily measurements use Ethiopia’s Tana Beles Integrated Water the time profile of benefits, partial appropri- of the quantity of water (in m3/s) and sed- Resource Development Project (TBIWRD). ability of benefits, and coordination failures. iment (in t/d), carried downstream at or We analyze the impact of three types of close to the point of main water discharge TBIWRD-supported interventions: (i) estab- from that micro-watershed from 2009, at SWC in Ethiopia lishment of SWC structures to limit run-off each of the five hydrological stations. These on land already cultivated, such as terraces, readings provide outcome measures that In Ethiopia, the total cost of economic bunds, and physical vetiver grass barriers; can help cross-check results based solely on losses from soil degradation due to changes (ii) conservation structures on non-culti- remotely-sensed data. in land use and land cover has been esti- vated land such as hillsides and land used mated at US$231 billion or 0.41 percent of for grazing and forestry, like construction of We defined a control area by drawing a total GDP, adding to the urgency of taking micro-basins, terracing, bunding, and area 5-km buffer around each of the “treated” action. Efforts to halt soil degradation are closures; (iii) new tree plantings to establish micro-watershed, which allows us to use a generally advocated as a promising strat- agroforestry systems for stabilizing land- difference-in-differences (DID) specification egy to not only increase farmer resilience scape and producing timber and fuelwood. with pixel-level fixed effects for identifica- to climate shocks in the short term, but also tion, to assess how the cumulative area to deal with the underlying causes in the Annual information for these treatment vari- treated affects mean photosynthetic activ- medium-term. Although large amounts ables is provided by the project monitoring ity, vegetation cover, and plant or soil water have been spent on SWC programs, their system: share of a micro-watershed covered content in different seasons. Variations impacts are not always evident. by SWC measures on cultivated or degraded over time in micro-watershed coverage, land and new tree planting respectively. together with the geographic discontinu- From 2009 through 2015, TBWIRD sup- Data on the precise location and timing of ity created by micro-watershed boundaries, ported watershed planning and institutional project interventions are unfortunately not can provide a basis for identifying project strengthening in the entire Tana-Beles available, imposing limitations in identifying impacts on NDVI, SAVI and LSWI, which can sub-basin and specific SLM activities in effects of specific measures and the extent then be evaluated beyond the effects of the five critical watersheds covering some 860 to which they vary with watershed features. project’s planning and institution-building km2. Each watershed comprises 25 to 40 Instead, the evaluation is done at the level components or other initiatives. The esti- micro-watersheds, defined as geographical of the micro-watershed for which we have mated effects of SLM investment are thus units with a single point of water discharge. aggregated annual data, which we use in likely to be conservative. Findings Our results support that TBIWRD project during the last year, consistent with the Two factors may impart additional down- activities stimulated improvements in soil notion that project benefits are sustained ward bias on our coefficient estimates: First, water content and vegetation cover. Results and accumulate over time. institution-building and training activities from the DID approach suggest that the three types of project-supported SWC investments had significant effects on soil Figure 1. Project Levels and Control Areas Before and During the Project water content and photosynthetic activity, a. NDVI the size of which increases significantly over time, implying that TBIWRD activities 0.36 improved soil water content and vegetation cover (see Figure 1). 0.355 Estimated impacts are largest for changes in soil water-holding capacity (approximated by the LSWI) and estimated elasticities are 0.35 highest for SWC on degraded land, followed by SWC on cultivated land and tree planting. 0.345 Analysis of daily volume and sediment content of water flows in the project area 0.34 showed a declining trend overall, and though less rigorous, suggests that the 2005_S1 2007_S1 2009_S1 2011_S1 2013_S1 2015_S1 impacts are not only statistically significant Time (Year_Season) but economically meaningful. This responds b. SAVI to the project objectives of reducing erosion 0.22 and improving soil water-holding capacity. The magnitude of estimated elasticities is 0.215 comparable to results from similar studies which argue that changes of this size in NDVI are associated with sizeable carbon sequestration that, at commonly assumed 0.21 carbon values, translate into significant benefits. 0.205 Variation also emerges between treatment types and outcome indicators and esti- mated impacts vary by season and index in 0.2 plausible ways. SWC activities are estimated 2005_S1 2007_S1 2009_S1 2011_S1 2013_S1 2015_S1 to increase soil water content in rainy and Time dry seasons, which allows more vegeta- c. LSWI tive cover to endure through the pre-rainy season. However, planting trees results in an 0.11 increase in vegetative cover that is spread equally across seasons. 0.105 Whether SLM interventions—in Ethiopia and elsewhere—are sustainable in the long term has been the subject of considerable 0.1 debate. Although too little time has elapsed since project closure to make any mean- ingful inferences on long-term effects from 0.095 the data, we calculated estimates of sus- tainability by using variation in the speed of project roll-out across micro-watersheds. 0.09 For all indices considered here, the esti- 2005_S1 2007_S1 2009_S1 2011_S1 2013_S1 2015_S1 mated impact of interventions completed Time two or more years ago is two to three times Outside project area Project area larger than the effect of those implemented Note: Three seasons per year (dry, S1; pre−rainy, S2; and main rainy season, S3). covered the entire river basin rather than Conclusion and policy assess not only overall impacts but also their just the intervention areas. Moreover, implications variation with initial conditions. anecdotal evidence points towards consid- erable contamination as the government This study demonstrates that freely available Georeferenced data on changes in local bio- also supported SLM-related initiatives in imagery, when combined with informa- physical and economic conditions can be the non-project watersheds we used as a tion on the location, type, and timing of based either on surveys at household- and control, creating a downward bias. Second, SWC efforts, offers a new way to assess the plot-level or on local readings of climatic our inability to more precisely locate the impact of these interventions at a scale and variables, either by community members areas covered by specific SWC activities resolution superior to what was available in along the lines reported here or by sensors suggests that the coefficients are averages the past, at a fraction of the cost. Applying that may be controlled remotely. Linking that conceal possibly vast heterogeneity this methodology to the TBIWRD points remotely sensed vegetation indices to in terms of treatment type and watershed towards significant magnitude of impact, such geo-referenced data offers opportu- characteristics. which was most pronounced for soil water nities to better understand the biophysical content. correlates and economic meaning of devel- Disaggregated data to more precisely opments, as well as the channels through describe the nature of these interventions The fact that we obtain highly significant which they may affect individuals’ behavior would have made it possible to explore results, despite limitations due to the high in response to climate change. how specific SWC activities affect soil level of aggregation of treatment variables water moisture and vegetation cover in the by space, time, and activity type, highlights Treatment data quality, in particular defi- short- and long-term. They would also have the potential of remotely sensed data to ciencies in geo-coding and disaggregation allowed to assess how these impacts vary complement traditional data sources to by type of these data, thus emerges as with biophysical, socioeconomic, and insti- improve understanding of climate change the main constraint on the ability to use tutional characteristics in ways that could impacts and ways to address them at micro remotely sensed data for more in-depth inform identification of hotspots for SWC level. This can help inform both retrospec- analysis. For this reason, routine generation intervention, and improve understanding tively and for the future the design and of geographically explicit, disaggregated, of the underlying relationships. Information evaluation of interventions such as SWC and clearly documented intervention data on the marginal cost of specific measures activities in near real time and at a scale is likely to have very high payoffs and offer would have allowed us to test how effi- that is more aligned with how such inter- opportunities for fruitful follow-up research. ciently project resources were allocated. ventions are designed. and would allow to This policy brief is based on Ali, Daniel Ayalew; Deininger, Klaus; Monchuk, Daniel. 2018. Using Satellite Imagery to Assess Impacts of Soil and Water Conservation Measures: Evidence from Ethiopia’s Tana-Beles Watershed. Policy Research Working Paper; No. 8321. World Bank, Washington, DC.  https://bit.ly/2nVAqhX. Daniel Ali is a senior economist, Klaus Deininger a lead economist, Daniel Monchuk a consultant, all at the World Bank, Washington DC. ✦  CONTACT: kdeininger@worldbank.org. The views presented are those of the authors and do not necessarily represent those of the World Bank, its Executive Directors or the member countries they represent.