from EVIDENCE to POLICY Learning what works for better programs and policies February 2016 What’s so hard about improving access to water and sanitation? Safe drinking water and proper sanitation are critical to ernments know that clean water matters, they don’t healthy human development. For much of the world’s know how to ensure everyone has it. poor, a lack of clean The World Bank is committed to ending extreme WATER AND SANITATION water has led to dis- poverty and boosting shared prosperity. Providing ease and high rates clean, safe water to people wherever they live is critical of premature death. to achieving these goals. A World Bank research team Children often fail analyzed more than 130 water, sanitation and hygiene to thrive because ill- evaluations to understand what evidence there is nesses sparked by the for successful programs and what still needs to be spread of fecal mate- learned. The researchers found that evidence is clear rial and otherwise- that improving sanitation and handwashing reduces contaminated water diarrhea. The evidence is less clear when it comes to saps their ability to expanding services to large populations or changing absorb nutrients. This behaviors, such as getting people to add chlorine to can hurt long term drinking water or to wash their hands. More research cognitive and physi- and more innovative programs are needed to establish cal development. While development groups and gov- the path forward. Context Globally, some 630 million people lack access to safe review to identify and analyze the impact evaluation drinking water and some 2.4 billion people—about one evidence in water and sanitation and draw lessons third of the world’s population—do not have access to and a knowledge base for policymakers and develop- a toilet. Lack of clean water affects children in particu- ment organizations. A World Bank research team, with lar, because diarrhea depletes the body of key nutrients, support from the Strategic Impact Evaluation Fund leading to stunting and chronic malnutrition. Each year (SIEF), conducted a systematic review of water and diarrhea kills around 760,000 children under five. sanitation impact evaluations to provide a basis for fu- Despite the importance of clean water and sanita- ture policymaking and research. tion for healthy kids, there hasn’t been a systematic Evaluation Researchers looked at studies that examined the effec- fully accessible. They also had to have a statistically- tiveness of water and sanitation programs, drawing from valid counterfactual. The research team didn’t include impact evaluation and medical databases, as well as re- qualitative evaluations, cost/benefit analyses, literature positories from research institutions like the World Bank, reviews or protocol studies. regional development banks, universities, non-govern- In total, the team identified 136 rigorous impact mental organizations and other research organizations evaluations and these were divided among five working in this field. They limited the search to papers categories according to the types of interventions from 1969 and onward. The researchers identified 1,200 evaluated: water quality, water supply, sanitation, water and sanitation related studies out of about 850,000 hygiene (handwashing), and the fifth was for evaluations studies in these databases and repositories. that looked at programs with more than one type of WATER AND SANITATION Among these, the research team looked at impact intervention. More than 70 percent of the evaluations evaluation studies, and limited their search to included in the systematic review were experimental, randomized controlled trials, experimental designs usually randomized controlled trials. The review looked and quasi-experimental evaluation methods. To be at a range of factors for determining effectiveness, such included in the systematic review, studies had to have as better access to water and better health. produced final results and these had to have been made Results The rise in number of impact evaluations of The evidence across all the impact evaluations water and sanitation programs reflects the of water and sanitation clearly shows that relevance of water and sanitation for healthy programs to improve sanitation, increase human development. handwashing and ensure clean water can reduce intestinal disease among young children. About 80 percent of studies of water and sanitation is- sues were published over the past 16 years (starting in Almost all of the impact evaluations measured incidence the year 2000), an indication of the increasing attention of diarrhea and 78 percent of evaluations found that on the issue. In fact, in the last five years, impact evalu- programs led to a drop in occurrence in young children. ations accounted for nearly half of all studies that re- The biggest impact was reported in programs that searchers located dealing with water and sanitation. The included a component to improve the quality of water— interest in impact evaluations highlights the importance often through purification systems or better storage being placed on developing an evidence base for poli- methods. Close to 70 percent of these evaluations cies and programs. Despite this increase in the number reported a decline in diarrhea. Nearly 50 percent of of impact evaluations in this sector, more still needs to programs with a component to increase handwashing be done to fill in important knowledge gaps and the ef- reported that diarrhea dropped, with similar results for fectiveness of these types of interventions. programs that sought to improve sanitation (and reduce This policy note summarizes “Evidence Mapping: Overview of Water, Sanitation and Hygiene Impact Evaluations,” Luis Andres, Christian Borja-Vega and Ronald Gomez-Suarez, Oct. 19, 2015 (unpublished). open defecation) and programs with components to increase in people actually washing their hands, even if improve the supply of and access to water. Likewise, soap and water is available. almost half of programs that combined handwashing with improving water quality led to a decline in diarrhea More also needs to be done to understand the in young children. effects water and sanitation programs can have While there isn’t as much evidence collected on inci- on socioeconomic issues like household poverty, dences of illnesses such as dysentery, respiratory diseas- income and children’s performance in school. es and influenza, for example, randomized controlled trials of combined programs to encourage handwash- Most evaluations look at health results like diarrhea, ing and improve water quality did show significant re- but as researchers found, few examine the effects of ductions. But the evidence is mixed. For example, in a water program on education-related outcomes such one of the largest water evaluations to date, examining as school attendance and test performance. Similarly, the effects of the Total Sanitation Program in India, evaluations often look at a household’s spending on only diarrhea rates, not other water-borne diseases, water and sanitation, but few look more broadly at were shown to have dropped. At the same time, im- potential socioeconomic gains from improved access pact evaluations rarely study the effects on widespread to water. For example, connecting families to piped-in waterborne conditions, such as cholera, hepatitis, and water might lower household spending because now adenovirus, although diarrhea may sometimes result they don’t have to buy water from distributors. A few from infection from another condition. impact evaluations that researchers reviewed did try to analyze the effect of improved water supply and clean Programs generally succeeded at improving the water on employment and broader social opportunities quality of water. for children, but results were inconclusive. This would definitely be an area where impact evaluation The review showed that 75 percent of impact evalu- researchers might want to take note. These areas of ations that tested bacterial contamination reported knowledge may require impact evaluation designs with strong declines in contamination. Typically, these were a long-term perspective of effects. programs in rural areas where they collected the water on site. The systematic reviews show us that when it comes to improving water quality—as However, there’s still not a lot of evidence on measured by a drop in diarrhea—filtration is how to change people’s behavior, especially very effective. when it comes to handwashing. Filtering water to remove contaminants leads to a 60 Washing hands with soap and water is one way to re- percent drop in diarrhea rates. Using a system called duce the spread of fecal matter and other contaminants. flocculation—in which the water is gently stirred so that Encouraging people to do this hasn’t been easy. Pro- particles stick together and can be removed—can reduce grams will often incorporate some measure of behavioral diarrhea by almost 50 percent. Adding chlorine to water changes, specifically around handwashing, to reduce the leads to around a 25 percent decline in diarrhea. incidences of diarrhea in young children. These programs Overall, the systematic review found that programs to usually include an education component to teach people improve water quality—regardless of the method used— about the importance of washing their hands at critical led to a 50 percent reduction in deaths from diarrhea; moments, such as after changing a baby’s diaper or before just over a 30 percent drop in diarrhea deaths for young preparing food. While impact evaluations show that these children from the age of one to five years old; a 20 percent programs usually do improve people’s knowledge about decline for children aged one to two months; and a drop when and how to wash their hands, there usually isn’t an of some 30 percent in stunting. Programs to improve water quality are the infrastructure programs—information that is critical most common but an increasing number of to policy makers looking to expand access in especially researchers are looking at how to effectively urban areas. combine water and sanitation programs and integrate them with other sectors. While research in this field is increasing, it’s not happening at the same rate around the world. Thirty nine percent of the studies evaluated programs to improve water quality by providing affordable purification About eighty percent of the evaluations were treatments and storage options. These programs often concentrated in just three regions: South Asia, Sub- were in rural or remote areas with few resources and a Saharan Africa and Latin America. More research needs small sample size. Hygiene programs account for 17 to be done to better understand ways to improve water percent of the evaluations, and water supply comes in and sanitation services in under-studied countries third with eight percent, followed by sanitation programs. across East Asia, the Middle East and North Africa, and WATER AND SANITATION Twenty nine percent of all the evaluations were of Europe and Central Asia, which to date account for just combination programs, such as those that sought to 20 percent of the evaluations. Also, the vast majority— encourage people to wash their hands with soap and water some 67 percent—studied rural communities, with and to install and use toilets or other improved sanitation. just 29 percent focused on urban populations. More Combined programs can potentially achieve even greater impact evaluations of programs in urban areas would impact on access, health and behavior change because they help policymakers understand how to reduce the spread have similar messages and both handwashing and improved of water-borne diseases in fast-growing cities where, sanitation seek to halt the spread of fecal material. especially in poor areas, infrastructure can be weak or One thing still lacking in the research is a body non-existent. of knowledge around scaling up water and sanitation Conclusion As the results of this analysis show, much more needs methods are needed to produce clear, unambiguous to be done to understand the ways that water and san- results. Indeed, highlighting the gaps in our knowl- itation programs can end extreme poverty and boost edge helps carve out a clearer path ahead so that re- shared prosperity. Delineating what we know—and searchers, along with the broader development com- what we don’t—is a critical first step to creating bet- munity, better understand where to focus their efforts ter policies and as the survey showed, more rigorous and resources in the years ahead. The Strategic Impact Evaluation Fund, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help alleviate poverty. The goal is to collect and build empirical evidence that can help governments and development organizations design and implement the most appropriate and effective policies for better educational, health and job opportunities for people in developing countries. For more information about who we are and what we do, go to: http://www.worldbank.org/sief. The Evidence to Policy note series is produced by SIEF with generous support from the British government’s Department for International Development. THE WORLD BANK, STRATEGIC IMPACT EVALUATION FUND 1818 H STREET, NW WASHINGTON, DC 20433 Produced by the Strategic Impact Evaluation Fund Editor: Aliza Marcus Writer: Daphna Berman