A Conceptual Framework Crowdsourcing Water Quality Data A Conceptual Framework November 2016 1 Crowdsourcing Water Quality Data 2 A Conceptual Framework Crowdsourcing Water Quality Data A Conceptual Framework November 2016 3 Crowdsourcing Water Quality Data Acknowledgements This report was produced Carter Borden, Ph.D. and Sarah Borden from Centered Consulting International, LLC, under the SAWI financed activity to improve capacity for water quality management, managed by Pratibha Mistry, Senior Water and Sanitation Specialist, the World Bank. Nishtha Mehta, Water and Sanitation Specialist and Dany Jones, IT Officer, provided valuable inputs. 4 A Conceptual Framework Table of Contents Overview...................................................................................................................................................... 7 Introduction.................................................................................................................................................. 9 Research Question. ............................................................................................................................................ 9 Theory of Change........................................................................................................................................... 10 Desired Outcomes........................................................................................................................................... 10 Pilot: Social, Political, And Environment Contexts:................................................................................................ 10 Inputs............................................................................................................................................................ 14 Citizen Science. .......................................................................................................................................... 15 Participation................................................................................................................................................... 16 Citizen Motivators: Initial Participation. .............................................................................................................. 16 Citizen Motivators: On-going Participation.......................................................................................................... 17 Scientist Motivators.......................................................................................................................................... 17 Demotivators.................................................................................................................................................. 17 Crowdsourcing........................................................................................................................................... 17 M-Apps. ........................................................................................................................................................ 18 Performance Indicators. .................................................................................................................................... 21 Project Design Considerations.................................................................................................................. 21 Social Design................................................................................................................................................. 21 Technical Design............................................................................................................................................. 22 Program Design.............................................................................................................................................. 26 Water Quality Indicators. ......................................................................................................................... 28 Human Activities Influencing Water Quality........................................................................................................ 29 Analyte Selection Consideration........................................................................................................................ 30 Crowdsourcing Project Methodology. ...................................................................................................... 31 Phase 1. Investigation and Scoping. ....................................................................................................... 32 Task 1.1. Site Selection. .................................................................................................................................. 32 Task 1.2. Local Team Identification and Chartering............................................................................................. 33 Task 1.3. Stakeholder Needs Assessment.......................................................................................................... 34 Task 1.4. WQIs and Sampling Methods Selection. ............................................................................................. 35 Task 1.5. Cost Estimate................................................................................................................................... 36 Task 1.6. Risk Assessment and Mitigation Plans. ................................................................................................. 37 Task 1.7. Performance Indicator Selection.......................................................................................................... 37 Task 1.8. Contracting..................................................................................................................................... 37 2. Initiation and Customization............................................................................................................... 38 2.1. Form an m-App User Subcommittee............................................................................................................ 38 2.2. m-App Design, Prototyping, and Testing. ..................................................................................................... 38 2.3. Data Dissemination Applications’ Design, Prototyping, and Testing. ................................................................ 39 2.4. Distribution/Training Water Quality Sampling Systems.................................................................................. 39 3. Implementation.................................................................................................................................... 39 3.1. Water Quality Screening. .......................................................................................................................... 39 3.2. Technical Maintenance. ............................................................................................................................. 40 4. Reporting and Evaluation..................................................................................................................... 40 4.1. Tracking and Data Collection. .................................................................................................................... 40 4.2. Evaluation and Reporting.......................................................................................................................... 41 4.3. Project Refinement and Improvement........................................................................................................... 41 Conclusion.................................................................................................................................................. 42 References.................................................................................................................................................. 42 Annex 1. Interview Questions................................................................................................................. 45 Annex 2. Performance Indicators – Process and Outcome Measures................................................. 50 5 Crowdsourcing Water Quality Data 6 A Conceptual Framework Overview Over recent decades, millions of people gained access to improved water sources. However, these gains are being undermined by increasing contamination of water sources from intensive agricultural practices, unregulated industrial discharges and inadequate sanitary infrastructure in both urban and rural areas. The contamination of drinking water, whether by microbes or chemicals, persistently remains one of the world’s predominant human health hazards. Poor health is a drain on household finances that could be used for food, transport, education, business or savings. The consumption of contaminated water threatens the physical and neurological development of children, compromising the development potential of future generations. As governments begin to tackle water quality issues, water quality monitoring is a logical starting point. After all, you can only effectively manage that which you measure and understand. However, traditional water quality monitoring infrastructure is expensive, requiring laboratories, sophisticated testing technologies, specialists to conduct tests and comply with sampling and testing protocols, and information systems to control and manage data flows and reporting. Recent technological developments in sensors and telemetry are revolutionizing the monitoring landscape, enabling the collection of high frequency data from remote locations with high accuracy. At the same time, the ubiquity of mobile phones and the internet enables citizens to participate actively in social causes. Information and communication technologies are providing citizens and communities with greater access to data and a means of communication that allow for more informed decision making. By harnessing the power of technology, the ability of citizens to assist in the collection of information and to expand our understanding of water quality issues has the potential to increase our ability to provide these citizens with knowledge of the water they are consuming in order to make more informed decisions. Developing a conceptual framework for achieving this goal is the first step in the process of realizing this future. This report documents the theoretical background that is being used to pilot the concept of crowdsourcing water quality data. It explores the theory of change, citizen science and crowdsourcing, and provides a short overview of water quality indicator selection as well as an outline for conducting a crowdsourced water quality study. 7 Crowdsourcing Water Quality Data Abbreviations and Acronyms ARD Acid Rock Drainage BOD biochemical oxygen demand CoC constituent of concern HCD Human Centered Design ICT Information and Communication Technology m-App mobile phone application m-ARD mobile phone application for agriculture and rural development m-WASH mobile phone application for water, sanitation, and health NGO nongovernmental organization SMS short messaging service TSS Total Suspended Solid WMA Water Management Agency WQ Water Quality WQI Water Quality Indicator WUA Water User Association 8 A Conceptual Framework Introduction Using mobile phone technologies coupled with water quality testing, there is great opportunity to increase the awareness of water quality throughout rural and urban communities in developing countries. Whether the focus is on empowering citizens with information about the quality of water they use in daily life or providing scientific data to water managers to help them deliver safe water to the citizens, the integration of citizen science, crowdsourcing, and innovative technologies has the potential to create positive and lasting change. The methodology proposed herein combines empowerment of the public through participation in the scientific process (citizen science) with modern technologies to collect, gather, and disseminate data (crowdsourcing). Citizen science, defined as “the practice of public participation and collaboration in scientific research to increase scientific knowledge” (National Geographic Society, 2016), has proven to be effective in: i) enabling large data sets to be collected at costs not feasible with conventional sampling programs; ii) informing citizens of the local and regional conditions; and iii) providing a common platform from which citizens, resource managers, and scientists can discuss issues and develop management alternatives (Rotman et al. 2012, Buytaert et al. 2014). When citizen science is coupled with crowdsourcing, “the practice of obtaining information or input into a task or project by enlisting the services of a large number of people, either paid or unpaid, typically via the Internet” (Oxford Dictionary 2015), the quantity of data and speed of reporting increases while the cost of data collection decreases. By employing a crowdsourcing approach with innovative technologies, there is potential to harness large amounts of data in areas previously considered either too remote or costly to access. The purpose of this conceptual framework is to outline the considerations and activities to be undertaken for a successful water quality monitoring project using citizen science and crowdsourcing. Specifically included in the framework are: 1) the research question to be answered through the project; 2) the theory of change that will lead to desired outcomes; 3) project design considerations to promote a successful pilot; and 4) the methodology outlining implementation steps. Though this conceptual framework is being developed to support a pilot project in India, the theories, considerations, and methods presented herein can be applied to other citizen science/crowdsourcing projects globally. Research Question How can crowdsourcing water quality monitoring tools be used to increase community participation and understanding of water quality? This fundamental research question spurs a series of subsequent questions that must be addressed for a successful project: • What are the design principles that encourage participation and support quality science? • What are the focus areas to test the crowdsourcing approach and tools? • What level of participation will citizens be involved in with regard to data collection, analysis, and feedback? • What are the most effective processes for collection, analysis, and feedback? • Which stakeholders play what roles in water quality monitoring? • How can the citizen science approach and crowdsourcing tools be transferred to other crowdsourced water quality projects? 9 Crowdsourcing Water Quality Data Theory of Change Given the complex nature of integrating citizen science and crowdsourcing to better understand water quality, the Theory of Change serves as the structure and identifies considerations necessary for project activities to effect the desired outcomes. Employing the Theory of Change starts with the end in mind and works backwards. It begins with identifying the outcomes to be achieved; moves through understanding the social, political, and environmental context of the study area; and finishes with identifying the inputs that can be used to influence the context to achieve the desired outcomes (Figure 1). Note, it is very important to characterize each contextual element thoroughly before embarking on the project as failure to properly understand them can lead to limited success or failure of the project. Desired Outcomes While developing a project, identifying the desired benefits and outcomes of each group has proven to be important when implementing citizen science methodologies. The desired outcomes from this crowdsourcing water quality project are expected to benefit citizens and communities, resource management agencies, and policymakers and scientists. Citizens are expected to receive real-time water quality status on the source of water they are about to use, feedback on the local and regional water quality conditions, and, potentially, actions from water quality incidents from management agencies. Water management agencies are expected to benefit from real-time water quality status reports from a much denser network of sampling locations, less costly water quality screening as the citizens are supplying labor, ability to identify and respond to water quality incidents in a more timely manner, and a common platform from which to educate and work with communities on water management. Finally, depending on the ancillary data collected, scientists and policymakers can benefit from a data set that provides not only water quality conditions throughout the study area but the type of water use, locations of water sources accessed, timing of use, seasonality patterns of water use, and so on. These data can also be mapped to other data sets such as disease, use of fertilizers and pesticides, and investments in water distribution systems to provide a more holistic understanding of how anthropogenic activities are influencing local water quality and community well-being. Pilot: Social, Political, and Environment Contexts To achieve the desired outcomes, social, political, and environment contexts must be well understood in order to effectively apply citizen science methodologies and crowdsourcing technologies. Failure to understand and accommodate any of these three contexts will likely lead to limited success. For example, when considering the political context for a specific site, if there is no belief in transparency on the part of governmental leadership, the ability of the pilot to effect change through citizen-involved science may not be possible. The following section outlines these contexts in greater detail. Social Context: When operating in a pilot site, it is critical to understand the nuances of the local community to ensure the pilot design is aligned with the needs of the community. Table 1 lists the social elements included in the Theory of Change. Further analysis will need to be completed regarding the site specific content of each element, why it is important to understand this information, and potential ways to access the information. 10 A Conceptual Framework Figure 1. Theory of Change framework used to guide the methodology for implementing a crowdsourcing water quality project INPUTS • Agencies/Partners • Strategic Resources – Human, Financial, Technical • Innovative Technology • Experience and Expertise PILOT Social • Demographics • Health Statistics • Literacy Rate • Community Structures • Technology Use Crowdsourced Water Quality Monitoring Environmental Political • Water Quality Issues • Water Laws • Constituents of • Political Structure Concern • Transparency • Water/Land Use • Political Will Practices • Technology Use • Technology Use OUTCOMES Understanding of water quality in real-time Community empowerment and knowledge of water quality Actionable data for water quality managers Cost of water quality monitoring 11 Crowdsourcing Water Quality Data Table 1. Elements to consider when evaluating the social conditions Social Content Why Important? Potential Elements Sources Demographics Age, gender, education, Implications for stakeholder Census data, household income, number interviews, technology government in household, religious design, training approach, agencies affiliation, language(s) communication strategies Health Statistics Mortality rates, morbidity To understand health Government rates, access to healthcare, implications related to agencies, quality of healthcare, water quality, need to hospitals, non- health behaviors, physical know overall health of the governmental environment community organizations (NGOs) Literacy Rate Percentage of population Configuration of the Census data, that can read and write technology, training design, government in native language and and communication are agencies English dependent on basic literacy. In the absence of that, alternative approaches will need to be developed Community Formal and informal For effective citizen Research, Structures political structures, involvement, need to interviews, community organizations, understand roles within NGOs, educational institutions, the community, potential networking methods of communication partner organizations, how throughout the community information is shared, and structures potential ways to access the community Technology Use Mobile phone ownership, Prevalence of sensors/ Government technology use for data loggers (e.g., mobile agencies, obtaining information phones), mapping the NGOs, private community readiness to sector incorporate new technology Political Context: In developing and executing this pilot program, there is great dependency and need for support from Water Management Agencies (WMA) and community leaders on multiple levels. Due to the nature of the information being gathered, analyzed, and reported in crowdsourcing water quality projects, it is necessary to have clarity regarding the purpose, processes, roles, and desired outcomes of the project. Without complete buy-in into the operation of the project, the ability to be transparent and open with information may not be possible. Given that the design of the project is intended to provide citizen empowerment around water quality, if buy-in into transparency and openness is not achieved, the ultimate success of the pilot will be at risk. Table 2 outlines considerations within the political context. 12 A Conceptual Framework Table 2. Elements to consider when evaluating the political conditions Political Content Why Important? How to Get? Elements Water Laws Federal, state, and Need to understand water WMAs local water laws quality regulations to address the issues and as a means to garner support for the work Political Federal, state, and For support of the project, WMAs, NGOs, water Structure local governmental decision making authority, users and community structures, agencies, and access to resources, leadership water users, and need to understand structure communities Transparency Support for For citizen science and Approval and visible open access crowdsourcing to work, two- support throughout to information way communication about WMA and the throughout the status of water quality is community managing agencies required and the community Political Will Ability and There must be clear and Assurances from willingness to make consistent support for open leaders in WMA and difficult decisions access to both positive the community and maintain and potentially negative transparency information if transparency and trust are to be maintained Technology Use Use of real- Need to understand and WMAs and service in Resource time monitoring address potential gaps providers Management networks, use of between current resource databases, reports management infrastructure generated, web and what is needed for a portals used, early successful pilot warning systems, communication with public Environmental Context: Given that the purpose of the pilot is to identify ways to empower communities to engage in, understand, and assist in improving water quality, understanding the environmental elements of the pilot site is necessary. Outlined in Table 3 are the considerations necessary when attempting to understand the environmental context. 13 Crowdsourcing Water Quality Data Table 3. Elements to consider when evaluating the environmental conditions Environmental Content Why Important? How to Elements Get? Natural Physiological, hydro- Natural conditions drive Hydrometeorological Conditions meteorological, geological, the hydrologic cycle monitoring data, and ecological setting which influences the geologic maps, quantity and quality of natural resource water reports, ecological reports, etc. Water and Land Land use; commercial, Anthropogenic activities Landuse maps, sector Use Practices industrial, municipal, and act upon the hydrologic water use records, irrigation use; structure cycle which influences the agricultural and operations (e.g., dams, quantity and quality of irrigation records tanks, canals) water Water Quality Benefits and limitations that Known water quality Water quality Issues are the product of water issues will guide the monitoring data and quality conditions water quality indicators reports selected Technology Use Monitoring networks Characterizes current and Central, state, and programs collecting trending conditions and municipal hydrometeorological water governments; quality and quantity, as academic and well as ecological data research institutions; scientific studies Inputs To achieve the desired outcomes of the pilot, there are a variety of inputs that help to positively position the work for success. The involvement of agencies and partners early in the process assists with both effective design and buy-in of the pilot. Ensuring proper resource availability and allocation are necessary for successful project execution. Innovation technology solutions provide a means for faster, more cost effective, and higher quality results. Staffing the project with experts in water quality, technology, and community engagement at both the global and local levels brings the power of experience to bear on the pilot. Given the iterative nature of this process, maintaining focus on the desired outcomes while remaining flexible in the approach will increase the likelihood of success. 14 A Conceptual Framework Citizen Science For a project to be successful in empowering citizens with respect to water quality, it is necessary to determine a level of engagement that will contribute to likely success. The level of involvement must be sustainable over the long term and scalable to other geographies after completion of the pilot. Bonney et al. (2009) outline three tiers of participation that are typical of scientific research involving the public: contributory, collaborative, and co-created, as defined below. • Contributory projects: these are the most common form of participation. These projects are usually designed by scientists and citizens participate mostly through data collection; • Collaborative projects: these are also mostly designed by scientists and citizens contribute through data collection, but the public might also assist with project design, data analysis, and/or communication of study findings; and • Co-created projects: the design of these projects is shared by scientists and community members with some of the public involved in the scientific process. For purposes of the pilot, it is recommended that community engagement be modeled around the collaborative approach. For citizens to be truly empowered and have the potential for positive change, it is important for them to be active participants in contributing data and in understanding the results of that information. At a minimum, the public should be involved in both data collection and communication of results. Building in a sense of ownership of the process is critical for on- going engagement on the part of the community. In terms of data analysis, it is likely that the technologies being tested during the pilot will have analysis built in. Therefore, it is not likely that specific data analysis will be necessary on the part of the citizen; however, trend data or other larger scale analysis may be relevant. More specifically, citizens could assist with refining the design of the pilot and determining protocols for data collection. Once the data have been collected and analyzed, discussion of the results with local government agencies could lead to water quality improvements. These improvements might come from greater awareness on the part of the community to activities that are causing disturbances in water quality. A campaign to educate other members of the community on water management practices could be a positive and very visible way to show how a change in behavior can result in improved water quality. Using a collaborative approach is likely to be an effective approach if the desired result is community empowerment toward improved water quality. While the benefits of collaboration with citizens are evident, for truly successful collaboration to occur, it is critical to understand the motivators that will encourage and keep citizens engaged. Dana Rotman states: “Existing projects have demonstrated the value of using volunteers to collect data, but few projects have reached the full collaborative potential of scientists and volunteers. Understanding the shared and unique motivations of these two groups can help designers establish the technical and social infrastructures needed to promote effective partnerships .… Identifying the pivotal points of motivational shift and addressing them in the design of citizen-science systems will facilitate improved collaboration between scientists and volunteers.” The use of the collaborative approach for the pilot study is a viable option, especially when attention is given to clarity and support for motivating factors. 15 Crowdsourcing Water Quality Data Participation Within the collaborative approach, motivators are key in terms of generating initial interest in being involved, but also in the decision to continue to participate. As stated in Building and Sustaining Critical Mass Engagement in Crowdsourcing, Idea Wonks (2015), it is possible to predict participation levels along a spectrum: latitude of rejection, latitude of non-commitment, and latitude of acceptance. To be successful, it is imperative to understand the motivators that move people to the latitude of acceptance and keep them there throughout the engagement. Citizen Motivators: Initial Participation Rotman et al. (2009) investigated the motivating and demotivating factors for initial and continual participation in citizen science projects. They found that citizens initially decide to participate typically based on four elements, with the first element clearly being the leading motivator (Figure 2): 1. Understanding of or curiosity about a specific factor of the project that stems from personal interest or potential personal gai 2. Prior work on scientific projects 3. A hobby related to the project 4. View of the work as a potential building block toward a scientific career Figure 2. A process model of citizen volunteers’ and scientists’ motivations in citizen science projects Start Here Citizens – no continued Citizens Scientists recognition/ (Personal (Need for attribution of work interest) data) Personal Interest (Initial, ongoing) Continued Initial involvement (Ongoing attribution, inclusion in involvement scientific work/outcomes, (Interest, Thirst for community involvement and Knowledge) advocacy ) Scientists – apprehension about quality of work and commitment of Active Collaboration citizens (Training, attribution, acknowledgment, and recognition) Citizens – lack of Scientists – altruistic recognition from support for public scientists education Green = Motivators Red = Demotivators Source: adapted from Rotman et al. 2009. 16 A Conceptual Framework Citizen Motivators: On-going Participation For citizens to continue to participate after the initial work of the project is complete, there are different factors in play: • Need to be recognized and their work attributed to them; • Desire for feedback and to know that the work they did was making an impact and what that impact included; • The sense of community involvement and that their participation was making a difference for members of their community; and • The ability to advocate based on the education gained from initial participation. Given these particular motivators, the need for clear, targeted communication and feedback to participants throughout the pilot cycle is critical. Setting up these mechanisms prior to launch will be a key step in setting up the pilot. Without continued oversight and engagement with the community to ensure they see value in participating, they could become demotivated, non- committal, and/or reject continued collaboration efforts. Scientist Motivators The motivators for scientists to participate have some overlap with citizen motivators when it comes to personal gain. 1. Ability to gather large amounts of data – to be successful, scientists are often dependent on the ability to gather relevant data and citizen science projects are typically designed with this in mind. 2. Leading the scientific process – for the majority of projects, scientists preferred to limit the scope of citizen involvement to field data collection. With the design of the pilot to include a broader role for citizen participation, attention must be paid to understanding specific motivations and how best to influence the expansion of the citizen role. 3. Contributing to the greater good through education and policy – this motivator included both an introduction to the specific scientific focus areas, as well as the scientific process as a whole. Demotivators 1. Mutual apprehension and a sense of distrust – for volunteers, the idea of working with scientists can be intimidating. Scientists can have concerns about the quality of work and willingness to maintain commitment to the project. They want to ensure the volunteers are well matched both in terms of capability and capacity prior to beginning their work together. 2. Scientists not understanding the key motivations of the volunteers – this results in false assumptions on the part of the scientists in terms of the initial motivator to participate and the need for strong communication and recognition of the participants. Crowdsourcing When undertaking a crowdsourcing project, it is important to consider the technology capabilities and how people interface with it. Information and Communication Technologies (ICTs) are benefiting citizens, governments, and organizations by providing access to information, enabling communities to coalesce around issues of concern, and making communication more immediate (Hutchings et al. 2012). A subset of the ICT is the use of mobile phones to help collect and disseminate data, as well as promote communication between stakeholders in order to alleviate poverty and improve development (Qiang et al. 2011). 17 Crowdsourcing Water Quality Data m-Apps Mobile phone applications (m-Apps) cover a variety of sectors. In reviewing mobile use in eastern Africa, Hellström (2010) identified m-Apps that have been developed for agriculture and rural development (m-ARD), education (m-learning), finance (m-Transactions, m-Banking), governance (m-Governance), and health (m-Health). Qiang et al. 2011 evaluated 92 m-ARD initiatives with respect to their potential to increase development. Within agricultural and rural development, the m-ARD initiatives ranged widely from agriculture (support, marketing, trade, research, innovation), resource management (water, climate), human development (employment, education and training, rural youth, rural woman opportunities, micro business development), governance (administration), and rural finance, infrastructure, and ICT (transactions, micro-finance, agricultural insurance, transport). Finally, Hutchings et al. (2012) evaluated 10 mobile phone crowdsourced applications that target water, sanitation, and health (m-WASH). Of these m-Wash applications, the researchers identified four main themes: assess poor or non-existent access, post-construction failures of systems, service reliability, and long-term sustainability of the resource. Thus, m-Apps cover a wide range of development initiatives and plenty of models exist to draw upon for developing crowdsourcing projects. Important in developing a crowdsourcing project is to identify the key actors interfacing in the mobile ecosystems, how they relate to the m-App, what information they require, and their capacity with respect to the technology available, financial situation, and education levels. Primary categories of key actors include policy makers and regulators, mobile network operators and service provides, content providers, m-App developers, NGOs, the private sector, researchers, media, and citizens (Hellström 2010, Qiang et al. 2011, Hutchings et al. 2012). The roles and uses of m-Apps for key actors are listed in Table 4. All should be considered when developing an m-App, though some may have lesser importance. For example, private sector actors were less important in m-WASH applications. Table 4. Change agents, role in mobile technology ecosystem, data needs Key Actor Role Data Relationship WMAs/Service Data collector/ Collect and disseminate data to communities. Use data Providers user/provider collected for planning, management, and emergency response. Also uses m-App as a communication tool with the communities. Mobile network Enabler Provide and manage the mobile network, as well as operators provide the hosting platform. Content providers/ Enabler Support the development and implementation of m-App developers m-Apps, as well as enables transactions. NGOs Data collector/ M-apps support an NGO’s mission of advocacy, raising user/provider awareness, and monitoring and evaluation. 18 A Conceptual Framework Table 4. Change agents, role in mobile technology ecosystem, data needs (continued) Key Actor Role Data Relationship Private Sector Data collector/ Commercial applications primarily involve supplying user/provider consumers with data or analysis for payment or to encourage customer loyalty or promotional service. Commercial applications are also developed to promote transactions between parties (e.g., banking services). Researchers, Data user Incorporates data into research and discovery. Innovators Media Data user Report current conditions or trends to the public. Citizens Data collector/ Citizens collect data to inform government agencies user (e.g., faulty service), as well as use information to make decisions concerning their community (e.g., notifications) or activities in which they are engaged (e.g., current market conditions). Amount citizens access disseminated data is a function of relevance, perceived benefit, education, and financial level. Source: Hellström 2010. As observed in Table 4, the flow of data and communication methods vary based on the purpose and uses of the m-App developed. This variation is illustrated in the primary means of short messaging service (SMS) messaging in m-Governance, which include notification, pull SMS, listen, and transaction (Susanto and Goodwin 2010). Notifications are one-way transmissions such as government broadcast of weather warnings. Pull SMS enables citizens to request particular information such as local crop reports. Listen service is an avenue for citizens to send information to government agencies such as flow observations and service complaints. Finally, transactional SMS support services include such transactions as paying bills or taxes. The typical informational flow in m-WASH applications is from the user to a server (Figure 3). Once received, the server then collects, packages, and disseminates the information to a variety of users via different methods including mobile phone messages, websites, dashboards, and social media. It is envisioned that this is the model that will be adopted for crowdsourcing water quality projects in India. Associated with data flow, consideration of the technology platform, network coverage, and messaging type are important elements in developing m-Apps (Hellström 2010, Qiang et al. 2011). The technology developments required include m-Apps for collection and technologies (for example, websites, emergency messaging) for dissemination. For adoption by citizens, the m-App purpose must be in line with the project but also perceived as easy to use, useful, reliable, adding value, trustworthy, and responsive to the users (Susanto and Goodwin 2010). Incorporating the key actors (especially citizens) early in the development process builds trust and technologies that are customized for user needs and, thus, is more likely to be adopted (Mechael et al. 2010, Hellström 2010, IDEO 2011). Similarly, data dissemination needs to be developed in relation to the end user needs. For some users, new web-portals or dashboards will need to be developed or adapted from existing technologies. For WMAs, service providers, and stakeholders with existing business processes that rely on water quality data, linking the m-App data server with the user system augments the data upon which business decisions can be made and requires little to no additional training. Platforms for developing m-Apps vary between developed and developing countries (Qiang et al. 2011). In developed countries, platform providers (for eample, Apple’s App Store) allow for posting of m-Apps and provide users with many services and access to substantial digital data. In developing countries, mobile network operators control the systems and the content provided 19 Crowdsourcing Water Quality Data Pilot Considerations: Data Flow. The crowdsourcing water quality project will likely involve citizens reporting water quality testing results to a regulating agency with the agency then analyzing and publishing the data to a variety of end users. For the pilot, the platform, network coverage, and method of exchanging data will need to be determined during development. tends to be hyper-local often not available in digital forms. Thus, developing m-WASH apps in developing countries will frequently involve designing and implementing local solutions. This can be an impediment to scaling the m-App application to broader areas. For long-term sustainability, it is important to have an effective supporting infrastructure, strong business model, and performance indicators for assessing the effectiveness of the m-App. For the business model and funding, the development horizon for an m-App can be broken into three stages: pilot, scalability, and sustainability (Qiang et al. 2011). Funding for each stage is different. Commercial applications are primarily driven by profit and thus a sustainable business plan will balance pricing structure with material, marketing, and management expenses. That said, initial stages may require seed money from donors or government agencies (Hellström 2010, Qiang et al. 2011). Qiang et al. (2010) found that 85 percent of m-ARD apps received donor, government agency and corporate social responsibility funding during the pilot stage. When moving from the pilot stage to the scalability stage, access to donor funding diminished which should be taken into account. Non-commercial m-Apps determine value from the social and/or economic benefits from the targeted audience (Qiang et al. 2011). Funding streams for non-commercial m-Apps tend to operate as long as donor, government agency, and corporate social responsibility funding exists. Therefore, for non-commercial m-Apps it is important to have clear goals for the business model in order to show benefit to the funding organization (Hellström 2010). Reliance on funding organizations may limit scalability if the funding is insufficient to expand the m-App services. Figure 3. Data collection and dissemination for mobile phone applications Source: Hutchings et al. 2012. 20 A Conceptual Framework Performance Indicators For evaluating project effectiveness, as well as determining areas that need improvement, performance indicators are crucial. Categories of performance indicators include citizen testing, technology (testing, m-App, dissemination applications, and server performance), impact on water management, and financials. Performance metrics include process indicators to track if the technology and its implementation are performing well and outcome indicators to determine if the crowdsourcing project is benefiting citizens and WMAs (the outcomes in the Theory of Change). Often, combinations of qualitative and quantitative indicators provide effective means of characterizing the processes and outcomes of the project. Annex 2 provides a list of appropriate performance indicators for crowdsourcing water quality using m-Apps. Project Design Considerations When pursuing a crowdsourcing project, it is advised to consider social design, technical design, and program design from the outset (Hutchings et al. 2012). The following text lists the important components to consider, potential barriers restricting long-term sustainability of the project, and actions to take for each design element. Social Design For successful adoption, it is crucial to develop m-Apps that are relevant and beneficial to the key actors. Developing this awareness involves many of the elements of citizen science but also includes defining socio-cultural context. In defining the socio-cultural context, the crowdsourcing literature overlaps many of the same design considerations that are outlined in citizen science, including literacy rates, household incomes, and motivators. In addition, who uses technology in a community or household may be a factor. For example, within a household, mobiles may be used by men only, thus an m-App developed for this community may inadvertently omit input from the women who collect water for consumption and washing. User perception is a key factor in user adoption or rejection of m-Apps. Susanto and Goodwin (2010) received 159 responses from surveys, questionnaires, and interviews in 25 countries concerning people’s perceptions of m-Governance apps. From this effort, the researchers identified 15 perceptions that influence m-App adoption. These perceptions include: • Ease of use • Efficiency in time and space • Availability of devices and • Responsiveness • Usefulness infrastructure • Trust in SMS • Relevance, quality, and • Value for money technology reliability of data • Convenience • Risk to user privacy • Reliability of mobile network • Trust and quality of public • Risk to money services • Self-efficacy in using • Compatibility SMS For increased likelihood of adoption, the m-App needs to be developed with regard to the user perceptions and thus it is important to determine user needs at the outset of the project. Additional training, education, and marketing can also be used to increase the likelihood of adoption as they help to inform and dispel misconceptions (ibid). Therefore, collaboration between key actors (for example, citizens, agencies, stakeholders, m-App developers) during the m-App development and implementation is a key factor in m-App adoption 21 Crowdsourcing Water Quality Data (Mechael et al. 2010). Methods exist for assisting in the development of technology to suit user needs. The Human Centered Design (HCD) toolkit (IDEO 2011) provides a structured method to evaluate community needs, create solutions, and design m-Apps. The first step in the HCD method is “hearing”, which stresses involvement of the target communities in learning about the issues and background, as well as creating and designing solutions. Findings from the “hearing” stage guide the design of m-Apps that fits the user needs. Development projects where the HCD toolkit methodology was successfully applied include providing children’s eye care in rural India; the Ripple Effect project, which aims to improve access to safe drinking water for the world’s poorest and most underserved people by stimulating innovation and capacity building; and the Fitted Ear Project which developed a systemic solution to providing hearing tests and fitting hearing aids (https://www.ideo.com/expertise/social-innovation/). Potential barriers to adoption of m-App are trust in the service provider or WMA developing the m-App and data privacy. Positive trust in service providers and WMAs manifests in expectations that the regulating organization will respond to requests or complaints sent in by the citizens (Susanto and Goodwin 2010). Systems that respond to users’ submittal of data are deemed to be more trustworthy than the agency receiving the data and having the ownership of responding. It is also important that citizens have access to the data being generated. Not providing notification of the receipt of submitted data nor access to the data create trust issues around transparency and can be a demotivator in participation (Rotman et al. 2009). Transparency and access need to be balanced against data privacy. Citizens may not wish for personal information collected during reporting sampling to be released. Care must be taken to ensure that the service providers and WMAs collecting the data do not distribute this information as doing so can be a barrier towards adoption (Susanto and Goodwin 2010, Hutchings et al. 2012). Technical Design Technical design is focused on the development of data collection (m-App) and data dissemination, as well as for crowdsourcing water quality projects, selection of the sampling methods and equipment. Considerations and barriers to address when selecting the data collection methods include phone type, messaging method, cost, and network coverage and reliability. Mobile Phone and Messaging Options: Mobile phones and messaging options vary from simple unstructured to sophisticated smartphone applications (Table 5). In rural communities in Pilot Consideration: Social Design. When developing and implementing the proj- ect, the following considerations should be addressed. • Socio-cultural context of communities within the potential project areas needs to be understood; • Roles and data needs of key actors involved in the project need to be identified and documented; • Th project is to be developed and implemented collaboratively with all key actors; • Citizens’ perceptions of the ease of use, reliability, relevance to their lives, trustworthiness, and benefit of the m-App should be determined; • Potential trust barriers including transparency, data sharing (for example, WMA sharing with public) need to be identified; • Data privacy and protection concerns of key actors should be addressed; and • End users can be informed and educated on the technology application and benefits, as well as the purpose and relevance of the data being collected. 22 A Conceptual Framework developing nations, basic phones are most prevalent whereas smartphones are increasingly used in urban communities in developed nations. For m-ARD projects in developing countries, Qiang et al. (2011) noted that the information provided to the farmers must be highly localized in order for them to know where to sell or trade crops and livestock. Similar inclinations are likely for monitoring water quality conditions. Websites, such as ICTWorks.org, provide information on ICT with respect to sustainable development of applications for use in rural and underserved communities across the developing world. Table 5. Technical design options for data collection Category Technical Design Advantages Disadvantages/ Barriers Simple SMS One-way transmission Little or no training Difficult to interpret Data Entry of data via SMS to the result or incomplete Any mobile phone capable (Unstructured) database information of SMS transmissions Not automatically geo-tagged or user metadata Simple SMS One-way transmission Any mobile phone capable Training required for Data Entry of data via SMS to the of SMS transmissions data entry (Structured) database. Includes Input to database Not automatically codes or symbols structured and complete geo-tagged or user to enable server to metadata decode the message Simple SMS One-way transmission Any mobile phone capable Training required for Data Entry of data via SMS to the of SMS transmissions data entry (Multiple database. Input to database Not automatically Choice) structured and complete geo-tagged or user metadata Allows for more comprehensive data inquiry Interactive SMS Two-way SMS Any mobile phone capable Training required for messaging of SMS transmissions data entry Input to database Not automatically structured and complete geo-tagged or user Allows for more comprehensive data inquiry Offers a means for communication and response 23 Crowdsourcing Water Quality Data Table 5. Technical design options for data collection (continued) Category Technical Design Advantages Disadvantages/ Barriers Native Forms Forms use built-in Most mobile phones Dissemination of on Basic m-Apps to structure the m-Apps in developing Interface is easier to use, Phones text message content countries may be requiring less training and format difficult Data input in correct format Stores data when not connected to network Smartphone m-Apps built to Interface is easier to use, Limited to Applications send and receive requiring less training smartphones which transmissions on are more expensive Geotag locations, user smartphones. to purchase/operate metadata Not as frequently Enhanced capabilities of used in developing m-Apps world Interactive This method involves • Any phone • Limited in range of Voice Re- a regular voice phone responses sponse call by user to a server • Data input in correct which houses an “inter- format active voice response” that guides the user through a series of responses Source: adapted from Hutchings et al. 2012, Hellström 2010. Design criteria for the m-App are ease of use, relevance, convenience, and reliability for the user along with achieving the data collection goals of the project. Susanto and Goodwin (2010) observe that perceived ease of use, value for money, convenience, and availability of the device and infrastructure were the most important in adoption. Hellström (2010) noted that simple, easy to use interfaces were an important factor when designing m-Apps, in addition to the m-App achieving the primary purposes of the project. Thus, m-App development should be simple and pragmatic. Citizen users need to be confident that the technology supporting the data collection method is accurate and reliable. Conversely, methods should be put in place that ensure the data collected are valid and correctly entered. In forms, minimizing manual data entry can help prevent errors associated with mistyping and data validation criteria can help catch errors. Otherwise, service providers and WMAs may question data accuracy and thus be hesitant to respond, creating a potential trust barrier (Frefield et al. 2010, Hutchings et al. 2012). Combining HCD with rapid prototyping methods in the design and development of the m-App creates greater opportunities for achieving a usable and relevant m-App. In these methods, user needs are determined in defining the design challenge and investigation that provide context for m-App design. The m-App design is created with the end user’s needs at the forefront and may involve key users. Once the m-App has been designed, rapid prototyping and piloting methods involving users can be employed to expedite the development and customization of the m-App. An effective approach to rapid prototyping and piloting is the Lean Startup Method. This method involves providing users a “minimum viable product” that is functional, but has not been built to final functionality (Ries 2011). The user tests the “minimum viable product” over a short period of 24 A Conceptual Framework time, providing feedback to the developer who then uses the feedback to rapidly alter the product. The new version of the product is then returned to the user for another iteration with the process continuing until a desirable product has been created. This rapid prototyping method minimizes the effort to build unnecessary m-App functionality that developers originally conceived as being necessary but the end user does not require or want. Costs: Costs of technology include the phone purchase, services charges, and m-App development. Hellström (2010) indicated the costs of SMS, voice, and data transfer are the biggest barrier to widespread adoption of m-App in eastern Africa. Users will weigh the cost of using an m-App if they feel the perceived value it provides is not worth it (Susanto and Goodwin 2011). Thus, careful consideration of the user benefits and costs is required when selecting the platform and type of messaging for the m-App. Networks Coverage and Reliability: Effectively being able to connect to the mobile network influences the trust and usability of m-Apps. Rural areas that do not have mobile network coverage are potentially limited in using m-Apps. Networks that are frequently out of service may require data to be stored on the mobile for sending when the network returns to operation. Furthermore, during natural disasters (for example, flooding during large precipitation events) or crises, mobile networks are less reliable than satellite based networks. Data Dissemination: Following collection, results need to be disseminated to key stakeholders, which may include citizens, service providers, WMAs, NGOs, and researchers. The content and format in which end users receive m-App data should vary based on their needs (Table 6). Similar to the m-App development, dissemination should be developed collaboratively with end users and follow the rapid prototyping methods. For developing m-Apps for government agencies, service providers and stakeholders involved in water management, business operations should be reviewed to determine how the m-App data augment their management processes and in what format it is best consumed. Often, these organizations have existing systems (for example, dashboards, graphs, reports) that can be used for disseminating the m-App data in a relevant and familiar format for decision makers, water managers, and operators. In addition, the technical infrastructure (which includes the data server and ancillary connections to networks, data exchange protocols, and software and hardware requirements) needs to be considered in the data dissemination and design. When developing the disseminating application, privacy should also be taken into account. That said, it is important for the citizens and communities involved in data collection to have access to the results. Failure to report m-App data to these groups will create a barrier as the crowdsourcing project will be perceived as not relevant or useful and untrustworthy, thus the likelihood of long- term adoption could be put at risk (Susanto and Goodwin 2010). Table 6. Technical design options for data dissemination Category Technical Design Users Web-based Web-based dashboards display • Government agency, service Dashboard the collected data to convey useful providers, stakeholder group information to the user Web-based Places instances of reports on maps • Anyone: general public, Interactive Map allowing for visualization of reports, government agency, NGOs, different categories service providers, stakeholder group 25 Crowdsourcing Water Quality Data Table 6. Technical design options for data dissemination (continued) Category Technical Design Users Broadcasting or Information can be disseminated by • For general information: general Bulk SMS sending out identical information public, government agency, through SMS to a wider group. NGOs, service providers, During emergencies, SMS warnings stakeholder group can be sent to responders and local • Warnings: emergency personnel, residents system operators, local residents Interactive Interactive communication systems • Any mobile phone capable of Communication handle collection, analysis, and SMS transmissions dissemination of data Reports Reports analyze and present in the • Reports are generated for general form of graphs, plots, spreadsheets, consumption and targeted photos, etc. audiences (e.g., political figures, scientific papers, investors) Social Media Reports, analysis, and messages • For general information: general can be disseminated through social public, government agency, media sites such as Facebook, NGOs, service providers, Twitter. stakeholder group Source: adapted from Hutchings et al. 2012, Hellström 2010. Water Quality Indicator (WQI) Sampling Methods: As these projects are using citizen science, water quality sampling will be conducted by citizens. Therefore, selection of the technology must be simple and accurate enough for citizens with limited training to screen for water quality conditions, and not overly complex and expensive that testing is prohibitive. Criteria for WQI sampling method selection include: 1. Relevance to water quality issues of the project area. 2. Applicable to the sampled water source: local groundwater wells, distributed systems, or surface water bodies. 3. Easy of use: this is an important factor when asking citizens to collect data. Additionally, how much training and practice is required to effectively conduct a test? 4. Accurate, reliable, and repeatable: additionally, is the method subject to interpretation? 5. Analytical range extends over the range required for reporting. 6. Cost per test and financial feasibility. 7. Stores well and is easily distributed. Program Design Program design involves evaluating the political and agency supporting the infrastructure, financial model, and metrics for evaluating the project. In addition, where the data server is housed, who manages it, and how it is financed will also need to be considered. Political and Agency Supporting Infrastructure: Mapping the governmental, water management agencies, and research institutions as well as the policies, laws, programs, and operations involved in monitoring and managing water resources provides context for the m-App project. Done thoroughly and effectively, this activity will identify opportunities and barriers to implementation 26 A Conceptual Framework of the crowdsourcing project. Special attention should be paid to organizations collecting similar data as well as the use of water quality data in managing water resources. Financial Models: An important task is to develop a financial model of the m-App project. This involves computing the revenues and expenses associated with the project. Funding a project comes from revenues generated (commercial) and donors, government agencies, and corporate social responsibility funding (non-commercial). Potential pricing models for commercial m-Apps include (Qiang et al. 2011): • Non-chargeable model: the user does not pay and receives only basic market information provided. Examples are government based weather forecasts; • Transactional model: the provider charges the subscriber for using m-App based on the number of transactions conducted; • Embedded model: the m-App is provided for free, but used to generate demand and strengthen customer loyalty to the provider’s primary, chargeable product or service; • Freemium: basic services are provided for free with users paying for additional premium services; and • Revenues must be balanced against expenses: typical expenses to consider are presented in the budget checklist (Table 7). Table 7. Budget checklist for developing an m-ARD Stage Category Expense Set-up Costs Human resources Planning, marketing research, project coordination, costs technology preparation, content development, other campaign preparation, contracted server provider/ consulting fees Technology Hardware Phones and SIM, personal computers, servers, modems/ internet connections, other equipment Software/online Operating systems, publishing, messaging, project services and campaign management, information safety (virus protection), application integration, web-site hosting Communications Mobile network connectivity, internet connectivity Other Additional security measures, training (courses, fees), administrative costs Operation and Human resources Project coordination, campaign implementation, content Maintenance costs development, other campaign tasks, financial and HR management, technology and information systems, contracted server provider/consulting fees Marketing costs Content development and design, materials, media advertising Technology Hardware, replacement/maintenance of equipment, communications Software, online Technology support service Other Additional security measures, Insurance fees, training (courses, fees), administrative costs Source: Hellström 2010. 27 Crowdsourcing Water Quality Data Pilot Considerations: Technical Design. When developing and implementing the project, the following technical considerations should be addressed: • Data needs for all key stakeholders are defined; • m-App is developed and implemented collaboratively with citizens through HCD and rapid prototyping methods; • m-App is developed such that users perceive it to be easy to use, convenient, reliable, and relevant in order to increase the likelihood of long-term adoption; • Trust is ensured by notifying users of test submittal and response to requests. At a minimum, users need a return notification that the server has received their data message. Ideally, they have access to results, as well; • Data privacy and protection of key actors has been addressed. This may include willingness of WMAs to share data with the public. Failure for the WMA to share data is a potential barrier to adoption; • Similar to the m-App development, data dissemination applications should be developed collaboratively with end users and follow the rapid prototyping methods; • Business processes should be evaluated when developing dissemination methods for government agencies, service providers, and stakeholders. Often, existing platforms can be used saving time and effort in development and training, as well as expense in hardware; and • WQI sampling methods need to be selected based on the criteria presented above. Metrics: Performance metrics are used to evaluate the project and guide modifications. Performance metrics relevant to crowdsourced water quality projects are categorized into citizen adoption, technology performance (m-App, data dissemination applications, WQI sampling methods), financial performance, and development impacts. The performance metrics are collected before, during, and after project implementation. Annex 2 lists performance metrics relevant to crowdsourced water quality projects. Water Quality Indicators Properly assessing water quality conditions in relation to identified user needs involves the selection and use of water quality indicators. Selection of water quality indicators will depend on the water quality conditions, current and potential influences that affect water quality conditions, and testing technologies available that support crowdsourcing objectives of real-time water quality monitoring. Water quality is influenced by natural factors and anthropogenic (human) influences (UNEP and WHO 1996), both of which must be considered when selecting the analytes to monitor and the study area for conducting the crowdsourcing project. Natural factors include the geological, topographical, meteorological, hydrological and biological systems in the drainage basin. In addition, water quality conditions can vary from diurnally to seasonally to annually due to differences in runoff volumes, weather conditions and water levels. Primary influences of surface and ground water quality in natural settings include leaching of heavy metals (for example, arsenic, cadmium, chromium, lead, and selenium), major ions (sodium, potassium, calcium, chloride, sulfate), fluoride, and radionuclides (for example, uranium and radium) from interaction with soils and rock, microorganisms living in surface water bodies and soils, nitrates and nitrites in organic rich soils, tannins in forests, Total Suspended Solids (TSS) carried into water bodies, and thermal warming associated with hot springs. 28 A Conceptual Framework Human Activities Influencing Water Quality Anthropogenic activities that influence water quality include discharges of domestic, industrial, urban and other wastewaters into the watercourse (point source), and the spreading of chemicals on agricultural land (diffuse) in the drainage basin (Table 8). Point sources load water quality constituents at discrete locations (for example, outfalls to major rivers). Typical sources include sewage treatment, industrial waste effluent, solid waste disposal, animal feedlots and mine-adit runoff and processing plant effluent. Diffuse (non-point) sources arise from many small inputs over a wide area. Examples include unsewered fecal pollution, agricultural land runoff, solid waste facilities runoff, and mine waste pile runoff. Types of constituents associated with anthropogenic activities include: • Urban sewage and wastewater runoff: Sewage can run off or leach to groundwater introducing petroleum products, bacteria (including pathogens), fecal matter, food waste, fertilizers/pesticides from lawns, and household products such as cleaning solvents, used motor oil, paints, paint thinners, and detergents. • Industrial effluent and wastewater runoff: A wide variety of chemical, bacterial, and organic matter is used by businesses and industries for production (raw materials and processing) and cleaning and sanitation, as well as resulting by-products. Common analytes associated with industrial productions include: o Filling stations and refineries: petroleum, gasoline, motor oils, benzene; o Manufacturing: heavy metals, ammonia, volatile organic compounds, organic solids (biochemical oxygen demand - BOD); o Dry cleaners: cleaning solvents containing trichloroethaline and tetrachloroethaline; o Tanneries: chromium, sulfides, ammonia, volatile organic compounds, organic solids (BOD); and o Food processing: ammonia, cleaning solvents, bacteria, organic solids (BOD). • Agriculture: Farmers apply nutrients such as phosphorus, nitrogen, and potassium in the form of chemical fertilizers, manure, and sludge as insecticides, herbicides, and fungicides containing organic compounds and heavy metals to kill pests (US EPA 2005). Runoff from precipitation or excessive irrigation can transport pathogens, nutrients, heavy metals, pesticides and oxygen-demanding organics and solids. In arid areas, evaporation of irrigation water can concentrate salts limiting crop yields, and cause a buildup of selenium that can harm waterfowl reproduction. • Mining: Mineral extraction uses water to extract minerals, process ore, control dust, store ore/waste and general operation. Associated with metal mines, water contaminated with high concentrations of metals, sulfide minerals, low levels of pH (Acid Rock Drainage - ARD), processing chemicals, and suspended and dissolved solids can negatively affect surface  and groundwater quality  in the catchment. Mining exposes sulfide minerals to oxygen, breaking down the minerals and potentially releasing aluminum, arsenic, cadmium, cobalt, copper, iron, mercury, magnesium, nickel, lead, selenium and zinc (Environment Canada 2009). In addition, oxidation of sulfides produces sulfuric acid and when drained, creates ARD, which is low in pH and high in metals. In the process of separating gold, ore, mercury and cyanide are commonly used. 29 Crowdsourcing Water Quality Data Table 8. Sources and significance of pollutants resulting from human activities Sources Bacteria Nutrients Trace Pesticides Industrial Oils and Metals and Organic Greases Herbicides Micro- pollutants Point Sources Urban sewage xxx xxx xxx x xxx Industrial effluent x xxxG x xxxG xx Mining effluent x xxx x xx Diffuse Sources Agriculture xx xxx x xxxG Urban waste and xx xx xxx xx xx x run-off Industrial waste x xx x xxx x disposal Mining waste and x xxx x xx run-off Source: modified from UNEP/WHO 1996. Note: “x” denotes low local significance, “xx” denotes moderate or regional significance, “xxx” denotes high local or regional significance, and “G” denotes global significance. Analyte Selection Consideration The list of water quality indicators varies depending on the monitoring purpose and setting. A general list of water quality indicators is presented in Table 9. Common water quality field tests include alkalinity, ammonia, chlorine, color, dissolved oxygen, electrical conductivity, hardness, iron, lead, nitrite, nitrate, odor, pH, phosphorous, and temperature. These provide a general overview of the water quality conditions, but lack the ability to quickly evaluate biological indicators which must be grown. In addition, field tests for pesticides are not common. For drinking water, the Government of India has set the water quality standards for a complete suite of water quality indicators at IS10500 (http://hppcb.gov.in/eiasorang/spec.pdf). Table 9. General list of physical, chemical, and biological indicators for assessing water quality. Constituents in italics can be measured in the field Physical Indicators Chemical Indicators Biological Indicators Color Alkalinity BOD Electrical conductance Chlorine Fecal coliform bacteria (conductivity) (Escherichia coli) Odor Dissolved oxygen Total coliform bacteria Taste Hardness Temperature Heavy metals (arsenic, cadmium, chromium, iron, lead, mercury, and selenium) TSS Major ions (sodium, potassium, calcium, chloride, sulfate) Nitrate, Nitrate Pesticides, organic compounds pH Phosphorous 30 A Conceptual Framework Pilot Considerations: Program Design • Consider the social, political, and technical infrastructure supporting the project; • Develop a viable business plan; and • Metrics are selected to measure citizen adoption of technology, impact to the citizens and communities, technology performance, and financial performance of the project. Crowdsourcing Project Methodology Using a combination of the fundamental principles of citizen science and crowdsourcing as it pertains to the use of mobile phones in assessing natural resources, a methodology has been developed to assess water quality conditions. The development of a crowdsourced water quality project is conducted in four phases: 1) investigation and scoping;, 2) initiation and customization; 3) implementation; and 4) evaluation and reporting. Primary activities in each phase include: 1. Investigation and Scoping: the preliminary investigation to determine the best site, methods to use, and feasibility of a crowdsourcing water quality project. Activities in this phase include site selection; identifying and chartering team members; stakeholder needs assessments; water quality indicators; cost estimates; risk assessment and mitigation planning; and performance metric selection. With respect to the Theory of Change, the context information and desired outcomes of the participating groups gleaned during this phase provide the foundation for moving ahead with the project and are crucial to its long-term success. 2. Initiation and Customization: building on the background information in Phase I, Phase II involves creating, developing, and implementing technological solutions, as well as water quality testing distribution and educational awareness programs. Technological developments progress along two fronts: i) m-App development for water quality screening; and ii) developing dissemination applications to provide information to citizens, agencies/service providers, and interested stakeholders (for example, media, researchers, private sector). Rapid prototyping is used to customize technological applications to meet end-user needs, thus increasing the likelihood of long-term adoption. 3. Implementation: the execution of the crowdsourcing project. During this phase, communities and management agencies are educated; testing kits are distributed; water quality is screened by citizens and uploaded to the project database; results are disseminated to the community and used by water management agencies to manage the resources and communicate with stakeholders. 4. Evaluation and Reporting: Compiling information and computing metrics, surveying and taking testimony of participants, and reporting results to the communities, managing agencies, and interested stakeholders. Outputs can be used to evaluate the effectiveness of the program and determine areas where improvements are required. If the program is to be continued, this is likely a periodic report. The following text outlines the tasks and associated objectives, activities, limitations, and outcomes. If not explicitly stated, development and implementation should be done with regard to the citizen science and crowdsourcing principles outlined above. This methodology assumes that mobile phones will be the technology to report the water quality analyses. Note, though the project methodology is being established for application in India, principles can be applied to crowdsourcing efforts in general. 31 Crowdsourcing Water Quality Data 1. Phase 1: Investigation and Scoping As stated, this phase is to provide the background and impetus for conducting the study along with laying the foundation for the other phases. The objectives, activities, assumptions, and outcomes of tasks associated with Phase 1 are presented in this section. Task 1.1. Site Selection Objectives: Select the geographic area for the project with consideration of the social and environmental factors. Activities: Site selection is an important first step in implementing a water quality crowdsourced project. Ideally projects are set in a water quality limited area where citizens have limited access to water quality results and resource managers have limited resources to monitor water quality conditions. When determining specific geographies to target during execution of a project, key factors for considerations include: Theory of Change Contexts: The baseline information regarding the social context (Table 1), po- litical context (Table 2), and environmental context (Table 3) as outlined in the Theory of Change. • Water Quality Limited Areas:  Selection of areas that include both impacted and non-impacted water quality conditions could provide an additional means of testing the technologies.  At a minimum, the area should be water quality limited as monitoring in non-impacted areas will result in a limited evaluation of how crowdsourcing water quality testing could impact daily lives. Additionally, water quality limiting sources should be identified such as area of agricultural fields, tanneries, manufacturing plants, power plants, mining operations, and so on. Land use/land cover maps identifying irrigated, commercial, Pilot Considerations: WQI Selection • The standard water quality issues identified as influencing rural drinking water include microbial contaminants, fluoride, arsenic, iron, nitrate, salinity, heavy metals (cadmium, lead, zinc), organic pollutants, and pesticides (Wateraid 2008); • The Central Groundwater Board and State Groundwater and Public Health and Engineering Departments in India monitor water at discrete locations, thus providing the background on the extent and magnitude of water quality issues in proposed study areas; • Monitoring data should be gathered and compared against the citizen screening test data to ensure data quality, as well as determine the efficacy of all monitoring systems in characterizing water quality conditions; • Crowdsourcing water quality is intended to educate citizens (in real-time) of the quality of the water resources they are using. Thus, the selection of WQI needs, at a minimum, to target the WQIs that influence and are meaningful to local citizens. Citizen education of water quality conditions and risks can be used to create informed selection; and • Spatio-temporal frequency of sampling should to be designed to characterize the land and water use practices given the local hydrology. As the pilot is meant to investigate the most effective sampling frequency, variable sampling schemes should be tested. For example, one village should be asked to measure daily while another might be asked to measure every time villagers use a water source. Note, it is preferred to “over sample” the monitoring effort as data points can be removed from the dataset during evaluation to determine the most effective sampling frequency. It is impossible to add data locations if the pilot is under sampled. 32 A Conceptual Framework municipal/domestic, and industrial land types are good screening tools for identifying potential sources of pollutants; • Managing Agency Readiness:  Much of the data collection, storage, and analysis will occur in partnership with managing agencies. In addition, there must be openness and transparency with the data if the crowdsourcing approach is to be successful. For these reasons, it is crucial that the partner managing agencies have willingness to partner in developing and using the data, as well as developing the infrastructure (cultural and technology) to support the project. • Organizing Entities: For distribution, education, and selection of monitoring individuals and information distribution, it is important to select an area where supporting organizations are active. These organizations might include Water User Associations (WUAs), NGOs, community outreach groups, farmer organizations, and so on.  • Watershed Based:  It would be advantageous if the location included both rural and urban settings that influence and use the same water body. It is not required to look at the entire catchment, just representative areas of rural and urban settings. It is preferable that the site locations are in close proximity to each other because maps showing the influence will be more illustrative. Assumptions/limitations: Actively seek a cross-section of participants based on demographic factors, for example, gender, economic sectors. Outcomes: A list of potential project areas and associated background information. Task 1.2. Local Team Identification and Chartering Objectives: Develop the Local Team that will sponsor, promote, manage, and participate in the project, as well as the organization that will manage the infrastructure supporting the project. Activities: The core Local Team should be made up of community members, managing agency officials, service provider representatives, NGO representatives, and other water user stakeholders. The size of the team should be large enough for effective representation of the various stakeholder groups, but right-sized to ensure timely implementation. It is critical that the specific roles and responsibilities of each of the stakeholder groups be clearly delineated at the outset. Typical questions to ask when selecting team members include: • Who will serve as the project sponsor? • Who will provide direct project management and oversight? • How will the team interact (meetings, briefings, updates)? • How will decisions be made? • What does it mean to serve as a representative on the team? Assumptions/limitations: Local Team members have the time, commitment, expertise, and authority to participate and contribute to the project. Outcomes: Selection of Local Team members. 33 Crowdsourcing Water Quality Data Task 1.3. Stakeholder Needs Assessment Objective: Identify the needs of key stakeholders to ensure the project design and implementation are aligned to meet those needs and desires. Activities: To sustainably engage key stakeholders in collecting, analyzing, and disseminating information, it is necessary to understand their motivation in participating in the monitoring program. To determine stakeholder needs and motivations, the use of HCD processes and techniques offers an effective means for identifying the needs, desires, and behaviors of stakeholders (IDEO 2011). Throughout the project, it is critical that the needs of the stakeholders are well understood so that the potential innovative solutions brought forth meet those needs. In addition, it is necessary to understand the technical and procedural readiness of the community to participate as citizen scientists in this crowdsourcing pilot. Lastly, these exercises serve as the beginning of a process of engagement that will occur throughout the pilot and provide participants with the chance to experience what this kind of engagement entails. Examples of assessment techniques include: interviews, immersion, card sort, and photo documentation (IDEO 2015). A description of each technique is provided below. • Interviews: Tables in Annex 1 outline the interviewees and questions for initial site visits. • Immersion: To fully understand the needs of the community, experiencing how citizens, agency personnel, and other water users interact with and use water and technology in their daily lives provide a powerful context for clarifying needs. If it is possible to shadow individuals as they go about their day and document what is seen and heard, this information can be of high value for use throughout the planning and execution of the pilot. It is important to simply listen and learn without trying to interpret what is occurring at the time of the immersion. Once there is a complete picture of water and technology use and needs, greater interpretation and application of the knowledge gained can occur. • Card Sort: In this exercise, stakeholders are provided with a deck of cards with a word or image on each card. They are asked to sort the cards in priority order depending on what is most important to them. This can be an effective exercise for those where literacy is a challenge. It can also serve to begin a deeper discussion about overall values. In addition, because the cards can be tailored to the specific needs of the project, customization of the exercise is relatively simple. The result of this activity is clarification on what is important to each of the stakeholder groups, which will indicate areas of overlap, as well as divergence. This is important knowledge given the interdependencies these stakeholder groups will maintain throughout the pilot. • Photo Documentation: With the prevalence of smartphones in many communities, requesting that stakeholders provide photos of how they use water and technology, as well Pilot Considerations: Site Selection The pilot project is intended to test methodologies and tools, therefore it is advised to focus on specific locations that possess a breadth of conditions to provide the greatest opportunity for learning. For the proposed project, two additional factors should be considered: • Rural Area Includes a Command Area:  Ideally, the rural community would be in an irrigated area receiving water from a reservoir as these often have salinity and fertilizer impacts to surface and ground water; and  • Urban Diversity: It would be beneficial to have residential neighborhoods of different economic means as well as local industrial activities. Covering an entire city is not required, but including one or two wards of varying socioeconomic conditions is important. 34 A Conceptual Framework as what is important to them in that use is another means of gaining an understanding of desires and needs. Asking them to describe the photos from their perspective augments that understanding. This photo documentation can also serve as a physical example of community engagement if the photos are posted for community members to see. These visible symbols of the work of the project and the engagement of the community in participating in this work are critical to ensure initial participation and continued engagement throughout the pilot and beyond. Assumptions/limitations: Stakeholders are willing and available to participate in the needs assessment and resources are available to conduct the assessment. Outcomes: The numerous outcomes from this task that are crucial to the project success include: 1. Identification of the participants and communities that will be engaging in the citizen science project. 2. An understanding of: • Community water uses, as well as the water sources and quantity used; • Level of community awareness of water quality issues. If aware, method used to receive information; • Processes, protocols, and needs of agencies, NGOs, and the scientific community; • Drivers to foster long-term participation in the project; and • Mobile devices that are used, technical knowledge of participants, data plans and cost of usage. 3. Dissemination of data: The preferred method to receive and provide information. This will provide the foundation for adapting technology in the initialization and customization phase. 4. The training method most likely for success. Task 1.4. WQIs and Sampling Methods Selection Objectives: Select the WQIs to be monitored during the crowdsourcing project. Identify the potential sampling methods that citizens can use to measure the WQIs. Activities: This task builds upon information gleaned in the previous tasks and will be shaped by known water quality limitations, types of anthropogenic and natural pollution sources in the study area, use of water by the communities, interests of the agencies, NGOs, and scientific community. Steps for selecting the suite of analytes to be monitored include: 1. Determine the project area and the associated communities. 2. Determine the sources where people obtain water for drinking, as well as sanitization and health. Is the water primarily extracted from local groundwater wells, distributed systems, or surface water bodies? 3. Review existing water quality monitoring efforts to determine the water quality constituents that have been collected and documented. Potential sources for water quality monitoring data include the state government’s Department of Public Health and Engineering, central and state Departments of Surface Water and Groundwater, and the sate’s Department of Agriculture. 35 Crowdsourcing Water Quality Data 4. Inventory the current and future anthropogenic activities that may affect water quality conditions in the project area. Watersheds dominated by agriculture will have a different set of analytes than one that is primarily industrial. If the area has potential to have change in landuse or development, parameters should be included that represent the future conditions. 5. Determine the audience for the analysis. As stated, the citizen science/crowdsourcing approach involves, at a minimum, individuals using water resources for drinking water and those managing it. Analytes should be chosen that are relevant to both, though priority should be given to the individuals conducting the sampling as they will need incentives to remain in the program. Collection of data that the samplers view as irrelevant to their well-being will likely not support continued commitment to the monitoring effort. Once WQIs have been identified, a survey of testing methods is performed: 6. Determine the sampling methods and equipment available to screen WQIs at appropriate concentrations to match the desired conditions. The Government of India has established the regulatory standards for drinking water in IS10500. 7. Determine the requirements, cost per test, and feasibility of distribution of sampling equipment. 8. Obtain and perform a trial of each type of sampling equipment to determine the appropriateness for the project. Specific emphasis should be placed on ease of use, accuracy, reliability, repeatability, and training requirements of the sampling method. Sampling methods subject to user interpretation will need additional training. Assumptions/limitations: This assumes the communities have been characterized and that the information is available outlining WQIs. Outcomes: List of relevant analytes and associated sampling methods to be employed in the project. Task 1.5. Cost Estimate Objectives: Develop a cost estimate for the Initiation and customization, implementation, and reporting phases of the crowdsourcing project. Activities: Conduct a financial evaluation for the pilot project to assess financial performance, as well as assess the project’s scalability and sustainability. Revenues include those coming from profits for commercial projects and funding from donors, government agencies, and corporate social responsibility funding for non-commercial projects. Expenses should be calculated based on those outlined in Table 7. Depending on the identified project needs and conditions, additional revenues and costs may need to be considered. Note, if proven useful for citizens and agencies, the pilot project could be continued past the one-year period. Therefore, the budget for the implementation and the evaluation and reporting phases should be written such that it is the annual expenditure to continue the water quality screening in subsequent years. Assumptions/limitations: Accurate and up to date costing data is available. Outcomes: A cost estimate for the four phases of the crowdsourcing project. 36 A Conceptual Framework Task 1.6. Risk Assessment and Mitigation Plans Objectives: Understand the potential risks and develop mitigation plans prior to project launch. Activities: With the Local and Project Teams, conduct a SWOT analysis. Once risks have been identified, develop mitigation plans to provide guidance during planning and implementation. Assumptions/limitations: Local and Project Team members are willing and available to develop the SWOT analysis and mitigation plan development. Outcomes: Identification of the potential risks that may occur during the project and mitigation plans should the risks come to fruition. Task 1.7. Performance Indicator Selection Objectives: Select performance indicators for measuring citizen adoption of technology, impact on citizens and communities, technology performance, and financial performance of the project. Activities: Methods for data collection include collecting numerical data (for example, number of samples taken by farmers, number of visits to websites) as well as interviews of participants, community members and leaders, agency personnel, and interested stakeholders. Numerical data should be collected through time to illustrate trends in the use of the technology, who is adopting it, in what regions, with what frequency. Interviews should be conducted to elicit perceptions on the performance and relevance of the technology in the citizens’ daily lives, and to determine what is working well and what needs to be changed to improve and incentivize participation in screening water quality. Finally, results from the screening tests should be compared with state and central government water quality monitoring network results to determine the effectiveness of the screening techniques. Using the performance indicator list in Annex 2 along with other resources, the Project Team will develop a list of appropriate indicators. The list will be refined in consultation with the Local Team. A document outlining the indicators; protocols, sources, and methods for obtaining information; and format for reporting the results will be created. Survey questions should be outlined at this time with the expectation that the survey will be given before, during, and after the project in order to measure impacts. Generic survey questions are provided in Annex 1; however, these should be refined to reflect context of the project. Assumptions/limitations: Quality, quantity, and format of primary data are sufficient for evaluation. Outcomes: Document outlining the performance indicators, source of the data, how it will be collected, and the reporting mechanism. Task 1.8. Contracting Objectives: Contracting with vendors for services and equipment. Activities: Develop terms of reference for services and equipment to support the project. Contracting services can include, but are not limited to: Pilot Considerations: Cost estimate. During the cost estimate, the long-term outlook for the sustainability of the project should be considered. Therefore, clear objectives should be determined as this will define the performance indicators that can be used to determine the value of the project during the Project Refinement and Improvement Task. 37 Crowdsourcing Water Quality Data 1. Project implementation and management. 2. Community outreach. 3. Water quality testing equipment. 4. Lab validation expenses. 5. Technical services: m-App developers, website developers, server specialist. 6. Servers. Assumptions/limitations: Project funds have been secured from the funding agency (non- commercial project) or venture capital (commercial project). Funding procedures and schedules will be dependent on the rules for the funding agency. Services for m-App development may need to be contracted during Phase 2. Outcomes: Contracted vendors for services and equipment. 2. Initiation and Customization This phase require the formation of a technical user subcommittee to customize the m-App and dissemination technologies. 2.1. Form an m-App User Subcommittee Objectives: Develop the m-App User Subcommittee that will be responsible for the m-App conceptualization, design, development, and testing. Activities: The core m-App User Subcommittee should be made up of, at a minimum, community members that will use the m-App, managing agency officials, and m-App developers. The size of the team should be large enough for effective representation of the various community groups, but right-sized to ensure timely implementation. Assumptions/limitations: m-App User Subcommittee members are willing and available to participate in m-App development. Outcomes: Selection of m-App User Subcommittee members. 2.2. m-App Design, Prototyping, and Testing Objectives: Design, develop, and test the m-App and supporting server and infrastructure. Activities: Develop and customize the user interfaces for the mobile phone app. The design guidelines should follow the recommendations listed in the Technical Design section. This process will be iterative, employing HCD and rapid prototyping methods. The m-App developer will work with the supporting agency to implement the server, hardware, internet connectivity, software, data protocols, and other infrastructure supporting the m-App development. Additional contracting may be required to support the relevant infrastructure. Assumptions/limitations: Members of the m-App User Subcommittee actively test the m-App and provide information to the m-App developers. m-App developers can make changes and return an updated version to members. Outcomes: A customized and tested m-App that is useful for the community and achieves the goals for the project. 38 A Conceptual Framework 2.3. Data Dissemination Applications’ Design, Prototyping, and Testing Objectives: Design, develop, and test the data dissemination applications for citizens, WMAs, service providers, and other stakeholders. Activities: Develop and customize the data dissemination applications for the WMAs, service providers, and other stakeholders. The first step is an evaluation of the data needs and reporting methods required to support the business processes from each group. In addition, the servers, data protocol, and network connections from each user system will need to be evaluated and connections mapped to the m-App server. For each data dissemination application interface to be created, the development process will be iterative, employing HCD and rapid prototyping methods. Assumptions/limitations: Members of each group actively test the data dissemination and provide information to the data dissemination application developers. The data dissemination application developers can make changes and return an updated version to group members. Data protocols from each organization are available to exchange data. Outcomes: Customized and tested data dissemination that is useful for the each user group. 2.4. Distribution/Training Water Quality Sampling Systems Objectives: Distribution of water quality sampling equipment and education on its use. Activities: Develop the system for distributing water quality sampling equipment to the citizens and communities. Develop training materials and methods for water quality sampling and use of the m-App and citizen version of the data dissemination application. For the WMAs, develop training for use of the data dissemination application as well as it inclusion into their business practices. Assumptions/limitations: The availability of participants to receive both equipment and training. The availability of sampling equipment for distribution. Outcomes: Participants trained in the use of the equipment now in their possession. 3. Implementation Building on the information learned in the Investigation and Scoping Phase and solutions and technology developed in the Initiation and Customization Phase, the Implementation Phase of the full program is executed. The information discovered and the project formulated in the first two phases will dictate how the third phase will be conducted. While specific factors and aspects of the project will be determined in Phases I and II, the primary tasks required include water quality screening by citizens and technical maintenance. Listed below are the general terms of elements to consider for each task. 3.1. Water Quality Screening Objectives: Facilitate water quality screening by citizens and incorporation of the data collection into the WMA business. Activities: Citizens are actively screening water quality using sampling methods and m-App. This task involves supply and distribution of sampling equipment, determining that training is maintained, and public outreach is continued. Data are collected by the m-App server and performance indicator data are actively being collected. WMA have incorporated the data to support decision-making and water resource management. 39 Crowdsourcing Water Quality Data Assumptions/limitations: Citizens are adequately supplied, trained, and incentivized to conduct the tests. Data dissemination applications are maintained and kept current. Outcomes: Results during implementation include: 1. Active citizen networks that are screening for water quality, reporting results, and receiving water quality reports beneficial to their needs. 2. Data dissemination applications for citizens, WMAs, service providers, and other stakeholders. 3.2. Technical Maintenance Objectives: Support the m-App server database and reporting websites, phone apps, and links to agency databases. Activities: Throughout the project, support will be required to ensure that the technology supporting the crowdsourcing remains functional. Activities include updating hardware and software, troubleshooting power outages and hardware failure, installing and using anti-viral software, developing and maintaining data transfer protocol between organizations, and periodically backing up databases. Assumptions/limitations: Technical staff will be allocated time and trained to support the m-App application, associated servers, and data dissemination applications in each user group. It is assumed that these staff will remain through the project. Outcomes: Maintained infrastructure supporting the m-App and data dissemination applications. 4. Reporting and Evaluation A primary component of the Theory of Change is to measure and evaluate the progress and effectiveness of the project. This is envisioned to occur at the six-month and one-year intervals for the project with annual reporting to occur, should the project be continued. Results will be used to illustrate the effectiveness of the program and identify aspects of the project that need improvement for its sustainability moving forward. 4.1. Tracking and Data Collection Objectives: Gather information for use in evaluation and reporting (Task 1.7). Activities: Collect and process data for the performance indicators, as well as collect stories and testimonials from participants. The performance indicators selected will change depending on how the project is defined in Phases I and II. Assumptions/Limitations: Data collected will be processed and made available to participants. It is assumed that data collection and analysis will involve agency, scientists, and citizens involved in the Local Team. Outcomes: The database of the water quality data collected, valuation of performance indicators (defined in Task 1.7), and compilation of stories and anecdotal data for use in evaluation and reporting (Task 4.2). 40 A Conceptual Framework 4.2. Evaluation and Reporting Objectives: Report the project performance to the various participating groups involved in the project. Activities: Using the data collected in Task 4.1, reports are generated for several audiences. At a minimum, the reports generated include: a project overview for communities of the citizen participants; technical report for the agencies, NGOs, and scientists involved in the project; and effectiveness and financial reporting to funding agencies. Other audiences and reports may be determined in the course of the project, but the data should be comprehensive enough to customize the analysis for new audiences. Assumptions/limitations: The report formats will likely change based on the intended audience. Outcomes: Performance reports describing the water quality project tailored to the audiences involved in the project. 4.3. Project Refinement and Improvement Objectives: Understand areas for improvement that can be implemented during the next project cycle. Activities: Using the performance metrics and participant feedback regarding their experiences with the project, define refinements and improvements to be implemented during the next phase project. Assumptions/limitations: Availability of data and participants to provide feedback. Outcomes: Improvements to the project based on lessons learned. Pilot Considerations: Implementation Duration. Ideally the implementation phase of the pilot project would extend over a year. This allows the project to be evaluated through all phases of the hydrologic cycle, as well as in rural communi- ties, the Kharif, Rabi, and summer growing seasons. 41 Crowdsourcing Water Quality Data Conclusion Presented are the Theory of Change, design considerations, and methodology for conducting a crowdsourced water quality pilot project. Testing water quality using mobile technology in a crowdsourced framework with citizens assisting in the process is an exciting, yet complex, propo- sition with potentially great reward. To be successful will require thoughtful and well considered planning and execution. This document incorporates research specific to this kind of work, and suggests a Theory of Change and methodology to guide the efforts of the pilot. Given the critical importance of citizens and agencies understanding the quality of water they use, conducting the pilot project and continuing to learn and improve on this innovative approach is essential and has the potential to bring about lasting change. 42 A Conceptual Framework References Bonney, R., Ballard, H., Jorday, R., McCallie, E., Phillips, T., Shirk, J., and Wilderman, C.C. 2009. Public Participation in Scientific Research: Defining the Field an Assessing Its Potential for informal Science Education. A CAISE Inquiry Group Report. Center for Advancement of Informal Science Education. Washington, D.C.: CAISE. Buytaert, W., Zulkafli, Z., Grainger, S., Acosta, L., Alemie, T.C., Bastiaensen, J., De Bièvre, B., Bhusal, J., Clark, J., Dewulf, A., Foggin, M., Hannah, D.M., Hergarten, C., Isaeva, A., Karpouzoglou, T., Pandeya, B., Paudel, D., Sharma, K., Steenhuis, T., Tilahun, S., Van Hecken, G., Zhumanova, M. Citizen science in hydrology and water resources: opportunities for knowledge generation, ecosystem service management, and sustainable development. Frontiers in Earth Science (2014) 2: 21 pp. [DOI: 10.3389/feart.2014.00026]. Environment Canada, 2001. Guidance Document for the Sampling and Analysis of Metal Mining Effluents. EPS Report 2/MM/5 – April 2001, Minerals and Metals Division, Environmental Protection Service, Environment Canada. 46 p. Environment Canada, 2009. Environmental Code of Practice for Metal Mines. 1/MM/17, Mining Section, Mining and Processing Division, Public and Resources Sectors Directorate, Environmental Stewardship Branch, Environment Canada. 46 p. Freifeld, C. C., Chunara, R., Mekaru, S. R., Chan, E. H., Kass-Hout, T., Ayala Iacucci, A. and Brownstein, J. S. 2010. Participatory Epidemiology: Use of Mobile Phones for Community- Based Health Reporting. PLoS Medicine, 7(12), e1000376. http://doi.org/10.1371/journal. pmed.1000376. Hellström, Johan. 2010. The Innovative Use of Mobile Applications in East Africa. Sida Review 2010:12, Commissioned by Sida, Department for Human Development, Stockholm, Sweden. 140 p. Hutchings, M., Dev, A., Palaniappan, M., Srinivasan, V., Ramanathan, N., Taylor, J. 2012. mWASH: Mobile Phone Applications for the Water, Sanitation, and Hygiene Sector. Pacific Institute, Oakland, California. 114 p. IdeaWonks, LLC. 2012. Building and Sustaining Critical Mass Engagement in Crowdsourcing. September 8, 2015. October 30, 2015 . IDEO. 2011. Human Centered Design Toolkit, 2nd Edition. IDEO. 187 p. IDEO. 2015. The Field Guide to Human Centered Design. IDEO. 189 p. McKinley, Donelle. 2015. Design Theory of Crowdsourcing. October 2015 . Mechael, P. 2009. “The case for mHealth in Developing Countries.” Innovation: Technology Governance, Globalization. Winter 2009, Vol. 4, No. 1: pp. 103 -118. 43 Crowdsourcing Water Quality Data National Geographic Society. 2016. Oxford Dictionaries. 2015. Oxford University Press. 30 October 2015 . Ries, Eric. 2011. The Lean Startup. Crown Business Publishing, New York. 320 p. Rotman, D., Preece, J., Hammock, J., Procita, K., Hansen, D., Parr, C., Lewis, D., Jacobs, D. “Dynamic Changes in Motivation in Collaborative Citizen-Science Projects.” The 2012 ACM Conference on Computer Supported Cooperative Work. Seattle: ACM, n.d. 1-10. Qiang, Christine Zhenwei, Siou Chew Kuek, Andrew Dymond and Steve Esselaar. 2011. Mobile Applications for Agriculture and Rural Development. ICT Sector Unit, World Bank. Washington DC. 120 p. Susanto, T. D. and Goodwin, R. 2010. “Factors Influencing Citizen Adoption of SMS-Based e-Government Services”. Electronic Journal of e-Government, Volume 8, Issue 1 2010, (pp55 - 71), www.ejeg.com. United Nations Environment Programme (UNEP) and World Health Organization (WHO), 1996. Water quality monitoring: A practical guide to the design and implementation of freshwater quality studies and monitoring programmes.J. Bartram & R. Balance (Eds.). London: CRC Press. US EPA. 2005. “Protecting Water Quality from Agricultural Runoff.” Fact Sheet No. EPA-841-F- 05-001. March 2005. WaterAid. 2009. “Drinking water quality in rural India: Issues and approaches.” Wateraid report. www.wateraid.org 44 A Conceptual Framework ANNEX 1. Interview Questions Interviewee/Duties Questions GOVERNMENTAL AGENCIES Senior Officials • What are their needs/concerns regarding water quality? (Directs the water resources • Determine water quality (WQ) issues being addressed by the programs) agency. How are the WQ issues managed? How are water management decisions/policies put into action? • How open is the agency to sharing data? • Do they currently use mobile phones in their water management? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? • Note: important to get support for implementation of project Mid-level Officials • What are their needs/concerns regarding water quality? (Manages the water resources • Determine WQ issues being addressed by their department. How programs for the government are the issues managed? in the study area, e.g., Executive Engineers) • Do they measure WQ? If so, by what method, what equipment, how many locations, and how often? How many staff are involved in WQ management and what are their roles? • How do they communicate with stakeholders? Policy makers? NGOs and academic organizations? • Could this process be improved with new technologies? If so, how? How open are they to sharing data with the communities? • Do they currently use mobile phones in their water management? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? • Note: these individuals will likely be the project coordinators from the site area(s) 45 Crowdsourcing Water Quality Data Interviewee/Duties Questions Field Personnel • What are their needs/concerns regarding water quality? (Manages water resources in • What is their role in managing water? Do they measure WQ? If the field) so, by what method, what equipment, where, and how often? • How do they communicate with the community? With whom do they communicate? How often? By what media? • How are they helping to manage WQ issues? Could this process be improved with new technologies? If so, how? • Do they use mobile phones? What models? For what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? Data Centre Staff • What are their needs/desires regarding managing water quality data? (Manages the information technology of data: collect • What water quantity/quality data do they collect/store, from field/laboratories, disseminate? What parameters and from whom do they receive processing, quality data? How is the data received and in what format? Where is the assurance/control, storage, data stored, processed? and dissemination of data) • How do they communicate with the communities, stakeholders, policy makers, other staff, and other departments? In what format? How often? By what media? How open is the leader with sharing data with communities? • How are they helping to manage WQ issues? Could this process be improved with new technologies? If so, how? • Do they incorporate mobile phones to collect data or disseminate data to managers? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? 46 A Conceptual Framework Interviewee/Duties Questions Communities Community Leader(s) • Determine the community the leader oversees and what they manage. • What are their water quality needs/desires? • Determine WQ issues being addressed in their community. How is the leader managing the issues? • Is the leader alone or are others involved in WQ management? If so, what are their roles? • Do they measure WQ? If so, by what method, what equipment, how many locations, and how often? • How does the leader communicate with the stakeholders? Government officials? Could this process be improved with new technologies? If so, how? • How open is the leader with sharing data with the community? • Do they use mobile phones? What models? For what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies to assess WQ conditions in their community? • Note, the leader will likely be the community project sponsor from the community side. Typical Domestic User • Determine household uses of water. What are their sources of water? Do they have to travel to get it? How far? How much water do they use? • What water quality issues are of concern? How do they receive information on WQ? How often? By what media? Could this process be improved with new technologies? If so, how? What is the literacy rate? • Do they monitor their WQ? If so, what technologies do they currently employ for determining WQ? • Do they use mobile phones? What models? For what purpose do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies to assess WQ conditions in their community? 47 Crowdsourcing Water Quality Data Interviewee/Duties Questions Farmers • Determine irrigation practices including source of water, quantity used, and timing of use. What crops do they grow: kharif, rabbi, (Growing crops using summer? irrigation, exclude dry • What kinds of applications do they use on their crops? land farmers in evaluating irrigation schemes.) • What water quality issues are of concern? Do they have problems with salinity? How do they receive information on WQ? How often? By what media? Could this process be improved with new technologies? If so, how? • What technologies do they currently employ for determining WQ? • Do they use mobile phones? What models? For what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies to assess WQ conditions in their fields? Water User Association • What role do they have in managing water quantity and quality in the community? (Coordinates water • What are the primary WQ issues being observed in the communities management within command with which they are involved? How are they helping to manage areas or communities) WQ issues? • How do they communicate within the community? With whom do they communicate? How often? By what media? Could this process be improved with new technologies? If so, how? • Do they currently use mobile phones in their water management? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies into these communities? Aid Agencies NGOs, Academia • What is their role within the community? Do they play a role in Organizations managing water? If so, what role do they play? (Assisting the communities • What are the primary WQ issues being observed in the communities with water and land with which they are involved? management) • How do they communicate with the community? With whom do they communicate? How often? By what media? How are they helping to manage WQ issues? Could this process be improved with new technologies? If so, how? • Do they currently use mobile phones in working with the communities/farmers? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? 48 A Conceptual Framework Interviewee/Duties Questions Businesses Industries • Determine the industries’ uses of water. What are their sources of water? How much do they use? How much and where do they (Industries to include are discharge effluent? Storm water runoff from facilities? those that access freshwater or discharge waste water • What water quality issues are of concern to the industry? from similar sources as local communities and farmers) • How do they measure WQ in their processes? • How do they receive information on WQ? How often? By what media? Could this process be improved with new technologies? If so, how? • Do they currently use mobile phones in their business processes? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? Water Distribution • Determine household uses of water. What are their sources of Companies water? How do they treat the water? How do they distribute the water? How much does each party use? (Provides water through piped distribution systems) • What water quality issues are of concern? Could this process be improved with new technologies? If so, how? • What technologies do they currently employ in evaluating WQ? • How do they communicate WQ conditions with the communities, stakeholders, policy makers, government officials, others? In what format? How often? By what media? How open is the leader with sharing data with communities? • Do they currently use mobile phones in their water management? If so, for what purposes do they use mobile phones? • Would they be willing to participate in implementing crowdsourcing technologies in the study area? 49 Crowdsourcing Water Quality Data Annex 2. Performance Indicators – Process and Outcome Measures Performance indicators are used to determine the effectiveness of the crowdsourcing effort to measure and effect change. From the Theory of Change, the desired outcomes have been defined as better understanding of water quality, improved community empowerment and knowledge concerning water quality, increased actionable data for water quality management, and reduced cost of water quality monitoring. The list of indicators in this annex offers process indicators to track if the technology and its implementation are performing well, as well as outcome indicators to determine if the crowdsourcing project is benefiting citizens and water management agencies (the outcomes in the Theory of Change). A combination of qualitative and quantitative indicators is presented as both provide effective means of characterizing the processes and outcomes of the project. The table below presents a generic list of indictors for consideration and due to overlapping objectives, there is duplication in the use of indicators amongst categories. For example, the m-App Use category within the Citizen Adoption group has a similar set of outcome indicators as the m-App category within the Technology Performance group because both are determining the effectiveness of the technology. The indicators suite selected for a project will need to be customized given the cultural, technological, political, and environmental conditions of the study. Categories Process Indicators Outcome Indicators Citizen Adoption m-App Number of samples per user of m-App Perceived ease of use (navigation, logic, Use Number of m-App users (start/end) style) Trends in m-App adoption Perceived usefulness Spatial distribution of sampling Perceived convenience Frequency of sampling Perceived efficiency in time and distance Demographics of users (male, female, Perceived risk to user privacy economic status, job description) Perceived relevance, quality, and reliability Types of phones used Perceived availability (cell phone Types of water sources sampled network coverage, monitoring equipment availability) Types of water uses (drinking water, cooking, bathing and sanitation, Self-efficacy in using m-App irrigation, washing) Perceived value for money See m-App Technology Performance Perceived ease of obtaining water quality sampling equipment Citizen reporting a change in water consumption due to m-App data/return messages (change of sources, use from a source, timing of use) 50 A Conceptual Framework Categories Process Indicators Outcome Indicators Reporting Number, frequency of community (non- Correct amount of information provided Portal WMA) visits to the reporting portal Perceived ease of use (navigation, logic, Demographics of users (male, female, style) economic status, job description) Perceived usefulness See Reporting Portal Technology Perceived convenience Performance Perceived risk to user privacy Perceived relevance, quality, and reliability Perceived availability (access to the internet) Self-efficacy in using the interface Number of links by external parties (e.g. NGOs, other agencies, WUAs) to the reporting portal Communi- Number of training courses provided Percentage of citizens knowledgeable of cation for community leaders, NGOs, WUAs, water quality conditions (CoCs), sources of (Outreach) academics, general public pollution, local water quality, where to find Number of newsletters produced information). using m-App data or reporting portal Citizens perceive better knowledge of water information quality conditions Number of individuals subscribed to Citizens perceive improvement of receive newsletters government in management of water Number of visits to reporting portal quality conditions Number of public meetings Citizens perceive improvement in government transparency Updated pollutant management policies (household, commercial, municipal) Observed change in fertilizer/pesticide application Observed change in water disposal, sanitation Inclusion of water quality data in community water management plans Number of WUA, collectives, communities posting water quality information Number of conference presentations, scientific journal articles by WMAs, NGOs, WUAs, community leaders regarding WQ using m-App data/number of attendees 51 Crowdsourcing Water Quality Data Categories Process Indicators Outcome Indicators WMA Adoption Supporting Number of WMAs using m-App Number of senior decision makers using Management Number of WMAs linked to results m-App, reporting portal Decision/ portal Number of field officers using m-App, Organization reporting portal Number/frequency of WMA staff use of reporting portal Types of management decisions based on Responses per contaminated samples m-App data and improved understanding of water quality conditions Response time to contaminated samples Perceived improved understanding of water quality conditions, water use by demographic, sector, source, purpose due to m-App data Management decisions incorporating citizen use of water (sources, timing, frequency, uses) Perception of changes in public support greater for water management decision Number of reports using m-App data, results portal graphics/statistics Identification of new locations, times of contaminated water quality Trend in response rates to contaminated samples Communication Number of training courses provided Percentage of citizens knowledgeable of (Outreach) (community, NGOs, WUAs, etc.) water quality conditions (constituents of Number/frequency of public concern (CoCs), sources of pollution, local presentations to WMAs, NGOs, water quality, where to find information). WUAs, community leaders using m-App Citizens perceive better knowledge of water data (include number of attendees) quality conditions Number of conference presentations Perceived improvement of government in using m-App data management of water quality conditions Number of newsletters produced using Perceived change in quality of interactions m-App data using m-App data with communities, WUAs, NGOs, citizens Number of individuals subscribed to Perceived improvement in government receive newsletters (if established by transparency WMA) Improved pollutant management policies Number of visits to website (household, commercial, municipal) Change in fertilizer/pesticide application Inclusion of water quality data in community water management plans Number of WUA, collectives, communities posting water quality information Trend in costs associated with mitigation/ prevention water quality issues. Reduced illnesses/deaths due to water borne diseases 52 A Conceptual Framework Categories Process Indicators Outcome Indicators Staff Number, frequency of field staff and Perceived usefulness, relevance, and quality Competency decision makers using m-App of training Number, frequency of field staff and Perceived convenience of training decision makers using reporting portal Self-efficacy in using the interface Number staff training courses offered/ See Supporting Management Decisions attendance (technical staff, decision makers) Number of technical staff supporting m-App, reporting portal Duration of technical staff assignment to support m-App, reporting portal Technology Performance m-App Accuracy of measurements (per See Citizen Adoption: m-App Use constituent) See WMA Adoption: m-App Use Speed of measurement Accuracy of spatial location Accuracy of response message Percentage time with adequate connection to network Number of complaints on technology Relevance of analytes to CoCs Number/duration when m-App was inoperable due to software glitch, mobile network outage Response time to reported bugs Reporting Speed of site upload See Citizen Adoption: m-App Use Portal Number of clicks to relevant information See WMA Adoption: m-App Use Mapping functionality/presentation of results Accuracy of spatial location Time lag from m-App to reporting portal Support of multiple platforms (computer, tablet, mobile) Number/duration when reporting portal was inoperable due to software glitch, mobile network outage Response time to reported bugs Financial Indicators for Project Revenues Total profits/funding to support project Project has sufficient budget to continue Costs Total outlay to support project Decision makers perceive the technology Cost of technology (hardware and is beneficial economically (e.g. prevents software) expensive responses to water quality issues, cost effective means of collecting data, cost Cost of website, intranet service charge per capita of water quality education) Cost of staffing support Cost of supporting infrastructure Cost of educational, communications, publications 53 Crowdsourcing Water Quality Data Notes 54 A Conceptual Framework 55 Crowdsourcing Water Quality Data 56