A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 0 1 83305 ICT FOR DATA COLLECTION AND MONITORING & EVALUATION Opportunities and Guidance on Mobile Applications for Forest and Agricultural Sectors DECEMBER 2013 A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 0 1 ICT FOR DATA COLLECTION AND MONITORING & EVALUATION: Opportunities and Guidance on Mobile Applications for Forest and Agricultural Sectors © 2013 International Bank for Reconstruction and Development / International Development Association or The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Any queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. Cover Images: Left: © AES Department/World Bank; Right: © Flore Martinant de Préneuf/PROFOR C ontents iii CONTENTS Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Chapter 1: Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.1 Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.2. Survey Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Chapter 2: Assessing Project Needs and Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5 Chapter 3: Data Collection Implementation Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.1 Frontline Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.2 Automated Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.3 Crowd-Sourcing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.4 Passive Capture and Data Harvesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 3.5 Qualitative Data Analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Chapter 4: Many Applications, Many Opportunities: Key Choices for ICT to Collect Data in Rural Areas . . . . . . . . . . 11 4.1 Connectivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 4.2 Data Input Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2.1 SMS vs. Form-based Digital Data Input . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2.2 Basic Phones, Smartphones, and Tablets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 4.2.3 Digital Pen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.3 Budget and Costs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.4 Dashboard: Analytics, Data Management, and Stakeholder Access . . . . . . . . . . . . . . . . . . . . . . . . 16 4.5 Additional Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 4.6 Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.7 Technical Support . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 4.8 Language Capability, Security, Privacy, and Data Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R iv C ontents Chapter 5: Service Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.1 What ICT Cannot Do: Thoughtful Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.2 Driving Adoption . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.2.1 Adding Value to the End User . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.2.2 Properly Characterizing Users and Clients in Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 5.2.3 Rewarding Excellence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.2.4 Providing Non-financial Incentives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.3 Managing Risks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.3.1 Misplaced Focus of Training Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.3.2 Context Matters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 5.3.3 Efficient Cheating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Chapter 6: Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Successful Reliance on Tried and True Technology—The International Small Group 6.1  and Tree Planting Program in Kenya . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 FAO’s Open Foris Initiative—Interoperable, Modular Monitoring, 6.2  and Evaluation Tools for Forestry . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 The Way Ahead—The Effective Use of Satellite Imagery and 6.3  Object-Based Image Analysis Software in Laos . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Community-Managed Sustainable Agriculture—A Bottom-Up 6.4  Revolution Assisted by Mobile Technology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Thwarting Drought—Mobile-Based Data Collection for Drought 6.5  Preparedness in Uganda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 Chapter 7: Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 List of Tools Mentioned in the Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Annex 1:  Annex 2: Overview of Tool Capabilities and Considerations to Address in a Tool Selection Process . . . . . . . . . . . . . 43 Annex 3: Full Life-Cycle Costs Estimation Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Annex 4: Technology Features of Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 FIGURES Figure 1.1: Mobile Phone and Internet Penetration in Selected Regions (2000–11, per 100 persons) . . . . . . . . . . . . . . . . . 2 Figure 1.2: Framing the Use of ICT in Data Collection and M&E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Figure 3.1: Community Meeting. Girl on Cellphone. Aurangabad, India. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Figure 3.2: Women of Takalafiya Lipai Village. Niger. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 Figure 4.1: Telecommunications Network. Cambodia. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Figure 4.2: Platform Usage Based on Companies Surveyed, in Order of Frequency . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 Figure 7.1: Macro-level Effects of ICT in Data Collection and M&E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N CONTENTS v TABLES Table 4.1: Product Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Table 4.2: Suggested End-User Training Package Time Estimates for Specific Data Collection Applications . . . . . . . . . . . . 15 Table 4.3: Data Output Capacity and Software Integration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Table 4.4: Spatial Visualization Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 BOXES Box 2.1: Project Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Box 2.2: People Needs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 Box 4.1: Items to Consider When Selecting an M&E Tool for a Rural Development Project . . . . . . . . . . . . . . . . . . . . . . . 11 Box 4.2: Connect Online, Connect Offline: An Open Source Tool for Tackling Poor Connectivity . . . . . . . . . . . . . . . . . . . 13 Box 4.3: Questions to Include: Thinking Through What Is Needed . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Box 4.4: Digital Green’s Innovative M&E Data Reporting Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Box 5.1: Boosting Non-financial Incentives and Decentralized Collaboration through Social Media . . . . . . . . . . . . . . . . 25 A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R G LO S S A R Y vii GLOSSARY Custom The solution relies on special analysis software developed or customized specifically for a given client. Custom installation The software or system is installed to client’s premises and is expected to be serviced for maintenance physi- cally at the client’s site or remotely (see also Remote Management). Custom SW The software is developed or tailored specifically for customer needs and is usually made available for that particular client only. Intellectual Property Rights (IPR) to the software in full or in part may be with the service provider (developer) or with the client, depending on the explicit agreement between the two. Background IPR (anything existing or developed prior to the client project) remains typically with the developer or service provider. eCognition Image analysis software for geospatial applications. External GPS Mobile device or application relies on external device that is able to record geographical coordinates and transfer these data to a mobile device via wired or wireless data transfer mechanisms. GIS The Geographical Information System provides the ability to perform spatial analysis on the data set. This usually requires geographical location information being part of the data set at some level (coordinates, street addresses, etc.). GPS Support of Global Positioning System in mobile device allows mobile application to obtain geographical loca- tion information directly from within the mobile device. Hosted The software or system is installed and run usually at the service provider’s premises, but the client has access to the system via regular data communications networks such as Internet or telephony networks. The service provider is responsible for maintenance. ICT Information and communication technology In-house support Client organization’s own personnel provide support. This is usually the case when open-source systems are used. iOS Formerly the iPhone Operating System IT Information technology License fee The software or system is made available for use against a fee (a copy of the system is sold to the customer). Client modification of the software is usually not allowed. Support and maintenance are usually not included in the license fee (inclusion or exclusion of these should be clearly stated in the licensing terms or licensing agreement). M&E Monitoring and evaluation Offline use Capability to collect data when a mobile device has no active mobile data connectivity (for example, in remote areas). A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R viii G LO S S A R Y Online use Capability to collect data directly into a central database by communicating the inputted data immediately over an active data connection (usually a cellular data or Wi-Fi network). This usually involves a data connection being available to collect data. Open source Software licensing model in which the software is offered for use for free and can usually be modified at will; depending on the specific terms of the open-source license, the modifications may need to be offered for free to all other users of the software. Outsourced support Support is provided by a specialist third-party company or organization that is not providing the software or a service. PDA Personal digital assistant Remote management The software or system is usually installed at the client’s premises but the service provider can perform main- tenance activities remotely by accessing the system via computer data network such as the Internet. SaaS/cloud Software as a Service concept usually relies on the software or system being installed in third-party computer hosting facilities, with both the client and the service provider accessing the system remotely. The system is usually maintained and serviced by the service provider. Service provider Support is given by the service provider that is the source of the software or service. Signature Capability to input graphical signature digitally by writing with a finger or a stylus, usually on a mobile device that has a touch screen. SMS Data collection relies on use of mobile device’s SMS capability, either to transmit human-composed SMS mes- sages (that may or may not be specifically encoded or formatted according to project specifications) or as an underlying data transmission platform for a mobile application. SPSS A statistical analysis software suite developed and marketed by IBM. Subscriptions The software or system is available for use only when subscription is active. When the subscription ends, the right to use the system usually ends. Two-way sync Data can be uploaded from device to server and also downloaded from server to device. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N A c k nowledgments ix ACKNOWLEDGMENTS ICT for Data Collection and Monitoring & Evaluation: Opportunities and Guidance on Mobile Applications for Forest and Agricultural Sectors was funded by the World Bank Agriculture and Environmental Services (AES) unit and the Program on Forests (PROFOR). PROFOR is a multi-donor partnership managed by a core team at the World Bank. It finances forest-related analysis and processes that support efforts to improve people’s livelihoods through better management of forests and trees, enhance forest governance and law enforcement, finance sustainable forest management, and coordinate forest policy across sectors. In 2013, PROFOR’s donors included the European Commission, Finland, Germany, Italy, Japan, the Netherlands, Switzerland, the United Kingdom, and the World Bank. (See www.profor.info.) This report was made possible by a number of key individuals and consultants who authored and reviewed the publication, the information and communication technology group within AES that led and guided development of the report, and NetHope, with whom the accom- panying cloud-based database product was developed. The analytical study was conducted by Cory Belden and Priya Surya, who significantly contributed to the design of the study, finalization of the analysis, study findings, and writing of the report. Analysis was further supported by Aparajita Goyal and Peeter Pruuden. Forestry case studies were compiled by Troy Etulain. We are also very grateful to the service providers who answered surveys and provided information on their application or software and to those who participated in the associated workshop in March 2013. The team also acknowledges the peer reviewers of this work—Carol Bothwell, Caroline Figueres, Tim Kelly, and Krishna Pidatala—for their constructive critique and guidance throughout the process. Eija Pehu, Task Team Leader of this study from the AES Department of the World Bank, and Tuukka Castren, the co-Task Team Leader, provided continuous guidance. The support of Alison Mills and Jim Cantrell in managing the publication of the report is appreciated. All omissions and inaccuracies in this document are the responsibility of the authors. The views expressed do not necessarily represent those of the institutions involved, nor do they necessarily represent official policies of PROFOR or the World Bank. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R P R E FA C E xi PREFACE technology This study was developed to assist development practitioners in assessment and selection of information and communication ­ (ICT) applications for monitoring and evaluation in rural projects, specifically in agriculture and forestry, with an emphasis on mobile technology for data collection. Particularly in highly decentralized projects, data collection can be challenging, and the large number of ­ options and specific project needs makes selecting technology a challenge. This report was developed in response to an identified need for development practitioners to be able to stay current with changing technology and identify appropriate avenues for assessing and selecting technology to support monitoring and evaluation (M&E) as well as project outcomes. ­ The report proposes guidance in selecting and applying technology for data collection and monitoring and evaluation through the lens of agriculture and forestry projects. It is designed to be a deep-dive, operational piece that tackles how governments and development practitioners can use ICT to enhance their data collection and M&E efforts in rural development projects and programs. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R C hapter 1 — O vervie w 1 Chapter 1: OVERVIEW Tracking progress in sustainable agriculture and forest ­management is To support the growing interest among practitioners in using ICT ­nterventions challenging: distances are long, populations are sparse, i in agriculture and forestry sectors, the World Bank published two range from policies to crop and livestock practices, and the voice ­ reports on information technology (IT) in rural landscapes in 2011. of the farmer is critical for success. Recent approaches f ­ocusing on The ICT in Agriculture e-Sourcebook1 explores how digital tools— climate impacts and land use pressures (climate-smart agriculture mobile devices, applications, software, and geographic information and landscape approach) add to the complexity and require efficient ­ griculture systems (GIS), among others—can be used in 14 different a data collection and analysis methods. Heightened unpredictability subtopics ranging from productivity to risk management. The 2012 and changes in weather patterns have affected the productivity and Information and Communication for Development report dives into risks associated with agriculture and forestry activities and therefore mobile applications across different sectors, including agriculture, the lives of the communities who depend on them. The urgency to with a focus on value chains.2 In addition, the eTransform Africa obtain reliable data and their analysis and distribution to different report provides data and insights for the transformational power of stakeholders have increased substantially—given the global uncer- ICTs with sectoral examples, including agriculture.3 Within the for- tainty around food production, commodity trade, food prices, and est sector, Forest Governance 2.0: A Primer on ICTs and Governance4 the effects and speed of climate change. Furthermore, sustainable examines the role of technologies ranging from radio and mobile agriculture and natural resource management solutions are becom- phones to hi-tech satellite imagery in increasing public participa- ing interlinked and more knowledge-intensive, requiring reliable tion, enhancing economic efficiency, and improving law enforce- data for decision making. ment. All of these reports aim to highlight the opportunities found within the ICT innovation space for agriculture and forests. Alongside increasing climate change concerns is a promising trend: fast-moving, cost-effective, widespread information and com- Since the publication of these reports, significant progress has been munication technologies (ICT)—especially mobile phones. Their made on multiple fronts. Improvements in infrastructure have made affordability and pervasiveness has made them viable tools for data the mobile phone the most common and most adaptable tool used collection. With near real-time feedback from the field, t ­echnology worldwide. The leapfrog effects of ICT have increased access to is facilitating the ability to oversee operations across dispersed quality information, eased knowledge sharing among practitioners geographic locations, obtain complete data sets at a faster and ­ and resource-constrained governments, and created opportunities more efficient pace, and evaluate results more often and with a to improve accountability. The expansion of ICT has also made tighter and clearer feedback loop to practitioners implementing programs. The systemization of ICT in the monitoring and evalua- 1 www.ictinagriculture.org. 2 http://siteresources.worldbank.org tion (M&E) process also enables accountability—from field staff to /EXTINFORMATIONANDCOMMUNICATIONANDTECHNOLOGIES/Resources regional and central governments and development partners. It /IC4D-2012-Report.pdf. 3 http://www.infodev.org/infodev-files/resource/InfodevDocuments_1162.pdf. also supports evidence-based decision making and the effective 4 http://www.profor.info/sites/profor.info/files/docs allocation of resources in order to maximize social impact. /Forest%20Governance_web.pdf. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 2 C hapter 1 — O vervie w FIGURE 1.1:  Mobile Phone and Internet Penetration in Selected Regions (2000–11, per 100 persons) 110 100 90 80 70 Per 100 people 60 50 40 30 20 10 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 East Asia & Pacific, mobile subscriptions East Asia & Pacific, Internet users Europe & Central Asia, mobile subscriptions Europe & Central Asia, Internet users Sub-Saharan Africa, mobile subscriptions Sub-Saharan Africa, Internet users Source: World Bank 2013 and World Development Indicators 2013. Note: Data includes only developing countries in the regions. the work of development practitioners easier and more accurate. capacities and constraints. The number of technologies applicable In some regions there are already more mobile phone subscriptions collection and M&E is daunting; hundreds of apps, software to data ­ than people, and even Internet access has become more common. packages, platforms, services, features, and business models exist ­ (See figure 1.1.) and are constantly developing. Designs and implementation strat- egies vary. Costs range from open source and free to commercial Efficient and precise data collection is an integral component of systems and expensive. Not only does a practitioner have to learn M&E of projects and programs. The enhanced ability to monitor, how to use the technology and demonstrate it to staff and partners, measure, and adjust to impact—through visualizing data on maps he or she must also justify why a certain product was selected and using GPS coordinates, accessing research published in previously paid for in place of others and consider its long-term sustainability inaccessible locations, providing rich information to farmers who and utility in generating tangible outcomes. could otherwise not be reached quickly, or recording beneficiaries who gain access to services—improves capacity to meet goals aimed at reducing poverty and improving productivity and resil- 1.1 SCOPE ience. This report identifies where ICT has expanded the capacity questions This report seeks to propose solutions to some of these ­ to perform good M&E and, more importantly, it identifies where it concerning data collection and monitoring and evaluation. has not. It identifies where and how it can expand data collection ­ esigned to be an operational piece that addresses how It is d and M&E, but also why and how technology is not a replacement for ­ governments and practitioners can use ICT to improve their data human agency and involvement in analysis and interpretation tasks. collection and M&E efforts in rural development projects. Although The interest in deploying ICT for data collection and M&E has the report focuses on agriculture and forest activities, the principles also led to a plethora of tools and platforms with a variety of discussed can largely be extended to other sectors. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 1 — O vervie w 3 FIGURE 1.2: Framing the Use of ICT in Data Collection and M&E Needs Implementation Technology, Service design assessment model budget and cost The second section of the report focuses on the most important software, ­ ­ training required, GIS capability, survey limits, and data aspect of ICT use: articulating the needs of the project and users. ­ validation processes. After information was sourced from the com- The third section provides an overview of five models currently panies, a series of product and or technology considerations for used to implement and integrate information technology into M&E practitioners were developed. Additional sources addressing data ­efforts. The crux of the report centers on choosing the right product collection in developing countries using ICT, such as the work done or set of products for the project, and it includes cross-comparative by Humanitarian Nomad,5 were also consulted. guidance on application features such as data validation, offline The case studies in section 6 describe how a particular technol- capacity, dashboards, and built-in analytics (see section 4). The ­ ogy was adapted for use in various rural contexts. This information service design section deals (see section 5) with issues inherent to was sourced directly from practitioners involved in the projects. the provision of ­public services, such as how to provide ­appropriate Questions regarding technology selection, implementation of the incentives for the participation necessary to sustain the program ­ project, challenges, and impacts were explored in each case. The and why post-data collection e ­ fforts are critical to success. Along cases were selected with the intention to provide practitioners with with these practical approaches to deploying ICT, the report a diverse snapshot of how different data collection models and ­ describes five case studies on mobile-based data collection in the technologies have been employed to achieve specific project goals ­ ection 6). The conclusion section agriculture and forest sectors (see s in agriculture and forestry. follows the case studies. As stated earlier, this is a dynamic field, and thus the features It is important to note that the logic of the sections in this report captured in this publication will evolve and change quite rapidly ­ and the process suggested in figure 1.2 is an evolving one. The over time. To address this evolution, the information on applica- feedback loop is a critical component of any project that uses ICT, tions generated for this publication is also provided and updated and updating technology and the service design that supports it on a cloud-based public database developed by NetHope.6 This is based on the trends in the sector, which are constantly shifting. cloud portal is intended to provide a space for agencies and Paying attention to this feedback loop increases sustainability and organizations to research ICT solutions, exchange information on ­ maintains cost-effectiveness. the quality of services provided, and share experiences. The portal maintains a searchable product catalogue that includes the applica- 1.2  SURVEY METHODOLOGY tions discussed in this publication among many others. The portal The publication draws upon information sourced from over 20 also hosts discussion forums, user reviews, and accumulates case ­ companies that have developed widely used systems and apps for studies. Practitioners can access and use this database by going to ICT-enabled data collection in rural areas (see annex 1 for a list of http://cloudportal.nethope.org/supersearch/#q/keywords=&num= the tools referenced throughout this publication). The companies 10&channel=products&orderby=relevance&sort=desc&category= surveyed are those that have deployed their application or product 37&&38&&27&inclusive_categories=yes&pagination=P0 in more than one challenging context and on a large scale. These companies were surveyed on 34 application features, including 5 http://humanitarian-nomad.org. platforms used, dashboard analytics, interoperability between 6 http://solutionscenter.nethope.org. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R C hapter 2 — A ssessing P ro j ect N eeds and C onstraints 5 Chapter 2: ASSESSING PROJECT NEEDS AND CONSTRAINTS With a plethora of available technologies with unique combinations BOX 2.1: continued of capabilities and features, assessing the needs for any particular project is the first step in selecting the right set of ICT tools. To de- • Survey-based or another model: What model does the project require? (for example, crowd-sourced cide whether a particular technology—or any technology for that information, environmental or supply chain informa- ­ matter—is useful or not requires clearly defining project goals and tion through sensors, satellite imagery, or surveys) sector needs. Asking the questions typical to agricultural or forest • Level of reporting: To whom will reports go, and projects are important before even considering the technology. For how will they be used? (for example, multimedia for example: What is the project attempting to achieve? Who are the ­ beneficiaries, dashboards that allow multi-stakeholder access, auto-generated reports) targeted beneficiaries? What data are important to collect? What do project leaders want to draw from the analyzed data? What is the • IT resources: Are the ones needed available, and from where? (for example, in-house software development, best way to report the data for management and stakeholders? outsourced Software as a Service (SaaS) solutions) • Context: What constraints are inherent to ­ conducting The extensiveness of data collection and its method should be estab- projects in this environment? (for example, lished before the project begins. If technology is determined to meet ­ political conflict, inhibiting weather patterns, poor infrastructure) project and data collection needs, additional queries can be posed. These questions have to do with issues related to ensuring that ben- • Funding: Are realistic cost estimates available? Have ­ sufficient funds been allocated for this work? eficiaries are effectively reached and empowered through the use of ICT and as such are less concerned with the specifics of technol- BOX 2.2: People Needs ogy, such as connectivity, the length of battery life, costs, and offline capacity, and more concerned with timelines, whether the data col- ­ lection effort is singular or reoccurring, and the characteristics of data • Availability of technology literate enumerators or beneficiaries: How much training will be required to users, such as literacy, technology trust, and exposure to surveying. implement the data collection effort? ­ • Availability of technology trainers and support staff: The questions in boxes 2.1 and 2.2 highlight key project and people Will resources to train participants be available, and for a needs that warrant consideration. These questions are by no means sustainable price and sufficient duration? comprehensive but are meant to help assess project needs. • Well-constructed team: Can the project recruit and maintain a team with the diverse skill set required? Importantly, this includes specialists BOX 2.1: Project Needs with the ­necessary sector ­ expertise, professionals methodology (for example, survey techniques in ­ • Data: What types of data are required? (for example, such as random assignment), local experts with deep understanding through narrative stories, one- indigenous knowledge, and analysts who can ­ word answers on a number of different questions, or a outline essential questions and then perform ­ mix of these) ­ rigorous diagnostics once data have been gathered. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R C hapter 3 — D ata C ollection I mplementation M odels 7 Chapter 3: DATA COLLECTION IMPLEMENTATION MODELS Upon outlining project and people needs and determining that FIGURE 3.1:  Community Meeting. Girl on Cellphone. ­ selected technology is a good fit, an appropriate data collection Aurangabad, India. model can be designed. Using frontline workers—either commun­ ity-based professionals or selected surveyors—to collect information is the most common method of mobile-based data collection. In fact, ­ ethod. almost all providers discussed in this publication use this m Photo Credit: Simone D. McCourtie. This section outlines this method and other emergent ­ models in data collection, including automated capture, crowd-sourcing, passive capture and data harvesting, and qualitative data analytics. ­ 3.1  FRONTLINE WORKERS Frontline workers are enumerators, surveying the local population or target group on questions of interest. Using frontline workers to collect digital data requires similar design efforts as traditional Projects may benefit from testing data quality in one format versus methods such as random sampling. Local enumerators may have another—that is, testing whether the mobile phone interface helps limited experience with technology and survey methodologies, or hinders their particular data capture process. and they may thus require training in technology basics as well as survey administration in order to communicate questions and cap- 3.2  AUTOMATED CAPTURE ture pertinent information accurately. A mobile application is one Automated data capture through technologies like GPS, sensors, of many among the digital tools a frontline worker can use. Among satellites, and remote sensing has been around longer than other its advantages is the ability to make changes to the forms that can models. However, only recently has it become affordable, accessible, easily, and in some cases automatically, update field surveys. and tailored to development specialists and developing-country However, some projects with standardized forms and a large vol- governments (figure 3.1). Often GPS data are collected through the ume of data collection may benefit from digital pen technology, in same applications that frontline workers use for household data. which information is stored both digitally and on hard copy, which Records are marked by location data through a built-in or attached allows frontline workers to focus more on the interview process device. Modern smartphones and pocket cameras feature built-in than technology. While digital and hard copy storage in this manner GPS devices that allow geocoded photographs to be used as part may be perhaps more familiar to people, the quality of the data col- of the M&E process, often for evidence recording purposes. GPS lected can sometimes be substantially less than in mobile-phone- coordinates can also be used to log infrastructure points, such as ­ based systems that may allow for error checking at the source. farm location and size, or irrigated areas. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 8 C hapter 3 — D ata C ollection I mplementation M odels Sensors collect ecological data on soil, water, and other elements of FIGURE 3.2: Women of Takalafiya Lipai Village. Niger. interest. After installation, these are programmed to record data at certain times. This method, along with satellites, is especially useful for measuring changes over time. Satellites are most commonly used to monitor land use shifts such as deforestation and water patterns (for example, desalination, sea level rise), as well as production, such as ­ bservation yield output and rate of crop growth. However, raw earth o data require high levels of analysis before they are directly interpre- Photo Credit: Arne Hoel. table or actionable information. Remote sensing technologies such as LiDAR mapping allow analysis of vegetation mapping, including forest structure components such as crown density, crown volume, stand height, and tree density over large areas. These data can be used to estimate more complex vegetation characteristics, ­ including basal area, forest biomass, and forest volume, among others. section 5.2 on “Driving Adoption”). The utility of crowd-sourcing may also be limited by a lack of advanced phones that can send 3.3 CROWD-SOURCING complex ­ messages (of course, short message service [SMS] can be With the advent of widespread mobile phone access, crowd- used when simple, short messaging is all that is required), illiteracy sourcing for data collection is another emerging method for data that limits participation, and high levels of error involved in gather- capture. Crowd-sourcing is accomplished by allowing, request- ing messages from uncontrolled submitters (for example, without ing, and empowering rural people to send in their observations, surveyors or frontline workers), especially when participation is low. data, or information through their mobile devices (figure 3.2). This Crowd-sourcing is one component in wider data collection strate- approach can be used for many different purposes in rural develop- gies. It is essential that crowd-sourced observations are verified to ment. One common use is during crises and natural disasters. Data ensure that deliberate or accidental misinformation is identified, collection from the general public can also be used to help forest particularly in rural contexts where large numbers of participants and agricultural authorities identify emerging trends and phenom- (compared with urban areas) are not there to provide an additional ena. Agricultural applications such as pest outbreaks are a major level of accuracy. Last, if law enforcement is helped by citizens’ par- area for potential use, and in the forest sector, local monitoring of ticipation, as can be the case especially in forestry, it is essential that logging in forests can help reduce the prevalence of illegal logging. participants’ personal safety is not compromised and that informa- Crowd-sourcing allows individuals to contribute to the data collec- tion remains truly confidential. If mobile devices are to be used in tion process, making it more democratic and transparent, and also crowd-sourced data collection, it must be taken into account that enforcement and prevention activi- helps authorities to target their ­ already existing legacy mobile devices need to be supported by ties to areas of specific interest. selected data collection system, be it SMS or form-based. (See also While it has many applications, this method of data collection the discussion on technology support in section 4.) is not without significant challenges. It is often difficult to attract ­ participation of a large enough number of people to capture ­ adequate data needed for robust data collection. It may require 3.4  PASSIVE CAPTURE AND DATA HARVESTING heavy investment in continuous marketing and compelling incen- Data that can indirectly reflect a change in consumer habits, needs, tives for participation. One particularly essential incentive is visible or economic status can be captured from mobile phone use pat- and timely feedback to the information provided; participants need terns. For example, by analyzing patterns in mobile top-up amounts to be ­ confident that their contributions lead to action (see also and usage, researchers have been able to detect shrinking incomes I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 3 — D ata C ollection I mplementation M odels 9 well before the release of official statistics on this trend. Another of the policy or intervention, opinions on who should pay, or the example can be the flood of data generated through mobile pay- impact of a particular program. perceived ­ ments for subsidized and other agricultural inputs, which can allow This method highlights perceived outcomes beyond the focus governments and civil society organizations to better understand on outputs (for example, farmer adoption of new practices rather the use of these opportunities and the subsequent food produc- than the number of extension trainings given). Stories are collected tion patterns. This can lead to greater preparedness for preventing by volunteer or paid enumerators, fed into Sensemaker software, or responding to food shortages. Data from mobile usage patterns and analyzed to produce visualizations and analysis. This method is can also be combined with other methods to create a more robust not without its challenges—the cost of the software alone may be data collection strategy.7 prohibitive for some projects, but for complex problems involving a diverse set of stakeholders, such as wildlife conservation efforts or forest community-related projects, these emerging qualitative 3.5  QUALITATIVE DATA ANALYTICS analysis tools may be considered part of a broader M&E strategy to Analytical tools such as Sensemaker,8 created by CognitiveEdge,9 add a diverse human dimension to collected data. help quantify and analyze story-based data and use “stories” or ­ reports from myriad sources: users, experts, policy documents, Linked metadata automatically sourced and organized from multiple videos, and photographs, and then find patterns within these quali- ­ sources is another exciting form of an open data pooling initiative tative data. The analysis and associated visualizations are formed now also moving into agriculture space with a leading application of from micro-narratives to build a rich and diverse picture of the TotoAgriculture.10 This movement has great promise for qualitative questions of interest. For example, this could include perceptions data collection, but it is still in its early phase, and the dilemma for of the main problem or challenge, understanding of the purpose agriculture is how to select locally relevant data from global sources. 7 http://www.unglobalpulse.org. 8 http://www.sensemaker-suite.com/smsite. 9 http://cognitive-edge.com. 10 www.totoagriculture.org. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 11 Chapter 4: MANY APPLICATIONS, MANY OPPORTUNITIES: KEY CHOICES FOR ICT TO COLLECT DATA IN RURAL AREAS With needs assessed and an implementation model designed, the BOX 4.1:  I tems to Consider When Selecting an M&E Tool for a team can proceed to selecting a specific technology, application, Rural Development Project or platform. The selection process itself takes some attention. The digital opportunities available to practitioners working in the forest • Are there data collection technologies already being used in the target country? and agriculture sectors are far-reaching. Hundreds of applications • What platforms are used in the target country? exist, with varying connectivity capacities, hardware components, • Have the mobile devices that will be used to collect costs, and features. The proliferation of these applications, while cer- data already been procured? tainly promising for development work, has also led to hesitation • What is (are) the type of survey(s) required: one-off or continuous? and confusion in selection, design, and implementation. • Is there an existing reporting/analysis/visualization tool in use (such as ArcMap, Google Earth, SPSS)? Practitioners must answer a number of questions in order to ­determine whether an application suits their project’s needs. For ­ example, are • Are there opportunities to scale out existing systems? applications truly offline capable? What are the running costs of a • How much is it possible to align with the public service data collection effort? Are dashboards secure enough to store sensi- provider (for instance, in the agriculture sector)? tive data? How much training is required for local staff? What types • Is an SMS based system required? of hardware offer GPS functionality? Are there applications capable • Do the survey data need to be stored on your own servers? of collecting thousands of data points? Answering these questions, • Is real-time synchronization from the remote field or among others—and before technology roll-out—are critical to the field office location needed? project’s cost-effectiveness and success. Box 4.1 includes further • Are forms in non-western Latin character sets needed? questions to be addressed when selecting tools for a project, building on work of the Humanitarian Nomad Online Selection Tool.11 This also • Do the staff who create the forms have a basic ­ understanding of databases and data structures? includes taking into account e ­ xisting capacity in both personnel and • Is it acceptable to pay for a service solution that technology. (See annex 2 for a more thorough overview of product would host the solution and facilitate installation and development of forms? ­ capabilities and considerations when making an ICT selection.) • Is there a need to collect and display spatial data on Table 4.1 builds on the questions in box 4.1 and displays the mul- a map? tiple options available according to different product features. It is • What are the emerging trends in technology and methodology? a helpful visual in thinking through options available for platform, capability, storage, analysis, features, and other important aspects of The data sourced from companies revealed a number of key dis- the technology being selected. tinctions between the applications available for data collection and M&E in the rural sector. It also clarified relationships between 11 http://humanitarian-nomad.org/online-selection-tool. features of a given application (for example, more-complex data A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 12 C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas TABLE 4.1: Product Features Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Proprietary License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms Note: The table shows different options available in the market and does not imply any specific combinations of features in columns or rows. collection needs more higher-end hardware). This section ­highlights FIGURE 4.1: Telecommunication Network. the top considerations when selecting an application for a project, Cambodia. such as connectivity, data input technology, budget and costs, ana- lytics, data management, and stakeholder access. 4.1  CONNECTIVITY Though major strides have been made in connecting last-mile, ­ rural populations to telecommunications and broadband networks (figure 4.1), there are still many areas with unreliable connections. Practitioners and governments alike are often surprised to discover that mobile services are not feasible due to restricted telecommu- access. Innovative solutions to using ICT even in these nications ­ situations are possible, but they require careful planning. On the opposite end, some rural areas are gaining access to broadband. This increases the potential to use smartphones in data collection efforts and relieves the need for offline tools and the transaction Photo Credit: Chhor. costs resulting from travel to “connected” central locations. The overwhelming majority of providers surveyed have “offline capacity” (see box 4.2 for Digital Green’s method).12 This means ­ that the mobile devices used have the option of collecting data and storing it, then subsequently uploading it to the central server not typically successful if telecommunications networks are weak. or dashboard once an Internet or telecommunications network is Some applications offer specialized solutions. For example, iForm- within range. This is a good option for data collection efforts that builder created the “thunderplug,” a device that synchronizes data involve enumerators because they often have to travel to a central from multiple mobile handsets to a central location even without location in any case. Crowd-sourcing efforts are different and are networks.13 12 http://www.digitalgreen.org. 13 https://www.iformbuilder.com. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 13 BOX 4.2:  Connect Online, Connect Offline: An Open Source typically have smaller screens only capable of displaying one or a Tool for Tackling Poor Connectivity questions at a time on a single screen, whereas smartphones few ­ and tablets can accommodate a larger number of questions with In areas with poor Internet connectivity, uploading extensive survey data or media-files can be a debilitating challenge at the more descriptive question text. More complex answer structures, field level. To circumvent this challenge, an India-based non- such as selection lists and tables, typically also require either smart- governmental organization (NGO), Digital Green, has created an phones or tablets. Form-based tools often also include skip logic open-source platform, Connect Online Connect Offline (COCO) features that allow for data from previous answers to be used to that enables people to use the application continuously, and only requires connectivity when a user is ready to synchronize determine which of the subsequent questions need to be displayed with the global data repository. This customizable framework and answered. Simple SMS-based approaches do not suit well if such can be used to upload baseline survey data, photos, and videos questionnaires need to be used. It is worth noting that a form-based from the field without the need of IT or Engineering staff. system may also internally use SMS as a data transmission channel, Free download available at www.digitalgreen.org/technology. but such a setup typically has disadvantages on the costing side, as cost per character sent via SMS is typically much higher than when General Packet Radio Service (GPRS)/3G data connection is used. survey Another consideration, along with connectivity, is whether a ­ Usually an SMS-based system is assumed to use the regular SMS can be updated in real time. Survey uploads or changes must be inbox/outbox system available on each and every mobile device in done with a network connection (currently most applications even the lowest price points. transmit data over telecommunications networks, not broadband). However, there are differences in the levels of management. If using 4.2.2  Basic Phones, Smartphones, and Tablets SMS to collect data, the administrator usually must send an updated Most service providers are moving away from approaches that use survey to the data collector. If using tablets or smartphones, survey basic phones toward those that use smartphones. This is because changes will synchronize automatically with the handsets (as in the while basic phones have lower start-up costs, they have higher case of Cropster).14 There are middle grounds to these two options usage costs because SMS can be very expensive. Conversely, smart- ­ as well; for example, users are given the option to update the sur- phones might be more expensive initially, but they do not often veys. This allows them to finish data collection on one version of the incur such high data transmission costs. The cost of these devices is survey and then update to the second version when convenient. improving, given dramatic drops in prices, and many older hardware products maintain compatibility with applications even after a new Connectivity and the need for data transfer should be the first con- version has been released. In the same vein, smartphones are often siderations when designing an ICT-enabled data collection effort. easier to learn how to use than SMS. Whereas SMS data collection The strength of networks will significantly influence the technology, projects are forced to use code to fit information into the small text application, hardware, and level of administrator involvement. message format, many smartphones have intuitive touch screens. Tablets are also becoming more commonly used due to rapid price 4.2  DATA INPUT TECHNOLOGY decreases. This is especially true for Android devices. 4.2.1  SMS vs. Form-based Digital Data Input Some applications can run on multiple platforms and others are Devices to be used in actual data collection are often determined more restricted. Cropster can run on all platforms so long as there by the needs and complexity of the desired data set. One significant is an Internet connection. Freedom Fone,15 on the other hand, can factor is the physical screen size, which determines the amount receive calls and texts (crowd-sourcing) from any platform, but sur- of information that can be displayed at a time. Feature phones veys must be administered through Ubuntu 12.04 or Debian. 14 https://www.cropster.org. 15 http://www.freedomfone.org. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 14 C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas An additional component to selecting hardware is whether BOX 4.3:  Questions to Include: Thinking Through What Is geographical data should or need to be collected. Most applica- ­ Needed tions can be GPS-enabled. This feature is most often determined by Despite the capacity of digital tools to capture lengthy surveys, the capacity of the phone or tablet used and therefore influences questions should be kept at a minimum. The quality of data gath- the cost of hardware. For example, Android, iOS devices, and other ered deteriorates quickly if surveys are too long. Moreover, too smartphones have GPS features built-in. Simple Java phones do not. much data makes analysis difficult and overwhelming; the original intent of the data collection is drowned out in efforts that ­attempt Freedom Fone and FrontlineSMS,16 which do not use smartphones, to tackle multiple agendas simply because the tool a ­ llows for it. do not have GPS capacity. If location is absolutely critical but the Defining the scope of the project explicitly helps to narrow down phone used does not feature GPS, additional devices can be used the list of questions. Projects must therefore optimize for accuracy to attach GIS information to each survey, although this method is during surveys and interviews to achieve best results. time-intensive and prone to error. Integrating ICT commonly reduces the transaction costs of data 4.2.3  Digital Pen 17 collection, so in most cases, digitizing at least some steps of the In some cases, a mobile application can complement digital pen- process increases efficiency. based data capture. For instance, for a survey regarding livestock A number of cost and budget factors should be considered when disease, surveyors may find that only 30 percent of households choosing an application. The three most varying factors are hard- are experiencing livestock disease. In that case, a mobile applica- ware and associated platform, the level of complexity (and therefore tion may be used to capture data on the 70 percent of zero-valued the training and troubleshooting required), and the scope or scale households, and dot-printed or color-coded forms only need to be of the data collection effort. The third factor is intuitive: the larger used with households where there is an incidence of one or more the scale, the higher the cost. Of course, there are a multitude of livestock suffering from a disease. This combination allows for cost additional cost implications. A thorough bidding and selection pro- savings as well as a more descriptive feature set. Effectiveness of cess (see annex 3 for a costing template and a list of considerations) processes based on digital pen technology is also often challenged will help to clarify accruing costs. by the lack of automatic data validation capability at the source. Such capability can be made available when using other electronic The applications discussed in this publication have the c ­ apacity to input mechanisms (laptops, tablets, mobile phones). Also, digital collect substantially large amounts of data. Due to confidentiality pen battery charging should be considering when using digital agreements and “hands-off” business models, many companies pen–based solutions in remote areas. are unable to provide specific details on the scope of certain proj- ects. In general, however, the applications reviewed in this pub- lication are capable of creating surveys with over 100 questions 4.3  BUDGET AND COSTS (and some with as many as 400 questions)—see box 4.3—and Considering the project’s budget as well as the costs of a given hundreds of enumerators (iFormbuilder, Magpi,18 and Open Data application or technology logically follows the connectivity analy- ­ Kit [ODK]19 reported projects that used over 1,000 enumerators). sis, as connectivity limits options regardless of the budget size. Tens of thousands to millions of observations have been collected ­ vailable are indeed in a single project. Clearly, the applications a 16 http://www.frontlinesms.com. collection efforts. capable of implementing scaled-up data ­ 17 A digital pen is an input device that captures the handwriting of a user and converts handwritten analog information created using “pen and paper” into digital data, enabling the data to be digitized and uploaded Cost is related to the platform used because of the hardware to a computer and displayed on its monitor. The handwriting-capturing technology used by various vendors may be based on accelerometer, ­ requirements and the costs associated with using them. It is important positioning assistant, camera, or trackball. Digital pens typically contain a regular writing pen so the output can be seen on paper, as with any pen. Depending on the technology used, the paper may be plain or 18 https://www.magpi.com (Magpi was formerly known as Episurveyor) specifically formatted using miniature spotted patterns. 19 http://opendatakit.org/ I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 15 FIGURE 4.2:  Platform Usage Based on Companies Surveyed, TABLE 4.2:  Suggested End-User Training Package Time in Order of Frequency Estimates* for Specific Data Collection Applications20 LESS THAN ONE DAY ONE TO THREE DAYS FIVE TO SEVEN DAYS ANDROID: ************** (14) DataWinners iFormbuilder Magpi JAVA: *********** (11) doForms FrontlineSMS TechnoBrain iOS: ****** (6, with others on their way in the future) Mobenzi PoiMapper Freedom Fone Nokia Data Gathering Esoko Blackberry: ***** (4) Open Data Kit Cropster Windows: ***** (4) Text to Change Freedom Fone EpiCollect OpenXData mKrishi to note, however, that sometimes costs are accounted for elsewhere (for Source: Product Survey 2012. example, using a platform that is more intuitive for users will reduce the * If users are new to the technology, the amount of time needed to train may increase. costs associated with training even if that device is more expensive ini- tially). The most widely used platforms associated with mobile devices today are iOS, Android, and J2ME. The J2ME platform uses the cheapest Finally, the level of complexity of the hardware selected (along hardware and is the most simple in capacity, however it does not pro- with expectations of data collectors) will determine training needs vide many options that smartphones do. iOS and Android p ­ latforms, and capacity building, which is often the most expensive—and on the other hand, provide more complex functions, but they are also most important—part of a data collection effort. Enumerators and more expensive than a Java device. Blackberry and Windows are also self-reporters (crowd-sourcing) will need less training if they are platform options, but they are much less commonly used than the using basic Java phones or their own phones. With more-complex others. Figure 4.2 captures the platforms that providers use. devices, additional training is required. The most time-intensive training is usually needed for participants who are asked to perform Costs can also come from a variety of other sources. The database administrative or dashboard tasks, especially if they are unfamiliar host, management, training, and fees for SMS or data transmission with the technology. Table 4.2 is a simplified estimate of the amount services range in price. Whether the application comes packaged of time it takes to train data collectors on application or device use. with other services like consultation also influences the price. Training needs will increase if someone new to the technology is Generally speaking, applications that are not SaaS will be cheaper, charged with managing the administrative portal. Yet it is important but they may require additional IT support or planning time and that training not thwart enthusiasm for digital tools; most local par- thus increase costs through staff hire. The important note here is ticipants learn and adopt these technologies with ease. that projects that use ICT to enhance their data collection or M&E processes still necessitate human involvement for design and The survey used to gather company information in this study also implementation; this component will increase either the cost of the attempted to estimate costs for each application. The companies application or the costs supporting it. At the same time, integration surveyed were presented with a hypothetical situation. In its most of ICT systems needs to be built in existing and foreseen institutional basic form, companies were asked what it would cost to collect realities to be sustainable. For example, the structure and sources of 8,000–10,000 observations using 50 enumerators in Kenya. Yet budget funding matter. Many agencies may be cash-poor but staff- due to a plethora of options, presenting detailed information on rich, or there may be (donor) resources available for investment but responses has limited utility. In their aggregate form, packages that not recurrent budget. It is important to consider whether invest- include training, hardware, and dashboards are estimated to cost ment and recurrent budgets can substitute for each other. This has Solutions 20 These and a range of other tools are profiled in the Nethope ­ an impact on technology choice as well; an agency may have staff Center database: http://cloudportal.nethope.org/supersearch/#q for in-house implementation and support but not for payment for /keywords=&num=10&channel=products&orderby=relevance&sort= desc&category=37&&38&&27&inclusive_categories=yes&pagination external services and licenses, or vice versa. =P0. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 16 C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas less than $10,000 for almost all providers surveyed. An important TABLE 4.3: Data Output Capacity and Software Integration lesson learned here—beyond the cumulative and generalized TEXT OR PROVIDER PDF XLS CSV SPSS STATA XML estimate—is that there is a multiplicity of options and customized Cropster * * * * solutions on the market even within companies that have well DataWinners * established and widely used applications. This means that articulat- doForms * * * * ing needs, context, and goals are indispensable tasks before con- EpiCollect * * tracting a provider. Without such articulation, costs cannot and will Esoko * * * not be accurately assessed during bidding processes. ESRI * * * Freedom Fone * 4.4 DASHBOARD: ANALYTICS, DATA FrontlineSMS * * MANAGEMENT, AND STAKEHOLDER ACCESS Grameen * The dashboard or portal, where collected data are stored and avail- iFormBuilder * * * * * Kimetrica * * * * * able for analysis, is a critical component to a data collection effort. Magpi * * * All applications host the data somewhere, and where they host mKrishi * them—along with the data manipulation features offered, the plat- Mobenzi * * * * form’s organization, and data output—affects the types of analysis, Nokia Data * * * sharing, and reporting available to users. Two fundamental options Gathering are an off-the-shelf solution that might accompany the application ODK * * * selected or a customized system. Open source solutions like MySQL OpenXData * and PostgreSQL, as well as solutions created by a paid developer, Poimapper * * * * * are good options—depending on factors like budgets and needs. TechnoBrain * * * Text to Change * * The first and probably most important consideration regarding the Source: Product Survey 2012. dashboard is whether it is hosted on a stationary hard drive or in the cloud. In the past, many applications were installed and hosted on software, and some dashboards have analytical software built in. one computer. This made it difficult for multiple users to view the data. Exporting to xls, pdf, Word, and csv are some of the most common Most applications now offer their services in the cloud or will soon do analysis output options. Applications can also export to visualization so (FrontlineSMS, for example, is currently developing a cloud-based software (if using GPS data) such as KML, Google Earth, Bing Map, server). Some providers leave the stationary or cloud option open for and ArcGIS online. How the data will be used after they are collected administrators to decide. If selecting the cloud option, two important should be considered before selecting an application. Table  4.3 questions are how many people can access the data and how they shows the options for a selection of providers, demonstrating the will get that access. Some providers have restrictions on the number many similarities in their export capacities.22 For heavy analytical of viewers (such as Cropster, which only allows four users). Different work, applications that export to STATA or SPSS may save time by applications also offer a variety of “usership” options: Poimapper, for ­removing the need to transfer data from multiple software. However, example, allows one admin user to create others and assign them transferring data from one software to another is not all that ­difficult rights (for example, to create forms, manage data, and view data).21 with solutions like StatTransfer.23 As such, it could be argued that Data output and management are also considerations. Different data formats compatible with many others, such as csv, are the ­applications can export data to a variety of analysis and visualization 22 These and a range of other tools are profiled in the Nethope Cloud Portal database: http://cloudportal.nethope.org. 21 http://www.poimapper.com. 23 http://www.stattransfer.com. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 17 TABLE 4.4: Spatial Visualization Tools more-rapid analytical output (for monitoring during different project PROVIDER KML BING ARCGIS GOOGLE phases). A common but important application function is “search.” Nokia Data Gathering * parameters Searching or organizing data in the dashboard (or setting ­ DoForms * ­ educes to do so) by date, contact, and survey question, among others, r Kimetrica * * * the manual data manipulating often needed for analytics. Magpi * Techno Brain * * Kimetrica is a special case, as it offers end-to-end project manage- Poimapper * * * ment especially designed for M&E needs, which makes it a versatile Mobenzi * * platform for generating standardized national, policy, or project- iFormBuilder * level metrics.26 The system interfaces with Google Earth, and at the ODK * same time users can overlay country administrative maps. The sys- EpiCollect * * tem comes with administrative maps for every country in the world ESRI * * * * down to administrative level 3. It also allows users to upload their Source: Product Survey 2012. Note: Cropster plans to integrate ArcGIS in the short-term. own administrative and EA maps. Kimetrica also includes a large number of routine data entry and field enumeration management ideal formats to choose during application selection. Companies do functions that make it easy to monitor survey performance and ­ offer customized solutions and are sometimes willing to develop progress in near real time. a new tool or process (such as exporting to a different software) if requested. Thus this area of functionality is in constant evolution. 4.5  ADDITIONAL FEATURES Visualization and data representation can play a major role when There are special features of applications that may save time and using data to argue for change, which makes the use of spatial data be critical to certain M&E efforts. The first is whether an application relevant. There are a number of spatial tools to map data through has signature or photo capacity. Recording signatures can be useful GPS. Spatial visualization software includes KML, ArcGIS, Google for a variety of projects, but it is particularly useful in data collection Earth, and Bing. Table 4.4 shows the providers that integrate this efforts that may require participants to confirm or acknowledge type of software into their application and dashboard directly. Some their involvement or the release of their personal data. Photos can applications, like Poimapper, can export to other visualization soft- be useful in smaller data collection efforts that are designed to ware if licenses are purchased. record case studies or personal anecdotes. They are also useful for projects that require identification (for example, data collection for Dashboards also have a range of additional features. Providers offer subsidized fertilizer to selected farmers). a variety of other reporting or analytical tools within the dashboard ­ itself. However, because most applications allow data export to rel- Put simply, smartphones have these features and Java phones do evant software like Excel, some reporting features can be considered not. However, just because the device has the capacity does not more or less bonuses. The DataWinners dashboard provides the user mean the application can logically store and organize the photos or with basic calculations (sum, average, min, max) for quantitative data,24 signatures. Twelve out of 20 providers surveyed have photo-taking FrontlineSMS offers chart views for polls, and Mobenzi integrates capacity, but only 4 out of the 20 have signature capacity. Most pro- chart-making for reports into their system.25 These features are not viders have plans to offer both services in the future. as critical as other features, such as capacity to record spatial data or record data offline, but they could save time for projects that require Interactive-voice records (IVR) are also an important feature of digitized services in the agriculture and forest sectors, but they are 24 https://www.datawinners.com. 25 http://www.mobenzi.com. 26 http://www.kimetrica.org. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 18 C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas not as relevant to data collection efforts. The primary purpose of efforts through IVR could be a practical way to get analyses back to including IVR in applications is to give illiterate users access to the local stakeholders. same information as people who can read text messages. Users can Some of the best M&E programs collect client or beneficiary infor- call a specified number and select pre-recorded options to receive mation at the individual level, in aggregate forming a rich, detailed information on important farm practices, for example. Though data database of work completed. Digital Green has gone a step farther collection through IVR may not be feasible due to airtime costs and to connect geographical and user data through Farmerbook—a difficulties with research methods in rural locations (for example, visual geo-mapped database to provide a connection between end selecting certain geographically located households based on a rep- clients, service providers, partner organizations, donors, and the resentative sample size), disseminating the results of data collection general public. (See box 4.4.) BOX 4.4: Digital Green’s Innovative M&E Data Reporting Model The main page of Farmerbook features the leaderboards of the how their questions, interests, and adoptions change over time farmers, community intermediaries, and partner organizations. A and how that relates with other members in their community. different metric, called adoption rate, ranks community members, Farmerbook images of farmers and service providers also con- facilitators, and partners based on the percentage of ­ practices nect with Digital Green’s Facebook game Wonder Village, which they have adopted relative to the total number of extension brings Digital Green’s work with rural communities to a wider ­ videos they have viewed. A more composite metric, based on ­international village. attendance rates, screening frequency, and adoptions, is used to In addition to this user-centric view, the organization’s analytics grade the villages, depending on their level of activity and per- dashboards (http://analytics.digitalgreen.org) share aggregate formance. This feature is meant to stimulate some healthy com- statistics and visualizations of these data along time, geographic, petition among service providers and communities to improve and partner-based dimensions and “Our Videos Library” (http:// participation and adoption. videos.digitalgreen.org) shows these data from a video-centric The timeline view on a village or individual allows users to see the perspective to identify videos that are most or least viewed, integrated nature of the practices that farmers are watching and adopted, and queried to improve content quality and relevance. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 19 4.6 INTEROPERABILITY application up-front cost of single-provision tools (for example, an ­ Data exchange standards allow organizations to share data within or a database), the latter often excludes maintenance and technical their own bodies and with external data experts and stakeholders. support. However, customized solutions do also require continued Although it is not a specified feature of a given application, the IT support after initial provisioning, and as these services are often level of desired interoperability—or the capacity to easily transfer separately sourced at a later stage, the overall cost may also be data and outputs to various systems and dashboards—should be quite a bit different. Practitioners should therefore carefully weigh considered, as it prevents convergence complications in the future. the benefits of a bundled off-the-shelf approach against a more Complications could, for example, be derived from an attempt to customized solution. aggregate data from similar surveys conducted in different regions. One final note: not all projects necessitate interoperability. Many Without data exchange standards and formats, governments and organizations that have pioneered ICT in their projects, such as agencies cannot effectively share data or add to analysis across Catholic Relief Services,28 have tried many options and are now departments. Surveys must be “reinvented” when applications use increasingly converging on a centralized model in order to have specialized software that is non-transferable to other types. For large comparable, standardized data across the organization. development organizations, this is highly problematic and leads to the same inefficiencies found in simple, paper-based data collec- 4.7  TECHNICAL SUPPORT tion. Not only is maintaining and requiring the same data standards critical to the sustainability of digitized processes, it also has sub- Technical support is crucial to new ICT4D projects, and it can vary stantial payoffs when tools can be used or built in from other proj- widely among different service providers. Some companies include ects. All new initiatives, and even ongoing ones, should consider tech support in their fees while others do not. Open-source appli- the interchangeable nature of their systems—and attempt to make cations do not normally maintain a strong technical support team, them increasingly open to sharing and transferability. and users are often left to their own devices to solve problems. Community forums are one way users have answered questions Formhub, for example, uses the industry standard open-source and resolved issues. However, forums do not always lead to timely ­application, Open Data Kit (ODK), which uses xls forms (Excel). Using or sustainable solutions. IT teams that can provide regular, expert ODK Collect through formhub allows users to build surveys through solutions to technical problems like coding error are critical to proj- an Excel file, upload them onto formhub, and download them ects that use free applications. It is important to realize that using onto enumerators’ phones, where data are then transferred to an open-source applications does not automatically mean there is no xls-compatible server.27 This solution takes some learning, is free, cost involved throughout the full life cycle of deployment of such and, importantly, maintains common standards throughout the application software. Open Data Kit is a good example. Underlying data collection process. software is offered free to use, but the operational model may Due to the increasingly recognized importance of a total ecosystem ­ require support from either service providers or an in-house ICT approach and interoperation, many providers are using the Software team. This makes up the business model for the service providers. additional as a Service model. These are services that require no ­ investment in infrastructure outside of purchasing the end-user 4.8 LANGUAGE CAPABILITY, SECURITY, PRIVACY, device. Even maintenance and technical support is included in fees. AND DATA VALIDATION In cases where in-house IT support is limited, these models can be As development specialists begin using technology in their great solutions. It is important to note that although costs over the projects, there are always a variety of concerns raised, one of which ­ lifetime of the SaaS tool usage may seem at first higher than the is language. Oftentimes, common languages like English, Spanish, 27 http://formhub.org. 28 http://www.catholicrelief.org. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 20 C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas French, and Arabic are not useful in initiatives in rural spaces. in place and in active use. Service providers offering hosted services Technology needs to support local and indigenous languages; even usually have taken measures to offer these functions as part of their if local people are not directly reading or responding to messages service. (which is sometimes the case due to illiteracy), enumerators who SMS is a feature most prone to security challenges. Technically the collect the data normally speak local languages. Fortunately, many SMS security is often compromised not at the mobile network level applications can master most languages. iOS products support 34 that is used to transport the SMS messages, but at the end-user ­ pplications can spend a few days to cus- languages, and many a device usage level. For instance, sent messages are usually stored in tomize systems to support specific languages. The most common the device and are accessible to anyone gaining access to it unless capacity is the platform, where data are restrictions to language ­ the device usage itself is controlled by security measures such as hosted along with the data visualization and stakeholders’ access PIN codes. FrontlineSMS, a service provider that maintains that the portal or dashboard. best protection comes from user care, recently published a guid- ­ecurity. Two additional common concerns are viruses and data s ance note on data security.29 While both concerns are valid, the virus-related threats appear Privacy in the context of data collection can be seen from at least limited, according to service providers. None of the companies ­ two perspectives: privacy of the collector or enumerator and privacy queried report that they have had problems with viruses. Viruses are of the data collected. In the former, the main concerns are protect- more common in situations when computers or laptops are con- ing the identity of the party collecting the data, be it crowd-sourced nected to the Internet. If users download items from the Internet or or a frontline worker. For instance, when data gathering is used to their e-mail, a virus can be included. These threats should be consid- provide field information about activities such as illegal logging or ered and prevented through the use of firewalls and other means if unlawful land use, it must be ensured that the information cannot using these types of tools. freely be linked back to the person reporting it. In case of privacy Data security is another problem, one that most service providers of the data, the collected data may contain sensitive information have overcome by providing hosted services. If data are hosted at a such as personal details. Therefore proper mechanisms must be put client’s own data center or sometimes even stand-alone servers, the in place to limit or possibly prohibit access to identifiable pieces of security of the whole hardware may become a concern. Encryption, information in the collected data set. login passwords, and backups are all used to protect sensitive Data validation—making sure that errors are caught and cor- ­information. However, in some circumstances the computer systems rected—is another issue that practitioners are concerned about. may be compromised and data can be unlawfully decrypted with However, service providers have discovered and used methods adequate computing resources in a certain amount of time. Use of that control potential problems with errors. For example, many non-common passwords at all levels is usually a good measure to applications have skip conditions (for example, if a respondent is help make this type of attack less successful. Thus the information is male, and there are questions specific to females, the survey will still only as safe as the hardware and the physical environment it is “skip” over the female-related questions), automatic date recordings, stored in. Another aspect related to data security is data safety, the specified ranges for numerical questions, and constraint validation overall end-to-end system reliance against data becoming acciden- (which does not permit the user to go to the next question if blank tally or deliberately destroyed. This could happen simply because values exist). Applications also use built-in mechanisms to prevent of human error or because of systems or hardware failure or even numbers or letters from entering certain spaces (by specifying the disasters such as fire or floods. To deal with challenges related to data security and safety, it is important that proper backup capabili- ties and, if necessary, disaster recovery strategies and procedures are 29 http://www.frontlinesms.com/user-resources/user-guide-data-integrity. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 4 — M an y A pplications , M an y O pportunities : K e y C hoices for I C T to C ollect D ata in R ural A reas 21 “type” of information to be entered), multivariate validation (which apply ArcGIS also provides tools to help validate user data. Users can ­ prohibits implausible combinations of data), and the prevention of topology-based validation to make sure that features conform to double entries. Most applications will not allow users to input or pre-determined spatial restrictions. Users can also define coded val- send data that falls outside of the specified criteria. Not all applica- ues, domains, or subtypes to restrict attribute data to specific choices, tions offer the same range of data validation tools. These are impor- helping you maintain data accuracy. Additionally, you can create tant questions to ask providers, particularly when data are complex and use feature templates to streamline data creation by defining or sensitive. required attribute fields, default attribute data, and default symbols. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R C hapter 5 — S ervice D esign 23 Chapter 5: SERVICE DESIGN It is widely acknowledged that technology is not a stand-alone data and analytics back to relevant stakeholders. Communication solution in most development settings. The most successful appli- ­ experts are useful in tasks that involve the provision of feedback. cation approaches develop and design integrated systems—mak- ing the mobile application, hardware, backend software, dashboard, 5.2  DRIVING ADOPTION and sharing mechanism, as well as the collection methodology, and As in any landscape project or program aimed at improving liveli- field worker incentives, all key aspects of the overarching data initia- hoods, long-term adoption by users and by those urging uptake of tive. In addition to the technology infrastructure, a strong service new technologies (for example, community extension workers) is design is needed to ensure that the technology will be adopted both critical and often challenging. The use of ICT does not auto- widely, add maximum value, and be sustainable beyond the initial matically overcome these inherent challenges. However, there are stages. This section addresses three of the most important com- solutions to adoption barriers. Many of them that are centered on ponents in effective service design: thoughtful analysis, ensuring the structure of incentives are similar to those used in projects that adoption, and managing risks. Just like in the design and imple- do not use ICT. These strategies to increase sustainable adoption mentation of development programs that do not use ICT, these most closely apply to frontline workers who implement the survey, components—the human components—remain critical. but their application can also be extended to other users, such as those involved in crowd-sourcing efforts. 5.1 WHAT ICT CANNOT DO: THOUGHTFUL ANALYSIS 5.2.1  Adding Value to the End User Technology is not a stand-alone solution for M&E. Even after select- In projects where extension officers and community professionals ing, building, and deploying the technology, successful projects carry out trainings, mobile-based data collection applications can require human capital to translate the data into meaningful infer- help isolated service providers to structure their work and to obtain ences and to decipher development implications. Statisticians and fact-based feedback regarding their performance. By giving them methodologists, who are trained to accurately derive conclusions targets and making them feel part of a larger effort with achievable about causality and outcomes, cannot be replaced by technology. benchmarks, features that send analysis back to service providers Moreover, it is dangerous for organizations working in this space to could provide a better context and connectedness, especially in believe that technology can be used for deeper analysis of the data. rural projects. This value can also support and sustain continuous Local M&E expertise—or capacity to correctly obtain conclusions from use of data entry systems. Moreover, training that focuses on “why it data—may be difficult to find and retain. Practitioners should there- matters” and how the data are used is crucial to motivate field staff fore include the hire of analysts within their project plans and budget. to strive for the most accurate results. Skilled analysts are not just required when data have been collected. As in projects without ICT, they should also be included during survey 5.2.2  Properly Characterizing Users and Clients in Projects design, sampling methodologies, and other advanced evaluation Many agricultural applications deal directly with farmers and other processes. Similar thinking should accompany processes to report end-users of information and services. In forestry—and in forest A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 24 C hapter 5 — S ervice D esign governance and resource assessments in particular—the situation enumerators. (See box 5.1.) Targeting women and minorities in the is often different. Often forest sector development interventions selection process also provides opportunities for those traditionally aim at influencing the performance of the forest institutions and excluded from gainful employment to gain marketable skills and agencies as well as the forest users directly. Therefore public sector experience. For community professionals, incentives such as vis- information management is the focus, and forest users are indirect ibility and prestige, as well as the motivation to work on hardwired clients of development investments. Consequently the data collec- challenges in their communities, can be a major driver, especially on tion methods and incentives are different. In essence, action often community-driven development projects. happens at the “wholesale” rather than “retail” level. This has two specific implications in forest sector data management 5.3  MANAGING RISKS and M&E: First, it is essential that data and information flow in two A final component of service design is the management of risks. directions; information dissemination on forest resources and agen- Mobile-based data collection has a great number of benefits that cies activities is just as important as information collection from makes information transfer automated and efficient, as well as rich the public. Second, in public administration and service delivery, with the use of location data and photo capabilities. But data collec- the assessment of sustainability differs from private transactions. tion technology is only as good as the information that is collected Assessing sustainability from the perspective of a private transac- and the abilities of those who collect it. Therefore, accounting for tion and financial revenue capture is not enough. Public services risks is imperative during both service design and implementation cannot be assumed to be entirely self-financing. stages. One final comment on characterizing the users and clients involved pertains to participation. Participatory design—that is, piloting, and 5.3.1  Misplaced Focus of Training Programs user-testing software with actual users—creates a product more Even paper-based data collection requires rigorous training of likely to be adopted by taking into account uniquely local con- enumerators in eliciting information from target individuals and straints and perspectives. households. With a mobile interface that could potentially distract from the data collection process, this training becomes even more 5.2.3  Rewarding Excellence essential. As projects turn to mobile and digital technology, training Projects have various incentive structures for frontline workers, must focus on not only the technology itself but, more importantly, hired exclusively as enumerators to collect survey data or as service on helping enumerators understand the context of the project and providers who also collect data on program participants. These how the information they collect will be used in order to enable structures may include a lump-sum payment or a fee per day of them to focus on accurate results. data submitted. Some projects, such as the Sustainable Agriculture project of the Society for Elimination of Rural Poverty (SERP) take a 5.3.2  Context Matters different approach. SERP’s project provides a base salary plus a sub- In switching from manual to mobile-based data collection, it stantial performance bonus (30–50 percent of the monthly salary) becomes easier to coordinate and track field staff—such as exten- ­ for sending in complete, accurate data sets 90–95 percent of the sion workers. When these data begin to be used to judge perfor- time. This strategy is highly effective in aligning incentives between mance of field staff without offering added value in the front end, the organization and the frontline staff in improving data quality location capabilities can remove autonomy and the project can and building capacity. subsequently suffer from lack of adoption. 5.2.4  Providing Non-financial Incentives One example of this is when mobile applications capture informa- Training and improving employability for future opportunities by tion regarding attendance at agricultural trainings. The attendance obtaining experience is often a big incentive, especially for younger could also be a factor that the frontline worker does not necessarily I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 5 — S ervice D esign 25 BOX 5.1: Boosting Non-financial Incentives and Decentralized Collaboration through Social Media Aggregating buyers and producers of eco-friendly organic young facilitators, who are also organic growers, get the chance ­ produce, the Association of National Ecological Producers (ANPE) to promote their own products in the ecoferias. in Peru has integrated youth involvement and social media tools rural In line with this strategy, the project has leveraged growing ­ to promote decentralized coordination and local ownership and access to the Internet in order to use social media tools such as as an added marketing platform to create effective demand for Facebook and You Tube in two main ways: to provide an oppor- ­regional ecoferias. tunity for exchange of ideas and information among youth facili- With the financial and advisory support of the International tators and opportunities for leadership and recognition and to Institute for Communication and Development, the orga- position the “Fruits of the Earth” brand to attract new customers ­ nization has embarked on a three-year pilot that currently and improve consumer awareness of eco-conscious consumption. works with 45 young facilitators and over 500 organic farmers The project is currently in its third year and has begun scaling up selected from local communities to collect data from farmers to three more districts in Peru. ANPE is striving to make the ICT regarding supply and in turn facilitate marketing and busi- initiative and youth involvement sustainable through partner- ness decisions. ships with both the Peruvian restaurant industry and exporters of organic products. As the project covers only costs of traveling to the ecoferias, much of the incentive for participation is non-financial: the opportunity Additional Resources: to learn and gain experience, a chance to expand their networks ANPE Peru http://www.anpeperu.org/ and obtain exposure to their peers and to external organizations Frutos de la Tierra – Ancash https://www.facebook.com and farmers, and recognition through programs on rural radio /Frutosdela TierraAncash and through their commitment and involvement, which gives Frutos de la Tierra – Caramoja https://www.facebook.com them a more active role in their associations and families. The /FrutosDeLa TierraCajamarca control. With real-time feedback, if this information were to judge some extent, but the problem can ultimately only be addressed by the service provider’s performance in a vacuum without taking ensuring that the application adds value to the end user, provides into account the relevance and usefulness of the program and the resources necessary to compensate performance, and enables local context, it could become a disincentive for workers to con- rather than undercuts the autonomy of service providers. Another tinue to send in information. The section on improving adoption method to improve accurate reporting is cross-validation methods addresses how adding value that empowers frontline workers with that double-check data for red flags and ensure system-wide qual- feedback and information, coupled with incentives for improving ity. For instance, attendance at trainings that does not correlate with performance, can increase adoption of mobile-based interventions, adoption rates might be an indication of false reporting. Peculiar especially at scale. data should automatically be flagged and brought to the attention of project coordinators. 5.3.3  Efficient Cheating On the other hand, mobile applications can also provide additional In a different setting, where manual systems had previously required controls not available in traditional processes, such as automatic signatures or thumbprints of participants and where locations are location recording, time stamps, and timing of form filling pro- not mapped to field sites, suboptimal usage of the capabilities of cess even to the level of time taken for an individual field to be mobile applications could make cheating more efficient and could completed. Later analysis of such data (for instance, assessing the lower the ethical barrier for reporting fictional entries. Programs distribution of time taken per field over larger number of forms) are especially vulnerable to poor data of this kind if incentives for may provide additional information to help identify occurrences of adoption are misaligned. Requiring geotagged or photographic evi- potential cheating. Tying incentive mechanisms to correct submis- dence that is checked randomly could help with these problems to sions of data can enhance the quality of the data collected. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 26 C hapter 5 — S ervice D esign It is important to ensure that when ICT is considered in place of a approach as well. If cheating was of concern, tools that provide relevant traditional pen-and-paper system, the control mechanisms need to be ­ detection mechanisms should be used. Similarly, the acceptance of put in place equal to or better than those of traditional M&E systems. If data may require a gatekeeper role with authentication (signatures), signatures were required according to traditional processes, geotagged and so on. All such aspects would need to be designed into the model data or digital signatures would need to be required in an ICT-based properly when considering use of ICT in place of traditional M&E tools. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 27 Chapter 6: CASE STUDIES The following case studies capture the experiences of attributes deforestation to the energy, tourism, agriculture, and tea ­organizations—from large to small—that have developed and used industries. The country’s forest problems were brought to light in technology innovatively for data collection and M&E. They have 2004 when Wangari Maathai, founder of the Green Belt Movement, been selected to highlight a variety of off-the-shelf, open source, won the Nobel Peace Prize largely in recognition of her work sup- and customized software solutions, as well as innovative service porting peace and stability through tree planting. design that highlights the best practices in implementing ICT in TIST focuses on teaching farmers tree planting techniques that, by agriculture and forestry. The case studies also highlight the diversity design, propagate from person to person, and it uses well-worn in scale and the challenges faced by organizations in achieving sus- data collection technology to measure and monitor the progress tainability beyond the project period. of the incremental growth, root mass, and health of the trees these farmers plant. 6.1 SUCCESSFUL RELIANCE ON TRIED AND TRUE TECHNOLOGY—THE INTERNATIONAL SMALL Project GROUP AND TREE PLANTING PROGRAM (TIST) In addition to training on irrigation and beekeeping, TIST trainings IN KENYA30 cover planting and caring for indigenous tree species as well as fruit Properties of Featured Technology: Palm, PDA, external GPS, and nut trees that could help diversify residents’ income. Henneke custom installation, frontline workers, proprietary, in-House, online, said that the TIST technical approach is entirely based on organiz- two-way sync (see also annex 4). ing local groups whose communal interactions help ideas travel Context by word of mouth. The demonstrably useful techniques, shared by one neighbor to the next, have successfully led to the planting of a For more than a decade, Ben Henneke has been helping people diverse range of economically and environmentally important trees. in Kenya plant trees to support both the environment and farm- As an example, in the Meru District TIST has predominantly sup- ers’ incomes. His organization—The International Small Group and ported the planting of eucalyptus, grevillia, cypress, mango, acacia, Tree Planting Program (TIST)—developed technology specifically avocado, mukwego, cordia africana, macadamia, and orange trees. designed to verify the progress made in planting and tree growth According to TIST, so far approximately 4.5 million of these trees in TIST project areas. Henneke knows that tree planting can provide have been planted and 1.7 million seedlings have been raised. economic as well as environmental benefits only when measured correctly and that his organization’s work can make a considerable The community reforestation activities provide residents the benefits difference to the livelihoods of small farmers. of generating income from the carbon credits earned from the trees they have planted. Approximately 100 TIST field monitors (called As in many countries, Kenya has experienced rapid deforestation “quantifiers”) follow the UN Framework Convention on Climate over the past several decades. The Kenya Forestry Working Group Change standards for Monitoring Afforestation/Reforestation by 30 http://www.tist.org. Small Groups or Individuals to measure the carbon sequestered A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 28 C hapter 6 — C ase S tudies by the farmers’ trees. Carbon values are verified according to the In Kenya alone TIST is tracking 39,000 individual tree groves. The standards of both the Verified Carbon Standard (VCS) and the Clean organization states “present models show that the trees planted … Development Mechanism (CDM), allowing the Kenyan farmers to should achieve between 500,000 tons and 3,000,000 tons of CO2 receive annual payments for the certified emission reduction credits sequestration.” they have earned. TIST, which operates similar programs in Uganda, In order to meet its data collection demands, TIST developed its Tanzania, and India, expects its programs cumulatively to sequester own phone-based forms in C++ as well as software for receiving 5 million tons of carbon by 2020. and sorting data. Information sent in from the Palm devices is auto- matically sorted into searchable databases. Henneke said that over Challenges the years TIST has mined these data very successfully to identify TIST faces the challenge of making its work relevant to a large trends and needs for further interventions. number of small farmers who may have only planted a limited ­ ­number of trees. Holdings of “0.04 hectares are the reason we set out Perhaps surprisingly, power rather than software has presented to do this in the first place,” Henneke explained. Still, their ­ numbers a more significant challenge because quantifiers are often in the are massive. There are approximately 45,000 individual plots that are field away from mains electricity for days at a time. More than once owned or managed by TIST members in Meru and Nanyuki near Mt. TIST has demonstrated an ability to “hack” power systems—using Kenya, the two areas where TIST operates. the old, positive meaning of the word. Some computer equip- ment that TIST formerly used would lose all its data when it lost The extensive field monitoring and amount of data involved, power. To keep the Palm devices working longer in the field, TIST together with the stakes involved for subsistence farmers, have ­ quantifiers use “pigtails,” which are four rechargeable AA batteries raised the incentive for TIST to efficiently and accurately collect data. wired into one circuit, then taped together and plugged into the The supplemental earnings from accurately tallied carbon credits devices. provide the farmers with a cushion against the vagaries of weather or price fluctuation. Solution In order to map the location of specific trees and specific tree spe- The beauty of TIST’s customized, even if dated system, is that it has cies, and to capture tree volumes for carbon credits, TIST has long continued to work reliably well beyond the expected expiration used stand-alone GPS devices in combination with older Palm date. The mobile data collection devices have kept functioning Centro 680 and 650 and the Zire 71 mobile devices for data collec- for several years, and the custom software has required minimal tion. The Palms are no longer for sale via the usual channels (TIST updates. Henneke said they have studied other, open-source data finds them on secondary markets at a discount), but they continue collection platforms, particularly those that operate on the Android to function well and with low operating costs. “We can outfit some- operating system. Yet they found the alternatives lacking in the spe- one for 150 bucks,” Henneke explained. cific functionalities TIST needs in the field. The data on trees and tree groves are collected in person using Although they are not using enterprise-level data collection tools, the Palm devices, with trained enumerators filling out set forms they present a counter-narrative to free and open-source pro- with ­ details of tree type, health, and measurements. Diameter is tagonists: If we build in only the functionalities we need, to our ­measured at breast height (1.3 meters/4.27 feet). The quantifiers use specifications, we save time and money over the long run. Typically, formulae to calculate the biomass of large stands of trees. Three- customized, proprietary software is associated with high cost and quarters of the enumerators are men, most who come from the unsustainability. Open-source tools, on the other hand, are extolled provinces where TIST works. They are trained repeatedly, and their for the ability of a community of volunteer users to refine and field calculations are audited. advance a tool that anyone can use. ­ I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 29 Yet even paying for a staff person who would be a part of a p ­ articular 6.2 FAO’S OPEN FORIS INITIATIVE— open-source community for the sake of TIST exceeded the long- INTEROPERABLE, MODULAR MONITORING, AND EVALUATION TOOLS FOR FORESTRY31 term sustainability plans of TIST. Rather than demonstrating the most modern technological gadgets work in the field of forestry, Properties of Featured Technology: Android, smartphone, GPS, TIST demonstrates how forest projects can be successful when they external GPS, frontline workers, open source, online, offline, two-way are “scoped” according to local financial capability realities rather sync (see also annex 4). than built upon large but temporary international organization Context budgets. TIST eschews what Henneke considers to be typical devel- Inspired by the idea of harnessing ICTs to dramatically enhance opment project expenditures, emphasizing project spending levels forest management globally, in 2013 the Food and Agriculture that will be sustainable after his project funds end. Organization (FAO) is concluding the first phase of its Open Foris Eventually the organization developed its own site that was opti- Initiative, an effort to empower national and subnational forest mized for mobile phone screens, allowing quantifiers to query the agencies to collect, process, analyze, and share forest resource and database from the field. As of summer 2012, TIST’s custom platform related socioeconomic and governance data. began providing information on tree grove names and location According to FAO, 80 percent of countries face difficulties acquiring in relation to a quantifier’s physical location. This information has data and reporting on the state of their own forests due to a lack of proved especially interesting to the farmers with whom the quanti- organizational capacity, particularly with ICT tools. These countries fiers interact. are particularly vulnerable to illegal logging, environmental degra- TIST uses the data primarily for two purposes, both of which are dation, and threats to wildlife. To address this issue, FAO has com- related to carbon credits. First, the organization wants to use the mitted to supporting 30 countries, and Open Foris tools are already aggregate numbers to demonstrate and promote the cumulative being used in Ecuador, Indonesia, Paraguay, Peru, Tanzania, Vietnam, effect of its tree planting work. Second, TIST wants to help individual and Zambia. Soon they will start working with forest management farmers receive remuneration from global carbon credit schemes, bodies in Bhutan, Mongolia, and Papua New Guinea. no matter how few trees they have planted. For both aims, TIST Vietnam’s long history of comparatively sophisticated nationwide ­ adheres to VCS and CDM. forest management and technically capable personnel has made Yet as their devices wear out, TIST faces an inevitable challenge it particularly suitable for Open Foris. Vietnam has maintained a of updating the hardware and software of its operations. Ben nationwide forest management system since 1990, when it first Henneke said that TIST is now changing its data collection forms collected satellite imagery. A National Forest Inventory is mandated often enough that they have initiated a migration to HTML5. This by law every five years, and each cycle has included a complete will eventually mean updating devices, re-training quantifiers, and re-taking of high-resolution satellite imagery of the country’s for- heavily modifying their database platform. est cover—currently 323,000 square kilometers, or 39 percent of the country’s total size. This forest cover is divided into three clas- Henneke downplays his organization’s role, emphasizing that sifications: protection, which allows for limited harvesting that does “the trees do all the work.” And rather than looking to buy the lat- not impair any major forest asset such as a river; production, which est devices, he continues to find that developing the capacity of includes state forest enterprises, the country’s 44 land concessions, farmers—“human technology,” as he calls it—is the best possible and private plantations; and special use, which includes national investment TIST could make. parks, nature reserves, and species and habitat conservation areas. Source: Ben Henneke, Director, The International Small Group and Tree Planting Program, interview on 8 February 2013. 31 http://www.openforis.org/OFwiki/index.php/Main_Page. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 30 C hapter 6 — C ase S tudies According to Dr. Pham Manh Cuong, Deputy Director of the ƒƒ Open Foris Geospatial Toolkit allows GIS/RS experts to Department of Science, Technology and International Cooperation prepare cloud-free field maps, and land use maps and to at VN Forest, a huge amount of data are generated by the more than perform stratification and other tasks. 11,000 forest rangers in Vietnam, almost all of whom have mobile FAO is also working on a mobile data collection platform for the phones and near 100 percent nationwide signal coverage, from for- Android operating system in partnership with the Arbonaut com- est protection stations outfitted with e-mail, fax, and at least one pany. The platform would enable the collection of tree and forest GPS device, and from the e-mail capability of more than 600 district data in the field that could be easily transferred into other Open offices and nine Vietnam Forest Industry and Planning Institute Foris programs for analysis and mapping. (FAO’s partner in Vietnam) sub-centers throughout the country. FAO has been developing a mobile version of Open Foris Collect Information from all of these locations is sent to Hanoi via e-mail or for portable field data recorders. Miceli said FAO considered exist- fax on a regular basis. All of this makes for a large amount of data ing mobile data collection platforms but found they were not that must be sorted and made useful for national-level forest man- flexible enough for the much more complex and varied purposes agers as well as policy makers. tional-level forest inventory. They wanted users to have more of na­ Dr. Pham believes the system’s large amount of information flow control of the forms and data structures. “We’re making it so easy for ­ ­ requires carefully calibrated allocation of resources for data man- the user to set up Collect that they don’t even realize that they’re agement and analysis activities. “We need to define what level of designing a relational database,” he explained. information is necessary,” he said, explaining that the type of and The new Collect user interface will take a forms approach, but amount of data shared with VN Forest in Hanoi must be “cost-effec- the back end will have all of the metadata, invisible to the person tive” and must involve quality control of field reporting. ­ inputting the data. The metadata allows for greater integration across Open Foris apps and with other systems. Project “It is our mission to enable countries to work with their own data to FAO envisions three classes of Open Foris Calc users: people who produce information essential for decision making and reporting,” ­ analyze and query data—such as a ministry official or whoever said FAO’s Gino Miceli, one of the founders of the Initiative, explain- needs actual result data; forest scientists who can create new calcu- ing that his team simply wants to improve the management of lation models (for example, a new height model by which foresters ­national forest agency data so that the countries can get knowledge could calculate a tree’s height from a position on the ground); and and information from the data. Open Foris tools, Miceli explained, people engaged in setting up the system and executing country- aim to support all phases of the inventory process, from inventory specific computations. design and data collection to calculation and production of reports Solution and maps. Training and support is provided by FAO and partners to Open Foris tools are built precisely to provide efficiency and ensure that national staff are able to perform these essential tasks accuracy through flexibility and comparability, which are attri- ­ with little or no outside support. butes the FAO hopes VN Forest will make full use of in Vietnam. The current Open Foris software components include the following, Open Foris Collect can be configured by national staff to per- which FAO has deliberately kept as open source in order to allow for form flexible data checking. This is accomplished by configur- modifications and improvements by ICT and forest experts worldwide: ing the platform to compare different sets or types of data. It ƒƒ Open Foris Collect allows you to collect site-specific data on also provides a well-defined workflow, to help users manage paper or with portable data recorders. and perform data entry and data cleansing. And after data are ƒƒ Open Foris Calc can import the data and calculate volume, entered and cleaned, they are locked and may be exported to biomass, and carbon levels and can be used for mapping. various formats (csv and xml). I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 31 Open Foris tools aim to be modular and flexible, allowing each tool may become a target for illegal logging. Similarly, if inventory data to be configured and customized to country needs. This has been reveal the location of high-value trees, it could make them targets necessary because their systems and data needs differ greatly. for illicit harvesting. Despite the customization or data format, however, all deployments Acknowledging the sensitivity of locational data, FAO has designed seek to exploit the clear potential of visualized information for stron- Open Foris with security in mind. Not only does it include certain ger monitoring and evaluation work. Forest data that are geo-coded permission levels for various management responsibilities, Open can be included as one of multiple layers of data in any GIS-like Foris apps have the capability of protecting specific forest plot loca- mapping program. Layering different types of data on a map can tion information while still releasing enough information to allow allow the identification of facts that would otherwise likely have for statistically significant research. Researchers can still analyze for- been missed if the data remained in spreadsheet form. The Open est data without creating opportunities for outsiders to exploit the Foris suite supports biophysical, governance, and socioeconomic data to the detriment of the forests. data, allowing for layering that provides important insights into the Sources: socioeconomic ramifications of national forest policies. Gino Miceli, Forestry Information Systems Specialist, FAO. Manh Cuong Pham, VN Forest, Vietnam. New tools such as those offered by Open Foris introduce new efficiencies for national forest management and analysis. Yet they ­ also introduce new opportunities to provide useful information to 6.3 THE WAY AHEAD—THE EFFECTIVE USE OF the public—information that could help the media cover the govern- SATELLITE IMAGERY AND OBJECT-BASED IMAGE ANALYSIS SOFTWARE IN LAOS32 ment’s management of a country’s forest, assist conservation efforts, Properties of Featured Technology: Custom installation, GIS, and provide a way for independent scientists to conduct research. eCognition, automated, license fee, service provider (see also On the other hand, while Open Foris tools give forest institutions the annex 4). means to improve data collection and analysis, their use requires a Context significant commitment of human and budgetary resources. Field data collection itself can be quite costly due to the amount of staff From multi-million-dollar Lidar data acquisition to GSM-enabled hours required. These challenges highlight a potential need for remote sensors to portable bar code scanners, there are many technical assistance for forest agencies—not with the technology new costly devices that promise more-efficient and accurate itself, but with the development of plans and processes for effec- ­ forest management processes. Many of these tools rely upon tively using that technology. the details and precision of high-resolution satellite imagery that itself requires a new set of options and choices for effective use. Major Takeaways This ever-increasing diversity of choices makes it more difficult for Any government planning to eventually open its forest data to the forest agencies to conduct cost-benefit analyses on potential tool ­ public saves time and resources by choosing platforms, data for- investments. mats, and systems that are best able to facilitate internal use and Smartphones and other mobile devices are having increas- transparent external access. Open Foris tools allow for a smooth diverse features previously found in only dedicated high-end ingly ­ transition once a country like Vietnam decides to adopt an open devices. The convergence of technologies has led to the gradual ­ data policy. Specifically, Open Foris Calc allows governments to replacement of low-end GPS devices with smartphones with ­ release subsets of data in a controlled way. appropriate apps in recreational and non-commercial navigation ­ Despite the benefits of open data, making the entirety of a country’s markets. In professional forest management and law enforcement, forest information public involves several different types of vulner- abilities. If it is revealed that a particular area has less monitoring, it 32 http://www.jica.go.jp/project/english/laos/006. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 32 C hapter 6 — C ase S tudies most of the technology applied is rugged professional-grade. their monitoring. They mainly rely on the network of informants that However, these are often made available by donor-funded projects. the inspection office maintains throughout the country. The goal Once usually budget-constrained agencies start procuring devices therefore for FIPD has been to detect the removal of individual trees ­esources, it is possible that consumer market-oriented with their r on satellite images covering hundreds of thousands of square kilo- smartphones and relevant apps will gain more attraction as the meters, even though they sometimes can be very difficult for the budget-friendly second-best option. ­ human eye or pixel-based image classification software to notice. In some situations an agency’s information needs are so clear and Solution the potential tools are so much the right fit that it becomes easier High-resolution color satellite imagery combined with object-based to set priorities for the resources that are available. One example image analysis software is very well suited for this type of challenge of this is the Forest Inventory and Planning Division (FIPD) in Laos, and has proved especially useful in Laos. The Japan International which has found high-resolution satellite imagery33 and object- Cooperation Agency–funded Forest Information Management based image analysis software very useful in its work monitoring Project, which started in 2009, supported the creation of a nation- illegal logging. wide high-quality imagery baseline for Laos through which 5m Laos is a country of 230,800 square kilometers that has long suf- resolution RapidEye multispectral imagery was taken of the entire fered from the scourge of trees unlawfully being cut and taken from country over three months. The quality imagery enabled remote its forests. According to the International Union for Conservation identification of selective logging that was confirmed in situ by the of Nature (IUCN), the forest cover has shrunk to 35 percent of the Lao PDR’s Department of Forest Inspection (DOFI) with the support country’s area from a figure of 71 percent in 1940. IUCN attributes of the Sustainable Forest and Rural Development Project funded by part of this to illegal logging. the World Bank and the government of Finland. Challenge Aruna Technology Co., Ltd., a company based in Lao PDR and Cambodia, integrated the baseline data with newly acquired Illegal logging in Laos, where the lack of road infrastructure and dif- imagery and was able to identify locations of selective logging, ficult terrain make it logistically difficult to monitor forests, mainly and it even demonstrated the ability to detect the individual tree centers on 10–15 high-value woods such as multiple varieties of removal. According to Aruna owner Jeffrey Himel, the ability to rosewood. These can fetch $1,500 per cubic meter before export detect ­ cutting of individual trees visually enables development of and are sold for up to $5,000 per cubic meter online. The illegally more-automated solutions that could provide real-time monitoring harvested wood is mostly cut into rough timber and exported to of illegal logging in Laos and benchmark maps for future REDD car- Vietnam, Thailand, or China for processing into furniture that is then bon credit projects. In the future, these benchmarks will be used as exported to markets in China, Russia, North America, and Europe. baselines against which new carbon credits can be allotted. Often the loggers aim to remove only the trees of certain species and sizes. If they cut down a tree and find its interior less than ideal, Software they often will not bother removing it from the forest. Discovering With the imagery successfully gathered, FIPD procured eCognition these trees is often beyond the capacity of current forest monitor- object-based image processing software and worked with Aruna ing approaches, which only involve irregular on-the-ground obser- to quickly prepare a very detailed “segmentation” of small polygons vations by forest inspection officers. They do not conduct regular from the imagery that can then be classified into a forest and land foot patrols, nor do they regularly use other technologies to assist in cover map to serve as the benchmark for the whole country. The eCognition software was developed specifically to make sense 33 In this case study, “high” resolution refers to imagery between 2 meters of detailed and complicated data, originally for medical imaging and 10 meters resolution, whereas “very high resolution” or VHR imagery is less than 2 meters. technology used for scanning the human body. It was initially I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 33 developed by a think tank founded by Gerd Binnig, who shared on quality control. The institutions require a systemic vision for the 1986 Nobel Prize for physics for his work on a specialized data management and use; remote sensing analysis should not be microscope. thought of as simply a purchasable product. While in most cases the richer and more diverse the imagery and derived data are, the According to Himel, image analysis and classification previously more potentially insightful the information product becomes, here relied on pixel-based techniques. “This involved taking the digital achieving a quality result remains difficult and challenging. Not number values of individual pixels from the imagery and using a surprisingly, powerful software like eCognition is not simple to use. range of mathematical algorithms and techniques to characterize Currently it requires customizing image import and export rules for each pixel based on its similarity to others,” he explained, mention- each project and adjusting settings to accommodate for different ing several challenges for imagery accuracy, including atmospheric pixel sizes in the various images. conditions and complexity of the landscape. “Object-based image classification takes a completely different approach. Rather than The challenge for forest agencies therefore is to have informed focus on pixels, the method groups them into regional objects expectations of what insights this type of remote sensing could based on aspects such as color, texture, tone, shape, size or context. provide with available resources—a challenge with a chicken- By ‘segmenting’ the image into these larger objects, a more accu- and-egg dynamic: In order for them to know ahead of time rate analysis is enabled, then the objects can be more accurately whether the remote sensing is worth its cost, they essentially classified.” need to know what the analysis will reveal before they commit to pay for it. Yet if they knew the results, they wouldn’t need to pay As noted by Trimble, a provider of location based solutions, the for the service. technology is particularly useful when small or otherwise unob- servable changes in the forest would otherwise remain unnoticed. Remaining Challenges According to Trimble materials, eCognition will process and layer The frequency with which a forest agency processes satellite imag- a variety of forms of geographically accurate data, including raster ery through object-based image analysis software implies different images, LiDAR cloud points, GIS vectors, radar data, and hyper- ­ policy approaches. The more frequent it processes the information, spectral data. The software can also extract vertical and horizontal the more it places an evident priority on the information and the features; integrate images of different scale, resolution, and spectral more reactive it can be. Less frequent information processing may bands; and enable correcting for atmospheric-induced mistakes. come as a result of limited budgetary resources or a low prioritiza- tion, but it nevertheless limits monitoring ability and the potential Most critically, eCognition can integrate such information as acces- for the information to implement any actions that could help curb sibility or proximity to human dwellings into the segmentation and illegal activities. While the discovery of some objects intended classification. “Rather than examining individual pixels in isolation,” for long-term use, such as illegal logging roads, allows for longer the company says, “it distills meaning from the objects’ connotations reaction periods, infrequent image collection will only provide the ­ and mutual relations.” On a technical level, this distillation involves un-actionable facts of what has already happened. ­ the software employing fuzzy logic to determine the probabilities of values that are not binary. On a simple level, the advanced software Himel of Aruna Technology believes most solutions will rely on a helps answer a seemingly straightforward yet guilefully complex combination of lower-resolution imagery such as RapidEye in question: “What is happening, where?” combination with field survey and higher resolution imagery. He pointed out that as resolution and frequency of resolution increase, Policy Choices acquisition costs can increase exponentially. The costs and logistics While powerful, the software requires that forest agencies make of the associated activities of confirming tree felling in the field, many strategic decisions and streamline their data processing and data processing, and the capacity for interdiction must also be information flows accordingly while maintaining a strong focus considered. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 34 C hapter 6 — C ase S tudies Major Takeaways information from these images needs to be translated into useful Costly image analysis software does not make sense for all countries geo-referenced information for enforcement. In this case, the illegal or situations. Other countries may have starkly different illegal log- logging sites were identified from the images and the coordinates ging issues than Laos, with similarly different varying potential for recorded. After that, DOFI officials verified the sites with GPS- technology. In those cases, instead of organized and precise target- directed field visits. ing of valuable individual trees chosen for their export value, illegal With the increased capabilities and availability of highly capable loggers usually will cut down entire stands of trees to provide local cellphones, these questions have to be answered in an increasing households with firewood. These clear-cuts can be seen with lower number of cases and by an increasing number of users both inside resolution imagery and are often discovered by the naked eye by and outside the forest profession. The Laos case demonstrates one forest rangers on the ground. way of working with professional-grade devices. It may also show Any forest agency considering the use of satellite imagery in com- the way for later deployment of multiuse retail devices. bination with object-based image analysis software has a tough Sources: set of choices to face. The skill sets involved are rare enough that Interview with Jeffrey Himel, Aruna Technologies Owner, Vientiane, Lao they risk overburdening or losing a well-trained staff member to a PDR. higher-paying outside employer. The software and satellite imagery are not cheap. These factors often lead to the decision to outsource 6.4 COMMUNITY-MANAGED SUSTAINABLE the service, even if such consultancy services are not cheap. At least AGRICULTURE—A BOTTOM-UP REVOLUTION ASSISTED BY MOBILE TECHNOLOGY34 then the cost is a “one-off” and does not present the challenge of future recurring costs. Properties of Featured Technology: Java, basic phone, SaaS/ Cloud, GIS, crowd-sourcing, service provider (see also annex 4). In Lao PDR, a middle ground has been available: local technological Context expertise has provided the support and expertise to enable the FIPD to develop its capacity step-by-step rather than depend on outside With more than 1 million farmers in Andhra Pradesh, India, the consultants. In addition, forest agencies need to continually think Community-Managed Sustainable Agriculture (CMSA) program through the life cycle of data collection and utilization. In essence empowers members of rural communities with information and they need to begin with the desired effect on forest governance training to grow natural fertilizers and use non-pesticide farm- and then work backward to initial data collection. For example, if an ing methods to dramatically improve yields and increase farmers’ agency plans to collect data often, the establishment of the i ­magery incomes. CMSA deploys initiatives such as farmer field schools ­ baseline will enable lower-cost monitoring over time and will im- (FFS) to provide hands-on demonstrations, training, and extension prove the accuracy of results. services in over 11,000 villages. The program works with over 2,000 village trainers and community resource persons to provide training Laos’ experience with the software and imagery shows both the and coordination. potential of the tools and the complex planning they require. It ­ could be said that technology presents a double-edged sword Due to the immense scale of operations, the CMSA project faced because it has given the country’s forest inspectors the gift of sight implementation challenges in coordinating and monitoring a large and yet has presented them with newly challenging policy ques- statewide program through isolated rural community profession- tions. Hopefully for them, the way forward for this technology will als. Low-cost java-based mobile phones provided the opportunity likely involve decreasing costs, increasing user-friendliness and to connect various field trainers and coordinators under a uniform more-widespread and effective application. training and data capture program, provide them a structure to carry out their work, monitor their operations, and collect data Earth observation images and their interpretation provide only a starting point for investigations. As in the case of Laos, the 34 http://65.19.149.140/pilots/cmsanew/ab_us/aboutus_modify.html. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 35 regarding the impact and effectiveness of interventions as well as In the back end, the monitoring and evaluation platform helps the major roadblocks to greater adoption of sustainable agricultural collate and analyze collected data to monitor operations real-time methods. and evaluate effectiveness and impact of interventions over time. Information is also relayed in real-time on a public website to main- Since the adoption of the CMSA mobile application in September tain transparency and accountability of field operations both within 2011, some 1,300 community professionals have been trained, ben- and outside the implementing agency.35 efiting large numbers of smallholder farmers directly. Data collected have been analyzed to capture effectiveness of alternative methods The CMSA mobile application is designed by Bluefrog, a technology such as the System of Rice Intensification (SRI) and non-pesticide service provider based in Andhra Pradesh. The application is built management. in J2ME to accommodate basic feature phones. Data received are hosted in the service provider’s server, through software as a service For nearly a decade, the Society for Elimination of Rural Poverty model. (SERP)—a semi-autonomous government agency in Andhra Pradesh—has been actively promoting mobilization of the rural The various menus of the application are meant to capture different poor into self-help groups and connecting them to markets and initiatives in the CMSA program for which each trainer is respon- last-mile services. The agency currently works in 38,550 villages and sible. In general, trainers are responsible for running programs in reaches 11 million households. Federated at the group level, village five different villages five days a week, starting farmer field schools, level, and block and district levels, one of the main goals of the inspecting non-negotiable flagship components, inspecting organization is to build leadership capacity and facilitate demand- botanical fertilizer shops, and attending community group meet- ­ driven efforts to improve livelihoods through effective agriculture, ings to run extension video service—Digital Green—to promote livestock, dairy production, non-farm skills, financial literacy, and further awareness of methods and adoption. health and nutritional awareness, among others. The following is a description of some of the components covered SERP initiated CMSA in 2004 in order to address the major causes under the mobile application. of agriculture distress: extensive use of chemical inputs, high costs Field-based Extension Program—Farmer Field Schools: Farmer field of agriculture, displacement of local knowledge, unsustainable schools are field-based activities for groups of farmers to engage agricultural practices, and lack of market access. CMSA supports ­ with best practices. The FFS menu in the mobile application enables poor farmers in adopting sustainable agriculture practices, to the user to capture the attendance of the farmers, the image of the reduce the cost of cultivation and increase net incomes by moving ­ group, and the image of the land where the meeting has been held from input-centric model to a knowledge- and skill-based model. for different parts of the discussion. This enables the project to cap- The program is therefore focused on knowledge transfer, capacity ture the adoption of CMSA practices in various geographic areas. building, and empowering farmers and community resource pro- Ultra-poor36 Plot Program—36*36: Intended to address the nutri- fessionals in order to promote cost-saving, yield-boosting, sustain- tional needs of the poorest of the poor households, the 36ft*36ft able agricultural practices. model proposes a diverse range of crops—from fruits and tubers to Mobile Application vegetables and pulses—to promote nutritional security and year- The goal of the CMSA mobile application is twofold. On the round income. The mobile application allows field coordinators to front end, it seeks to help community resource professionals capture data from farmers experimenting with this seven-tier crop- organize their time and operations each day by providing a ping model and to track the income generated from each crop and structure for carrying out operations and an incentive to cap- the expenses incurred. ture data regarding work completed, along with a method to follow progress. 35 http://65.19.149.140/pilots/cs/dashboard.aspx. 36 Ultra-poor is a term used to describe the poorest of the poor in India. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 36 C hapter 6 — C ase S tudies System of Rice Intensification Methods: This revolutionary method Promoting Sustainability: In addition to training thousands of of rice cultivation is intended to be cost-effective and resource- community professionals in collecting information, and using it efficient for the most popular crops produced in Andhra Pradesh— in turn to structure their work, the project also coordinates with paddy. Under this method, a minimum amount of water is used five elected members for district subcommittees to take over instead of continuous flooding, and seedlings are replanted with coordination 10 days a month—6 days in the field, 2 days at ­ more room for root growth, producing a greater crop yield. The district-level meetings on staff activities, and 2 days in the main mobile application enables manual entry of the enumerator to cap- office at the state level. These activities allow members within ture the amount spent on agricultural inputs for this method, the community institutions (federated self-help groups) to begin shar- yield, and income from the crop. This is meant to ensure that there ing in operational duties, and eventually taking ownership of the is adherence to SRI methods and that the methods are effective in project in the future. producing desired effects on income. Cross-validation potential in The tool has effectively helped in the following: this system is quite limited. Adherence to “Non-negotiables”: In order to capture the level of Overseeing Operations. CMSA is a knowledge-intensive, farmer- ­adherence to sustainable practices, community enumerators record centric program where individuals must relearn new methods to data on the methods followed by the farmer. These include com- protect their crops from bad pests and learn to encourage ben- munity bonfires, seed treatment, bird perches, border crops, trap eficial worms and other insects and natural process to r ­ ejuvenate crops, yellow and white plates, intercrops, light traps, pheromone the soil. Due to its knowledge-intensive nature, much training is traps, delta traps in ground nut, alleys in paddy, cutting of the tips required before farmers can successfully transfer to non-pesticide in paddy crops at the time of transplantation, and application of farming systems. Therefore the mobile application has a strong botanical extracts as needed. This is meant to enable the project to focus on structuring ongoing training activities with farmers understand adherence to methods by individual farmers in relation to grow natural fertilizers and adopt non-pesticide farming to their yields, as well as horizontal adoption to various methods practices. by region. While the system provides an overall picture of adoption, individual adherence measures are not cross-validated through the Quantifying Impact. By capturing attendance and participation in application. individual programs by farmers, the CMSA program is building the ability to understand changes in yields, artificial input use, adher- Service Features ence to different programs, and the impact on overall income and Incentives for Excellence: The service features of the CMSA mobile well-being. This type of rich data on a large scale allows impact application strive to obtain quality and complete information by evaluation at a randomized scale to test the effectiveness of various incentivizing frontline staff. The honorarium received by these com- interventions. munity workers to compensate for wage loss is Rs 4,000 (~$80) per month. For those who receive 91–99 percent performance grades Evaluating Effectiveness of Intervention. Sustainable methods that in data submission, a Rs 1,500 bonus is offered. Those who submit avoid chemical fertilizers and pesticides involve a paradigm shift complete data receive a substantial Rs 3,000 bonus on top of their from input-centric agriculture to a knowledge-based model. This honorarium. The project also offers bi-monthly refresher training, entails considerable risk for the majority of farmers, whose crop may where staff can improve their data entry skills, and basic trouble- be their only source of income each growing season. Observation of shooting. Rewards for top performance, combined with continuous crop-cutting experiments, captured through mobile phones, allows technology training, rather than a daily incentive for submission, the systematic collection of data on the impact of these sustainable incentivizes frontline trainers to improve their performance. So far, methods on farmer income (lowered costs, increased outputs), pro- approximately 200 of 1,300 staff receive the monthly bonus, with viding proof of concept for farmers and promoting broader adop- about 10–20 receiving the 100 percent grade. tion of these methods. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 6 — C ase S tudies 37 6.5 THWARTING DROUGHT—MOBILE-BASED DATA severely delayed information transmission and thereby rendered COLLECTION FOR DROUGHT PREPAREDNESS IN the early warning system ineffective. The new early warning algo- UGANDA37 rithm was used to generate a drought bulletin used throughout the Properties of Featured Technology: Java, basic phone, smart- district for drought preparation and relief efforts in the Karamoja phone, hosted, frontline workers, open source (see also annex 4). region of Uganda. Context When the Drought Early Warning System (DEWS) was first intro- As part of a regional initiative to reduce the risk of drought in East duced, parish chiefs conducted the survey by hand, noting the Africa with Early Warning Systems, community chiefs in 55 village information on paper, which was then delivered from the sentinel centers are collecting specific and tangible data in resource avail- to the subcounty chief, and then to the DEWS Focal Person. It was ability and behavior to identify indicative patterns among the rural entered manually into the system and then analyzed and dissemi- pastoral communities of the Karamoja region. nated in the form of drought bulletins, delaying the process by five to seven business days. A year and half into the project, the impact Located in the arid northeast of Uganda, Karamoja has the lowest of the delay of paper-based data collection was evident—forecasts human development indicators of any region in the country.38 The and predictions were less relevant than before. With the emergence region suffers from chronic poverty, malnutrition, and food short- of greater network connectivity and affordable mobile devices, ages, as well as frequent droughts, due to generally poorly distrib- ACTED saw an opportunity to bring mobile-based data collection uted, unreliable, and low rainfall amounts. Unlike other regions in to the DEWS project. Uganda, which have a bimodal rainfall pattern, Karamoja’s pattern of rainfall allows for only one planting season, and the unpredict- The mobile-based data collection project to inform monthly ability of this pattern further exacerbates agricultural livelihoods. drought bulletins is the result of a partnership between three Given heavy reliance on cattle, sheep, goats, and poultry in the groups: local government partners who collect the information pastoral and agropastoral communities as food, investment, and monthly through their area chiefs at 55 parishes (village clusters) safety net, tracking vulnerability to drought requires indicators and publish the monthly bulletin; ACTED, the international NGO such as water availability, agriculture, and livestock conditions so that was able to bring together best practices and the stakeholders that communities may efficiently make the best of the land’s low to develop the Early Warning System for Karamoja; and FAO, which primary productivity. was able to design and work with the technical team to develop the Project mobile application using Nokia Data Gathering and which provided In an attempt to tackle the challenges of delays in data collection operational and trouble-shooting capacity training to the project. for preparedness and relief in vulnerable drought-prone regions, The Drought Early Warning System used in Karamoja relies on FAO and the Agency for Technical Cooperation and Development monthly weather forecasts from the Department of Meteorology of (ACTED), in partnership with a local district government in Uganda the Ministry of Water and Environment. The vulnerability indicators and inspired by the Kenyan Drought EWS, created a mobile applica- are collected from households, kraals, and markets by the village tion that enabled early signals to be collected and collated instantly chiefs. online and fed into an early warning algorithm. The prior delay in manually collecting, aggregating, digitizing, and analyzing data had In order to obtain the information in a timely manner and to sup- port communities and organizations in preparing for drought, 37 http://www.acted.org/en/uganda. three main factors played a key role: designing the optimal data 38 The administrative area in Karamoja has seven districts, which are fur- ther divided into subdistricts and then into parishes. Parish chiefs are collection parameters, a symbiotic partnership that enabled com- selected by the local government and are responsible for a number of munity ownership, and accessible mobile technology with network duties. Parishes have a market, where crops are sold, and kraals, where livestock are traded. connectivity. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 38 C hapter 6 — C ase S tudies Designing the survey was a collective effort between stakeholders getting information and collecting data, and provides access to a to arrive at a comprehensive yet efficient set of questions that can network of similar community leaders. be administered through a basic feature phone: Prior to technology training, the FAO trainers conducted sensitiza- ƒƒ Household survey (October 2012): Collects data from the tion to teach the interview process first. When enumerators are same 10 households in each survey location every month, aware of how the data will be used, and why it is important to have including type of water source and time spent to fetch accurate numbers, they are able to better establish a relationship water from each, quantity of water fetched, security, type with an interviewee, especially when disclosing potentially sensitive and source of food, crop conditions and type of crops, and household information. Because enumerators are usually already alternative livelihood indicators such as price of casual labor. familiar with texting on mobile keypads, only a half-day of initial ƒƒ Kraal survey: Tracks the same five herds of cows of about technology training was required prior to implementation. 20 animals each, monitoring livestock body condition and access to grazing areas. Main Takeaways ƒƒ Market survey: Administered monthly, tracking type and Empowerment of communities and local implementers: Periodic number of animals available in the market and market prices user training and capacity building generates greater awareness of for the main sources of grain, meat, and energy. digital tools and ownership of the process. Once collected and uploaded in a location with adequate con- Symbiotic partnerships can combine resources with capabilities: nectivity, the data are processed on a Nokia server and exported FAO brought in technical knowledge and training, and ACTED in CSV format and imported to the DEWS database through a con- provided implementation capacity in coordination with local com- version applet. The DEWS is a web-based centralized application, munities and government. on a server hosted by the Ministry of Agriculture, Animal Industries and Fisheries of Uganda. As the FAO involvement in the project Increased timeliness of early warning: Real-time data collection and ends in 2013, an integrated solution between Nokia data gather- drought bulletin production increase the timeliness of drought ing technology and DEWS is currently being planned. The project warning and the preparation response put in place by communities is also developing new web-based products to further information and partners. dissemination. Decrease in costs leads to greater likelihood of sustainability: As The parish chiefs are selected to be enumerators based on motiva- transport costs for carrying questionnaires from the field to sub- tion, accessibility, level of literacy, and availability of kraal and market country and district offices is eliminated, data collection becomes in their parish. Employed by the local government, they are nomi- more efficient and more viable for government adoption. nated by subcounty chiefs to be DEWS data collectors, enabling the Sources: government to run data collection sustainably beyond the funded Phillip Fong, FAO, interviews on 13 February 2013 and 20 March 2013. project period. Malika OGWANG, ACTED, interview and survey documents shared on 20 February 2013. Despite the fact that no additional compensation was offered, enu- K. Gelsdorf, D. Maxwell, and D. Mazurana, “Livelihoods, Basic Services and merators who were nominated were eager to participate because Social Protection in Northern Uganda and Karamoja,” Working Paper 4 (London: DFID, 2012). of the additional training, phone use, and the connections that the E. Stites, L. Fries, and D. Akabwai, “Foraging and Fighting: Community project offered. Working in isolated project areas, the periodic train- Perspectives on Natural Resources and Conflict in Southern Karamoja” ing brings together various parish chiefs to discuss challenges in (Medford, MA: Feinstein International Center, 2010). I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N C hapter 7 — C onclusion 39 Chapter 7: CONCLUSION There has been extensive expansion in the number of ICT4D incentivizing Hiring analysts who can perform quality data analysis, ­ products and services available in recent years, along with ­ adoption, and managing risks appear to be some of the most ­ substantial digital experimentation at all stages of the monitor- important tasks when deploying ICT. ­ ing and evaluation process in agriculture and forest sectors. As Finally, with increases in the effective use of information and experiments have matured into scaled projects, evidence these ­ unication technology, macro-level phenomena are being comm­ of their efficiency is being observed. Accrued knowledge on observed. (See figure 7.1.) As more and more projects tap into using ICT ­ specifically in data collection and data management ICT for data collection and M&E, large agencies are choosing to is increasing the capacity of organizations and governments to develop organization-wide strategies that lay out ICT policies, best technology into their projects and programs more incorporate ­ practices, and interoperable access. This has led to increased con- effectively. vergence around the most effective technologies, mobile applica- Yet despite the growth in active use of ICT in agriculture and tions, and ecosystem strategies. The sector is reaching a plateau, forest projects, the technology still remains a latent consider- making it easier for non-technologists to grasp and use ICT in many ation in many rural development projects. Confusion on how to of their projects. mobile applications and other digital tools remains properly use ­ However, new frontiers will continue to emerge as convergence fairly widespread. Like boarding a moving train, the constantly inspires innovation and improved solutions to new challenges. To evolving and fast-paced ICT sector appears difficult to carefully, prepare for these new frontiers, projects in agriculture and forests slowly, and thoughtfully engage in. Yet this should not discour- age practitioners’ use of ICT. Strategies to effectively think about and incorporate these technologies are emerging. This opera- tional report is one ­ attempt to aid practitioners in that decision- FIGURE 7.1:  Macro-level Effects of ICT in Data making process. Collection and M&E In summary, technology itself is not an end but a means. The spe- cific project and people needs may or may not be a good fit for Innovation Scale the use of ICT. If this technology is deemed suitable to achieve data collection and M&E goals, implementation models that maximize efficiency—such as the use of frontline workers or automated cap- ture—can be considered and systems designed. Only then should Org-wide Convergence specific features of technology be deliberated. And as in any other strategies public goods project, service design must be carefully considered. A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 40 C hapter 7 — C onclusion should remain nimble and use technologies that can easily transfer as the widespread and continued use of ICT is what leads to that. data from one platform or provider to the next. Feedback loops Rather, practitioners should engage with ICT as often as it meets between operations, staff, managers, policy makers, and users ­ ­mproving project needs and contributes to goals associated with i should help to clarify when it is time to “update” ICT approaches. rural livelihoods and achieving climate-smart landscapes. Using this Most importantly, practitioners should not wait for further conver- guiding report and other operational tools should provide ample gence of agreement on technology before using ICT in their projects, assistance in reaching those goals. I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N A nnex 1 — L ist of Tools M entioned in the R eport 41 Annex 1: LIST OF TOOLS MENTIONED IN THE REPORT Cropster www.cropster.org Text to Change www.texttochange.org DataWinners www.datawinners.com Grameen www.grameenfoundation.org doForms www.doforms.com Sensemaker www.sensemaker-suite.com/smsite EpiCollect www.epicollect.net CognitiveEdge cognitive-edge.com Magpi www.magpi.com TotoAgriculture www.totoagriculture.org Esoko www.esoko.com PROFOR www.profor.info ESRI www.esri.com Connect Online Connect Offline (COCO) www.digitalgreen.org Freedom Fone www.freedomfone.org ArcGIS www.esri.com/software/arcgis FrontlineSMS www.frontlinesms.com Google Earth earth.google.com iFormBuilder www.iformbuilder.com Bing Maps www.bing.com/maps Kimetrica www.kimetrica.org STATA www.stata.com mKrishi www.tcs.com/offerings/technology-products/ SPSS www.ibm.com/software/analytics/spss/ mKRISHI/Pages/default.aspx StatTransfer www.stattransfer.com Mobenzi Researcher www.mobenzi.com/researcher Formhub formhub.org Nokia Data Gathering nokiadatagathering.net OpenForis www.fao.org/forestry/fma/openforis/en/ Open Data Kit opendatakit.org NetHope Solutions Center http://solutionscenter.nethope.org OpenXData www.openxdata.org Humanitarian Nomad Online humanitarian-nomad.org/online-selection-tool/ Poimapper www.poimapper.com Selection Tool TechnoBrain www.technobraingroup.com/products/techno- brain-project-monitoring-evaluation-system.aspx A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R A nnex 2 — O vervie w of Tool C apabilities and C onsiderations to A ddress in a Tool S election P rocess 43 Annex 2: OVERVIEW OF TOOL CAPABILITIES AND CONSIDERATIONS TO ADDRESS IN A TOOL SELECTION PROCESS In addition to general questions, a more detailed checklist is available, indicating possible variables and attributes that should be considered when selecting or preparing tender documentation for a proper toolset for projects use. Mobile features •  offline/online capability •  one-way or two-way sync (upload or also download of data) •  form/survey updates Data and user management features •  user roles •  user groups •  hierarchy of groups •  access right features • backups • restore GIS features • coordinates • routes • areas •  location hierarchies •  use of map service providers •  interfaces to GIS systems •  map-based visualization features Data reporting and visualization •  integral part of the solution •  relies on external software Exporting and importing interfaces • MDML, Excel, Word, CSV, Tab delimited, ODBC, Text custom, SOAP, XML, API provided, Custom Interface, ESRI Shapefile, Google Fusion, HTML , JPG, KML, GeoRSS, JSON, SQL, PDF, TXT, PNG, RTF, Open Office, Google Docs, MS Access, MDML, SPSS Security •  in-device data encryption •  database encryption •  data transmission encryption •  access control mechanisms Costs • training • support • development • license • maintenance • subscription Business model •  open source with consulting •  license fee •  user-based subscriptions •  transaction-based subscriptions Language support •  in device for forms (single or multilanguage switch) •  in device for application • website A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 44 A nnex 2 — O vervie w of Tool C apabilities and C onsiderations to A ddress in a Tool S election P rocess Data content and modeling capabilities • text, images, single/multichoice questions, conditional subquestions, static and dynamic number of subforms, static and dynamic tables, (form features) location hierarchies, GPS, image, audio, video, numeric calculations, validation lists, validation ranges, calculated ranges, logic expressions for conditional questions, dates, time, etc. Supported platforms and hardware •  supported server databases •  supported operating systems •  supported devices and device operating systems Delivery options •  custom installations • hosted • SaaS/cloud •  remote management capabilities Provided services •  provided training, support and customization services and their geographic coverage Performance •  complexity of forms •  number of forms stored in device •  number of forms stored on the server •  number of concurrent users •  response times •  performance monitoring solutions in use •  load balancing solution Reliability •  formalized test and release processes •  release notes •  fail-over/redundancy solution I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N A nnex 3 — F ull L ife - C ycle C osts E stimation T emplate 45 Annex 3: FULL LIFE-CYCLE COSTS ESTIMATION TEMPLATE For costs comparison, a more detailed structured template can be used for determining the projected full life-cycle cost of the product set to be procured. Hardware costs UPFRONT RECURRING OVER PROJECT LIFE SPAN PROCUREMENT WARRANTY HOSTING MAINTENANCE REPLACEMENT Data collection Storage Analysis Software costs UPFRONT RECURRING OVER PROJECT LIFE SPAN SETUP USAGE USAGE MAINTENANCE SUPPORT Data collection Storage Analysis/Visualization Other costs (mostly recurring) Communication cost: Cellular Data/SMS Communication cost: Field-Office Training cost In-House IT labor cost Transportation and paper storage cost Labor cost for data collection Labor cost for data entry Labor for data cleansing A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R A nnex 4 — T echnolog y F eatures of C ase S tudies 47 Annex 4: TECHNOLOGY FEATURES OF CASE STUDIES Shading indicates features included in the case study. CASE STUDY 1: TIST Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Proprietary License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms CASE STUDY 2: FAO Open Foris Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Custom SW License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms CASE STUDY 3: Satellite Imagery in Laos Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Custom SW License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms A G R I C U LT U R E A N D E N V I R O N M E N TA L S E R V I C E S T E C H N I C A L A S S I S TA N C E PA P E R 48 A nnex 4 — T echnolog y F eatures of C ase S tudies CASE STUDY 4: Community-Managed Sustainable Agriculture Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Custom SW License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms CASE STUDY 5: Mobile-Based Data Collection in Uganda Device platform Android iOS Java Palm Device type Tablet Smart Phone Basic Phone PDA Device capability Camera GPS Signature External GPS Storage/stakeholder access Custom installation Hosted SaaS/Cloud Remote management Analysis GIS SPSS eCognition Custom Implementation model Frontline workers Crowd-sourcing Passive Automated Business model Open source Custom SW License fee Subscriptions Support In-house Outsourced Service provider Other Mobile features Online Offline Two-way sync SMS Data security In-device encryption Database encryption Encrypted connectivity Access control mechanisms I C T F O R D ATA C O L L E C T I O N A N D M O N I TO R I N G & E VA LUAT I O N Agriculture and Environmental Services (AES) 1818 H Street, NW Washington, D.C. 20433 USA Telephone: 202-477-1000 Internet: www.worldbank.org/ard