Investment Climate l World Bank Group Nuts & Bolts Technical Guidance for Reform Implementation Enabling Private Sector Feedback on Public Services through Mobile Devices Lessons from Recent International Experience World Bank Group The rapid diffusion of increasingly sophisticated mobile devices, What is mobile data particularly in the developing world, has created many new collection? opportunities for online interactions between government, Mobile data collection is the targeted gathering of information using devices businesses, and civil society. Many examples already exist of mobile such as mobiles phones or tablets. applications that allow citizens and businesses to request services, MDC makes it possible for citizens or report problems with public infrastructure, and even declare and pay businesses to provide feedback quickly, cheaply, and at their convenience, as tax liabilities. While for many years governments have utilized their long as they have access to a mobile websites to collect general feedback on service delivery or proposed device and the necessary software, laws and regulations, moving these capabilities to mobile devices either on their devices or a central web server. MDC provides a structured can enable immediate feedback on customer experiences after the framework and tools for collecting delivery of a particular service. This provides management of public information from target audiences, agencies greater ability to quickly identify performance issues, obtain compiling it in a central database as well as assessing and presenting the constructive suggestions for reform, and communicate progress back data using a variety of analytical and to the customers who provided the feedback. Mobiles also offer the visualization tools. opportunity for public agencies to disseminate information about reforms faster and more reliably since mobile network coverage, How can MDC tools support particularly in developing countries, is much more widespread than investment climate reform? terrestrial Internet access. MDC tools can provide the means to elicit several types of information, which can monitor public service delivery and support regulators in their regulatory reform or oversight efforts. Examples include: Based on research of several reform programs. Many of these organizations that have implemented mobile data collection (MDC) tools 1. Feedback on the delivery of mobile feedback capabilities as well were pioneered in the health sector specific government services, as recent experience implementing within developing countries to such as business inspections a mobile feedback project in Jordan, support disease reporting and patient and issuance of licenses—This the World Bank Group has identified monitoring in remote areas. They are involves the deployment of software a number of emerging best practices now increasingly being leveraged to through which businesses can and open-source software tools for gather feedback and crowdsource1 data respond to brief online surveys that data collection, summarization, and from citizens and businesses to support collect quantitative and anecdotal visualization which can be leveraged improved public service delivery in a feedback summarizing key aspects to support future investment climate range of areas. of service delivery such as the Enabling Private Sector Feedback on Public Services through Mobile Devices 1 quality of the customer experience, the knowledge and Design Develop Collect Analyze Resolve professionalism of government staff, and the compliance burden (for example, queue times and costs incurred). Design In the design phase of an MDC project, there are The software tools, which are easily downloaded from application stores or other sources, can be designed to several strategic, institutional, and financial issues to consider: collect data using a wide range of mobile devices from low-end feature phones to smartphones running Android What are the objectives? and iOS operating systems. They can also be structured Does management of the public agency wish to monitor so the aggregated data is displayed on an online its overall performance, the quality of individual services, or “management dashboard” summarizing the performance even individual service delivery locations? Or, is the objective of specific operating units over time. The Bank Group is to collect feedback on proposed reforms or new regulations, currently piloting a mobile-based feedback tool in Jordan as may be the case when the government is undertaking focusing on the business inspection services delivered by an investment climate reform program? Answers to these three government agencies. questions will help determine the features and functionality of 2. Facilitating the collection and analysis of the tools required to collect and analyze the feedback. business environment survey data—Many business intermediary organizations, think tanks, and government What is the nature of the information or feedback? agencies conduct periodic surveys to monitor different Is the information to be collected subject to privacy laws in aspects of the investment climate or evaluate the the country? Some information that may be collected in social overall quality of service delivery provided by specific sectors is considered confidential, for example, personal health government agencies. The data collection process can information. It is important to understand legal and regulatory be greatly facilitated using mobile technology. Survey requirements regarding the collection and storage of different data input by enumerators in the field is transmitted to a types of information in the country where the MDC system is central database, which supports the summary reporting being deployed. and analysis of the information collected. For example, the King Abdullah II Center for Excellence in Jordan has developed a tablet-based application to support its What ability exists at the public agency or elsewhere periodic performance survey of Jordanian government within the government to collect and respond to feedback agencies. as well as manage technology? 3. Crowdsourcing data to support regulatory and How does the agency presently collect and respond to budgetary monitoring—Several governments have feedback? Some public agencies already have established developed web-based tools to enable citizens to report complaint hotlines or other mechanisms to collect feedback on the implementation of local infrastructure projects2 or from constituents; others require a culture change to enable environmental violations encountered. Given that even the effective use of such feedback channels. Does the public many low-end mobile phones have cameras and software agency have skilled information technology (IT) staff to support to take advantage of geospatial data, they can provide a and maintain the MDC system, or will the agency need low-cost means to collect such information. Also, the use external support? Is there a data center where the application of mobile phones broadens the reach of data collection can be hosted? efforts to citizens and businesses that do not have direct Internet connectivity or even access to a computer. How will the ongoing administration of the service The balance of this note focuses on practical advice for be funded? practitioners who wish to use MDCs to gather feedback on the Many mobile initiatives of this nature have been criticized for delivery of specific government agencies (1, above); however, lack of sustainability. Most projects start with a pilot that is then much of the discussion is also applicable to the other two types evaluated for scale-up; however, how this scale-up will occur of data gathering (2 and 3, above). should be addressed in the initial design. Even if donors fund the design, development, and deployment of the system (the costs of which are not substantial when compared to other financial Implementing a feedback application and management systems), an ongoing investment in staff and to monitor government service delivery technology resources will be required to administer the software Mobile feedback projects should be implemented in five as well as analyze and respond to the feedback. Also, will the phases: design, develop, collect, analyze, and resolve. feedback providers be required to pay for the airtime required to 2 Investment Climate l World Bank Group transmit feedback, or will there be some arrangement to furnish FIGURE 1: MOBILE INTERFACE FROM JORDAN BUSINESS no-cost airtime or provide some reimbursement? INSPECTIONS SURVEY Develop In the develop phase of an MDC project, there are technology and device selection issues to consider. What type of technology should be employed? The choice among three technology approaches that can be employed in MDC depends on the amount and complexity of the data being collected and the sophistication of the mobile device used to collect and transmit the data. Alternatives include: 1. USSD sessions—Unstructured Supplementary Service Data (USSD) is a mobile telecommunications protocol through which a respondent using even a simple mobile phone can establish a “session” with the database server and be prompted to input data or make choices from an onscreen menu (for instance, select “good,” “fair,” or “poor”). This approach can be utilized for providing feedback in a very simple form, but it can be difficult for untrained individuals to master without instruction. 2. SMS messages—In this case, a short code is established to route Short Messaging Service (SMS) messages sent to it to a server where it will be read and stored in the database. Limited amounts of data also can be collected offline and transmitted later when the user has a phone signal. This is most often used for collecting simple feedback and has been successfully employed in many citizen feedback services, such as LAPOR3 in Indonesia. 3. Mobile applications—If more sophisticated smartphones and tablets can be utilized to submit data, this approach employs a mobile application that can be downloaded by the user for free from an application store. Data can be collected through forms with data fields or drop-down boxes for structured responses; images, recordings, or video can also be collected. This is a more user-friendly interface, and it permits larger amounts of data to be collected offline when the user is out of range of the telecommunications network. Figure 1 illustrates an example of a mobile interface for collecting feedback. What devices and operating systems will be supported? The software must run on the devices available in the local market, particularly if the program targets citizens and businesspeople using their own devices to provide feedback. This will involve considering the different devices (and their operating systems) prevalent locally and the other functionalities required such as global positioning system (GPS), camera, video, audio, and barcode readers. The main Figure 1 shows the user interface of the mobile-based operating systems utilized worldwide include: Android, iOS, feedback tool the World Bank Group is piloting in Jordan focusing on the business inspection services delivered by three government agencies. Enabling Private Sector Feedback on Public Services through Mobile Devices 3 Windows Mobile, Samsung Bada, Blackberry, Nokia Asha, If public agencies lack the technical resources to work with Linux, and Symbian OS. The product distribution in the local stakeholders to develop or customize the software, there are market may require the mobile application be developed on several options for procuring those resources from third parties. two to three operating systems to ensure the broadest reach. They can: Alternatively, the application could be based on USSD or SMS, which supports data input regardless of the operating • Tender for a firm that has a proprietary MDC system which system. can be configured and deployed to meet the project’s needs; this may be preferable if hosting by the vendor is also required; What type of software should be used? • Tender for a software developer using open-source tools to Recent technology trends support using open-source tools to develop software that runs on open-source platforms; doing provide the customization and support required to meet the so allows other developers to see how the system is set up project’s needs. This may be preferable if local support and and use the development as a basis for further customization. language capabilities are required; Among the positives of developing software using open-source • Utilize the “hackathon”4 or prize approach. This involves tools is the opportunity to exchange ideas with others working issuing a problem statement and convening or attracting to solve similar problems, since programmers who use open- computer programmers and other experts in software source tools often share their work within a community of development to produce a solution within a specified period developers. Another positive is avoiding the recurring license of time. Working with existing innovation hubs5 and solver expense. communities6 to organize and host the hackathon or issue the prize will expand participation. The most promising However, several robust proprietary packages, developed solution developed during this competition can then be in response to growing market demand, may be worth supported with further resources to complete and maintain the licensing or subscription costs; these packages may be a production version of the application. especially appropriate when the implementing organization lacks the capacity to manage software applications in-house. Collect In the collect phase, there are communication, Whether a system is open source or proprietary, usability environment, and technology issues to consider. is important. Software should be easy to use and take into consideration the level of literacy and technological What questions should be employed in the survey? sophistication within the target audience. Project leaders must think of the data to be collected and also A number of companies that have developed MDC systems must consider the questionnaire’s length and language, which use a software-as-a-service (Saas) model through which may impact the rate of response. Three indicative types of they provide a minimum level of service for free and then feedback that can be collected include: charge per user or per month to store information collected on a cloud-based server. Alternatively, public agencies that • Overall rating—similar to that provided by restaurant and have access to IT hardware and support can manage the hotel rating services, either rating the overall experience storage, maintenance, and support processes, thus avoiding or specific aspects (for example, timeliness, knowledge of subscription fees. government official). • Time or costs involved in the delivery of the service— For more information on selected MDC tools, see Table 1. particularly useful if there is a service standard established by the government agency (such as a business license issued How should the software selected be customized and within X days). supported? • Commentary on service delivery—providing free text More important than the particular software selected is information or answering questions about issues such as: the need to customize the software to meet stakeholders’ (i) knowledge and attitude of the individual providing the needs. Ideally, software customization requirements should service; (ii) consistency of service delivery with the relevant be designed in partnership with the stakeholders that guidelines and regulations; (iii) resolution of problems plan to use the tools, including the citizens and business encountered (for example, inspection violations); (iv) representatives who will be asked to provide feedback. indications of rent-seeking behaviors; and (v) suggestions Planning for periodic new releases of the software is also for reform. good practice, as new features and functionality requirements will inevitably be identified once the software application is in Public agency management should consider whether it is daily use. more appropriate to monitor (i) quantitative indicators (such as 4 Investment Climate l World Bank Group TABLE 1: OVERVIEW OF SELECTED MDC TOOLS Tool Description Can data be collected and stored in the cloud? CommCare HQ Proprietary software to collect and aggregate field data quickly, Yes; Dimagi offers software-as-a-service hosting with www.dimagi.com accurately, and securely on Android devices. full support. Demo is at http://www.youtube.com/watch?v=EHhi5t3w3YM Pricing is at http://www.commcarehq.org/service- packages/ EpiCollect Free, open-source software developed at Imperial College London Yes; Imperial College London allows organizations www.epicollect.net that runs on Android and iOS. For an independent evaluation to set up project websites for free, but approach of EpiCollect, go to http://www.phiresearchlab.org/downloads/ requires technical skills. epicollect-structured-evaluation.pdf http://www.epicollect.net/instructions/website/ NetWebsite.html FormHub Free, open-source tool, which was developed by Modi Research Yes, designed to be used with Open Data Kit formhub.org Group, Earth Institute, Columbia University. Allows users to create Aggregator; organizations can store data in the cloud surveys in Microsoft Excel and which are then uploaded to a using Google’s AppEngine. FormHub Account online. Runs on Android and utilizes Open Data Kit. Demo is at http://www.youtube.com/watch?v=jqpmMpkIXSQ FrontlineSMS and FrontlineCloud FrontlineSMS is desktop software enabling two-way Yes, through FrontlineCloud at $10 per month. www.frontlinesms.com communication to any mobile handset. FrontlineCloud, which is offered through the software-as-a-service model, enables users to send, receive, and manage SMS messages through an Android device. Magpi Proprietary software, formerly called EpiSurveyor, is in use by more Yes; Datadyne offers software-as-a-service hosting www.datadyne.com than 9,000 people in more than 170 countries. Runs on iOS, with full support. Pricing is at https://datadyne Blackberry, and Symbian. .zendesk.com/entries/21093588-Cost-of- Demo is at http://www.youtube.com/watch?v=KGAiCe5RORs& Magpi-formerly-EpiSurveyor- feature=youtu.be Open Data Kit Free, open-source set of tools which help organizations author, Yes; Open Data Kit Aggregator is the component opendatakit.org deploy, and manage mobile data collection solutions on Android that enables organizations to store data in the cloud devices. using Google’s App Engine. RapidSMS Free, open-source framework for dynamic data collection, logistics No. www.rapidsms.org coordination, and communication, leveraging SMS technology. uReport Free SMS-based communications technology developed by UNICEF No; It should also be noted that implementation ureport.ug Uganda to support social mobilization, monitoring, and response would require a formal partnership with UNICEF. efforts. By sending the text message, ”join,” to a toll-free number and submitting a few personal details, anyone with a mobile phone can become a volunteer ”U-reporter,” sharing his or her observations and ideas on a wide range of development issues. rating, cost) which can be summarized and presented visually The limitations of using agents are that they are expensive through a management dashboard; (ii) qualitative information and affect sustainability for continuous data collection (for instance, commentaries on the service delivery experience), processes. In cases where no agents are used, costs are lower, which requires individual analysis of each response; (iii) or a but lower data quality may result from limitations on user combination of the two. This choice will have a significant technology or basic literacy. If the public agency does not impact on the analytical resources required to administer the use agents, consideration should be given of whether the service and respond to feedback. telecommunication costs should be borne by the end user who is providing the feedback. In both cases, the project should How will feedback or other required information ensure that training of administrators and end users occurs and be collected? that training materials are available in different formats (one page user guides, YouTube videos, manuals). Will the public agency collect information directly from consumers or business representatives or will enumerators or agents be employed to collect the feedback? When using What communications channels and incentives will be used agents, identified persons go to the field to collect data and to elicit feedback? are usually educated and literate in mobile phone use. They are Once an improved feedback mechanism is operational, familiar with the population and topic, and they may receive there is a need to ensure that this new channel is marketed payment for the task or it may be part of their jobs. They either to potential users. A marketing campaign, which uses both use agency-provided devices and airtime or are reimbursed or targeted and public advertising, coordinates with press incentivized to use their own devices and airtime. outlets, distributes flyers or leave-behind cards, and works Enabling Private Sector Feedback on Public Services through Mobile Devices 5 through business intermediaries, should be employed to raise an example of a management dashboard for displaying awareness of the new feedback mechanism. Also, if feedback quantitative feedback on business inspection activities. is being solicited directly from users, promotional incentives such as free airtime to compensate respondents also should To utilize visualization and analysis features, the system be considered, particularly in the pilot phase. Preliminary requirements should be defined in the design stage to ensure research has shown that incentives are less important over that data formats can be easily imported and exported and the long term than responsiveness and accountability of the provide support for basic graphs, mapping functionality, and public agency to the feedback provided; however, if resources flexible queries. allow, incentives may be effective at building awareness and Collating and analyzing user comments and other non- jumpstarting use of the feedback tool. quantitative feedback is a more labor-intensive task requiring Analyze In the analyze phase, there are policy and ongoing staff resources. technology issues to consider. These include: Donors sponsoring reform programs under which these MDC solutions are implemented can also utilize this data to monitor How should the information be displayed? the outcomes of their regulatory reform interventions, for Virtually all MDC tools employ a server to host a data collection example, tracking reductions in compliance costs for businesses and analysis application, with an administrator interface that regulated by the public agency. allows the public agency to create and update forms and menus, import and export data, and set up management Resolve In the resolve phase of an MDC project, there dashboards and other tools for analyzing and visualizing the are institutional and communications issues to consider. These quantitative feedback in summary form. Figure 2 provides include: FIGURE 2: EXAMPLE OF MANAGEMENT DASHBOARD Feedback on Business Inspection Activities What was the duration of the Inspector was knowledgeable. inspection visit? Strongly Disagree 0–30 mins. Disagree 30–60 mins. Neutral 60–90 mins. Agree 90–120 mins. Strongly Agree >120 mins. Did the inspection contribute to Action Taken improved workplace safety?—Q1–Q4 No Action Strongly Agree Q4 Violation Agree Q3 Warning Neutral Q2 Closure Disagree Q1 Strongly Disagree 0 10 20 30 40 50 Percent 0 10 20 30 6 Investment Climate l World Bank Group How should the agency process and respond to feedback? being used, assessing their performance and advising them Once feedback systems have been IT-enabled and information on how to improve can increase ownership and effectiveness coming in from consumers and business representatives is of the data collection activity. If end users are providing the automatically organized in management dashboards, resources feedback directly, demonstrations can be provided by business should be devoted to analyzing the feedback collected, intermediaries or other partner organizations. identifying key issues that emerge, and developing action plans to address these issues. Examples of such issues may Challenge 3: Data fabrication include variation in service quality provided by type of service Fabrication of data is possible. Particularly when enumerators or location, or indications of weaknesses in the training of are used, cheating may be perceived as an easy way to reach client-facing staff. the target number of surveys completed. How should the agency share trends in feedback with Solution: Supervision through web and GPS tracking and consumers and business representatives? device management help identify such cases early on in the process of the data collection. Another solution is to require It is essential for long-term success of the project that—once authentication by end users; however, this may drastically the feedback system has been established and businesses are reduce the number of respondents. using it—the public agency report back to feedback providers on the issue identified and the organizational response. Information on aggregated responses to the feedback surveys Challenge 4: Lack of willingness to use the feedback should be made available through public channels such mechanism as agency websites, so that business people can see their Consumers or business representatives may be unwilling to feedback acknowledged by the agency. In addition, any policy take the time out of their business day or pay for the airtime change or service improvement resulting from feedback also required to complete a survey. should be communicated directly to feedback providers to demonstrate the agency’s responsiveness and accountability. Solution: Giving respondents free airtime, holding promotional events, or offering the chance of a prize may convince people to use the service, particularly during the pilot period. Challenges on MDC projects Various challenges can hamper mobile data collection projects. Challenge 5: Lack of trust among consumers and business The best way to combat these challenges is to be aware of representatives in providing feedback them at the outset and plan accordingly. Below are some If the government has a poor record of demonstrating of the most common challenges and recommendations for accountability to citizens and the private sector, potential addressing them. users may be reluctant to provide feedback for fear of unfair treatment by the public agency in the future. Challenge 1: Poor telecommunications network connectivity Solution: Ensuring users that their individual feedback will be Many countries have unreliable telecommunications networks protected and made available only in aggregate form within which make transmission of data problematic. the agency may help address this concern. Co-branding or offering the service through an independent partner or branch Solution: Conduct network assessments before system design of government also may help allay such fears. and ensure the technology solution selected is appropriate for the environment. For example, support for offline data entry Challenge 6: Need to secure funding for the ongoing with subsequent synchronization with the server can help administration of the MDC project address problems with mobile phone service interruption or Especially in public agencies in which there was previously no jammed networks. way to collect and respond to feedback, ensuring the viability of an MDC project to collect feedback on the nature of Challenge 2: Varying end-user capabilities government-to-business service delivery may be difficult. In collecting feedback from end users, their capacity to utilize such applications can often vary widely. Solution: Raising funds to support a long-term project in any context that employs annual budgeting can be challenging Solution: It is important to understand the baseline even when securing support for the most popular initiatives; characteristics of the people providing feedback, design the feedback projects also risk losing support over time from staff system, and modify training and marketing materials to ensure of the implementing agency, especially if feedback provided efficient mobile data collection. If enumerators or agents are through the system reveals weaknesses in the public agency’s Enabling Private Sector Feedback on Public Services through Mobile Devices 7 processes or staff. Therefore, public agency management represent official policy of the World Bank Group or of its Executive support must be secured before undertaking such a project, Directors or the countries they represent. and an internal champion and long-term budget should be identified prior to project kick-off. The Investment Climate Department of the World Bank Group helps governments implement reforms to improve their business environments and encourage and retain investment, thus fostering Challenge 7: Lack of government capacity to develop, competitive markets, growth, and job creation. Funding is provided by manage, and operate a feedback system the World Bank Group and over 15 donor partners working through Implementation of a feedback system requires some essential the multidonor FIAS platform. capabilities of public agencies including a culture of openness and responsiveness and a baseline of skills among public servants to provide competent customer service. In many Notes countries where the Bank Group is active in supporting 1 “Crowdsourcing” describes the act of reaching out to the public through improved public service delivery, it is necessary to work over a the internet or other media to collect information, identify ideas for solving problems, solicit funds for a particular cause, and so on. long period of time with governments to develop the requisite 2 As part of an economic stimulus program in Kenya, Google assisted the institutional culture and staff skills. Ministry of Finance in deploying a mapping application which displays information and photos uploaded by citizens reporting on the progress of local investment projects, such as schools, health centers, and roads. This Conclusion program has since ended. 3 Mobile data collection tools offer governments a cost-effective Layanan Aspirasi Dan Pengaduon Online Rakyat (LAPOR) is a web- and mobile-based system designed to enable citizens to report on the delivery of means to monitor their performance, support public-private public services, flag public infrastructure problems, and comment on policy dialogue on regulatory issues, and collect data required to issues. fulfill their regulatory oversight responsibilities. 4 A hackathon is a gathering of programmers and others with IT project skills, such as designers and usability experts, to work collaboratively to create (over a short and specified period of time) software designed to provide a Donor agencies can also leverage these tools to improve their solution to a problem. monitoring and evaluation of regulatory reform interventions, 5 A non-exhaustive list of examples in Africa includes AfriLabs, BongoHive particularly those focusing on improving quality of public (Zambia), Botswana Innovation Hub, Co-Creation Hub (Nigeria), Ice Addis service delivery and reducing compliance costs for businesses. (Ethiopia), iHub (Nairobi), iLab (Liberia), mFriday Lab (Ghana), Plug and Play Egypt. 6 Analysis and note were prepared by John R. Wille and Courtney S. A non-exhaustive list of examples includes Charodix, InnoCentive, OpenIdeo, PitchBurner, Skild, TopCoder, We ThinQ. Roberts (Investment Climate Department, World Bank Group). The findings and views published are those of the authors and should not be attributed to the International Finance Corporation (IFC), the World Bank, the Multilateral Investment Guarantee Agency (MIGA), or any other affiliated organizations. Nor do any of the conclusions Contact John Wille l Lead Private Sector Development Specialist l Investment Climate Email: jwille@worldbank.org TEL: 1-202-473-2707 www.wbginvestmentclimate.org 8 Investment Climate l World Bank Group