100047 Engaging the Community for Better Education COMMUNITY ENGAGEMENT Field Experiment MECHANISMS in Pakistan FIELD A Summary of the Full Report EXPERIMENT IN PAKISTAN Salman Asim • Ali Abbas • Mariam Adil A SIEF-funded research team attempted to measure the impact of different approaches for boosting community involvement in education. A pilot project, designed with World Bank assistance, sought to encourage and build closer relations SALMAN ASIM • ALI ABBAS • MARIAM ADIL among schools, school committees and local communities. The project was implemented in three districts in rural Sindh, Pakistan, as part of the World Bank-supported Sindh Education Sector Re- form Program (SERP-I & II). The Sindh Education Sector Reform Program gives annual grants to all public schools. Parents and other community members are supposed to decide together via school committees on how to spend the money for school improvement. The grant has two purposes. First, it increases existing funding for schools. Second, it gives parents and community members the opportunity to have a voice in deciding how money for schools can be best used. Over time, it is expected that by involving parents and the broader community in the functioning of the schools, schools will be better managed and learning outcomes will improve. However, giving school grants generally has not boosted community involvement in school management. The reason is twofold: First, only a select group of individuals in the community have ac- cess to information about the amount and delivery date of the grant. Second, even if community members are aware of the grant, they have limited knowledge about how they can engage with the school or school committee in terms of using the grant. The impact evaluation was deigned to help parents and communities directly participate in the improvement of education at the local level by reinvigorating school committees. Parents, school staff and other community members were able to interact with each other at a facilitated community meeting held at the community’s main school. Community members learned about the activities of the school, grants made available to it, and how they could use the school committees to engage with the schools. The project was designed around the realities of life in rural Sindh province, with an awareness of the particular problems faced by women and the very poor. In a feudal society like rural Sindh, socioeconomic power tends to be concentrated among a few community elites. Community members may This is a summary of Community Engagement Mechanisms: Field Experiment in Pakistan. The full project report is available at http://www-wds.worldbank.org/external/default/WDSContentServer/WDSP/IB/2015/09/30/090224b08 31163c4/1_0/Rendered/PDF/Community0enga0periment0in0Pakistan.pdf. not be comfortable talking openly, or critically, with school administrators during a public meeting. Some- times, the community’s elites – those who own the land and wield power – may not want poor community members to join the meetings. Poor families may not participate to avoid angering those who control their livelihoods. One part of the project used text messaging to create a virtual platform for people to discuss school issues and use of the school grant without being identified, thus, circumventing this constraint. Using information and communications technology, also known as ICT, proved key. The ma- jority of households in rural Sindh – 74 percent of households in villages and 78 percent of households in towns – have access to a cell phone, according to 2012 project census data. Taking advantage of the high mobile phone penetration rates, the research team developed a supplementary, virtual interface – the Community Dialogue Platform – to link community members to schools using their phones. The Platform opened an anonymous communication channel between community members, teachers and school committees. Virtual communication saved time and money by reducing the need for multiple community meetings. Arranging fre- quent public meetings can be costly, especially given low resources in a typical household in rural Sindh. Households that registered for the platform could send and receive messages using the num- ber provided by the facilitators. This created an anonymous and cost-effective way to encourage dialogue and share information. The project introduced multiple nudges through the Com- munity Dialogue Platform to maintain community focus on local education. Comments, opinions and complaints received from community members were summarized on a weekly basis by the facilitators and sent to all people registered in the participant database. The text messaging campaign also linked registered users to the School Management Committee’s executive body. The executive body is made up of elected community members, usually parents of students at the school. The executive body is supposed to lead efforts to create a School Improvement Plan to guide spending of school funds. Interactive Voice Response calls were key in improving implementation. Midway through the text messaging campaign, the research team used the ICT platform to collect feedback from the community. Interactive Voice Response calls, which are automated calls that allow people to respond using their keypad, asked people about their views on the program and how well it was working. Participants received phone credits and informational messages to reinvigorate dialogue, refocus discussion on school committees, and prevent drift from the core messages of the project. The discussions conducted through the Community Dialogue Platform provided researchers with micro-level insights into the nature, scope and extent of educa- tion-related problems faced by local communities in rural Sindh. To be successful, each intervention arm – facilitated community meetings and the Commu- nity Dialogue Platform – required that the community have a functioning school committee that could implement school improvement changes. The research project tested two approaches. The first focused on getting people more involved in school committees and in a community meeting with the hope that over time, better knowledge of a school committee’s procedural rules and its man- date, along with community monitoring of funds, would lead to better functioning school committees. A democratic process would lead to weak members being dismissed and replaced with more informed community leaders, thus strengthening the school committee as an institution. The second approach was a more active one. Instead of waiting for school committees to decide to hold an election or name new committee members, elections were scheduled and conducted. Elected members received training on their roles and responsibilities. The team tested both approaches separately, as well as together through a “crossover” design: almost half the villages in each intervention arm (the facilitated community meetings and the text messaging-based inter- vention) held elections for their school committee and new members and the committee’s executive body received additional training related to carrying out relevant responsibilities. The School Improvement Plans created by school committees showed that needs were similar across villages. The most common expenditures across villages included spending on repairs of ceilings, boundary walls, gates, and toilets, and on the repair and purchase of furniture and other classroom needs for students. The ability of field teams to get people in target communities to attend a meeting or use the text messaging platform was critical to the program’s successful implementation. On average, participation rates for village-level meetings exceeded the initial threshold of 60 percent. Overall, participa- tion rates across villages ranged from 21 percent to 100 percent. In the 28 villages where less than 20 percent of people participated in a community meeting, facilitators returned and held another meeting. In the case of the Community Dialogue Platform, an average of 63 percent of households registered to use the service, which was consistent with estimates of mobile phone penetration rates in rural Sindh. The project’s successful implementation relied on the ca- pacity of the facilitators in the field to implement the in- tervention. Two private firms were hired to support the design of materials and to help implement community mobilization efforts. Multiple safeguards were put in place to discourage firms from shirking their responsibilities. Periodic feedback from a third-party monitoring firm and regular supervision by the project team were critical in keeping implementation on track. Maintaining community interest was key to making the Community Dialogue Platform initiative work. Community interest was built up and maintained through various activities. An intermediary firm introduced the community to the Community Dialogue Platform and provided hands-on support; local champi- ons (community volunteers and School Management Committee members) were identified to support, spread and encourage the use of this Platform; and multilingual text support made it possible for text messages to be sent and received in the dominant local languages spoken in rural Sindh. The interventions were intended to repair what could be called the broken feedback loop be- tween local communities and schools. Communities needed to understand that they had a voice in how schools were managed and they needed to learn how to get involved. The solution was to give commu- nity members access to information and a platform that allowed them to voice their concerns and reinvig- orate the school committees. The project team followed a 360-degree design approach, targeting the lack of community participation from different angles. Mobilization efforts, village-level meetings, and the Community Dialogue Platform were used to improve people’s knowledge of school issues and to help them understand how to engage effectively with the com- munity school. These approaches allowed community members to relay their concerns about schools to the elected representatives of school committees. The community was also encouraged to proactively monitor the use of school funds. The project was deliberately designed to create a self-sustaining, community-managed loop to identify, manage and monitor school improvement activities without being dependent on government departments. The text messaging-based Community Dialogue Platform created a feedback mechanism between community mem- bers, school committee members and school officials. This platform falls within the broader class of citizen-feedback models that combine information technology with grassroots institutions such as school committees for improved service delivery. The direct involvement of the Reform Support Unit of the Sindh Education Department at all stages of the interven- tions – from design to implementation – led officials to in- corporate some of the pilot’s innovative features into the Second Education Sector Reform Program. Specifically, the Education and Literacy Department has started a text messaging initiative called ILMI (meaning “to study”) to connect teachers, parents, communities and students through a virtual platform. The goal is to provide a place for collaborative exchange of educational ideas and information, with the hope of creating a self-sustaining loop for improving education outcomes and encouraging dialogue, similar to the pilot program. The education department also created a text messaging system for school committees, specifically to keep teachers and school committee members updated about the transfer of funds and to give them a platform to raise concerns and complaints. The ongoing education reform program in Sindh intends to use the impact evaluation results to inform the scale-up of this pilot intervention. Text messaging and automated voice calls offer governments the opportunity to improve communication be- tween communities with relevant government offices in a cost-effective manner. Communities are able to make their views known directly to the relevant government offices – and they can do so anonymously as well – allowing government officials to respond more quickly to program problems and concerns on the ground. THE WORLD BANK, STRATEGIC IMPACT EVALUATION FUND 1818 H STREET, NW WASHINGTON, DC 20433 Produced by the Strategic Impact Evaluation Fund Editor: Aliza Marcus September 2015