Phone monitoring to improve service delivery A SIEF-supported nimble evaluation in India Problem. In many low-income countries, improving ser- beneficiaries–would lead agricultural officers to improve from EVIDENCE to POLICY vice delivery can be challenging, whether it’s making their performance so that more farmers received their sure that teachers are in the classroom ready to teach payments. The government had collected phone numbers or that cash transfers reach intended beneficiaries. One when it updated landholding records the year before, and it reason is that it’s often difficult to cost-effectively mon- hired a call center to ask farmers if and when they received itor programs, especially when they cover thousands of their check, if and when they cashed it, whether they faced communities and include very remote areas. This was the problems receiving or cashing the check, and how satisfied challenge faced by the Indian state of Telangana when it they were with the program. The study also was going to introduced a new program to make payments to farmers measure the impact of giving performance reports, based to help them buy seeds and fertilizer before each of the on the calls, to agricultural officers and their supervisors, two major planting seasons. The government wanted to but the reports weren’t available in time. ensure that people received their money in the expect- ed time frame and that farmers were not asked to make any payments for receiving their money. Intervention. Under the state’s new Rythu Bandhu (Friend of the Farmer) program, landholding farmers are supposed to receive the equivalent of about US$55 per acre before the summer and winter planting seasons. About 5.7 million farmers qualified and close to 90 percent of them owned less than three acres. The first distribution was made be- fore the 2018 summer planting season. Beneficiaries were Photo provided by study team. given checks that could be exchanged for cash at a local bank, regardless of whether a person had a bank account. Evaluation design. This study was a randomized control The checks were distributed during village meetings orga- trial. It covered 30 of the state’s 31 districts, excluding nized by the state’s agricultural officer in each subdistrict, urbanized Hyderabad, where there weren’t many benefi- known as a mandal, who supervised agricultural workers ciaries. In Telangana, districts are divided into mandals. sent to villages to deliver checks. Each agricultural officer oversees one or more mandals. The study randomly assigned 122 mandal agricultural Phone monitoring plan. The study tested whether us- officers to the treatment and 376 to the control group, ing phone calls to monitor the distribution of checks–and for a total of 584 mandals. In the treatment group, ag- telling agricultural officers that calls were being made to ricultural officers were told that the program would be monitored through phone calls to 150 randomly selected benefi- In addition, the program, which led to more farmers receiving ciary farmers in each mandal. In the control group, 50 randomly their checks and receiving them earlier, reduced farmers’ need selected farmers in each mandal were called, but the agricultural to borrow money before the planting season. Researchers esti- officers weren’t informed about the plan. mated $140,000 in additional benefits for the farmers, based on what farmers would otherwise have had to pay for loans to cover Results. Phone-based monitoring improved the rate at which their costs versus what the government would have earned in farmers received their checks based on the bank records show- interest by holding on to the $36,000. ing which checks were cashed. In the control group, 83 percent of farmers cashed their checks in the four-month period after Impacts. The government of Telangana is considering us- check distribution began. In the treatment group, there was a 1.3 ing this type of phone monitoring in other programs. In addi- percentage point increase in the number of farmers who cashed tion, the research team will now help the state government of their checks. Among farmers with the smallest landholdings, the Jharkhand apply phone monitoring to its implementation of increase was 2.2 percentage points. Overall, about US$1.0 mil- the central government’s heavily subsidized wheat and rice lion more reached farmers when agricultural officers were told distribution system. Researchers are also in touch with the they were being monitored. central government to discuss introducing similar programs at scale nationwide. Cost-effectiveness. The program was highly cost-effective. The government paid the call center about US$36,000, mean- Publications. NBER Working Paper Series, “Improving Last- ing that the cost-per-dollar of benefits delivered to beneficiaries Mile Service Delivery Using Phone-Based Monitoring,” Karthik was 3.6 cents, which is lower than the administrative cost of al- Muralidharan, Paul Niehaus, Sandip Sukhtankar, Jeffrey Weaver, most any anti-poverty program for which such data is available. Working Paper 25298. How can we learn better and faster for maximum policy impact? SIEF has chosen 20 rapid, low-cost evaluations for funding through its now closed fourth call for proposals. For the first time, SIEF focused on nimble evaluations, studies designed to produce rapid but rigorous information on implementation and program impact. The nimble evaluations chosen cover SIEF’s core areas of basic education, early childhood development and nutrition, health, and water and sanitation, and they will provide valuable evidence on how to improve programs in these sectors. The Strategic Impact Evaluation Fund, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help alleviate poverty. The goal is to collect and build empirical evidence that can help governments and development organizations design and implement the most appropriate and effective policies for better educational, health, and job opportunities for people in developing countries. For more information about who we are and what we do, go to: http://www.worldbank.org/sief. The Evidence to Policy note series is produced by SIEF with generous support from the British government’s Department for International Development and the London-based Children’s Investment Fund Foundation (CIFF). For more information SIEF visit www.worldbank.org/sief or email SIEFimpact@worldbank.org December 2018 Created by Aliza Marcus, SIEF communications