EVIDENCE September, 2010 from to POLICY a note series on learning what works, from the Human Development Network 57782 Does linking teacher pay to student performance improve results? Educators and education policy makers are concerned with ensure that children not only go to school, but also get creating the best possible learning environment for stu- a good education, the World Bank supported a study of dents. How to do this, especially in primary school, where government-run primary schools in the state of Andhra reading, writing and mathematical skills are first acquired, Pradesh in India. The project, incorporating 500 schools, is the subject of policy debates in many parts of the world. concluded that giving teachers cash bonuses based on the Should teachers be paid more? Can students be rewarded improvement in student performance was more effective for good test results? Do schools need more supplies and at raising test results than giving schools cash grants for better infrastructure? Should parents have access to bet- supplies or additional teachers. This project, which looks ter information about the quality of schools and parental at only one school system in one country, will not end rights and responsibilities? There are no clear answers yet. the debate over how to boost student performance. But At the World Bank, we are working to help achieve it does offer a piece of the puzzle to help steer policy universal primary education for every child, one of the makers and educators as they move forward with new United Nations Millennium Development Goals. To help educational programs and projects. Case Study Andhra Pradesh, India Researchers set out to study what works better at improving grade. The schools, which each had on average 80 stu- student test scores--paying teachers bonuses based on re- dents, were split into five equally-sized groups. In the sults or giving schools cash grants for supplies or cash grants first group, teachers were told they would receive a bo- to hire one extra teacher. The test site was Andhra Pradesh, nus based on the improvement in performance on assess- the fifth most populous state in India with more than 80 ment tests by their students ("individual incentive"). In million people, of whom 73 percent live in rural areas. More the second group, teachers were told they would receive than 80 percent of children in the rural parts of the state at- a bonus based on the improvement in performance of all tend government-run schools. students in the school ("group incentive"). The incentive The study encompassed 500 rural government-run payments were about 3 percent of a teacher's annual sal- primary schools running from first grade through fifth ary. Two other groups of schools received grants for sup- plies or for an extra teacher. The control group received no payments of any kind. The study covered two school Did You Know... years, beginning in June, 2005 and ending in May 2007. On any given day 25 percent of teachers in government-run The schools were randomly sampled from a represen- schools in India are absent tative population and then randomly assigned to one of At any given time less than 50 percent of teachers are teaching the five groups. Household surveys were conducted to obtain household demographic data for the 50,000 chil- Over 90 percent of spending on primary education, excluding dren in the study. The school year in Andhra Pradesh capital expenditures, goes to teacher salaries and benefits starts in the middle of June, and independently-admin- istered baseline assessment tests were conducted in late student and teacher attendance, teaching activity and to June and early July, 2005. In August, study coordinators observe in-classroom teaching. Based on the results of the visited the schools, providing them with the performance assessment tests at the end of the 2005­2006 school year, reports and explaining verbally and in writing into which qualifying teachers received bonus checks at the start of group the school was placed. During the school year, co- the next school year. The same process was repeated in ordinators made unannounced visits to collect data on the 2006-2007 school year. The Findings* Paying teachers extra if their students do well language skills could help students understand material in on tests is highly effective... other classes. The boost in the other subjects also could reflect an improvement in test-taking skills. Students in schools where teachers received either individ- ual-based or school-based incentive pay did better than Teachers who were eligible for bonuses students in control group schools in all five grades, all assigned more homework, class work and districts, and at all levels of test question difficulty. There practice tests. was very limited evidence that the program had different effects based on school characteristics such as the size of Teacher behavior was studied through individual in- the school or the quality of its infrastructure. There was terviews and direct observation. Teachers who were no difference in results based on student demographics, eligible for incentive pay did not come to work more including gender, household literacy, household caste and baseline score. Affluence, however, did play a role: students The average rural primary school in Andhra Pradesh has a total from more affluent families showed a better response to of about 80 students. The schools cover grades one through the teacher incentive program. five and have an average of three teachers. Teachers typically will teach more than one grade at a time, and they usually teach all subjects for the grade. Teachers are state employees ...and students also do better in subjects for and their salaries are based on experience, rank and some- which teachers do not receive incentive pay. times assignment location. There is no compensation based on performance. The average teacher salary at the time of the The study found that students in the incentive schools per- project was Rs. 8,000 per month, or Rs. 10,000 with ben- formed better in social studies and science, two subjects efits, compared with a per capita income in Andhra Pradesh of that were not included in the calculation of the bonus pay- about Rs. 2,000 per month. (U.S. $1 = 46 rupees) ments. The improvements may indicate that in the con- text of primary school education in a developing country, than teachers in the control group. Based on classroom where there are very low levels of learning, there is a "posi- visits by study observers, there also was no difference tive spillover" effect from increased teacher efforts in two in use of the blackboard, asking students questions, as- specific study areas. Students with stronger math skills, for signing homework or helping individual students. But example, are probably better able to handle science; better interviews with the teachers did highlight differences. This bulletin summarizes the results of the research paper "Teacher Performance Pay: Experimental Evidence from India" by Karthik Muralidharan and Venkatesh Sundararaman. The paper is based on a project known as the Andhra Pradesh Randomized Evaluation Study (AP RESt), a partnership between the Government of Andhra Pradesh, the Azim Premji Foundation, and the World Bank, with support from DFID. Full study can be found at: http://www.nber.org/papers/w15323 *All differences reported are statistically significant unless otherwise stated. pay for performance, so it is difficult to distinguish be- tween the impact of the size of the bonus and other teacher characteristics that influence base pay. It turns out that individual incentives are more effective than group-based incentives... Individual incentives and group incentives worked equally well in the first year of the program. By the sec- ond year, students in the individual incentive schools significantly outperformed the group incentive schools. ...while incentive bonuses in general work better than giving schools money for extra When asked unprompted questions about what they inputs that are unconditional on outcomes. did differently--and before being told the test results of their students--teachers in the incentive groups listed Cash incentives to teachers not only boosted student a number of activities that they undertook to prepare performance more than providing inputs, they were their students for the assessment tests. They were sig- also more cost-effective. Students in these schools per- nificantly more likely to have assigned more homework formed better than those in schools that received cash and class work, conducted extra classes after regular grants for supplies or an extra teacher. At the same hours, given practice tests and paid special attention to time, paying teachers based on the performance of weaker students. their students was more cost-effective. It cost an av- The disconnect between classroom observation and erage of Rs. 10,000 a year for an extra teacher or to teacher interviews can be explained, according to the give a school a cash grant, the same amount spent on researchers. Observers, who sat in only during school bonuses for teachers in schools in the individual incen- hours, might be unaware of after-hours classes, for ex- tive group. But students in these schools showed test ample. The observers also were active only from Sep- score gains that were three times larger than students tember through February, which means they would have in schools that received a grant or an extra teacher. missed practice tests or extra homework given closer to Students in the group incentive schools did not score the end of the school year, such as in March. Finally, it as well as students in the individual incentive schools, is hard to capture increased intensity of teaching effort but payments also were lower, making the group in- by observation alone. centive program as cost-effective as the individual in- centive program. The individual incentive program, However, not all teachers respond equally to however, would be more cost-effective after adding in incentive pay. the fixed costs of administering the programs. Teachers with higher base pay respond less well to the individual or group-based incentives. More experienced "The work the World Bank is doing on the effect of incentives on education in India is some of the best and most promising teachers--who are often those with higher base pay--also work in development today. Education is critical to the growth do not respond as well to incentives. The researchers sug- of the developing world and what we learn from these studies gested this could be because the bonus represents a higher can be quickly scaled to reach millions." Alex Tabarrok, Bartley percentage of base pay for younger (and usually less expe- J. Madden Chair in Economics at the Mercatus Center, George rienced) teachers. But they also note that young teachers Mason University and blogger at Marginal Revolution may respond better to any new policy initiative, including Teachers in general like the incentive program... ...and it could in turn lead to better teachers entering the profession. Close to 75 percent of teachers in the cash-bonus program reported that they were more motivated (25 percent report- The authors of the study suggest that paying bonuses based ed no change) and 95 percent of teachers had a favorable on performance could attract better teachers. By rewarding opinion about the program. A majority of teachers--more teachers, those who feel they can or do excel in the profes- than two-thirds--said that the government should consid- sion are more likely to be attracted to working in school er implementing a performance-based bonus system. systems that offer such an incentive program. Conclusion Making policy from evidence Efforts to improve education in both the developed and were not resolved in this study, such as the optimal ratio developing world typically focus on providing schools of bonus pay to base pay, the results show that perfor- with more money for additional supplies and teachers. mance pay for teachers did lead to significant improve- But there is substantial evidence that increased funding is ments in student test scores, with no apparent negative not sufficient to ensure better learning outcomes. Increas- consequences. Policymakers, educators and development ingly, policymakers are looking at pay-for-performance experts considering programs to increase student learning programs to improve student learning in schools. This may want to take a closer look at using financial resources study shows that even modest bonus payments to teach- to give teachers bonus payments based on how well their ers can boost education outcomes. While some issues students do in school. The Human Development Network, part of the World Bank Group, supports and disseminates research evaluating the impact of development projects to help alleviate poverty. The goal is to collect and build empirical evidence that can help govern- ments and development organizations design and implement the most appropriate and effective policies for better educational, health and job opportunities for people in developing countries. For more information about who we are and what we do, go to: http://www.worldbank.org/hdchiefeconomist THE WORLD BANK, HUMAN DEVELOPMENT NETWORK 1818 H STREET, NW WASHINGTON, DC 20433 Produced by Office of the Chief Economist, Human Development Network, Communications/Aliza Marcus