94649 AUTHOR ACCEPTED MANUSCRIPT FINAL PUBLICATION INFORMATION An Assessment of the Impacts of Sri Lanka’s Programme for School Improvement and School Report Card Programme on Students’ Academic Progress The definitive version of the text was subsequently published in Journal of Development Studies, 50(12), 2014-12-03 Published by Taylor and Francis and found at http://dx.doi.org/10.1080/00220388.2014.936396 THE FINAL PUBLISHED VERSION OF THIS ARTICLE IS AVAILABLE ON THE PUBLISHER’S PLATFORM This Author Accepted Manuscript is copyrighted by the World Bank and published by Taylor and Francis. It is posted here by agreement between them. Changes resulting from the publishing process—such as editing, corrections, structural formatting, and other quality control mechanisms—may not be reflected in this version of the text. You may download, copy, and distribute this Author Accepted Manuscript for noncommercial purposes. Your license is limited by the following restrictions: (1) You may use this Author Accepted Manuscript for noncommercial purposes only under a CC BY-NC-ND 3.0 IGO license http://creativecommons.org/licenses/by-nc-nd/3.0/igo. (2) The integrity of the work and identification of the author, copyright owner, and publisher must be preserved in any copy. (3) You must attribute this Author Accepted Manuscript in the following format: This is an Author Accepted Manuscript of an Article by Aturupane, Harsha; Glewwe, Paul; Ravina, Renato; Sonnadara, Upal; Wisniewski, Suzanne An Assessment of the Impacts of Sri Lanka’s Programme for School Improvement and School Report Card Programme on Students’ Academic Progress © World Bank, published in the Journal of Development Studies50(12) 2014-12-03 CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by- nc-nd/3.0/igo http://dx.doi.org/10.1080/00220388.2014.936396 © 2015 The World Bank An Assessment of the Impacts of Sri Lanka’s Programme for School Improvement and School Report Card Programme on Students’ Academic Progress Harsha Aturupane Paul Glewwe Renato Ravina Upul Sonnadara Suzanne Wisniewski April, 2014 Abstract This paper examines two education programmes in Sri Lanka, the Programme for School Improvement (PSI), which decentralises decision-making power, and the School Report Card Programme (SRCP), which was designed to provide parents and other community members with information on the characteristics and performance of their local schools. Using a difference in differences identification strategy, it finds the following results. First, the PSI programme significantly increased math and English reading test scores among grade 4 students, but not first language (Sinhalese or Tamil) test scores. However, PSI has had no effect on any test scores of grade 8 students. In contrast, the SRCP had no significant impacts on any test scores in either grade, and further inquiries revealed the that SRCP was never really implemented. Second, the paper examined the impact of both programmes on teacher and school principal variables. Overall, few effects were found, and in some cases effects were found that one would associate with reduced school quality. On a more positive note, the PSI programme does appear to have led schools to form School Development Committees (SDCs), as the programme stipulates, to establish a list of school priorities, and to implement projects funded through local fundraising. Corresponding author: Paul Glewwe, Department of Applied Economics, University of Minnesota, 1994 Buford Ave., St. Paul, MN 55108 USA. pglewwe@umn.edu. We would like to thank two anonymous referees for very helpful comments on an earlier draft of this paper. I. Introduction Sri Lanka has long outperformed other comparable developing countries in its educational attainment, as reiterated in a recent World Bank (2005) report. Yet that report also raised concerns that the quality of education in Sri Lanka is inadequate, in the sense that students’ learning is falling short of the goals set by the official curriculum. The Sri Lankan government has recently undertaken several initiatives to improve the quality of education in Sri Lanka (see Ministry of Education, 2007). One of them is the Programme for School Improvement (PSI), which provides a mechanism for collaboration between schools and the communities in which they are located through the appointment of school management committees with representation from education administrators, school principals and teachers, and members of the local community. Such programmes are often referred to as school-based management, or decentralised education. A second, more limited initiative is the School Report Card Programme (SRCP), which was designed to provide annual feedback to parents and the local community on schools’ performance against a set of criteria contained in the Report Card. The PSI was implemented in 2006 in all schools in eight selected education zones in Sri Lanka, and the SRCP was to be implemented on a pilot basis in selected schools in both those eight zones and eight other education zones. The implementation plans for both programmes were designed to allow researchers to estimate their impacts on students’ educational outcomes. More specifically, 200 schools in Sri Lanka were divided into four similar groups of 50 schools each. One group consists of 50 schools that implemented the PSI. The SRCP was to be implemented in a second group, and both programmes were to be implemented in a third group. Neither was implemented in the fourth group, which can serve as a ‘control’ group. The division of these 200 schools into the four groups was not done in a purely randomised manner, which 1 complicates estimation of the impact of the two programs. Fortunately, ‘base-line’ data were collected in 2006 for all 200 schools before either programme was implemented, and follow-up data were collected in 2008, after the programmes were to have been in place for about two years. The purpose of this paper is to estimate the impact of the PSI and SRCP programmes on the educational outcomes of Sri Lankan students in grades 4 and 8. In fact, as explained below, the SRCP was never fully implemented; in contrast, the PSI was implemented as planned. While it is not possible to estimate the impact of the SRCP, we present estimates that would measure its impact had it been implemented as a check of our estimation procedure. The remainder of this paper is organised as follows. The next section provides a review of the literature on school-based management programmes and school report cards. The following section describes the Programme for School Improvement (PSI) and the School Report Card Programme (SRCP). Section IV explains how the four different groups of 50 schools were selected, and Section V describes the data available for all 200 schools. Section VI describes the statistical methods used, and Section VII presents the results. The final section summarises and interprets the findings. II. Literature Review Decentralization of public services has long been advocated as a way to make those services more efficient and responsive to local, heterogeneous conditions (Behrman, 2010). For public education, it is viewed as a tangible way to improve school quality in less developed countries because central control of schooling can be ineffective in the presence of diseconomies of scale and can stifle local initiatives of teachers, principals and parents (Jimenez and Sawada,1999). Decentralization, along with community participation, is thought to provide better incentives for efficient allocation of education inputs by education service providers because 2 local stakeholders have better knowledge of local needs and stronger incentives to monitor service delivery (World Bank, 2004). The role of the community is important because the involvement of beneficiaries of public services is thought to be necessary to make the services work for the poor. The following paragraphs consider the literature on decentralization programmes (also known as school-based management programs) and learning report cards as a mechanism for community (or beneficiary) participation. Proponents of decentralization point out that there are several ways to decentralise a school system: ‘decentralization to subnational administrative units; school-based management in which some degree of control is transferred to principals and teachers in the school; and increased parental and community influence in schools by way of electing parent and citizen representatives’ (King and Ozler, 1998, p. 3). The expected outcome of school-based management reforms, for example, is better classroom instruction and student performance because what matters for learning are teacher motivation, school management and effective input allocation (King and Ozler,1998). In contrast, direct school inputs matter less for learning (Hanushek, 1995). However, studies suggest that the details of the program, and the incentives they create, matter for the success of the programme as measured by learning outcomes. For example, some authors find that decentralization can degrade service provision. In Argentina, where the central government transferred administrative power of secondary schools to provincial control, Galiani, Gertler and Schargrodski (2008) find that relatively poor communities did not benefit in terms of academic test scores, compared to wealthier communities. The authors conclude that the central government should monitor local school quality to guarantee a minimum standard. Kremer, Moulin and Namunyu (2002) also note the challenges of decentralization by pointing out how Kenya’s mix of centralised and decentralised 3 control over different aspects of the education system created incentives for misallocation. In particular, when some inputs were centralised (in this case teacher appointments and salaries) while others were decentralised (material inputs), there was an incentive for local communities to build too many small schools, to spend too much on teachers relative to non-teacher inputs, and to set school fees that exceed those preferred by the median voter and thus prevent many children from attending school. Other studies have found some success, in particular if the decentralization allows for the community to have some control over teacher staffing. King and Ozler (1998) examine decentralization in Nicaragua, which gives schools greater autonomy by shifting decision making away from the Ministry of Education directly to the schools. In the process of Nicaragua’s decentralization, the authors found a disconnect between the Ministry of Education’s official assignment of autonomy (de jure autonomy) and the decisions that were actually being taken by a particular school (de facto autonomy). The study finds that de facto autonomy is positively and significantly associated with student performance, in particular when the autonomy comes in the form of control over teacher staffing and monitoring. Jimenez and Sawada (1999) evaluate the EDUCO decentralization programme for schools in El Salvador. The key component of this decentralization programme was that schools in the EDUCO programme are managed by community education associations (ACEs), the members of which include students and parents; ACEs are responsible for hiring and firing teachers, monitoring teacher performance, and maintaining schools. One concern regarding this type of decentralization was that parents, as in El Salvador, were poorly educated and thus may lack the necessary management capabilities needed for this type of decentralization to work effectively. Though the data were not randomised, and thus there was a possibility of bias due to nonrandom selection into the 4 program, the authors found evidence that the programme succeeded in reducing student absence in El Salvador, although it had no impact on student test scores. A second way to address problems of low quality schools is to provide more information to parents and stakeholders on learning and educational achievement through learning report cards. According to the World Bank (2004), information is viewed as a mechanism through which beneficiaries of a public service, such as primary education, can hold providers accountable. Many, if not most, of the results from report card interventions are not encouraging; however, the findings again suggest that the details and the setting of the intervention matter for predicting whether the intervention would increase community participation and affect student learning (Banerjee et al., 2010). In a randomised experiment Andrabi, Das and Khwaja (2009) examine evidence of school and child learning report cards provided to both public and private schools across Pakistan. There are two important aspects of this particular intervention. First, the intervention was implemented in a context where households within each village had considerable school choice among both public and private school options (7.3 schools per village on average). Thus the authors argued that report cards would create competition. Second, the intervention provided parents information on their child’s performance, as well as average village and school scores for their child’s village and school. The results show an increase in student learning, and that the majority of the learning increases occurred among students who were at the lower end of the initial academic distribution. Further, the authors argue that the mechanism behind the report card intervention is likely to be driven by changes in school educational inputs, as opposed to changes due to children switching to better schools. 5 Using data from a randomised control trial on school management reforms in Madagascar (which included school report cards), Lassibille et al. (2010) investigate the impact of school management reforms at the district, sub-district and school level (including report cards) on educational outcomes, including student test scores. They find that the district and sub-district interventions, when combined with school level interventions, had a small positive impact on student test scores. Using the same data, Glewwe and Maïga (2011) investigate whether the results of the interventions varied by the type of teacher (civil service teachers, contract teachers and student teachers). They find no differences in the impact of the programme by the type of teacher, and more generally they find little or no impact on student test scores from any of the interventions. Finally, Banerjee et al. (2010) conduct a randomised evaluation of three primary school interventions in rural India, including learning report cards, which were intended to encourage both beneficiary participation and student learning. None of the interventions have any impact on community involvement, and thus no impact on student learning. III. Objectives and Implementation of the Two Programmes A key policy initiative introduced by Sri Lanka’s Ministry of Education in 2006 is a shared control model of school based management, the Programme for School Improvement (PSI). The aim of PSI is to improve service delivery through greater empowerment of schools and stronger partnerships between schools and their local communities. More specifically, the goal is to bring about a radical change in the culture of schools through the establishment of management structures and the provision of training and support services that increase the participation of parents and the local community, so that the quality of student learning becomes 6 the focus. Schools are to use their greater autonomy to raise funds from their local communities to pay for staff development, the maintenance and improvement of school facilities, and activities that promote learning among students. Forging links with local communities is also expected to improve public accountability. More specifically, under PSI schools should: (i) strive to actively involve the school community (parents, teachers, and past pupils) in running the school; (ii) develop a strategic and operational plan for the development of the school; (iii) seek to use resources effectively and to raise resources from local communities, where feasible and needed, to supplement government funds; (iv) improve performance in curricular and co- curricular activities; (v) establish consistency between staff training and school needs; and (vi) strengthen the school-community relationship. In pursuit of these objectives, each PSI school has established: (i) a School Development Committee (SDC), consisting of the Principal (as Chair), a deputy principal and representatives of teachers, parents, past pupils, and the Zonal Education Office; and (ii) a School Management Team (SMT) includes all school staff members of the SDC and other principals and sectional heads. 1 The PSI programme was introduced into Sri Lankan schools on a phased basis; it started with one education zone per province in 2006, and added a second zone per province in 2007. By the end of 2010, the PSI programme was scaled up to cover the entire country. The districts and zones in which the PSI was introduced in 2006 and 2007 are shown in the Table 1. The School Report Card Programme (SRCP) is an additional policy measure of the Sri Lankan government that was supposed to be implemented on a relatively small scale at the same time that the PSI was implemented. The idea behind the SRCP was that the school community, which includes the principals, teachers, parents, students and former students, should receive regular information on their school’s performance through a ‘report card’. This information 7 would enable school communities to improve their performance, either by stimulating them to action if performance was considered unsatisfactory, or by encouraging school communities whose schools were performing well to strive even harder. The report cards were to be filled out by school personnel at the end of the school year and distributed to parents and SDC members. They were to contain basic school information plus some information on teacher and student performance, such as the teacher attendance rate, student attendance, repetition and dropout rates, school pass rates on the grade 5, O level and A level examination results, and funds available to, and activities of, the school’s SDC. In fact, the SRCP was never implemented as planned, in part due to insufficient support from the Ministry of Education. Thus there is no reason that an evaluation should find an effect of that program. While one option would be not to present results for the SRCP, we decided to present the results as a check of whether our estimation strategy correctly finds no effect of the programme. In contrast, the PSI was fully implemented, with strong support from the Ministry of Education and the Provincial Education Authorities for the schools. There are many supporting documents that attest to its implementation, such as circulars sent by the Ministry of Education to schools to provide guidance on how to implement the PSI. In addition, a recent World Bank Report (2011, p.31) attests to the fact that the PSI was implemented as planned. 2 IV. Selection of the Four Groups of Schools The analysis in this paper is based on data from 200 schools in Sri Lanka. The original analysis plan was to divide these schools into four groups, three treatment groups and one control group. The following treatment groups were selected: 50 schools in which the PSI was implemented in 2006, 50 schools in which the SRCP was to be implemented in 2006, and 50 8 schools in which both programmes were to be implemented in 2006. The remaining 50 schools were to be ‘control’ schools, which were not selected for either program. This section explains how all four sets of schools were chosen. Sri Lanka has nine provinces (although for a short time in 2006, two were merged, so that there were only eight provinces). Each province is further divided into districts, of which there are 25 in Sri Lanka, and each district is further divided into education zones. In all of Sri Lanka there are 93 education zones. In each province, a single district was selected, and the 200 schools used in the analysis of this paper are from those selected districts. Within each selected district, one education zone was selected in March or April of 2006 to implement the PSI, and all schools in that zone implemented the PSI later that year, starting around July. The baseline data were collected in March of 2006, and they were intended to reflect school outcomes at the end of the 2005 school year, so the data were collected when the grade 4 students were in the first semester of grade 5 and the grade 8 students were in the first semester of grade 9. 3 Since all the schools in the selected education zones implemented the PSI in 2006, the 100 schools without the PSI (50 selected for SRCP only and 50 control schools) had to be selected from other education zones, but always from the same district. 4 In each province, the sole education zone that implemented the PSI was chosen as follows. Each province was told to select, in a random manner, one education zone. However, this selection was not rigorously supervised by Sri Lanka’s Ministry of Education, and so it is possible that other factors, such as political considerations, could have played a role in the selection of the education zone in some provinces. While this implies that the PSI schools may not be representative of all Sri Lankan schools, the random selection of PSI schools within these 9 education zones for data collection purposes (see below) means that the PSI schools in the data are representative of all schools in the set of selected education zones. After selecting the one education zone in each province for implementing the PSI, about 11 or 12 schools were randomly selected for the purposes of this study from among all schools in each selected zone. Since all schools in the selected zones were PSI schools, all of these randomly selected schools are PSI schools. This random selection of schools within education zones was done in a stratified way; in each zone schools were classified according to the four types of schools in Sri Lanka, 5 and in each of these four strata schools were selected with an equal probability. The total number of PSI schools selected over all nine provinces was 100. The 100 control schools (50 SRCP only schools and 50 control schools) were drawn from the same districts but, as explained above, from different education zones. While these schools were not chosen in a strictly random fashion, they were selected to match as closely as possible, given observable characteristics, the 100 PSI schools. More specifically, each PSI school was compared to all non-PSI schools in the same district (but in a different education zone) of the same type, ‘level’ and race, 6 and of those non-PSI schools the one with an enrolment level (as measured by the Ministry of Education’s annual School Census) closest to that of the PSI school was selected as the ‘match’. 7 Finally, 50 of the 100 PSI schools, and 50 of the 100 control schools, were randomly chosen to participate in the SRCP. Within each school, for grades 4 and 8, 20 students were randomly selected from each grade to take the exams and complete the questionnaires; if a school had less than 20 students, all of them were selected. Of the 200 schools for which data were collected in 2006, 198 have grade 4 and 161 have grade 8. More specifically, 159 have both grade 4 and grade 8, 2 have grade 8 but not grade 4, and 39 have grade 4 but not grade 8. The analysis below focuses on changes over time within 10 schools, so it uses only those schools for which test scores and survey data were available in both 2006 and 2008; thus 196 schools were used for grade 4 students and 158 for grade 8 students. To check whether the students in the 100 control schools from other education zones in the same district are well matched to the students in the 100 PSI schools, Table 2 compares the baseline (2006) values of the key variable, student test scores, for the PSI schools and the control schools. In all six cases (3 tests × 2 grades), there are no differences across the six combinations of grades and tests. This indicates that there are no major differences between the two types of schools. Table 2 also presents the same comparison for the 100 SRCP schools and the 100 non- SRCP schools. In only one out of six cases, English in Grade 4, is there a significant difference in the mean test scores between the programme schools and the control schools. While one would expect, given that assignment to the SRCP was genuinely random, no significant effects, it is quite possible that this single difference in Table 2 is due to random chance. V. Data Available for 2006 and 2008 A large amount of information was collected in 2006 and 2008 from the 200 schools from which data were collected for the purposes of this analysis. The data come from academic tests administered to students in grades 4 and 8 of those schools, and to their teachers, and from a set of questionnaires that were administered to students, teachers, section heads, school principals, zone directors and in-service advisors. 8 Table 3 summarises the type of information most directly related to learning and academic performance that is available from these 200 schools. In fact, the questionnaires contain more information than is shown in Table 3. 11 Three tests were administered to the students. Grade 4 students were administered academic tests that measured their skills in English, ‘first language’ (either Sinhalese or Tamil), and mathematics, and grade 8 students were tested in first language, mathematics and science. These tests were administered in March of 2006 (at which time the grade 4 students were in the first semester of grade 5 and the grade 8 students were in the first semester of grade 9) and for a new set of grade 4 and grade 8 students in October 2008 (again, the students were tested after they had entered the following grade). 9 These tests were designed by the National Education Research and Evaluation Centre (NEREC) at the University of Colombo, and were administered by the teachers and principals in the 200 schools, under the supervision of NEREC. NEREC hired employees from Zonal Education Offices to supervise the administration of the tests, and at least one such employee was present at all of the tests. For each grade and each year, the test scores for each student were standardised by subtracting the control group mean and then dividing by the control group standard deviation. Thus the control group test scores have a mean zero and standard deviation of one for each subject, grade and year. Tables 4 and 5 show the average test scores for each grade, by year and treatment group. Additional information on students is available from student and parent questionnaires. The student questionnaire collects basic demographic information from the students (age, sex, ethnicity, religion, number of siblings), language(s) spoken at home, food availability at home, transportation mode and travel time to school, availability of a desk and chair at school, availability of textbooks, workbooks and exercise books at school (separately for English, first language, math and science), attitudes about school, time spent doing various activities at home (watching TV, listening to a radio, reading), and whether any grades were repeated. Grade 8 students were also asked about extracurricular activities. The parent questionnaire collects basic 12 demographic information on both parents, some description of the home (type of building, ownership, utilities and ownership of durable goods), educational levels of parents and of the child’s siblings, parents’ occupation, household income, spending on educational items for the child, availability of reading materials in the home, time spent by the child in various activities, parent participation at school and helping the child with schoolwork, student participation in tuition (tutoring) classes, and aspirations for child’s level of education. The grade 8 parent questionnaire also has questions about parents’ attitudes toward education. In 2006, one parent from each student was surveyed. In 2008, data were collected from both parents, although in many cases, only one parent responded. Tables A.2 and A.3 in the appendix present definitions and basic descriptive statistics of the key variables from the student and parent questionnaires that are used in the regression analysis. The teacher questionnaire collects information on each teacher’s personal characteristics (age, sex, living accommodations, distance and commuting time to school, education and work experience), the teacher’s classroom (materials received, sufficiency of supplies, number of students), support provided by education administrators, teaching methods used, and teachers’ opinions on the official syllabus and curriculum. In 2006, two grade 4 teachers were surveyed from each school: the class teacher and the English teacher. For grade 8, one teacher from each subject was surveyed. In both cases, the data were averaged across all surveyed teachers in each school. In 2008, only the class teacher was surveyed. Appendix Tables A.4 and A.5 define and present basic descriptive statistics for the teacher variables included in the regression analysis. The principal questionnaire also begins by requesting personal information, and then asks questions on the teaching staff, school facilities, financial resources, opinions on various education issues, management training and practices, the activities and composition of the SDC, 13 and some information on student performance on recent national tests. Table A.6 in the appendix describes the variables from the principal questionnaire that were used in the regression analysis. The section head questionnaire also collects personal information, and then asks about facilities and teaching supplies, pedagogical practices, methods to evaluate teachers, satisfaction with the current teachers, and opinions on new educational policies, the current curriculum and other education matters. Data were also collected from an in-service advisor and zonal education director. Finally, teacher tests were administered to teachers in 2006 and 2008. These data are not used in the regression analysis as they were believed to play only a secondary role in the impact of the PSI and SRCP programs; thus a full description is omitted for brevity. VI. Methodology The objective of this paper is to estimate the average impacts of the PSI and SRCP programmes on Sri Lankan students’ educational outcomes. There are two distinct treatment effects that one could estimate, the average treatment effect (ATE) for all 200 schools in the data, and the average treatment effect on the treated (ATT), that is the effect of the programme only for the schools that participated in each of the two programmes. This section explains the methodology used, highlighting the assumptions needed to ensure unbiased estimation of ATT and ATE. A. Assumptions Required to Estimate ATT and ATE. To start, let Y1i denote the value of Y, an outcome variable of interest, if student i attends a school that participates in the program, and let Y0i denote the value of Y if student i attends a school that does not participate in the program. The average treatment effect can be defined as: 14 ATE = E[Y1i – Y0i] (1) Since ATE is not conditional on any student characteristics, ‘student I’ represents the average student’ in the population of students who attend the programme and control schools. Let Ps be a variable that equals 1 if school s participates in the programme and equals 0 if it does not participate. The average treatment effect on treated schools, ATT, can be defined as: ATT = E[Y1i – Y0i| Ps = 1] (2) This paper estimates both ATE and ATT using a standard ‘difference in differences’ strategy, which compares the changes over time in the observed values of Y in the schools that do not participate in the programme (schools with Ps = 0) to the observed values of Y in the schools that participate in the programme (Ps = 1). 10 The rest of this section explains the assumptions required for this estimation method to provide unbiased estimates of ATE and ATT. First, consider estimation of ATT. Denote the two time periods, 2006 and 2008, by T = 0 and T = 1, respectively. No schools have the programme in the first time period (T = 0); at that time Y0 is observed, and Y1 is unobserved, for all schools. Between the two time periods, the programme is implemented for participating (Ps = 1) schools, so in the second time period (T =1) Y1 is observed for those schools while Y0 is observed for non-participating (Ps = 0) schools. Strictly speaking, the only ATT that one can estimate in this scenario is the impact at time T = 1, which could differ from the impact at other time periods. Thus the goal is to estimate: ATT = E[Y1i – Y0i| Ps = 1, T = 1] (3) 15 = E[Y1i | Ps = 1, T = 1] - E[Y0i| Ps = 1, T = 1] The first term, E[Y1i | Ps = 1, T = 1], is easy to calculate because at time T = 1 the participating schools (for which Ps = 1) are in the programme so their observed values of Y are values of Y1. The fundamental problem is that the second term, E[Y0i| Ps = 1, T = 1], what Y would have been had those schools not participated, is unobserved. An assumption is needed to estimate that term. The assumption used in difference in differences estimation is that the expected change in Y0 over time is the same in the programme (Ps = 1) and non-programme (Ps = 0) schools. This parallel trends assumption is expressed as: E[Y0isT| Ps = 0, T = 1] - E[Y0isT| Ps = 0, T = 0] = E[Y0isT| Ps = 1, T = 1] - E[Y0isT| Ps = 1, T = 0] (4) where subscripts s and T are added to Y0i to highlight that it may depend on the school s attended by child i and on the time period. If (4) holds, it can be used to estimate the unobserved counter- factual in equation (3), E[Y0isT| Ps = 1, T = 1], since (4) implies that it equals E[Y0isT| Ps = 0, T = 1] - E[Y0isT| Ps = 0, T = 0] + E[Y0isT| Ps = 1, T = 0]. These three terms are observed in the data, since the observed Y for schools with Ps = 0 is always Y0, and the same holds in time period T = 0 for schools with Ps = 1. Thus the difference in difference estimate of ATT, denoted by ATTDD, is: ATTDD = E[Y1i | Ps = 1, T = 1] - E[Y0i| Ps = 1, T = 1] (5) = E[Y1i | Ps = 1, T = 1] – {E[Y0isT| Ps = 0, T = 1] - E[Y0isT| Ps = 0, T = 0] + E[Y0isT| Ps = 1, T = 0]} Figure 1 provides the intuition behind this estimation method. The lowest line shows the change over time in average Y0 for the non-programme schools. The next highest line, which is 16 solid around T = 0 but dashed around T = 1 (dashes indicate it is not observed at T = 1) shows the average Y0 for the programme schools. The highest line shows average Y1 for the programme schools around T = 1. The fundamental estimation problem is: Y0 is not observed for programme schools at time T = 1. Difference in differences methods estimate it by assuming that the change in Y0 for the programme schools from time T = 0 to time T = 1 is equal to the analogous change in the non-programme schools over that period. That is, one assumes that the lines showing how Y0 changes over time are parallel for both types of schools. One can then estimate Y0 for the programme schools at time T = 1 by adding the (observed) change in Y0 from T = 0 to T = 1 for the non-programme schools to the observed value of Y0 at time T = 0 for the programme schools. One can use standard statistical procedures to test whether ATTDD is significantly different from zero, but it is often more convenient to do so using regression methods. To see how this can be done, express Y0isT for student i in school s at time T as follows: Y0isT = α + βT + γPs + ε0isT (6) where the residual ε0isT measures student specific deviations from the expected value of Y0isT condition on T and Ps, that is deviations from α + βT + γPs. Note that equation (6) involves no assumptions yet; in particular no assumptions have been made about ε0isT. Consistent estimation of the parameters in equation (6) requires that the error term ε0isT be uncorrelated with T and Ps. In fact, the parallel trends assumption implies that this is the case (see Appendix for derivations). Next, write an equation for the process that determines Y1 for the students in the programme schools (that is, schools with Ps = 1) at time period 1 (that is, T = 1): 17 Y1isT = α + βT + γPs + δATT + ε1isT (7) = α + β + γs + δATT + ε1isT where the second line follows since this equation is only for programme schools in time period 1. Next, define δATT so that E[ε1isT| T = 1, Ps = 1] = 0; this involves no additional assumptions. This implies that δATT in (7) is the ATT for the treated schools (Ps = 1) at time period 1 (T = 1). 11 Combining equations (6) and (7) leads to the following regression equation: YisT(observed) = α + βT + γPs + δATTPsT + [ε1isTPsT + ε0isT (1 - PsT)] (8) = α + βT + γPs + δATTPsT + [(ε1isT - ε0isT)PsT + ε0isT] Consistent estimation of δATT using OLS requires that the error term (ε1isT - ε0isT)PsT + ε0isT not be correlated with T or Ps. As noted above, the parallel trends assumption implies E[ε0isT| Ps, T] = 0, so the only issue is whether E[(ε1isT - ε0isT)PsT| Ps, T] = 0. There are two cases: PsT = 1 and PsT = 0. Clearly, (ε1isT - ε0isT)PsT = 0 if PsT = 0, so the issue is whether E[(ε1isT - ε0isT)| Ps = 1, T = 1] = 0 when PsT = 1. As seen above, the parallel trends assumption implies that E[ε0isT| Ps = 1, T = 1] = 0, and the definition of δATT implies that E[ε1isT| Ps = 1, T = 1] = 0, so the error term in (8) is uncorrelated with all regressors in that equation, thus an OLS regression of observed values of Y on Ps, T and PsT yields consistent estimates of δATT, which equals ATT (in time period 1). Estimating the average treatment effect, ATE, requires one more assumption, which is that the average treatment effect on the non-programme schools is the same as the average treatment effect on the programme schools (ATT); this implies that the estimate of ATT is also an estimate of ATE. 12 The intuition behind this assumption is quite simple; one never observes 18 Y1 for any non-programme school, and ATE is an average over programme and non-programme schools. Thus one has little choice but to assume that the impact on non-programme schools is the same as the impact on the programme schools, which implies that E[ε1isT| Ps = 0, T = 1] = 0. 13 Thus, the regression based estimate of ATE is the same as the regression based estimate of ATT given in equation (8); the only difference is that one needs an additional assumption, that E[ε1isT| Ps = 0, T = 1] = 0. The intuition for this assumption is that, when estimating ATT, the value of Y1 for the non-programme schools is not needed, so no assumptions need to be made about it. However, when estimating ATE it is necessary to assume that the value that Y1 would take for the non-programme schools if they had been in the programme will not ‘veer off’ in some direction that is different from what is seen in the programme schools, for which Y1 is observed at time T = 1. The only way that the expression for Y1 in equation (7) can ‘veer off’ for non-programme schools is for ε1isT to ‘veer off’. This assumption for ε1isT rules out this possibility; in effect, it constrains the relationship between Y1 and Y0 at time T = 1 (the programme impact at T = 1) to be the same for programme schools and non-programme schools. B. Do the Required Assumptions Hold? Now that the assumptions for estimating ATT and ATE have been clarified, it is useful to consider carefully whether they are likely to hold, and what can be done to increase that likelihood, and to check whether those assumptions hold. This discussion focuses on estimation of the impacts of the PSI, because random assignment of schools to the SRCP should ensure that the assumptions hold for the evaluation of that programme, and because (as explained above) the SRCP was never fully implemented. The fundamental assumption required for consistent estimation of the ATT of the PSI programme is the parallel trends assumption, which in the regression model can be expressed as E[ε0isT| Ps = 1, T = 1] = 0. Intuitively, if E[ε0isT] ≠ 0 for PSI schools at time period 1 then the lines 19 in Figure 1 are not parallel. The issue is whether, in the absence of the program, the variable of interest (Y, or more specifically Y0) changes by different amounts in the PSI schools and the non-PSI schools. Such differential changes would occur if the school or child characteristics, whose effects on the variable of interest operate through ε0isT, are systematically different across the two groups of schools, and these differences lead to differential trends in Y0. There are two reasons, already discussed, why these two sets of schools are likely to be similar. First, as shown in Table 2, the initial test scores were not significantly different in five of the six tests. Second, as explained in Section IV, the non-PSI schools were matched to the 100 PSI schools based on school type, school ‘level’, race, and the number of students enrolled. Yet one could argue that there are other differences between these two sets of schools, or that differences may arise over time, that violate the parallel trends assumption. For example, one way the parallel trends assumption could be violated is that the characteristics of the students in the PSI and non-PSI schools change over time, for example students with more educated parents may be transferred into the PSI schools. This is unlikely because it would require students to transfer to a school in a different education zone, since each education zone has either all PSI schools or no PSI schools. But if student or school characteristics differ in PSI schools and non-PSI schools, the standard approach to minimise the possibility that such differences will lead to biased estimates is to enter school and student characteristics directly into the regression, that is, into equation (8). Then these student characteristics are no longer part of the ε0isT term in that equation and, to the extent that they are correlated with Ps or T, removing them from ε0isT reduces the probability that ε0isT is correlated with Ps or T. Thus, the regression (linearly) controls for all school characteristics by introducing school fixed effects, and controls for five child characteristics: gender, race, household income, mother’s education and father’s education: 20 S YisT(observed) = α + βT + S γsDs + δATTPsT + Xʹθ + [ε1isTPsT + ε0isT (1 - PsT)] (8ʹ) s =1 S = α + βT + S γsDs + δATTPsT + Xʹθ + [(ε1isT - ε0isT)PsT + ε0isT] s =1 where Ds is a set of dummy variables for each of the 200 schools in the data and X, a vector, represents the five student characteristics. Note that adding the school fixed effects and the student level variables has another advantage: they are likely to increase the precision of the estimates. Thus all the regressions in the next section include school fixed effects and student characteristics. 14 Finally, it is possible to check the parallel trends assumption indirectly. Note first that the estimated impact of the programme is the coefficient on PsT, the interaction of the programme school dummy variable and the time dummy variable. It is possible that the PSI schools were already on an ‘above average’ trajectory for other reasons, and if the regression does not control for those reasons, the PsT interaction term may be statistically significant even though there is no programme effect. If there were data collected at two points in time before the PSI programme was implemented, this difference in trajectories could be tested directly. Unfortunately, there are data at only one point in time before the programme began. Yet if one has reason to believe that certain types of schools were more likely to be on an ‘above average’ trajectory, and that those schools were more likely to be PSI schools, one could interact variables that correspond to those types of schools with the time variable (T) and check whether the inclusion of those interaction terms changes the estimated coefficient on PsT; such changes would indicate a violation of the parallel trends assumption. We implement this check for school level averages of the following 21 variables: household income, household spending on education, mother’s education, father’s education, teachers’ level of education, teacher absences and reported adequacy of teaching supplies. VII. Results This section presents estimates of equation (8′) for a variety of outcome variables of interest. The focus is on the parameter δATT, which under the parallel trends assumption estimates the ATT for the programme (either PSI or SRCP). The first subsection examines student and household level variables, beginning with students’ test scores, and the second subsection examines the impacts of the PSI and SRCP on school level variables. Note that the standard errors are clustered at the school level, which allows their correlation to have a very general form for students enrolled in the same school. A. Student and Household Variables. Table 6 presents difference in differences estimates – that is, estimates of δATT in equation (8ʹ) – of the impacts of the PSI and SRCP on student and household outcomes of interest for grade 4 students. Columns 1-3 examine the impact of the PSI programme on math, English and first language (Sinhalese or Tamil) test scores. Students in the schools that implemented the PSI have higher math and English test scores (10% and 1% significance levels, respectively), increasing those scores by 0.17 and 0.22 standard deviations, respectively. The estimated impact of the PSI on first language scores is smaller and negative, -0.08 standard deviations, but it is not significantly different from zero. 15 The lack of an impact on first language scores is not surprising; anecdotal evidence suggests that parents view English and mathematics skills as more important for their children’s future, and the PSI programme may have encouraged them to pressure schools to deemphasise first language skills. 22 In contrast to these effects of the PSI, the schools selected to implement the SRCP have lower test scores in all three subjects (columns 5-7), though none of these impacts is statistically significant. This is as expected given that the SRCP was essentially not implemented. 16 Columns 4 and 8 in Table 6 examine parents’ expenditures on the education of their children in grade 4. Neither the PSI programme nor the SRCP programme has a significant impact on parents’ expenditures on their children’s education. Table 7 repeats the difference in differences analysis of Table 6, but for grade 8 students instead of grade 4 students. In contrast to the results for grade 4 students, neither programme has any impact on any of the students’ test scores; the estimated impacts range from -0.12 to 0.05 standard deviations. 17 There are also no significant effects on student expenditures. One possible explanation for the lack of significant effects is that the programmes operated for about one half of grade 4 students’ time in school (two out of four years) while it operated for only about one fourth of grade 8 students time in school (two out of eight years). A second possible explanation is that about three fourths of both fathers and mothers had completed grade 4 or a higher level of schooling, while only about one third had reached grade 8. Thus while most parents may be comfortable trying to influence education policies at the grade 4 level, since most had of them had attained that level, it is also possible that most parents do not have the confidence to influence school policies for grade 8 since most of them had not completed that grade. A final point to keep in mind is that the standard errors of the estimated impacts are higher for the grade 8 students, presumably because the sample sizes are somewhat smaller than the sample sizes of the grade 4 students, but even so the point estimates are usually much closer to zero for the grade 8 students than for the grade 4 students. 23 As explained in Section VI, the finding that the PSI had a significant impact on math and English test scores in grade 4 is valid only if the parallel trends assumption holds. That section also provided one way to test that assumption, by adding an interaction term between the 2008 dummy variable (T) and school level averages of characteristics that are likely to affect test scores. The results of this test for seven such variables are presented in Table 8 for both grades. Including these interaction terms has virtually no effect on the estimated impacts of the PSI programme. In particular, in six of seven cases the impact of the PSI on math scores remains significant at the 10% level (or higher) and in all seven cases the impact of the PSI on English scores remains statistically significant at the 1% level. Thus this indirect test finds no evidence that contradicts the parallel trends assumption. B. School Variables. Tables 9-13 present difference in differences estimates of the impacts of the PSI and SRCP programmes on school level variables. Table 9 begins by examining teacher behaviour variables among grade 4 teachers. Neither programme had any impact on teacher absences, homework assignment or the teachers’ perception of whether money was allocated for quality inputs. Table 10 examines teacher variables for grade 8 teachers. As in grade 4, there is no statistically significant impact of either programme on teacher absence, nor is there evidence that the SRCP affected assignment of homework or that the PSI affected teachers’ perception of the amount of money spent on higher quality inputs. There was also no impact on whether the teacher received on time the class syllabus, textbooks and a teacher’s guide. However, the estimated impact of the PSI programme on teachers’ assignment of homework is negative and significant at the 5 percent level, and the estimated impact of the SRCP on teachers’ perception of the amount of money spent on higher quality inputs is positive and significant at the 10 24 percent level. Yet given that 12 estimates are presented in Table 10 it is possible that one or two significant results will be found even if the true values of all 12 parameters is zero. Indeed, joint tests for the six PSI regressions, and for the six SRCP regressions, cannot reject the null hypothesis that these programmes have no effect on all six variables. Thus we conclude that there is no evidence that either programme changed what teachers were doing, or the resources at their disposal, in the classroom. Tables 11-13 examine school principal behaviour; there is no distinction between grades 4 and 8 because most schools have both grades. The regressions include schools for which there were test scores in both 2006 and 2008. There were 198 schools with test scores in both years, for a total of 396 observations. Of this total, 315 observations are for principals of schools with both grade 4 and grade 8, 77 are for principals of schools with only grade 4, and 4 are for principals of schools with only grade 8. Beginning with the results in Table 11, the PSI programme had no effect on whether the principal had an appraisal system for teachers, had a system of rewards for teachers, had a self- evaluation programme for teachers, had introduced activities for the professional development of teachers or had observed teachers in the classroom. The sole significant impact of the PSI, and only at the 10% level, is that it increased the probability that the principal reviewed performance and/or monitored progress at the school. The SRCP programme had no significant impact on any of these principal variables, which is what one would expect given that it was never implemented. Overall, these results for the PSI are worrisome in that they suggest that principals were doing little to implement activities that should lead to increased school quality. Table 12 examines some variables concerning management of school needs. The PSI programme had no impact on whether the principal had conducted a needs analysis or had a 25 long-term plan. However, that programme did increased the probability that the principal had implemented some kind of project without financial support from the central or provincial government, had prioritised the school’s needs, and had increased the probability that a School Development Committee had been formed. The SRCP had no effect on any of these actions of the principal, which again is what one would expect since the SRCP was never implemented. Overall, in contrast to the results for teachers, it appears that the PSI programme did lead to positive impacts on principals’ efforts in the area of school management. Finally, Table 13 examines principals’ reports of financial assistance received. Neither programme had any impact on finances received from facility fees, from a school development society, 18 from a past student society, from an NGO, from the state, or from other sources. VIII. Summary and Conclusions Many international development agencies and education researchers have advocated education reform programmes that decentralise decision-making power and/or attempt to empower local communities to hold their schools more accountable. This paper has examined two such programmes in Sri Lanka, the Programme for School Improvement (PSI), which decentralises decision-making power, and the School Report Card Programme (SRCP), which was intended to provide parents and other community members with information on the characteristics and performance of their schools. Using difference in differences estimation, we find the following results. First, the PSI significantly increased math and English reading test scores among grade 4 students, but not first language (Sinhalese or Tamil) test scores. However, the PSI has had no effect on any test scores of grade 8 students. In contrast, the SRCP has had no positive impacts 26 on any test scores in either grade. The lack of a positive effect from the SRCP is consistent with reports that the programme was never fully implemented. Second, to better understand the reasons for the impacts of the PSI programme on students’ test scores in grade 4, the paper examined the impacts of both programmes on teacher and school principal variables. Overall, almost no effects were found of either programme on teacher variables. Yet on a more positive note, the PSI programme does appear to have led schools to form school development committees (which the programme stipulates), to establish a list of priorities, and to implement projects funded through local fundraising. While these results suggest that Sri Lanka’s PSI programme, which is an example of a school-based management programme, did improve student learning among grade 4 students, they are valid only if the parallel trends assumption holds. While an indirect test finds no evidence that it does not hold, this result does not guarantee that this assumption holds. Another weakness of the study is that it provides little or no information on which components of the PSI programme are the ones that make it most effective, and thus the study has little to say about how that programme could be improved. In addition, there is little evidence as to why the programme appears to have worked; in particular, there is little apparent change in teacher behaviour, although there is more evidence of change in the behaviour of school principals. Finally, almost all of the data on teacher and school principal behaviour is provided by the teachers and the principals themselves. The data they provide could be biased; more reliable data could have been attained by using trained observers to record their behaviour. As is often the case, these results raise additional questions. Perhaps most puzzling is why the PSI programme increased student test scores in grade 4, but not in grade 8. Part of the explanation could be the limited time between programme implementation and evaluation. This 27 may especially be true for grade 8 students since the PSI programme operated for only two out of the eight years those students were in school. Another possible explanation is that parents that typically have at most a primary level of education may be relatively confident when exerting pressure on schools to improve teaching in grade 4 but are less certain on what to do to improve student outcomes in grade 8. Another puzzle is why the PSI programme had effect on mathematics and English scores but not first language (Sinhalese and Tamil) scores. One possible answer is that parents may view English and mathematics skills as more critical to their children’s success than their first language skills, and so they pressured schools to de-emphasise teaching of first language. Also, as discussed in the literature review, previous studies suggest that the programme success is in the details. While the goals of the PSI programme encourage strong community involvement, it is not clear whether parents and teachers feel a sense of control over school management decisions. As in King and Ozler (1998), perhaps the most important distinction here is between the de jure autonomy of official decentralization and de facto autonomy in which the community really does take control. Future studies of the impact of school policies on student learning and other educational outcomes should collect even more detailed data on student, parent, teacher and principal behaviour, including direct classroom observation data. 28 1 School Development Committees (SDC) are headed by the principal and contain members of the teaching staff and other stakeholders, such as parents. SDCs are dominated by the principal, while School Management Teams (SMT) spread power more evenly between the principal, teachers and the local community. 2 One referee suggested that further confirmation could be found by examining parents’ reports of their school participation. Yet the available data, which may be noisy, show no significant effects of the PSI on parents’ participation in school meetings or in meetings with teachers; these results are shown in Table A.1 in the Appendix. 3 Sri Lanka’s school year runs from January to November. 4 The education zones chosen for implementing the PSI in 2007 were in different districts from the education zones chosen for PSI in 2006, so none of the control zones chosen in 2006 were selected for the PSI in 2007. Similarly, 16 additional zones implemented the PSI in 2008, and another 8 did so in 2009, but none of these 24 zones are the control zones selected in 2006. 5 All Sri Lankan primary and secondary schools are divided into four types: 1AB, 1C, 2 and 3. Type 1AB schools teach grades 1-13 and offer all three curriculum streams (arts, commerce and science). Type 1C schools teach grades 1-13 but offer only two streams (arts and commerce). Type 2 schools offer only grades 1-11, and small Type 3 schools offer only grades 1-5 or 1-8. 6 In addition to the school types described in footnote 5, there are five types of school ‘levels’: very congenial, congenial, uncongenial, difficult and very difficult. There are also three race categories: Sinhalese, Tamil and Muslim. 7 All ‘national’ and ‘Divisional Secretariat Division (DSD)’ schools were excluded from the sample because the Ministry of Education decided to implement the PSI programme in all national and DSD schools, so there is no control group for those schools. National schools, which make up about 3% of Sri Lanka’s schools, are operated directly by the national government, while other schools are operated by provincial governments. DSD schools are another 3% of schools that have recently been designated by the central government for a major improvement in their physical facilities. 8 In Sri Lanka, about half of the schools are in effect combinations of primary and secondary schools, and so have grades from 1 through 13. These schools have one principal, but are also divided into primary and secondary ‘sections’, each of which has a ‘section head.’ Education zones also have in-service advisors (school inspectors), who visit schools to supervise, and provide support for, teachers. 9 About 3.9% of students drop out between grade 8 and grade 9 (3.0% of girls and 4.8% of boys). This low attrition is unlikely to have a large effect on the estimates presented below. 10 See Angrist and Pischke (2009) for a detailed discussion of difference in differences estimation. 11 ATT (at T = 1) ≡ E[Y1 – Y0| Ps = 1, T = 1] = E[α + βT + γPs + δATT + ε1isT – (α + βT + γPs + ε0isT)| Ps = 1, T = 1] = E[α + β + γ + δATT + ε1isT - (α + β + γ + ε0isT)| Ps = 1, T = 1] = δATT + E[ε1isT| Ps = 1, T = 1] - E[ε0isT| Ps = 1, T = 1] = δATT. 12 More sophisticated methods allow ATT and ATE to be functions of observed student and school variables, which can be denoted by X. While those methods allow for ATE ≠ ATT, they still require that ATT = ATT for two schools, or two students, that have the same values for the X variables 13 More formally, the required assumption is that E[Y1 – Y0| Ps = 1, T = 1] = E[Y1 – Y0| Ps = 0, T = 1]. Using (6) and (7) yields a useful assumption for ε1isT: E[α + βT + γPs + δATT + ε1isT – (α + βT + γPs + ε0isT)| Ps = 1, T = 1] = E[α + βT + γPs + δATT + ε1isT - (α + βT + γPs + ε0isT)| Ps = 0, T = 1], which can be written as δATT + E[ε1isT| Ps = 1, T = 1] - E[ε0isT| Ps = 1, T = 1] = δATT + E[ε1isT| Ps = 0, T = 1] - E[ε0isT| Ps = 0, T = 1], so that 0 + 0 = E[ε1isT| Ps = 0, T = 1] + 0, which implies that E[ε1isT| Ps = 0, T = 1] = 0. 14 In many double difference estimates the same students are observed over time, which allows the regression equation to use the change in the students’ test scores over time as the dependent variables, which automatically differences out school fixed effects, and indeed student fixed effects. Yet our test score data from Sri Lanka are from different students in different years, although from the same schools, so we need to explicitly include school fixed effects in the regression equation (or, equivalently, express all variables as deviations from school means). 15 These results for the PSI are similar, though less precisely estimated for the math score, when the household variables are dropped from the regression. 16 In an earlier version of this paper, before we were informed that the SRCP had not been implemented, we estimated separate impacts for the 50 schools that were to have PSI only, the 50 schools that were to have the SRCP only, and the 50 schools that were to have both programmes. For first language, we found significant negative impacts for the 50 SRCP only schools and weakly significant positive impacts for the 50 schools with both programs. However, these two effects in opposite directions were not jointly significant. 29 17 One possible reason for the differences across grades 4 and 8 is that the samples differ; 39 schools have grade 4 but not grade 8. Yet this is not the case; when the sample is limited to schools with both grade 8 and grade 4 the estimated impacts on math and reading of grade 4 students are 0.189 (10% significance) and 0.250 (1% significance), respectively. One could also argue that two years is too short of a time period to find a significant impact. Yet several studies in the past 10-15 years have found statistically significant impacts of fairly similar programmes that had been operating for only one or two years. Examples include Jimenez and Sawada (1999), Galiani, Gertler and Schargrodsky (2008), and Banerjee et al. (2010). 18 School development societies are not the same as School Development Committees, which were initiated by the PSI program. School development societies are an older association that can receive local donations, but (unlike schools development committees) does not actively solicit donations from the local community. 30 References Allcott, H. and D. Ortega. 2009. ‘The Performance of Decentralized School Systems: Evidence from Fe y Alegria in Venezuela’ in F. Barrara-Osorio, H.A. Patrinos, and Q. Wodon, eds. Emerging Evidence on Private Participation in Education: Vouchers and Faith-Based Providers. The World Bank. Washington D.C. Andrabi, Tahir, Jishnu Das and Asim Ijaz Khwaja (2009) ‘Report Cards: The Impact of Providing School and Child Test-scores on Educational Markets’ Working Paper. Angrist, Joshua, and Jörn-Steffen Pischke. 2009. Mostly Harmless Econometrics. Princeton University Press. Banerjee, Abhijit, Rukmini Banerji, Esther Duflo, Rachel Glennerster and Stuti Khemani. 2010. ‘Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India.’ American Economic Journal: Economic Policy. 2(1):1-30. Behrman, Jere R. 2010. ‘Investment in Education – Inputs and Incentives’ Ch 73 in Handbook of Development Economics, Volume 5. Elsevier. Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muraldhiran, and F. Halsey Rogers. 2006. ‘Missing in Action: Teacher and Health Worker Absence in Developing Countries,’ Journal of Economic Perspectives. 20(1):91-116. Deaton, Angus. 1997. The Analysis of Household Surveys: A Microeconomic Approach. Johns Hopkins University Press: Baltimore. Galiani, Sebastian, Paul Gertler and Ernesto Schargrodsky. 2008. ‘School Decentralization: Helping the good get better, but leaving the poor behind.’ Journal of Public Economics 92:2106-2120. Glewwe, Paul and Eugenie Maïga. 2011. ‘The Impact of School Management Reforms in Madagascar: Do the Impacts Vary by Teacher Type?’ Journal of Development Effectiveness. 3(4):435-369. Hanushek, Eric (1995) ‘Interpreting Recent Research on Schooling in Developing Countries.’ The World Bank Research Observer. 1(2):227-246. Jimenez, Emmanuel and Yasayuki Sawada. 1999 ‘Do Community-Managed Schools Work? An Evaluation of El Salvador’s EDUCO Program.’ The World Bank Economic Review. 13(3):415-441 King, Elizabeth and Berk Ozler. 1998. ‘What’s Decentralization Got to Do with Learning? The case of Nicaragua’s School Autonomy Reform.’ Development Economics Research Group, Working Paper Series on Impact Evaluation of Economic Reforms, No. 9. Washington D.C.: World Bank. Kremer, M., S. Moulin and R. Namunyu. 2002. ‘Unbalanced Decentralization.’ Department of Economics. Harvard University. Lassibille, G., J-P Tan, C. Jesse, and T. Nguyen. 2010. ‘Managing for results in primary education in Madagascar: Evaluating the impact of selected workflow interventions’, World Bank Economic Review. 24(2):303-329. Ministry of Education. 2007. Education Sector Development Framework and Programme, Colombo, Sri Lanka. World Bank. 2004. World Development Report 2004: Making Services Work for Poor People. Washington, DC; World Bank. World Bank. 2005. ‘Treasures of the Education System in Sri Lanka: Restoring Performance, Expanding Opportunities and Enhancing Prospects.’ Human Development Unit. South Asia Region. The World Bank. Washington, D.C. and Colombo, Sri Lanka. World Bank. 2011. ‘Implementation Completion and Results Report (IDA-H1960 and IDA- 44230). Human Development Sector. South Asia Region. The World Bank. 1 Y Y1, P=1, T=1 ATT Y0, P=1, T=1 Y0, P=1, T=0 Y0, P=0, T=1 Y0, P=0, T=0 T=0 T=1 Figure 1: Difference in Differences Estimation 2 Table 1: Pilot Zones of the PSI Program, 2006 and 2007 2006 2007 Province Pilot Zone District Schools Pilot Zone District Schools Western Colombo Colombo 125 Kalutara Kalutara 148 Central Wattegama Kandy 82 Hanguranketa Matale 69 Southern Ambalangoda Galle 82 Tangalle Hambanthota 105 North-Western Chilaw Kurunegala 158 Kuliyapitiya Puttalam 164 Northern Vavuniya-South Vavuniya 97 Mannar Mannar 64 Eastern -- -- Sammanthurai Trincomalee 64 North-Central Tambuttegama Anuradhapura 69 Hingurakgoda Polonnaruwa 92 Uva Wellawaya Monaragala 87 Badulla Badulla 172 Sabaragamuwa Kegalla Kagalle 163 Embilipitiya Ratnapura 123 Note: In 2006, the Northern and Eastern Provinces were merged, and the Vavuniya-South Zone was selected from the (merged) North-Eastern Province. In 2007, the Northern and Eastern Provinces were de-merged and set up as separate provinces. Hence, separate zones were selected from each of these provinces. Table 2. Testing for Differences in (Non-Standardised) Baseline (2006) Test Scores Programme Control School Difference School Mean Mean [p-value] PSI Grade 4 Math 64.1 63.2 0.9 [0.683] English 48.2 45.7 2.5 [0.232] First Language 54.4 56.8 -2.5 [0.332] Grade 8 Math 41.8 43.1 -1.3 [0.405] Science 47.8 49.6 -1.8 [0.319] First Language 51.8 53.5 -1.7 [0.325] SRCP Grade 4 Math 62.1 65.3 -3.2 [0.140] English 44.6 49.4 -4.8** [0.021] First Language 53.8 57.3 -3.4 [0.169] Grade 8 Math 42.4 42.5 -0.04 [0.978] Science 47.7 49.6 -1.9 [0.303] First Language 52.5 52.8 -0.3 [0.848] Statistical significance at the 10, 5 and 1 percent levels is indicated by one, two and three asterisks, respectively. Statistical significance is based on standard errors that are robust and clustered at the school level. 4 Table 3: Selected Variables Available from the Sampled Schools Variable Source Student Indicators Test scores Student test Participation in tutoring (tuition) classes Student questionnaire Time spent studying Parent questionnaire Grade repetition Student questionnaire Teacher Indicators Subject knowledge (test scores) Teacher test Classroom supplies (books, texts, desks, etc.) Teacher questionnaire Teacher training (≥ 14 days in last 2 years) Teacher questionnaire Teacher absences Teacher questionnaire Adequate guides on student centred learning Teacher questionnaire Teachers allocated funds to buy school inputs Teacher questionnaire Parental and Community Indicators Parent/teacher meeting attendance Parent questionnaire Parent helps child with schoolwork Parent and Student questionnaire Parental expectations of child achievement Parent questionnaire Parent participation in school events Parent questionnaire Principal/Section Head/School Indicators School management practices Principal questionnaire Finances Principal questionnaire School facilities Principal questionnaire Teacher/principal meetings Section Head questionnaire 5 Table 4: Standardised Test Scores by Year and Treatment Group - Grade 4 2006 2008 Math N Mean SD N Mean SD PSI and Report Card 788 -0.105 1.041 751 -0.173 1.164 PSI Only 687 -0.286 1.080 654 -0.138 1.080 Report Card Only 756 -0.104 1.031 703 -0.233 1.247 Control 696 0.000 1.000 673 0.000 1.000 English N Mean SD N Mean SD PSI and Report Card 787 -0.108 1.048 750 -0.143 1.007 PSI Only 669 -0.285 0.979 659 -0.158 0.978 Report Card Only 752 0.060 1.050 708 -0.094 1.006 Control 699 0.000 1.000 663 0.000 1.000 First Language N Mean SD N Mean SD PSI and Report Card 784 -0.256 1.012 751 -0.590 1.192 PSI Only 688 -0.258 1.026 641 -0.500 1.117 Report Card Only 755 -0.210 1.030 707 -0.590 1.396 Control 699 0.000 1.000 673 0.000 1.000 Note: The sample is restricted to include only the schools that were in both rounds of data collection. Table 5: Standardised Test Scores by Year and Treatment Group - Grade 8 2006 2008 Math N Mean SD N Mean SD PSI and Report Card 669 -0.080 0.957 612 -0.180 0.925 PSI Only 635 -0.172 0.953 594 -0.171 0.952 Report Card Only 626 -0.198 0.953 609 -0.134 0.876 Control 631 0.000 1.000 609 0.000 1.000 Science N Mean SD N Mean SD PSI and Report Card 669 -0.185 0.984 616 -0.217 1.047 PSI Only 634 -0.157 0.995 594 -0.134 0.984 Report Card Only 621 -0.163 0.948 612 -0.093 0.975 Control 637 0.000 1.000 615 0.000 1.000 First Language N Mean SD N Mean SD PSI and Report Card 661 -0.108 1.034 617 -0.144 1.076 PSI Only 611 -0.189 1.091 590 -0.123 1.017 Report Card Only 640 -0.113 0.992 614 -0.058 1.041 Control 637 0.000 1.000 613 0.000 1.000 Note: The sample is restricted to include only schools that were in both rounds of data collection. 6 Table 6: Grade 4 Test Scores and Household Educational Expenditures (school-level fixed effects estimation with clustered standard errors) (1) (2) (3) (4) (5) (6) (7) (8) VARIABLES Math English First lang. Expenditure Math English First lang. Expenditure Year = 2008 -0.048 -0.111* -0.222** 0.949** 0.051 0.055 -0.186** 1.065*** (0.066) (0.057) (0.105) (0.374) (0.062) (0.053) (0.078) (0.362) 2008×PSI 0.170* 0.216*** -0.076 0.506 -- -- -- -- (0.087) (0.077) (0.135) (0.541) 2008×Report Card -- -- -- -- -0.020 -0.104 -0.150 0.292 (0.088) (0.078) (0.132) (0.535) Male -0.238*** -0.355*** -0.205*** 0.124 -0.239*** -0.357*** -0.206*** 0.123 (0.028) (0.028) (0.036) (0.171) (0.028) (0.028) (0.036) (0.171) Sinhala 0.378*** 0.001 0.177 1.446** 0.379*** 0.006 0.181 1.440** (0.122) (0.106) (0.117) (0.562) (0.122) (0.107) (0.120) (0.569) Tamil 0.120 -0.028 0.176 0.975 0.115 -0.028 0.191 0.931 (0.109) (0.087) (0.171) (0.849) (0.108) (0.083) (0.175) (0.869) Income 0.044*** 0.060*** 0.0113 1.106*** 0.043*** 0.061*** 0.013 1.100*** (0.015) (0.012) (0.014) (0.104) (0.015) (0.012) (0.013) (0.103) Mother’s Educ. 0.050*** 0.040*** 0.024*** 0.212*** 0.050*** 0.040*** 0.024*** 0.212*** (0.007) (0.006) (0.006) (0.036) (0.007) (0.006) (0.006) (0.036) Father’s Educ. 0.022*** 0.027*** 0.008 0.037 0.023*** 0.027*** 0.008 0.038 (0.007) (0.005) (0.008) (0.042) (0.007) (0.005) (0.008) (0.042) Constant -1.040*** -0.636*** -0.527*** 3.448*** -1.042*** -0.641*** -0.530*** 3.450*** (0.125) (0.097) (0.128) (0.632) (0.124) (0.097) (0.130) (0.638) Observations 4,660 4,643 4,654 4,680 4,660 4,643 4,654 4,680 R-squared 0.072 0.094 0.038 0.107 0.070 0.091 0.039 0.107 Number of schools 196 196 196 196 196 196 196 196 Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 7 Table 7: Grade 8 Test Scores and Household Educational Expenditures (school-level fixed effects with clustered standard errors) (1) (2) (3) (4) (5) (6) (7) (8) VARIABLES Math Science First lang. Expenditure Math Science First lang. Expenditure Year = 2008 -0.047 0.055 0.017 4.391*** -0.019 -0.007 -0.033 3.973*** (0.072) (0.070) (0.060) (0.358) (0.069) (0.061) (0.060) (0.390) 2008×PSI -0.034 -0.124 -0.046 -1.025* -- -- -- -- (0.088) (0.087) (0.079) (0.574) 2008×Report Card -- -- -- -- -0.087 0.003 0.053 -0.148 (0.089) (0.088) (0.080) (0.572) Male -0.203*** -0.154*** -0.538*** -0.233 -0.203*** -0.155*** -0.539*** -0.240 (0.034) (0.036) (0.034) (0.209) (0.034) (0.036) (0.034) (0.208) Sinhala 0.222* 0.308*** 0.412*** 1.173 0.219* 0.311*** 0.415*** 1.177 (0.114) (0.098) (0.125) (0.811) (0.113) (0.097) (0.125) (0.797) Tamil 0.007 0.037 -0.039 0.307 0.00005 0.040 -0.033 0.319 (0.115) (0.112) (0.131) (0.807) (0.114) (0.112) (0.130) (0.806) Income 0.027* 0.022 -0.0002 0.748*** 0.028* 0.023 -0.0004 0.757*** (0.015) (0.017) (0.016) (0.107) (0.015) (0.016) (0.016) (0.108) Mother’s Educ. 0.026*** 0.032*** 0.041*** 0.110*** 0.026*** 0.032*** 0.041*** 0.111*** (0.005) (0.005) (0.005) (0.026) (0.005) (0.005) (0.005) (0.026) Father’s Educ. 0.029*** 0.036*** 0.035*** 0.075** 0.029*** 0.036*** 0.035*** 0.074** (0.006) (0.006) (0.006) (0.032) (0.006) (0.006) (0.006) (0.032) Constant -0.610*** -0.844*** -0.709*** 6.028*** -0.607*** -0.849*** -0.713*** 6.006*** (0.111) (0.109) (0.126) (0.784) (0.110) (0.109) (0.126) (0.776) Observations 3,406 3,407 3,393 3,389 3,406 3,407 3,393 3,389 R-squared 0.047 0.054 0.127 0.199 0.047 0.053 0.127 0.197 Number of schools 158 158 157 158 158 158 157 158 Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 8 Table 8. Indirect Test of Parallel Trends Assumption (1) (2) (3) (4) (5) (6) Grade 4 Grade 8 Test score variables: Math English 1st lang. Math Science 1st lang. Original Estimates (from Tables X and Y) PSI 0.163* 0.204*** -0.085 -0.034 -0.124 -0.046 (0.088) (0.077) (0.133) (0.088) (0.087) (0.079) Add interaction of T with school-level average of household income in 2006 PSI 0.164* 0.218*** -0.096 -0.030 -0.122 -0.041 (0.087) (0.077) (0.134) (0.088) (0.087) (0.079) Add interaction of T with school-level average of household spending on education PSI 0.168* 0.216*** -0.088 -0.031 -0.125 -0.045 (0.086) (0.077) (0.127) (0.089) (0.087) (0.079) Add interaction of T with school-level average of mother’s education PSI 0.143* 0.217*** -0.087 -0.036 -0.126 -0.046 (0.086) (0.076) (0.135) (0.087) (0.086) (0.076) Add interaction of T with school-level average of father’s education PSI 0.159* 0.222*** -0.076 -0.036 -0.125 -0.049 (0.088) (0.076) (0.135) (0.088) (0.087) (0.076) Add interaction of T with school-level average teacher education PSI 0.191** 0.239*** -0.041 -0.036 -0.124 -0.037 (0.089) (0.078) (0.136) (0.089) (0.087) (0.079) Add interaction of T with school-level average days teacher is absent in past year PSI 0.166* 0.216*** -0.059 -0.047 -0.142 -0.054 (0.088) (0.077) (0.136) (0.089) (0.087) (0.080) Add interaction of T with school-level index of adequacy of teaching supplies 0.144 0.221*** -0.0568 0.012 -0.106 -0.019 (0.093) (0.083) (0.137) (0.092) (0.090) (0.083) Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 9 Table 9: Grade 4 Teacher Variables (school-level fixed effects estimation with clustered standard errors) (1) (2) (3) VARIABLES Teacher Homework Money Absence A. PSI Regressions Year = 2008 -8.179*** -0.052 0.030 (2.320) (0.059) (0.028) 2008×PSI 4.896 -0.091 0.065 (4.957) (0.125) (0.053) Constant 23.46*** 2.808*** 0.907*** (1.028) (0.026) (0.012) R-squared 0.064 0.017 0.028 B. SRCP Regressions Year = 2008 -7.106*** -0.074 0.045 (2.478) (0.059) (0.028) 2008×Report Card 0.573 -0.010 0.003 (4.286) (0.128) (0.055) Constant 23.47*** 2.809*** 0.907*** (1.029) (0.026) (0.012) R-squared 0.059 0.013 0.020 Observations 382 349 371 Number of schools 196 191 195 Robust standard errors, clustered at the school level, in parentheses. *** p<0.01, ** p<0.05, * p<0.1. Notes: ‘Homework’ refers to frequency that the teacher gives homework to students; =3 if always, =2 if seldom, =1 if once in a while. ‘Money’ refers to teacher’s opinion as to whether money was allocated for quality inputs; =1 if yes or =0 if no. 10 Table 10: Grade 8 Teacher Variables (school-level fixed effects estimation with clustered standard errors) (1) (2) (3) (4) (5) (6) VARIABLES Teacher Syllabus Class Text Teacher’s Homework Money Absence Guide A. PSI Regressions Year = 2008 -0.279 -0.131 -0.077 -0.282** 0.159 0.045* (3.337) (0.121) (0.084) (0.125) (0.099) (0.027) 2008×PSI 0.517 -0.255 -0.027 -0.210 -0.231** -0.031 (4.589) (0.167) (0.120) (0.184) (0.116) (0.043) Constant 23.01*** 2.451*** 2.707*** 2.501*** 2.671*** 0.933*** (1.133) (0.039) (0.029) (0.043) (0.027) (0.010) R-squared 0.000 0.084 0.017 0.131 0.033 0.017 B. SRCP Regressions Year = 2008 2.933 -0.308** -0.140 -0.460*** 0.032 -0.012 (3.528) (0.129) (0.091) (0.143) (0.081) (0.034) 2008×Report Card -5.985 0.101 0.097 0.153 0.033 0.083* (4.551) (0.168) (0.120) (0.184) (0.120) (0.043) Constant 23.01*** 2.452*** 2.707*** 2.501*** 2.671*** 0.933*** (1.125) (0.039) (0.028) (0.043) (0.028) (0.010) R-squared 0.011 0.070 0.021 0.126 0.006 0.039 Observations 310 289 296 284 289 298 Number of schools Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 The dependent variables in these regressions are categorical at the teacher level; however OLS fixed effects estimation was used instead of probit or order probits because the 2006 teachers data were averaged across the three teachers in each school. In 2008, only the class teacher was surveyed. ‘Homework’ refers to frequency that the teacher gives homework to students; =3 if always, =2 if seldom, =1 if once in a while. ‘Money’ refers to teacher’s opinion as to whether money was allocated for quality inputs; =1 if yes or =0 if no. 11 Table 11: Grade 4 and Grade 8 Principal’s Management of Teaching Variables (Probit and Ordered Probit estimation with clustered standard errors) (1) (2) (3) (4) (5) (6) VARIABLES Appraises Reward Self Activities Review Observe Evaluation A. PSI Regressions Year = 2008 0.293 0.265* 0.146 -0.001 0.069 -0.058 (0.203) (0.155) (0.172) (0.175) (0.194) (0.129) 2008×PSI -0.174 -0.272 -0.118 0.166 0.482* 0.130 (0.259) (0.214) (0.214) (0.230) (0.266) (0.155) Constant 0.849*** -0.526*** 0.182** 0.610*** 0.723*** (0.105) (0.097) (0.092) (0.104) (0.102) Threshold 1 -2.358*** (0.223) Threshold 2 -0.804*** (0.102) Threshold 3 -0.015 (0.094) Threshold 4 0.433*** (0.095) Threshold 5 0.845*** (0.097) B. SRCP Regressions Year = 2008 0.253 -0.002 0.079 0.055 0.263 0.009 (0.212) (0.165) (0.173) (0.167) (0.191) (0.127) 2008×Report Card -0.106 0.275 0.016 0.058 0.0345 0.001 (0.258) (0.214) (0.214) (0.230) (0.258) (0.156) Constant 0.849*** -0.526*** 0.182** 0.610*** 0.723*** (0.105) (0.097) (0.092) (0.104) (0.102) Threshold 1 -2.357*** (0.223) Threshold 2 -0.803*** (0.102) Threshold 3 -0.015 (0.094) Threshold 4 0.432*** (0.095) Threshold 5 0.844*** (0.097) Observations 330 334 329 309 322 329 Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 See the text for the definitions of the dependent variables. 12 Table 12: Grade 4 and Grade 8 Principal’s Management of School Needs Variables (Probit estimation with clustered standard errors) (1) (2) (3) (4) (5) VARIABLES Project Needs Priorities Long Term Formed Analysis Plan School Dev. Com. A. PSI Regressions Year = 2008 0.046 0.136 -0.256 0.693*** -0.617*** (0.160) (0.252) (0.212) (0.262) (0.237) 2008×PSI 0.415** 0.257 0.642** 0.621 1.577*** (0.210) (0.346) (0.321) (0.456) (0.343) Constant -0.256*** 1.358*** 1.358*** 0.907*** 0.765*** (0.094) (0.128) (0.128) (0.106) (0.102) B. SRCP Regressions Year = 2008 0.207 0.280 0.169 0.868*** 0.250 (0.163) (0.260) (0.246) (0.277) (0.221) 2008×Report Card 0.103 -0.060 -0.332 0.128 -0.142 (0.208) (0.339) (0.297) (0.402) (0.291) Constant -0.256*** 1.358*** 1.358*** 0.907*** 0.765*** (0.094) (0.128) (0.128) (0.106) (0.102) Observations 329 344 343 341 294 Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 See the text for the definitions of the dependent variables. 13 Table 13: Grade 4 and Grade 8 Principal Financial Assistance Received Variables (Ordered Probit estimation with clustered standard errors) (1) (2) (3) (4) (5) (6) VARIABLES Facilities School Past Other NGO State Dev. Students Society Assoc. A. PSI Regressions Year = 2008 0.090 0.211 -0.192 0.159 -0.182 0.221 (0.144) (0.138) (0.217) (0.194) (0.200) (0.155) 2008×PSI -0.087 -0.089 -0.089 0.096 -0.264 -0.030 (0.201) (0.203) (0.299) (0.246) (0.281) (0.182) Threshold 1 0.310*** 0.430*** -2.827*** 1.231*** 0.932*** -1.861*** (0.096) (0.094) (0.384) (0.129) (0.112) (0.160) Threshold 2 2.026*** 2.062*** 1.386*** 2.584*** 2.387*** 0.404*** (0.203) (0.167) (0.140) (0.279) (0.268) (0.093) B. SRCP Regressions Year = 2008 0.053 0.299** -0.233 0.288 -0.226 0.156 (0.140) (0.137) (0.250) (0.197) (0.184) (0.149) 2008×Report Card -0.016 -0.288 -0.010 -0.175 -0.180 0.103 (0.204) (0.204) (0.300) (0.249) (0.282) (0.182) Threshold 1 0.310*** 0.429*** -2.825*** 1.231*** 0.932*** -1.862*** (0.096) (0.094) (0.383) (0.129) (0.112) (0.160) Threshold 2 2.024*** 2.071*** 1.386*** 2.585*** 2.385*** 0.404*** (0.205) (0.168) (0.140) (0.281) (0.268) (0.093) Observations 311 335 302 310 309 339 Robust standard errors, clustered at the school level, in parentheses *** p<0.01, ** p<0.05, * p<0.1 See the text for the definitions of the dependent variables. 14 Appendix: Parallel Trends Assumption Implies that E[ε0isT| T = 1, Ps = 1] = 0 Using (6), consider the four conditional means for the four possible combinations of T and Ps: E[Y0isT| Ps = 0, T = 0] = α + E[ε0isT| Ps = 0, T = 0] E[Y0isT| Ps = 0, T = 1] = α + β + E[ε0isT| Ps = 0, T = 1] E[Y0isT| Ps = 1, T = 0] = α + γ + E[ε0isT| Ps = 1, T = 0] E[Y0isT| Ps = 1, T = 1] = α + β + γ + E[ε0isT| Ps = 1, T = 1] Next, define α = E[Y0isT| Ps = 0, T = 0], define β so that α + β = E[Y0isT| Ps = 0, T = 1], and define γ so that α + β + γ = E[Y0isT| Ps = 1, T = 0]. Note that these definitions imply that the conditional means of ε0isT in the first three conditional expctations all equal 0. Applying the parallel trends assumption to equation (4) implies that E[ε0isT| Ps = 1, T = 1] = 0: E[Y0isT| Ps = 0, T = 1] - E[Y0isT| Ps = 0, T = 0] = E[Y0isT| Ps = 1, T = 1] - E[Y0isT| Ps = 1, T = 0] (4ʹ) α + β + E[ε0isT| Ps = 0, T = 1] - α - E[ε0isT| Ps = 0, T = 0] = α + β + γ + E[ε0isT| Ps = 1, T = 1] – α - γ - E[ε0isT| Ps = 0, T = 0] E[ε0isT| Ps = 0, T = 1] – E[ε0isT| Ps = 0, T = 0] = E[ε0isT| Ps = 1, T = 1] – E[ε0isT| Ps = 0, T = 0] 0 = E[ε0isT| Ps = 1, T = 1] Thus the expected value of ε0isT is 0 for all possible values of T and Ps, so ε0isT is uncorrelated with those two variables. 15 Appendix Tables Table A.1. Impact of PSI on Parent Involvement with Schools. (1) (2) (3) (4) Grade 4 Grade 8 VARIABLES School event Meet teacher School event Meet teacher Year = 2008 0.835*** 1.093*** 0.062 0.012 (0.044) (0.048) (0.042) (0.031) 2008×PSI 0.052 0.035 -0.064 0.028 (0.065) (0.071) (0.057) (0.039) Male 0.011 0.006 -0.026 -0.028* (0.017) (0.018) (0.021) (0.015) Sinhala -0.038 -0.029 -0.022 -0.018 (0.055) (0.056) (0.081) (0.073) Tamil 0.025 0.014 0.027 0.012 (0.056) (0.056) (0.111) (0.085) Income -0.001 -0.002 0.0048 -0.001 (0.009) (0.010) (0.009) (0.008) Mother’s educ. -0.006* -0.002 0.013*** 0.009*** (0.004) (0.003) (0.003) (0.003) Father’s educ. -0.007* 0.003 0.013*** 0.012*** (0.004) (0.004) (0.004) (0.003) Constant 1.739*** 1.467*** 2.219*** 1.960*** (0.058) (0.061) (0.088) (0.069) Observations 4,603 4,607 2,829 3,373 R-squared 0.390 0.510 0.020 0.016 Number of schools 196 196 158 158 Robust standard errors, clustered at the school level, in parentheses. *** p<0.01, ** p<0.05, * p<0.1 School event: How often parent participates in various events (Parent survey) Meet Teacher: How often parent meets with teacher (Parent Survey) For both variables, 3 = very often, 2 = once in a while, 1 = never. 16 Table A.2: Child and Parent Control Variables – Grade 4 Description 2006 2008 N Mean SD N Mean SD Gender Child’s report: 1=male, 0= female 2881 0.500 0.500 2687 0.505 0.500 Sinhala Child’s report:1=yes, 0= no 2859 0.721 0.449 2662 0.739 0.439 Tamil Child’s report:1=yes, 0= no 2859 0.153 0.360 2662 0.148 0.355 Other Ethnic Child’s report:1=yes, 0= no 2859 0.127 0.333 2662 0.113 0.317 Group Household Parent report of household income =1 if 2714 1.919 1.150 2346 2.507 1.316 Income1 less than 3000, =2 if between 3000- 5000, =3 if between 5001=10,000, =4 if between 10,001-20,000, =5 if between 20,001-30,000, =6 if above 30,000 Mother’s Categorical variable ranging from 2720 9.869 2.964 2233 9.399 3.120 Education2 discrete values of 1=15. 1= post graduate degree, 15=not attended school Father’s Categorical variable ranging from 2705 9.481 3.065 2360 9.646 2.977 Education2 discrete values of 1=15. 1= post graduate degree, 15=not attended school Educational Which of the following needs of your 2783 8.921 5.680 2817 9.357 7.126 Expenses1,3 child’s education do you spend for. 1= less than 500, 2=between 500-1000, 3= more than 1000. Notes: The sample is restricted to include only the schools that were in both rounds of data collection. 1 In 2008 parent questionnaire, data were collected from both mothers and fathers. If there are data from both parents, the household income and the educational expenditures data were taken as the average. 2 In 2008 parent questionnaire, data were collected from both mothers and fathers. If there are data from both parents, the father’s education was taken from the father’s data and the mother’s education was taken from the mother’s data. 3 The variable was created by summing across the following expenditure categories: school fees, transport, books, pens and pencils, school uniform, shoes, instruments, additional reading books, hostel fees, repair and maintenance of school buildings, library fees, sports equipment, sports functions and concerts, educational tours, societies, other. 17 Table A.3: Child and Parent Control Variables – Grade 8 Description 2006 2008 N Mean SD N Mean SD Gender Child’s report: 1=male, 0= female 2528 0.478 0.500 2322 0.453 0.498 Sinhala Child’s report:1=yes, 0= no 2522 0.721 0.449 2322 0.725 0.446 Tamil Child’s report:1=yes, 0= no 2522 0.194 0.395 2322 0.186 0.389 Other Ethnic Child’s report:1=yes, 0= no 2522 0.085 0.279 2322 0.089 0.284 Group Household Parent report of household income =1 if 2333 1.920 1.085 1349 2.659 1.318 Income1 less than 3000, =2 if between 3000- 5000, =3 if between 5001=10,000, =4 if between 10,001-20,000, =5 if between 20,001-30,000, =6 if above 30,000 Mother’s Categorical variable ranging from 2348 8.578 3.389 1323 9.487 3.210 Education2 discrete values of 1=15. 1= not attended school, 15=post graduate degree Father’s Categorical variable ranging from 2315 8.622 3.221 1334 9.223 3.180 Education2 discrete values of 1=15. 1= not attended school. 15=post graduate degree Educational Which of the following needs of your 2333 9.669 5.723 1372 14.496 5.282 Expenses1,3 child’s education do you spend for. 1= less than 500, 2=between 500-1000, 3= more than 1000. Notes: The sample is restricted to include only the schools that were in both rounds of data collection. 1 In 2008 parent questionnaire, data were collected from both mothers and fathers. If there are data from both parents, the household income data and the educational expenditures data were taken as the average. 2 In 2008 parent questionnaire, data were collected from both mothers and fathers. If there are data from both parents, the father’s education was taken from the father’s data and the mother’s education was taken from the mother’s data. 3 The variable was created by summing across the following expenditure categories: school fees, transport, books, pens and pencils, school uniform, shoes, instruments, additional reading books, hostel fees, repair and maintenance of school buildings, library fees, sports equipment, sports functions and concerts, educational tours, societies, other. 18 Table A.4: Teacher Variables1 – Grade 4 Description 2006 2008 N Mean SD N Mean SD Teacher Number of days teacher took leave in 191 23.558 24.253 191 16.419 19.062 Absence 2005 or 2007 for vacation, medical, maternity, no pay, other Homework Frequency that the teacher gives 174 2.799 0.443 175 2.743 0.464 homework to students =3 if always, =2 if seldom, =1 if once in a while Money Teacher’s opinion if money was 188 0.904 0.295 183 0.956 0.205 allocated for quality inputs for grade 4 students in 2005 or 2007 = 1 if yes, =0 if no Note: The sample is restricted to include only the schools that were in both rounds of data collection. 1 In 2006 two teachers from each schools were surveyed, the class teacher and the English teacher. These data were averaged across the two teachers in each school. In 2008, only the class teacher was surveyed. 19 Table A.5: Teacher Variables1 – Grade 8 Description 2006 2008 N Mean SD N Mean SD Teacher Number of days teacher took leave in 157 22.961 10.192 153 23.030 26.071 Absence 2005 or 2007 for vacation, medical, maternity, no pay, other Syllabus =3 if received in time, =2 if received 156 2.443 0.600 133 2.203 0.851 late, =1 if did not receive Class Text =3 if received in time, =2 if received 156 2.703 0.356 140 2.621 0.593 late, =1 if did not receive Teacher’s =3 if received in time, =2 if received 153 2.516 0.620 131 2.099 0.867 Guide late, =1 if did not receive Homework Frequency that the teacher gives 153 2.675 0.545 133 2.714 0.470 homework to students =3 if always, =2 if seldom, =1 if once in a while Money Teacher’s opinion if money was 157 0.931 0.189 141 0.965 0.186 allocated for quality inputs for grade 8 students in 2005 or 2007 = 1 if yes, =0 if no Note: The sample is restricted to include only the schools that were in both rounds of data collection. 1 In 2006 three teachers from each school were surveyed, one from each subject. These data were averaged across the three teachers in each school. In 2008, only the class teacher was surveyed. 20 Table A.6: Grade 4 and Grade 8 Principal Variables Description 2006 2008 N Mean SD N Mean SD Principal’s Management of Teachers Appraisal Do you have an appraisal system for 189 0.803 0.398 143 0.853 0.356 your teachers? =1 if yes, =0 if no Reward Do you have a system of rewards for 189 0.301 0.460 147 0.347 0.478 teachers? =1 if yes, =0 if no Self Evaluation Do you have a self evaluation scheme 189 0.577 0.495 142 0.606 0.490 for teachers? =1 if yes, =0 if no Activities Have you introduced any activities for 168 0.732 0.444 143 0.755 0.431 professional development of teachers? =1 if yes, =0 if no Review Do you review performance and 185 0.768 0.424 139 0.842 0.366 monitor progress of your school? =1 if yes, =0 if no Observe How often do you observe teaching =1 188 2.872 1.532 143 2.783 1.322 if daily, =2 if weekly, =3 if fortnightly, =4 if monthly, =5 if occasionally Principal’s Management of School Needs Project Did you undertake any project, 185 0.405 0.492 146 0.500 0.502 programmes or repair without financial assistance from the Central Government or Provincial Government in 2005 or 2007? =1 if yes, =0 if no Needs Analysis Have you done a need analysis for your 197 0.914 0.281 149 0.946 0.226 school? =1 if yes, =0 if no Priorities Have you prioritised the needs that you 197 0.914 0.281 148 0.912 0.284 have identified? =1 if yes, =0 if no Long Term Do you have a long term (2005) or 5 194 0.814 0.390 149 0.966 0.181 Plan year plan (2007)? =1 if yes, =0 if no 21 Formed School Have you formed a School 191 0.780 0.412 105 0.826 0.379 Development Development Committee in your Committee school? =1 if yes, =0 if no Financial Assistance Received From: Facility Fees Level received: 3=highest, 2=average, 1 177 1.4011 0.546 136 1.419 0.524 = not enough School Dev. Level received: 3=highest, 2=average, 189 1.354 0.511 148 1.426 0.561 Society 1 = not enough Past Pupil Level received: 3=highest, 2=average, 171 1.094 0.347 133 1.053 0.256 Assoc. 1 = not enough Other Level received: 3=highest, 2=average, 174 1.115 0.354 138 1.159 0.267 1 = not enough NGO Level received: 3=highest, 2=average, 1 175 1.183 0.416 156 1.110 0.314 = not enough State Level received: 3=highest, 2=average, 1 193 2.316 0.558 148 2.399 0.491 Assistance = not enough Note: The sample is restricted to include only the schools that were in both rounds of data collection for grade 4 and grade 8. 22