100255 AUTHOR ACCEPTED MANUSCRIPT FINAL PUBLICATION INFORMATION Class Size, School Size and the Size of the School Network The definitive version of the text was subsequently published in Education Economics, (forthcoming 2015), 2015-03-16 Published by Taylor and Francis and found at http://dx.doi.org/10.1080/09645292.2015.1015405 THE FINAL PUBLISHED VERSION OF THIS ARTICLE IS AVAILABLE ON THE PUBLISHER’S PLATFORM This Author Accepted Manuscript is copyrighted by the World Bank and published by Taylor and Francis. It is posted here by agreement between them. Changes resulting from the publishing process—such as editing, corrections, structural formatting, and other quality control mechanisms—may not be reflected in this version of the text. You may download, copy, and distribute this Author Accepted Manuscript for noncommercial purposes. Your license is limited by the following restrictions: (1) You may use this Author Accepted Manuscript for noncommercial purposes only under a CC BY-NC-ND 3.0 IGO license http://creativecommons.org/licenses/by-nc-nd/3.0/igo. (2) The integrity of the work and identification of the author, copyright owner, and publisher must be preserved in any copy. (3) You must attribute this Author Accepted Manuscript in the following format: This is an Author Accepted Manuscript of an Article by Coupé, Tom; Olefir, Anna; Alonso, Juan Diego Class Size, School Size and the Size of the School Network © World Bank, published in the Education Economics(forthcoming 2015) 2015-03-16 CC BY-NC-ND 3.0 IGO http://creativecommons.org/licenses/by-nc-nd/3.0/igo http:// dx.doi.org/10.1080/09645292.2015.1015405 © 2015 The World Bank Class Size, School Size and the School Network Tom Coupé* Anna Olefir** Juan Diego Alonso** Abstract In many transition countries, including Ukraine, decreases in population and fertility have led to substantial falls in the number of school-aged children. As a consequence, these countries now have school networks that consist of many small schools, leading many countries to consider reorganizing their school networks by closing smaller schools and reallocating students. Using a rich dataset of almost the entire population of Ukrainian secondary schools, we estimate the effect of school size and class size on the performance of secondary schools on Ukraine’s Independent External Test. We find that larger schools tend to have somewhat better performance, both in terms of test scores and in terms of test participation. The size of this effect is relatively small, however, especially in rural areas for which the estimates are likely to be less biased. Class size is found to be insignificant in most specifications and, if significant, of negligible size. The importance of this finding for the ‘rationalization’ of the large school networks in Eastern Europe are discussed. Keywords: school size, class size, economics of education, Ukraine, transition countries JEL Codes: I21, I28, P2 *Kyiv School of Economics ** World Bank Contact: Tom Coupe, 1 Ivana Mazepy Str., 01010 Kyiv, Ukraine. Tel 38 044 492 8012, fax 38 044 492 8011, tcoupe@kse.org.ua 1 I. Introduction The substantial positive returns to education for both individuals (e.g. Psacharopoulos and Patrinos, 2004) and countries (e.g. Barro and Lee, 2013) are a fairly established fact in the literature of the economics of education. It then comes as no surprise that policy makers and academicians are so interested in understanding the factors that can influence the quantity and quality of education. While the academic literature on these factors is vast, most of it focuses on developed countries, and while there is some literature on developing countries, only recently have studies using data from countries in Eastern Europe and the Former Soviet Union (henceforth, transition countries) started to appear. The recent surge in this literature can be explained by the fact that performance measures for these countries have become available, both through the participation of these countries in international assessments of student learning and through the introduction of centralized nation-wide assessments in several transition countries. In terms of the development of their educational systems, transition countries may be considered to lie somewhere between developed and developing countries. Transition countries typically have had mandatory basic education and enjoy very high levels of school attendance. As a result, the issue is not how to get children to attend school, as it is in a big part of the developing world. Instead, transition countries face the challenge of how to modernize and reform their educational system so as to provide quality education at an affordable price. One possible opportunity to realize such quality enhancing reforms might come from taking advantage of the so-called “demographic dividend,” i.e. the sizeable demographic changes that took place in most of the transition countries, especially in the last 20 years. In fact, after the fall of the Soviet Union, transition countries have faced an unprecedented demographic shock, with increasing mortality and emigration, but also with a serious drop in fertility. This negative shock to fertility has translated into an increasingly smaller number of school-aged children, considerably reducing school size and class size over time (Berryman, 2000). In addition, given that this drop in children of school age did not go together with a decline in the number of schools, teachers or classes, student-teacher ratios and the size of schools and classes has decreased substantially. As a consequence, transition countries are now in the situation where they have a disproportionately large number of schools, teachers and classes. 2 And this oversized system does not appear to have led to an improvement in the quality of education. Many transition governments are now facing the question of what to do with these oversized systems, often characterized by a multitude of very small schools and very small classes within schools. From a budgetary point of view, many governments would like to merge small schools, as the budgetary burden of one big school with many students is typically smaller than the budgetary burden of having a network of many small schools with few students in each. Similarly, consolidating classes within schools appears as an attractive option to generate some savings that could then, at least partially, be reinvested in increasing the skills and performance of the remaining schools, classes and teachers, presumably leading to quality improvements in the educational system. In several transition countries, this consolidation movement is already ongoing 1. Kuddo (2009) describes the process in Armenia, Kallai and Manui (2004) in Romania, Herrmann (2005) in Hungary, the World Bank (2010) in Bulgaria, McGuinness et al. (2001) in Estonia, Hazans (2010) in Latvia, and Berdashkevich and Vlasov (2010) in Russia. Optimization of the school network 2 is also among the key steps for reforms in the Ukrainian education sector according to the Presidential Economic Reforms Program for 2010-2014 ‘Prosperous society, competitive economy, effective government’ 3. Schools were planned to be closed if they: • have less than 10 students and only offer grades 1-4 (level I schools); • have less than 40 students and only offer grades 1-9 (level I-II schools); • have less than 100 students and offer all grades (1-11/12, level I-III schools). School size was thus proposed as the key decision criterion to close schools. Given that pupils of closed schools would be reallocated to other schools, the remaining schools would see an increase in size and most likely would have the size of their classes increase. 1 Throughout this document, the terms “optimization”, “consolidation” and “rationalization” are used interchangeably. In all cases, they refer to the territorial reorganization of the school network in a way that could better use the available resources in the system maximizing the installed capacity of current institutions. 2 We use the term “school network” as short for the "geographical networks of schools" 3 However, there are still signs of a push-back from the Government’s original initiative due to fierce local opposition to school closures. While many declarations thus have been made by the government, little implementation has followed. 3 In this paper, we attempt to foresee the impact of these plans on the quality of education by studying how school size and class size affect the performance of schools in Ukraine, using data on educational performance from Ukraine’s Independent External Test and data on educational inputs from the Ukrainian Ministry of Education, Youth, Sports and Science. In Section II, we first review the academic literature on the evidence that links class size and school size to educational outcomes. Section III portrays a picture of Ukraine’s educational system. Section IV provides a detailed description of the data used in this study and presents our estimates of class size and school size effects within the ‘educational production functions’ of Ukraine. Section V, finally, summarizes the main messages and policy conclusions from this empirical study. II. Literature Review There are several reviews of the literature on the determinants of educational performance, both internationally (Hanushek & Woessman, 2011) and focusing on developing countries (Glewwe (2002) and Glewwe & Kremer (2006)). Two general messages emerge clearly from these reviews. The first one is that methodologically, it is very hard to obtain pure unbiased estimates of the impact of specific determinants on education quality; most studies, including ours, are plagued by issues like selection effects and endogeneity, and experimental studies are rare. The second is that the vast number of studies on inputs gives a mixed picture, with many studies showing an impact but also many studies finding insignificant effects. This second conclusion suggests country-specific studies can be valuable. The literature on schools in transition countries is much less developed than the literature on either schools in developed countries or schools in developing countries. There is one cross country study that focuses on transition countries. Ammermuller et al. (2005) estimate an education production function for seven Eastern European countries (excluding Russia) using the TiMMS 2003 wave. Their study finds that class size has a positive influence on test scores, only if one does not control for selection effects. Once selection effects are controlled for the coefficient of class size becomes insignificant. Other included school characteristics, such as shortage of materials and measures of teacher and school autonomy were also found to be insignificant in most cases. 4 Single transition countries studies so far focus on the Central European countries with, as far as we could find, little evidence being available about countries which belong to the Commonwealth of Independent States (CIS) 4. Three studies use Polish data to study the determinants of educational performance. Herczynski and Herbst (2005) find a small positive effect of class size. Bukowska and Siwińska-Gorzelak (2011) find a significantly positive effect of school size in almost all their specifications. They also find a positive influence of school competition on students’ test scores. Finally, Jakubovsky and Sakowski (2006) find a small positive effect of school size and an inverse u-shaped effect of class size. Kallai and Maniu (2004) estimate educational production functions for Romania. They found a positive effect of school size but no effect of class size on students’ test scores. Porta (2011) using PISA data for Romania also finds that the smallest schools (first quintile) perform less well than bigger schools, but that the size of this effect is small and significant only for reading scores. A similar study for Serbia (MacDonald et al., 2009) found similar small but significant effects for reading, math and science scores. Hermann (2004) finds a negative effect of primary school size on the likelihood of entering a better secondary school in Hungary. Overall, the large international literature on the effects of class size and school size suggest that while class and school size might have an effect, these effects are unlikely to be large on average 5. For transition countries, school size appears to be more significant than class size as a determinant of educational performance and, increasing school size appears to be, if any, beneficial for educational performance. At the same time, the literature also suggests that effects vary from one country to another, pointing to the need for more country-specific studies. III. The Educational System of Ukraine - Some Background 4 These studies typically do not address the issue of selection across schools. Table A1 in the appendix gives more details on each study. The few studies on Ukraine are covered in the next section which focuses on the educational system in Ukraine 5 At the same time, some subgroups of students might be more sensitive to changes in class or school size than other subgroups. We do not have data at the student level but we do present results below for subgroups like male/female or for students in different parts of the performance distribution. 5 Due to the demographic crisis the number of students in Ukrainian schools fell by 40 percent since independence while the number of schools only fell by 7 percent and the number of teachers by only 4 percent 6. As a consequence, Ukraine is maintaining a large network of small schools that have become smaller and smaller because of the severe drop in birth rates: 30 percent of level I schools have less than 10 students; 21 percent of level I-II schools - less than 40 students; and 17 percent of level I-III schools - less than 100 students 7. Overall 19.2 percent of schools are small as defined above and the overwhelming majority are located in rural areas. As a result budget resources while increasing over time mainly support the large network of institutions rather than being directed towards quality-enhancing inputs 8. Figure 1: Trends in number of students teachers and schools in GSE in Ukraine 1990-2010 120 110 Index, 1990 = 100 100 90 80 70 60 50 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 schools teachers students Source: authors’ calculations based on data from the Ministry of Education Class-size has also been steadily decreasing and reached 18.1 students per class (23.2 – in urban areas and 12.5 – in rural areas). In this paper, we investigate school-size and class-size effects on school performance in Ukraine using schools’ graduates in Ukraine’s External Independent Test (EIT). The External Independent Test test became the key basis for entrance to higher education institutions in the country beginning in 2008. This test was introduced to reduce corruption in education, corruption that traditionally took place at the secondary schools where grades on 6 Around 510 thousand teachers work in the sector with 4.2 million students (compared to 537 thousand teachers and 7.1 million students in 1990/1991 school year). 7 World Bank calculations based on a school-level database compiled on the basis of information provided by each of the oblasts and related jurisdictions for the 2009/2010 school year. 8 See for details the Ukraine Public Finance Review (phase II) in FY 2007-08. Currently education sector expenditures account for almost a quarter of consolidated budget (24.8 percent in 2010) and 7.1 percent of GDP. The bulk of the consolidated budget is earmarked for salaries and utilities which are mainly intended to maintain budgetary institutions and thus crowd out other expenditures. 6 the local tests could be bought or at the university level where good grades on the entrance exams could be bought. The EIT is the only standardized nation-wide test of student’s knowledge in Ukraine. For a given subject, all students take the same test, a test that aims to measure general knowledge in a specific subject. Everything is done to ensure the reliability of the outcomes. Exams are developed, printed and corrected centrally, and distributed to exam centers only at the last moment. All students are given an ID number, which is indicated on the exam rather than the name. Initially 8 subjects were offered, with Ukrainian language being compulsory for all and students being allowed to take up to 5 subjects . In later years the number of subjects was changed (in 2010, Math, Ukrainian language, Ukrainian history, geography, Physics, Chemistry, Biology, Foreign language) and students also had to take either history or mathematics. The exam consists of multiple choice questions and test scores can vary between 100 and 200. The standard language for all exams is Ukrainian, but students can opt to get exams in minority languages such as Russian and Hungarian 9. It is important to note that this test only covers part of the graduates of secondary schools. • The EIT exams are only taken by upper secondary education graduates (grades 11-12), so we miss around 40 percent of lower secondary school graduates in urban areas and almost 50 percent in rural areas. Most of them leave for vocational schools as an ultimate goal or as the transit route to higher education institutions and thus omit the EIT exams. • the EIT exams are only taken by those students who are interested in pursuing studies at the university level. For example, in 2009 84 percent of graduates participated in EIT in Ukrainian language and literature and in 2010 – 78 percent. A smaller number of students participate in the EIT in Ukrainian history, math and English. • EIT is the main but not the only way for entering the higher education system. According to the Ministry of Education and Science of Ukraine in 2008 overall 91.2% of students were admitted to higher education institutions of ІІІ-ІV level of accreditation on the basis of EIT certificates. So far there has been little analysis of the EIT data. In addition to the simple descriptive statistics in the annual EIT report, there are three studies that use EIT data. Kovtunets et al. (undated) provide some analysis of whether the EIT exam results predict performance during 9 For a detailed discussion on the language issue see Muravyev and Talavera (2010) 7 the first year at university. Muravyev and Talavera (2010) use the EIT to see how an announced (but not implemented) language policy change has affected the subject choice and performance of students at minority language schools. The closest to the current paper is Coupé et al. (2010) who use a sample of 300 schools to link EIT scores to a wide set of input variables. They find a positive effect of school size and no effect of class size. The current paper extends that analysis by drawing on a much bigger sample of thousands of Ukrainian schools. The size of the current dataset not only provides for more precise estimates but also allows for the analysis of more precise questions, like an analysis by gender or by type. In addition, the current dataset allows us to analyze the Ukrainian plan to close schools with less than 100 students. IV. Data In this paper, we use several datasets that were matched and combined into one comprehensive dataset. From the Ministry of Education, we obtained data for all Ukrainian schools, on the total number of students, the number of students by grade, the number of classes by grade and the total number of teachers and staff. Data about performance on the EIT were obtained from the yearly public data files posted on the EIT site that provide for each school the distribution of students’ scores over 10 intervals. In addition, from the Ukrainian Centre for Education Quality Monitoring we obtained mean and median scores data and information about the gender and the language choice of students 10. A more detailed description of these sources and the matching process can be found in the Coupe et al. (2011). For 11,683 Ukrainian schools we have information on both 2010 EIT test scores (from the EIT database) and information on input variables (from the Ministry of Education) 11. 10 The correspondence between those two sources of data was high but not perfect. For a few schools, we had no mean scores and for a somewhat greater number of schools the total number of students according to the two databases was not equal, though such deviations were typically very small. One possible explanation for these differences is that the database of the Ministry’s EIT Center is updated, f.e. correcting mistakes or including the results of appeals. 11 The descriptive statistics for 2008 and 2009 give a qualitatively similar picture. Note that in this section we will only look at differences in means between groups, delaying the question of significance to the regression analysis. 8 Given that our main variables of interest, school size and class size, but also other input variables are very different depending on whether or not the school is located in an urban or rural area, we provide the descriptive statistics for rural and urban schools separately. For the descriptive statistics we also focus on the most common type of schools, the 10,361 ‘ordinary’ schools 12, to avoid mixing the effect of size and school type. We further restrict our sample to those ordinary schools for which we have a complete set of ‘credible’ information, that is, they have scores on the Ukrainian language exam, do have teachers and students, have a student teacher ratio less than 50 and a class size less than 35 13. To illustrate the effect of school size and class size we divide the samples of rural and urban schools into four intervals, each interval having approximately about a quarter of the observations. School size is measured by the total number of students at the school with class size measured by the number of students in the final grade divided by the number of final grade classes (for a vast majority of schools this is the 11th grade, for some it is the 12th grade). We use several indicators of school performance • the mean score of students of a given school on the test of a specific subject • the median score of students of a given school on the test of a specific subject • the percentage of students of a given school scoring above 173 • the percentage of students of a given school scoring above 150 • the percentage of students of a given school scoring below 135.5 • we also compute a ‘student participation ratio,’ that is, for those schools that have students who take the exam on a specific subject, we compute the ratio of students who take the exam divided by the total number of students in the highest grade 14. We also provide summary statistics for the explanatory variables we will use 15. We have two more measures of school inputs. 12 We exclude ‘ordinary’ evening schools. For the regression analysis below, however, we include all types of schools, controlling for differences through type-specific dummies. 13 We lose about 5 % of the schools by doing this, mainly rural schools. 14 We have this indicator for those schools that we were able to match to the MoE database. For some schools, we clearly have unrealistic participation ratios. We keep them in the sample here but will exclude these schools in further analysis – see below for details. 9 • Student teacher ratio – the total number of students in the school divided by the total number of teachers in the school (these numbers are based on headcounts, not full time equivalents). • Student staff ratio - the total number of students in the school divided by the total number of teachers and non-teaching staff in the school (these numbers are based on headcounts, not full time equivalents). We also have two measures of the group composition of the exam takers of each school. • Language – the percentage of EIT exams (other than the Ukrainian language exam) taken in Ukrainian. Note that this reflects the language choice of the students taking the test and hence can differ from the language choice of the whole class. • Gender Composition – the share of boys among those taking the EIT exam. As not all students take the exam, this can differ from the share of boys in the class. And we have two measures of the economic situation in the area (rayon) where the school is located. • The registered unemployment rate at the end of 2009. • The average monthly wage in Ukrainian Hryvnia, the local currency, at the end of 2009. Finally, we have one measure at the level of the village where the school is located, which controls for the possibility that bigger villages can have both bigger schools (because they have a bigger population) and better schools (because better teachers or ‘better’ parents might move to bigger villages with more amenities). • The population in the village where the school is located based on the 2001 population survey. For the big cities, we typically have population data for the city district where the school is located. [Table 1a and 1b around here] 15 In a companion paper (Coupe et al., 2010), we did a more detailed analysis on a smaller sample of about 300 schools for which we had more background variables. Background variables included measures of the extent of children from problem families at the school; the extent of the library and the IT infrastructure of the school; the age, experience and training of the principal; and indicators of the self-assessed autonomy of the school (in Ukraine, the educational system is highly centralized leaving little room to the schools). Including these variables did not lead to conclusions different from the ones reported in this paper and many of these additional variables did not turn out to have a significant effect. 10 Based on the descriptive statistics in tables 1a and 1b, we find that the size distribution of urban and rural schools overlap little. For rural areas, the school size quartile thresholds are at 100 students, 141 students and 209 students while for the urban schools, the thresholds are at 305 students, 458 students and 646 students. Hence, the lowest size quartile for urban schools has an upper limit that is higher than the lower limit of the highest size quartile of rural schools. In rural areas, the relation between size and Ukrainian language exam scores is limited. Bigger schools have a slightly higher percentage of students among the top students, but also a somewhat higher percentage of low scoring students. The participation ratio, however, is somewhat lower in the lowest quartile. While performance does not change much as size increases in rural areas, input variables do change a lot: the schools in the top quartile have, on average, a class size that is about the double the class size in schools in the lowest quartile. The same is true of the student teacher ratio. In the cities, the relation between size and test scores is much clearer, with bigger size goes together with substantially better mean and median test scores, a higher percentage of high scoring students and a lower percentage of low scoring students. For example, while the lowest urban size quartile has an average mean score of 147.5 and 44.75 percent students scoring ‘above average’, the top quartile has an average mean score of 154.75 and 60 percent students scoring ‘above average’. In addition, schools in the upper quartiles have substantially higher participation rates than schools in the lower quartiles. Besides performance increasing with the size of the school in urban areas, also the class size and the student teacher ratio increase with the size of the school. Given that bigger schools in urban areas perform better, on average, than smaller schools in urban areas, and that, on average, schools in rural areas are smaller than schools in urban areas, it is no surprise that schools in rural areas perform worse, on average, than schools in urban areas. The participation ratio is also substantially smaller in rural areas. Note that if we would focus on performance in Mathematics or Ukrainian history, rather than on the Ukrainian test, we get similar results. 11 As explained in the introduction, Ukraine plans to close schools with less than 100 students 16. About a quarter of rural areas fall in this category, indeed, the upper bound of the bottom quartile of school size in our sample happens to be 100. The above results suggest that students at these smaller schools are not underperforming in test scores (nor are they over- performing), though they do require substantially more inputs and have somewhat smaller participation rates. Very few schools in urban areas have less than 100 students (about 1 percent of the urban sample). The descriptive statistics above, however, suggest that in urban areas increasing school size can increase both quality and save on inputs. This is unlike the descriptive statistics for the rural areas, which suggest that inputs can be saved (at a higher rate than in urban areas) by having bigger schools but also, that quality would not change much. Of course, none of the above results control for other explanatory variables that can influence the school’s performance measures. The regression analysis presented below will analyze whether these preliminary findings are confirmed once one controls for such confounding factors. V. Analysis a. Do school and class size affect participation in higher education? Only those students interested in continuing their studies at a university within Ukraine have an interest in taking the Independent External Test, hence the school averages we have are based on a selected group of students rather than all students graduating in a given year. Moreover, while students take the decision to participate or not themselves, schools have a possibility to try to influence this choice. For example, if a school would like to get a good average score on the EIT, it can try to persuade weaker students not to take the test. To study the degree to which there is selection, we use the ‘student participation ratio’, that is, for those schools that have students that take the exam on a specific subject, we compute the ratio of students who take the exam divided by the total number of students in the highest grade (for a vast majority of schools this is the 11th grade, for some it is the 12th grade). Given that all students that participate in the EIT take the Ukrainian language exam, a school’s ratio of the number of students that take the Ukrainian language exam on the number V. 16 Given our performance measure is based on a test taken by school graduates, we do not have schools of level I or level I-II in our sample. 12 of students in the highest year is a good proxy for the extent of selection. If this ratio is 1, it means all students participate; if it is 0 it means nobody participates 17. Table 2 tries to explain the differences between schools in terms of the 2010 participation ratio. We first run a simple OLS regression (1) of the participation ratio on our main variables of interest and a set of dummies reflecting the settlement status (urban versus rural), the oblast in which the school is located and the type of school. Then (2) we add a set of additional explanatory variables including the gender composition of the school’s EIT participants, their language choice, the school’s location population in 2001, and the average wage and the unemployment rate in the rayon where the school is located in 2009. Next (3) we allow for non-linear effects in our main input variables, adding squared terms of the total number of students, the student teacher ratio at the school and the class size of the graduating class. Finally, we use Tobit regression (4) to control for the fact that the participation ratio is constrained to be between zero and one. We run these four regressions first on the total sample and then we run separate regressions for urban and rural schools. We find that school size is positively correlated with the participation-ratio in both rural and urban areas, though the effect is somewhat smaller in rural areas. When allowing for a non- linear effect of school size (specification 3), we find, based on the complete sample, that the maximum participation ratio is reached at 1,156 students, which is close to the 98th percentile of school sizes in our sample. A school at this level has a participation ratio that is roughly 7 percentage points higher than a school of 455 students (75th percentile), 11 percentage point more than a school at the median (227) and 14 percentage point more than a school of 128 students ( the 25th percentile). Note that we get these effects after correcting for a wide range of other factors that can affect a school’s participation ratio. When restricting the sample to urban schools, we get the maximum at 1,060 students (95th percentile). A school at this level can expect a participation ratio that is 3, 6 and 10 percentage point higher than a school with 666, 465 and 305 students respectively. 17 In about 7 % of the cases, the participation ratio indicator is bigger than one, i.e. there are more students taking the test than there are in the 11th/12th grade. One explanation for this is that the MoE data were recorded at a different time than the EIT exams, and that some students might have changed schools between the recording of the MoE data and the EIT exams. A second possible explanation is that some graduates of previous years were allocated to the school where they graduated in the past. A third explanation is simply that there are mistakes in the database. In the regression analysis presented below, we include in the sample only schools with a participation ratio that is less than 101 percent. 13 When restricting the sample to rural schools, we get the maximum at 470 students (between 95th and 99th percentile). A school at this level can expect a participation ratio that is 3.5, 5.7 and 7 percentage point higher than a school with 208, 141 and 100 students respectively. The impact of class size varies from one specification to another, both in terms of significance and in terms of size. In any case, even if significant, the size of the effect is very small. We also find that the student teacher ratio has a positive effect in all specifications, indicating that schools with a higher number of students per teacher have higher participation ratios. However, this effect is small in general and even insignificant for the urban schools. In rural schools, we find some evidence of a non-linear effect, with the optimal student teacher ratio being around the 75th percentile at 8 students per teacher. But again, deviating from that optimum barely reduces the participation ratio. As far as other variables are concerned we find that evening schools have low participation ratios (70 percentage points less than the average school) and that urban schools have substantially higher participation rates ( about 8 percentage points). Also the type of school matters: lyceums, colegiums and gymnasiums have significantly higher participation ratios (5 to 10 percentage points higher than ordinary schools). Schools located in rayons with higher registered unemployment have somewhat higher participation ratios as do schools located in rayons with higher wages and as do rural schools located in bigger villages. Interesting to note is the finding that schools that have a high percentage of males among the EIT participants are also schools that have a somewhat higher participation ratio. Note that this gender effect is much stronger than the language effect – having a higher percentage of students choosing Ukrainian as test language participate in the EIT goes together with only slightly more students participating and this effect is not statistically significant. This suggests that the Ukrainian language test does not scare off those more comfortable speaking Russian. So far we have discussed the selection of students in the test. One additional selection issue is related to the selection of students into schools. Indeed, it is possible that better students prefer to study at the bigger (or smaller) schools or that better students are put into smaller (bigger) classes. In other words, correlation does not necessarily mean causality. Like Urquiola (2006) noticed for Bolivia, also in rural Ukraine there is very often only one school 14 with one class per grade, which reduces the possibility of students to select into schools, and of schools to select students into classes. Hence, our estimates for rural areas should provide ‘cleaner’ estimates of the effect of school size and class size on performance. Still, one should be careful when interpreting our results in a causal way. b. Estimating Educational Production Functions We next run regressions using the same set of explanatory variables but with the mean test score of the school as a dependent variable. Since our dependent variable has no observations at the limit, there is no need for a Tobit regression, instead we add the participation ratio in specification (4) to control for possible selection effects. We focus first on the score on the Ukrainian language exam. We find that (table 3) school size has a significantly positive effect on the school’s mean Ukrainian language exam score in all specifications, though the effect is clearly stronger for urban schools. Using the non-linear OLS specification, an urban school at the 25th percentile of the school size distribution has an expected mean score that is 2 points less than a school at the 50th percentile, and 4 points less than a school at the 75th percentile. For rural schools, the difference between the 25th and the 75th percentile is only 1 point. Class size has a negative effect in all but one specification and this effect is statistically significant. However, the size of these effects is small, moving from the 25th to 75th percentile does not change a mean by more than 0.5 points, both in rural and urban areas. We further find that student teacher ratio has a consistently negative effect but when disaggregating the sample into urban and rural schools, this effect is not significant. Gymnasia and Lycea score 5 to 10 points higher than other school types. Evening schools perform substantially less well (10 or more points less), urban schools do better than rural schools (about 2 points more) and there are substantial oblast specific effects. While introducing extra explanatory variables does reduce the importance of oblast specific dummies, some sizeable effects do remain. For example, the average score in urban schools in Lviv is expected to be 7 points more than the average score of schools located in Crimea. For rural schools this difference is 3 points. Gender composition matters with more boys among the test takers reducing the average school score. Going from the 25th percentile to the 75th percentile in terms of share of boys will decrease the average score by 1.5 to 2.5 points. 15 The unemployment rate is never significant but a higher average wage goes together with a slight decrease in points. Schools in bigger (more populated) communities have slightly higher mean scores. The language preference of students (measured by the percentage of EIT exams, other than the Ukrainian language exam, taken in Ukrainian rather than any of the minority languages) has a substantial effect on the performance on the Ukrainian language exam – moving from a school where about 40% of the exams are taken in a language other than Ukrainian (25th percentile) to a school where all students take all exams in Ukrainian (75th percentile) increases the average score on the Ukrainian language exam by about 4 points. Controlling for selection by including the participation ratio does not change the results much. For example, an urban school at the 25th percentile of the school size distribution has an expected mean score that is 1.5 points less than a school at the 50th percentile, and 3 points less than a school at the 75th percentile. For rural schools, the difference between the 25th and the 75th percentile is less than 1 point. The participation ratio itself has a positive and significant effect on the average score, especially for urban areas but also for rural areas 18. This suggests that better schools have higher grades and higher participation. Finally, our explanatory variables explain about 40 percent of the variation in mean scores for urban schools, but substantially less (15 percent) for rural schools. Robustness Checks 19 Besides data on the score on the Ukrainian language test (the test that is taken by almost all students), we also have data for the mathematics and the history scores. By and large, the results for the mathematics and history tests are similar to the results for the Ukrainian test. Bigger schools get better average scores while schools with higher student teacher ratios and higher class sizes get lower scores, though the latter effect is small and often not significant. From the EIT center, we also obtained school averages disaggregated by gender, allowing us to check whether different ‘types’ of students are affected differently by school size. Note that, based on descriptive statistics, girls do better than boys, on average, on all subjects, 18 One could argue that selection and performance are jointly determined and hence it would be better to estimate these variables jointly in a system of equations. Unfortunately, we do not have instruments to identify such system. 19 Table 6 gives an overview of the results for the main variables of interest – complete tables for these robustness checks can be found in Coupe et al. (2011). 16 except for Geography, with the gender gap to be most substantial on the Ukrainian language exam. When running regressions by gender, we use the average score of the female students, respectively male students, of a school and regress it on the same set of explanatory variables, with the exception that our language variable is now the percentage of exams (other than the Ukrainian language exam) taken in Ukrainian by the female students of a school rather than by all students. Using the Ukrainian language test scores, like for the total sample, school size has a positive effect on the school’s mean Ukrainian language exam score for both males and female students. Again, the effect is clearly stronger for urban schools. Class size has a negative effect for male and female students and this effect is statistically significant for male students and almost significant for female students. The size of these effects is bigger for males than for females but in both cases moving from the 25th to 75th percentile does not change the mean by more than 1 point. Student teacher ratio has a consistently negative effect but like for the whole sample, for females when disaggregating the sample into urban and rural schools, this effect is not significant. For boys there is a significant negative non-linear effect, but the size of this effect is very small. Gender composition matters little for girls but is significant for boys: more boys among the test takers reduces the average school scores of boys somewhat: going from the 25th percentile to the 75th percentile in terms of share of boys will decrease the average score by 0.3 to 0.6 points. One potential reason for this finding could be that boys are more disruptive than girls, and when disruptive target other boys more than they target girls. The fact that we found a negative effect of the share of boys on the school score when using the total school average (girls and boys combined) is thus mainly due to the fact that girls on average score better, but not (or only marginally so) because mixed gender classes have a negative effect on class performance. So far we focused on the mean score. However, we also have data on the percentage of students in different parts of the distribution. We therefor also ran regressions with the percentage of students scoring above 173 and the percentage of students scoring below 135.5. Overall, the results confirmed the findings we made so far (even though, as expected, the signs switch for the regression where we use the bottom students). One exception is that the 17 student teacher ratio has, for urban schools, a consistently negative effect on the percentage of students above 173. Given the low variation in that ratio, however, going from the 25th percentile to the 75th percentile, does not change the percentage of top scoring students by more than 1 percentage point. In addition to the data for the 2010 EIT, we also have data for the 2008 and 2009 EIT. Given that we have 3 years of data for both inputs and outputs one could argue we can use panel data techniques to estimate the determinants of educational performance in Ukraine. However, yearly test score data can be quite crude measures of educational performance. To illustrate this, we follow Kane and Staiger (2002) and look to what extent changes in scores are correlated. In case changes are purely random, one would expect increases in one year to be followed by decreases in the next year and the correlation would be -0.5. Kane and Staiger (2002) find a correlation of -0.37 for the US data. In our sample of Ukrainian schools, we find -0.465. This suggests that 93% (0.465/0.5) of the changes from one year to another are transitory rather than permanent 20. The latter implies that by averaging out yearly variations and taking averages over longer periods one will get closer to the stable underlying school quality variable 21. We therefore ran regressions using weighted averages of the mean score over the three EITs, 2008-2009-2010, using the students participating in each year as weights. We use these weights both for the dependent and the explanatory variables 22. Given that we take out part of the random variation by taking weighted averages, it comes as no surprise that the explanatory power of our regressions increases, adding roughly 10 percentage points to the adjusted R2. In general, the results are similar to what we found when just using 2010 data, albeit that the sizes of the class and school size effects are slightly smaller. VI. Conclusions 20 A similar sized correlation is obtained when restricting the sample to the bigger schools (above median number of students taking the Ukrainian test) 21 If instead we run Fixed effects regressions we, not surprisingly, get imprecisely estimated coefficients and signs that differ across specifications. 22 An implicit assumption here is that the production functions are stable over time- this is likely to be a harmless assumption, even though there were slight changes in the organizational procedure of the tests ( f.e. some very specific groups of students were not required to participate) from year to year. 18 The empirical analysis undertaken in this study points to the fact that the size of schools and classes does not appear to be very important for the educational quality of secondary schools in Ukraine, a finding that is broadly in line with what has been found for other countries and hence that there is little evidence for a trade-off between efficiency and quality. In the case of Ukraine, if anything, our findings suggest that school size has a small positive effect and that class size has no significant effect on school performance. Table 4a and 4b summarize the effects of class size and school size across specifications. Our results show that bigger schools tend to have higher participation ratios and tend to have higher mean test scores, more students among the top students and less students among the bottom students. This is especially true for urban schools, where the difference between the 25th percentile (about 300 students) and the 75th percentile (over 600 students) of the school size distribution is about 4 test score points and about 7 percentage points in terms of participation ratio. For the rural schools, the effect of moving from the 25th percentile (about 100 students) and the 75th percentile (over 200 students) of the school size distribution is substantially smaller, at roughly half the values found for urban schools. At the same time, the rural area estimates are likely to be less affected by a possible endogeneity bias. Since increasing the size of schools, on average, will result in an increase in the size of classes, we could wonder what would happen with performance. Indeed, not much. Going from the 25th percentile to the 75th percentile of the class size distribution does not affect the performance much, relative to going from the 25th percentile to the 75th percentile of the school size distribution – often the difference is even insignificant and sometimes the difference is even positive. Our estimates also imply that there seems to be an ‘optimal’ size, a point after which further increasing school size goes together with lower mean scores. Our estimates of that point (which varies from about 400 for rural schools to over 1000 for urban schools), however, show that few Ukrainian schools already have reached that point. Our estimated optimal point is also substantially higher than 100 students suggesting that the 100-student cut-off point used by the Ukrainian government for ordering the optimization of complete schools (levels I-III) may be very conservative, especially for urban schools, almost none of which are below this threshold and for which the estimated optimal point is often over 1000. This suggests there is a lot of room for consolidation. 19 From a policy point of view, our findings suggest that consolidating the network of schools through a downsizing in the number of small schools and a transfer process of students in those schools to neighboring ones is unlikely to harm school performance on the EIT. If anything it may even increase educational outcomes. Of course, an important caveat here is that this “no harm conclusion” is only likely to hold if the organization of the transfer and of the commuting to the receiving schools is organized efficiently 23. While our results are suggestive, they further need to be interpreted with caution as we have not been able to investigate in detail to what extent our findings reflect causal relationships rather than correlation. Nor can we claim that these empirical findings can be extended to schools other than complete schools (level I-III), since the analysis was undertaken with measures of school performance for 12th-graders. Both these areas remain avenues for further research as well as the analysis of the student-level EIT data as opposed to the school-level analysis presented here 24. 23 Of course, students of closed schools will incur additional transfer costs related to the longer commute 24 In a first attempt, Thoryk (2011) uses 2007 TIMSS data to estimate the effect of school size on school performance in transition countries. Overall his country specific estimates suggest there is little relation between school size and performance. His results for Ukraine suggest an insignificant effect of size on 4th grade science and 8th grade mathematics, an inverted U shape for 4th year math and a U-shaped relationship for 8th grade science. While the latter two results are significant, the size of the effect is found to be negligible. 20 Annex I - Tables Table 1a. Descriptive statistics for rural schools, by school size quartiles Rural Schools First Quartile Second Quartile Third Quartile Fourth Quartile # Mean Median # Mean Median # Mean Median # Mean Median Mean Score 1529 148.83 148.80 1495 148.28 148.40 1514 148.65 148.50 1498 147.68 147.80 Median Score 1529 146.98 147.00 1495 147.12 147.00 1514 147.77 148.00 1498 147.36 148.00 Percentage >173 1529 8.97 0.00 1495 9.08 0.00 1514 10.02 7.14 1498 10.12 7.69 Percentage >150 1529 48.41 50.00 1495 46.71 46.15 1514 48.23 50.00 1498 45.69 46.15 Percentage<135.5 1529 23.60 20.00 1495 25.36 23.81 1514 24.92 23.08 1498 27.79 25.93 Participation Ratio 1529 69.85 70.00 1495 73.91 75.00 1514 73.30 75.00 1498 74.57 74.46 Exams Per Student 1529 3.21 3.17 1495 3.22 3.18 1514 3.26 3.22 1498 3.31 3.29 Nr of Subjects 1529 5.21 5.00 1495 5.70 6.00 1514 6.12 6.00 1498 6.99 7.00 # Students 1529 77.22 80.00 1495 120.06 119.00 1514 171.27 169.00 1498 326.27 288.00 # Teachers 1529 17.01 17.00 1495 20.31 20.00 1514 23.13 22.00 1498 36.02 34.00 # Non-Teaching Staff 1529 10.77 10.00 1495 12.79 12.00 1514 14.57 14.00 1498 19.66 18.00 Students/Teacher 1529 4.59 4.57 1495 6.07 6.00 1514 7.61 7.48 1498 9.17 9.00 Students/Staff 1529 2.82 2.81 1495 3.71 3.67 1514 4.66 4.60 1498 5.88 5.78 Class Size 1529 8.70 8.00 1495 11.62 11.00 1514 14.76 14.00 1498 18.85 18.50 Share Ukrainian 1529 0.93 1.00 1495 0.92 1.00 1514 0.89 1.00 1498 0.85 1.00 Share of Males 1529 43.60 44.44 1495 43.66 42.86 1514 43.57 43.75 1498 41.77 41.67 Unemployment Rate in Rayon 1529 3.45 3.20 1495 3.24 2.90 1514 2.79 2.50 1498 2.34 2.00 Average Wage in Rayon 1529 1389.90 1330.00 1495 1390.30 1323.00 1514 1417.64 1338.00 1498 1457.92 1353.39 Population in 2001 1529 754.39 704.00 1495 1030.68 960.00 1514 1523.92 1350.00 1498 2718.11 2343.00 21 Table 1b. Descriptive statistics for urban schools, by school size quartiles Urban Schools First Quartile Second Quartile Third Quartile Fourth Quartile # Mean Median # Mean Median # Mean Median # Mean Median Mean Score 994 147.50 147.00 976 150.32 150.10 983 152.28 152.10 980 154.75 154.50 Median Score 994 146.75 146.00 976 150.00 149.50 983 152.35 152.00 980 155.18 155.00 Percentage >173 994 11.05 7.14 976 13.29 10.53 983 15.59 13.51 980 18.89 16.42 Percentage >150 994 44.75 43.48 976 50.47 50.00 983 54.68 54.29 980 60.06 60.00 Percentage<135.5 994 28.77 26.67 976 23.82 21.74 983 20.48 19.05 980 17.43 16.28 Participation Ratio 994 83.27 82.35 976 87.95 88.89 983 92.06 91.43 980 96.64 93.17 Exams Per Student 994 3.29 3.25 976 3.36 3.33 983 3.42 3.39 980 3.47 3.46 Nr of Subjects 994 6.62 7.00 976 7.48 8.00 983 7.85 8.00 980 8.04 8.00 # Students 994 213.67 222.00 976 378.99 380.00 983 544.26 542.00 980 855.35 802.00 # Teachers 994 25.47 24.00 976 36.90 36.00 983 48.30 47.00 980 69.40 66.00 # Non-Teaching Staff 994 16.74 15.00 976 19.92 19.00 983 22.63 22.00 980 28.99 27.00 Students/Teacher 994 8.66 8.81 976 10.64 10.56 983 11.64 11.53 980 12.59 12.55 Students/Staff 994 5.29 5.40 976 6.91 6.86 983 7.88 7.79 980 8.86 8.75 Class Size 994 17.07 17.00 976 21.04 20.50 983 22.34 22.00 980 24.00 24.00 Share Ukrainian 994 0.68 0.97 976 0.69 0.97 983 0.72 0.98 980 0.74 0.98 Share of Males 994 43.23 43.61 976 43.16 43.75 983 43.76 43.90 980 43.79 44.19 Unemployment Rate in Rayon 994 1.99 1.65 976 2.11 1.70 983 1.99 1.70 980 1.72 1.50 Average Wage in Rayon 994 1733.38 1707.00 976 1755.53 1679.00 983 1771.25 1732.60 980 1851.47 1795.00 Population in 2001 994 63189.65 32737.00 976 75172.65 49938.00 983 80115.54 52265.00 980 112160 103244 22 Table 2: Ukrainian Language, Participation Ratio Total Urban Rural (1) (2) (3) (4) (1) (2) (3) (4) (1) (2) (3) (4) # students 0.014*** 0.013*** 0.031*** 0.031*** 0.014** 0.014*** 0.038*** 0.038*** 0.010** 0.008** 0.048*** 0.053*** * * 16.05 15.35 11.22 8.66 16.61 16.3 12.55 11.87 2.66 1.97 4.34 4.51 2 (# students) -0.000*** -0.000*** -0.000*** -0.000*** -0.000*** -0.000*** -7.8 -5.49 -9.57 -8.33 -3.64 -4.07 Student/Teacher 0.173* 0.204** 0.626** 0.737*** 0.108 0.115 0.455 0.502* 0.565** 0.571*** 1.348** 1.478** * 1.83 2.15 2.4 2.83 1.09 1.15 1.44 1.8 2.68 2.72 2.03 2.41 2 (Stud./Teacher) -0.031*** -0.034*** -0.022** -0.023** -0.082** -0.088*** -2.94 -3.28 -2.13 -2.35 -2.2 -2.69 Class Size 0.017 0.016 0.057 -0.247* 0.147** 0.156*** -0.169 -0.444** -0.048 -0.059 0.727*** 0.398* * 0.49 0.45 0.39 -1.79 3.32 3.38 -0.59 -2.23 -0.85 -1.04 3.27 1.78 2 (Class Size) -0.003 0.003 0.006 0.012** -0.028*** -0.021*** -0.92 0.85 0.91 2.54 -4.29 -3.16 urban==1 8.773*** 8.529*** 7.295*** 8.164*** 17.39 16.81 13.33 13.41 Gender Comp. 0.089*** 0.089*** 0.093*** 0.041* 0.042* 0.039** 0.102*** 0.102*** 0.108*** 6.54 6.62 8.91 1.76 1.82 2.46 6.24 6.32 7.81 Unemployment rate 0.544*** 0.561*** 0.602*** 0.384** 0.285 0.308 0.555*** 0.628*** 0.688*** 2009 4.03 4.16 4.2 2.15 1.6 1.58 3.02 3.44 3.4 Average Wage 2009 0.002*** 0.002*** 0.002*** 0 0 0 0.003*** 0.003*** 0.004*** 3.34 3.54 2.66 0.02 0.24 0.09 3.85 3.88 2.99 Population 0.003 0.008** 0.010** 0.001 0.004 0.005 0.310* 0.334** 0.348*** 1.14 2.54 2.26 0.45 1.13 1.43 1.95 2.29 3.48 Share Ukrainian 0.016* 0.017** 0.021** 0.014 0.014 0.019* 0.016 0.018 0.017 1.87 1.99 2.3 1.26 1.27 1.93 1.15 1.25 1.19 _cons 75.101** 67.172*** 63.185*** 68.108*** 82.443* 79.378*** 76.311*** 81.290*** 70.735* 59.646** 47.878*** 50.930*** * ** ** * 63.76 43.18 31.22 30.41 52.81 37.89 22.19 28.73 23.71 18.3 12.1 9.75 23 sigma 18.866*** 14.110*** 21.698*** 134.82 90.1 100.17 R Adj sq. 0.309 0.316 0.32 0.447 0.448 0.459 0.083 0.098 0.107 N 10906 10906 10906 10906 4922 4922 4922 4922 5984 5984 5984 5984 The regression also includes oblast dummies, dummies for different types of schools and evening schools. The full table can be found in excel tables which can be made available upon request. *** means significant at the 1% level, ** at the 5 % level and * at the 10 percent level.T statistics are based on robust standard errors for the OLS specifications. Table 3 – Ukrainian Language, Mean Score Total Urban Rural (2) (2) (2) (1) (3) (4) (1) (3) (4) (1) (3) (4) Extended Extended Extended # students 0.009*** 0.009*** 0.011*** 0.009*** 0.009*** 0.009*** 0.016*** 0.009*** 0.002 0.001 0.013*** 0.012*** 19.21 18.5 7.9 6.46 16.83 17.05 9.41 5.6 1.18 0.86 2.98 2.7 2 (# students) -0.000** 0 -0.000*** -0.000* -0.000*** -0.000** -2 -0.96 -4.72 -1.76 -2.76 -2.56 Student/Teacher -0.183*** -0.178*** -0.318*** -0.364*** -0.05 -0.087 -0.216 -0.298* -0.056 -0.047 -0.274 -0.317 -3.76 -3.84 -2.74 -3.16 -0.78 -1.43 -1.21 -1.72 -0.66 -0.59 -1.2 -1.4 2 (Stud./Teacher) 0.007* 0.009** 0.004 0.008 0.006 0.009 1.66 2.21 0.73 1.47 0.56 0.8 Class Size -0.081*** -0.068*** -0.347*** -0.351*** -0.003 0.015 -0.443*** -0.413*** -0.052** -0.057** -0.16 -0.183* -4.6 -3.95 -5.08 -5.17 -0.11 0.55 -3.18 -3.06 -2.08 -2.36 -1.64 -1.88 2 (Class Size) 0.008*** 0.008*** 0.011*** 0.010*** 0.003 0.004 4.58 4.78 3.48 3.26 1.02 1.34 Urban 1.714*** 2.047*** 2.059*** 1.525*** 6.55 7.99 7.57 5.68 24 Gender Comp. -0.109*** -0.109*** -0.116*** -0.114*** -0.115*** -0.122*** -0.109*** -0.109*** -0.112*** -19.75 -19.8 -21.11 -10.69 -10.82 -12 -16.86 -16.86 -17.35 Unemployment rate 0.029 0.023 -0.018 0.031 0.015 -0.036 -0.032 -0.024 -0.044 2009 0.47 0.37 -0.29 0.3 0.14 -0.36 -0.41 -0.31 -0.56 Average Wage 2009 -0.001*** -0.001*** -0.001*** -0.001*** -0.001*** -0.002*** 0 0 0 -3.15 -3.4 -3.95 -3.64 -3.71 -3.94 0.15 0.07 -0.19 Population 0.009*** 0.007*** 0.007*** 0.007*** 0.007*** 0.006*** 0.122*** 0.115*** 0.105*** 4.83 4.2 4.01 3.54 3.5 3.36 8.35 7.88 7.07 Share Ukrainian 0.072*** 0.072*** 0.071*** 0.063*** 0.064*** 0.061*** 0.070*** 0.070*** 0.069*** 17.2 17.23 17.35 10.47 10.56 10.49 11.48 11.55 11.6 Participation Ratio 0.073*** 0.179*** 0.031*** 12.85 15.15 4.82 _cons 154.054** 158.393*** 160.926** 156.307** 153.541** 159.317*** 163.011** 149.337** 148.706** 152.515*** 152.942** 151.437** * * * * * * * * * 233.99 198.02 161.95 146.12 158.55 126.84 84.79 69.49 87.01 86.29 77.44 76.05 R Adj sq. 0.262 0.321 0.323 0.339 0.391 0.431 0.434 0.483 0.079 0.159 0.16 0.164 N 10906 10906 10906 10906 4922 4922 4922 4922 5984 5984 5984 5984 The regression also includes oblast dummies, dummies for different types of schools and evening schools. The full table can be found in excel tables which can be made available upon request. *** means significant at the 1% level, ** at the 5 % level and * at the 10 percent level.T statistics are based on robust standard errors for the OLS specifications. 25 Table 4a: Effect of school size, difference in dependent variable, relative to median school size Dependent Variable type First Quartile Median Third Quartile At Max Max at Participation Ratio (Ukrainian) urban -3.8 0 3.5 6.3 1060 Participation Ratio (Ukrainian) rural -1.5 0 2 5.6 470 Average Ukrainian Score urban -2 0 2 6.4 1600 Average Ukrainian Score rural -0.4 0 0.5 1.3 427 Average Mathematics Score urban -2 0 2.1 6.8 1641 Average Mathematics Score rural -0.4 0 1.3 3.1 443 Average Ukrainian History Score urban -1 0 1.2 8.5 2159 Average Ukrainian History Score rural -0.3 0 0.5 1.1 422 Average Ukrainian Score - Female Students urban -1.8 0 1.9 5.9 1584 Average Ukrainian Score - Female Students rural -0.6 0 0.8 2 446 Average Ukrainian Score - Male Students urban -2 0 2.1 7.5 1762 Average Ukrainian Score - Male Students rural -0.6 0 0.8 2 446 % students having > 173 on the Ukrainian Exam urban -2 0 2.3 15.6 3065 % students having > 173 on the Ukrainian Exam rural -0.7 0 1 3.1 538 % students having < 135.5 on the Ukrainian Exam urban 3.2 0 -3.2 -7.5 1291 % students having < 135.5 on the Ukrainian Exam rural 0.36 0 -0.4 -0.7 316 Participation Ratio - weighted average variables urban -2.9 0 2.7 4.9 1105 Participation Ratio - weighted average variables rural -0.9 0 1.1 2.5 430 Average Ukrainian Score- weighted average variables urban -1.8 0 1.8 4.8 1408 Average Ukrainian Score- weighted average variables rural -0.3 0 0.3 0.5 344 Differences are percentage points for the participation ratios and points for the test scores For the 2010 regressions, school quartiles are at about 100, 140 and 210 for rural areas and 305,465 and 666 for urban areas. For the weighted average regressions, quartiles are at 110,154 and 225 for rural areas and 322, 484 and 685 for urban areas 26 Table 4b: Effect of class size, difference in dependent variable, relative to median class size Dependent Variable Type First Media Third at Max/Min Signif. Participation Ratio (Ukrainian) urban -0.2 0 0.4 -0.3 14.9 NS Participation Ratio (Ukrainian) rural -0.5 0 -0.4 0 13.1 S Average Ukrainian Score urban 0.1 0 0.2 0 20.8 S Average Ukrainian Score rural 0.4 0 -0.3 -0.6 28 Almost Average Mathematics Score urban -0.1 0 0.2 -0.1 17.9 Almost Average Mathematics Score rural 0.3 0 -0.3 -1.3 55.2 NS Average Ukrainian History Score urban 0.1 0 0.2 0 21.5 S Average Ukrainian History Score rural 0.7 0 -0.5 -0.7 22.4 S Average Ukrainian Score - Female Students urban 0 0 0.22 0 20.1 S Average Ukrainian Score - Female Students rural 0.4 0 -0.3 -0.6 25.6 Almost Average Ukrainian Score - Male Students urban 0.1 0 0.3 0 21 S Average Ukrainian Score - Male Students rural 0.6 0 -0.6 -1 28.4 Almost % students having > 173 on the Ukrainian Exam urban -0.1 0 0.6 -0.1 19.1 S % students having > 173 on the Ukrainian Exam rural 0.4 0 -0.3 -0.5 23.8 NS % students having < 135.5 on the Ukrainian urban -0.2 0 -0.4 0 21.2 S % students having < 135.5 on the Ukrainian rural -0.7 0 0.6 1.9 35.3 NS Participation Ratio - weighted average variables urban -0.2 0 0.5 -0.2 20.3 S Participation Ratio - weighted average variables rural 0.9 0 -1.4 1.4 5.2 Almost Average Ukrainian Score- weighted average urban -0.2 0 0.6 -0.2 20.3 S Average Ukrainian Score- weighted average rural 0.4 0 -0.5 3.2 -38 NS Differences are percentage points for the participation ratios and points for the test scores For the 2010 regressions, school quartiles are at about 100, 140 and 210 for rural areas and 305,465 and 666 for urban areas. For the weighted average regressions, quartiles are at 110,154 and 225 for rural areas and 322, 484 and 685 for urban areas 27 References Ammermüller, Andreas, Hans Heijke and Ludger Wößmann (2005). Schooling Quality in Eastern Europe: Educational Production during Transition Research. Economics of Education Review, Volume 24, Issue 5, pp. 579–599. Barro, Robert J. and Jong-Wha Lee (2013). A New Data Set of Educational Attainment in the World, 1950–2010, Journal of Development Economics, Volume 104, pp. 184-198. Berdashkevich, A. and V. Vlasov (2010). Prospects of the Development of the Rural Schools. Russian Education and Society, vol. 52, issue 11, pp. 59–71. Berryman, S.E. 2000 Hidden Challenges to Education Systems in Transtion Countries. Washington D.C.: The World Bank. Bukowska, Grazyna and Joanna Siwinska-Gorzelak (2010). School Competition and the Quality of Education: Introducing Market Incentives into Public Services. The Case of Poland. Economics of Transition. Volume 19, issue 1, pp. 151–177 Coupé, Tom, Anna Olefir and Juan Diego Alonso (2010), Are Location and School Size Important for Educational Achievement? – An Investigation of the Determinants of the Performance of Secondary Schools in Ukraine, mimeo Coupé, Tom, Anna Olefir and Juan Diego Alonso (2011), Is Optimization an Opportunity? An Assessment of the Impact of Class Size and School Size on the Performance of Ukrainian Secondary Schools, World Bank Policy Research Working Paper 5879 Glewwe, Paul (2002). Schools and Skills in Developing Countries: Education Policies and Socioeconomic Outcomes. Journal of Economic Literature, Volume 40, issue 2, pp. 436-482 Glewwe, Paul and Michael Kremer (2006). Schools, Teachers, and Education Outcomes in Developing Countries. Chapter 16 in Erik Hanushek & F. Welch (ed.), Handbook of the Economics of Education, Elsevier, edition 1, volume 2, number 2. Hanushek, Eric A. and Ludger Woessmann (2011), The Economics Of International Differences In Educational Achievement, Chapter 2 in Eric Hanushek, Stephen Machin & Ludger Woessmann (ed.), Handbook of the Economics of Education, Elsevier, pp. 89-200. 28 Hazans, Mihails (2010). Teacher Pay, Class Size and Local Governments: Evidence from the Latvian Reform. IZA Discussion Paper No. 5291, October 2010 Herczynski, Jan and Mikołaj Herbst (2005). School Choice and Student Achievement. Evidence from Poland. MPRA Paper No. 6138. Hermann, Zoltan (2004), Demographic Change and the Efficiency of Primary Schools in Hungary, working paper. Hermann, Zoltan (2005). Small village-schools and efficiency loss due to diseconomies of scale. In Hermann Z. ed. Efficiency problems in the public education. Budapest: National Institute for Public Education, 2005, pp. 73-87. Jakubowski, Maciej and Pawel Sakowski (2006). Quasi-Experimental Estimates of Class Size Effect in Primary Schools in Poland. International Journal of Educational Research, Volume 45, Issue 3, pp. 202–215. Kallai, Ella and Mircea Maniu (2004). Input Efficiency in Publicly Provided Education: The Case of Romania. Babes-Bolyai University, Cluj-Napoca, Romania. Kane, Thomas and Douglas Staiger (2002). The Promise and Pitfalls of Using Imprecise School Accountability Measures, Journal of Economic Perspectives, Volume 16, Issue 4, pp. 91-114 Kovtunets, Volodymyr, Igor Likarchuk, Sergiy Rakov,Victor Gudzynsky (undated). Quality of Universities Admission Based on External Independent Assessment in Ukraine, working paper Kuddo, Arvo (2009). Structural Educational Reform: Evidence from a Teacher’s Displacement Program in Armenia. World Bank, SP Discussion Paper No. 0902. MacDonald, Kevin, Patrinos, Harry and Emilio Porta (2009). Using PISA to Understand the Determinants of Learning. School size and achievement in Serbia. World Bank education working paper, 2009. McGuinness, Aims, General Rapporteur, Steven Bakker, Neils Hummeluhr, Graham Reid, Ana-Maria Sandi, Péter Soltész, Evelyn Viertel, and Ian Whitman (2001). OECD Centre for Co-Operation with Non-Members. Reviews of National Policies for Education. Estonia. 29 Muravyev, Alexander & Talavera, Oleksandr, 2010. "Can State Language Policies Distort Students' Demand for Higher Education?," IZA Discussion Papers 5411, Institute for the Study of Labor (IZA) Porta, Emilio (2011). Using PISA to Understand the Determinants of Learning. School size and achievement in Romania. World Bank working paper draft, 2011. Psacharopoulos, George & Harry Anthony Patrinos, 2004. "Returns to investment in education: a further update," Education Economics, Taylor and Francis Journals, Volume 12, Issue 2, pp. 111-134 Tkhoryk, Oleg (2011), “School Size as a Determinant of Educational Performance in Transition Countries”, Kyiv School of Economics, Unpublished Master of Arts Thesis. Urquiola, Miguel (2006), Identifying Class Size Effects in Developing Countries: Evidence from Rural Bolivia, The Review Of Economics And Statistics, Volume 88, Issue 1, pp. 171– 177 World Bank (2010); “A Review of the Bulgaria School Autonomy Reforms”, Report No. 54890-BG, Human Development Department, Europe and Central Asia Region, The World Bank. 30 Table A1. Empirical findings on school size, class size and student-teacher ratio for transition economies. Country Study Methodology Dependent Findings School size Class size Student- variable teacher ratio Czech Ammermuller Clustering- TIMSS Math The most significant Not considered Significant Not Republic, et al (2003) robust linear and Science is student but the sign considered Slovak regression test scores background, with of effect Republic, (least squares family background differs by Hungary, and fixed next. School countries Slovenia, effects) autonomy is negative Lithuania, but mostly Latvia, insignificant. Romania Poland Herczynski OLS Log of School concentration Not considered Marginally Not and Herbst average math (H index) has significant considered (2005) and science negative impact. (positive) but test score Number of schools very small has positive effect but it diminishes after 4th school is added. Interschool standard Only H index is deviation of significant (negative), score number of schools has positive and increasing effect. Poland Bukowska OLS, data A national H index – negative, Positive Not Not and Siwińska- envelopment school test PIT income positive, considered considered Gorzelak analysis score social and transport (2011) expenditures, and population – negative. Poland Jakubovsky OLS, 2SLS, Class mean Increasing rural Positive Positive Not and Sakowski IV score for 6th schools classes to significant but significant, considered (2006). (exploiting grade exam equal those of urban very small but becomes maximum schools will widen negative class size exam-scores gap when rules) between them, controlled contrary to the policy- for total makers belief. school enrollment Hungary Hermann Fixed effects Choice of Share of students in Negative Not Not (2004) logit either general child-care after impact on considered considered or vocational lessons and share of probability of secondary students with special better school or a needs positively secondary technical influence choice of school choice school after better schools 8th grade (primary school) Romania Kallai and Random National Share of qualified Positive both Insignificant Negative Maniu (2004) effects for exam test teachers positive. for city and for urban panel data score, school rural/town schools averages schools Romania Porta (2011) Panel data, PiSA scores Correlation of schools After Not Not correlation size and scores is controlling for considered considered family 31 analysis small but negative. background, school size effect is significant only for reading. Serbia McDonald et Panel data, PiSA scores Mean scores for small Controlling for Not Not al (2009) correlation schools are lower and student and considered considered analysis significant for all family subjects characteristics, small schools have negative significant effect on scores. 32 Table A1. Empirical findings on school size, class size and student-teacher ratio for transition economies. Country Study Methodology Dependent Findings School size Class size Student- variable teacher ratio Czech Ammermuller Clustering- TIMSS Math The most significant Not considered Significant Not Republic, et al (2003) robust linear and Science is student but the sign considered Slovak regression test scores background, with of effect Republic, (least squares family background differs by Hungary, and fixed next. School countries Slovenia, effects) autonomy is negative Lithuania, but mostly Latvia, insignificant. Romania Poland Herczynski OLS Log of School concentration Not considered Marginally Not and Herbst average math (H index) has significant considered (2005) and science negative impact. (positive) but test score Number of schools very small has positive effect but it diminishes after 4th school is added. Interschool standard Only H index is deviation of significant (negative), score number of schools has positive and increasing effect. Poland Bukowska OLS, data A national H index – negative, Positive Not Not and Siwińska- envelopment school test PIT income positive, considered considered Gorzelak analysis score social and transport (2011) expenditures, and population – negative. Poland Jakubovsky OLS, 2SLS, Class mean Increasing rural Positive Positive Not and Sakowski IV score for 6th schools classes to significant but significant, considered (2006). (exploiting grade exam equal those of urban very small but becomes maximum schools will widen negative class size exam-scores gap when rules) between them, controlled contrary to the policy- for total makers belief. school enrollment Hungary Hermann Fixed effects Choice of Share of students in Negative Not Not (2004) logit either general child-care after impact on considered considered or vocational lessons and share of probability of secondary students with special better school or a needs positively secondary technical influence choice of school choice school after better schools 8th grade (primary school) Romania Kallai and Random National Share of qualified Positive both Insignificant Negative Maniu (2004) effects for exam test teachers positive. for city and for urban panel data score, school rural/town schools averages schools Romania Porta (2011) Panel data, PiSA scores Correlation of schools After Not Not correlation size and scores is controlling for considered considered family 1 analysis small but negative. background, school size effect is significant only for reading. Serbia McDonald et Panel data, PiSA scores Mean scores for small Controlling for Not Not al (2009) correlation schools are lower and student and considered considered analysis significant for all family subjects characteristics, small schools have negative significant effect on scores. 2