Report No: 77722-TR Promoting Excellence in Turkey’s Schools March 1, 2013 Human Development Sector Unit Europe and Central Asia Region Document of the World Bank Promoting Excellence in Turkey’s Schools 1 Report No: 77722-TR Promoting Excellence in Turkey’s Schools March 1, 2013 Human Development Sector Unit Europe and Central Asia Region Document of the World Bank 2 Promoting Excellence in Turkey’s Schools Vice President : Philippe H. Le Houérou Country Director : Martin Raiser Sector Manager : Alberto Rodríguez Task Team Leaders : Cristian Aedo and Naveed Hassan Naqvi Promoting Excellence in Turkey’s Schools 3 Table of Contents EXECUTIVE SUMMARY.....................................................................................................................................................v CHAPTER I: INTRODUCTION..........................................................................................................................................1 CHAPTER II: TURKEY’S PERFORMANCE IN PISA........................................................................................................3 II.A TURKEY’S PERFORMANCE IN PISA IN A REGIONAL AND TIME PERSPECTIVE..............................................4 II.B IDENTIFYING THE FACTORS THAT DRIVE TURKISH STUDENTS’ PERFORMANCE IN PISA......................8 II.C TURKEY’S INEQUALITIES IN A REGIONAL AND NATIONAL PERSPECTIVE.................................................. 12 CHAPTER III: AREAS OF POLICY FOCUS....................................................................................................................17 III.A ENCOURAGING AN EARLIER START IN SCHOOLS............................................................................................... 18 III.B IMPROVING TEACHING METHODS........................................................................................................................... 19 III.C REDUCING INEQUALITIES OF OPPORTUNITIES IN UPPER SECONDARY EDUCATION.......................... 23 SUMMARY OF RECOMMENDATIONS...........................................................................................................................25 PROPOSED FUTURE RESEARCH AGENDA...................................................................................................................27 REFERENCES.....................................................................................................................................................................28 ANNEX 1.............................................................................................................................................................................30 ANNEX 2.............................................................................................................................................................................33 ANNEX 3.............................................................................................................................................................................34 TABLES: TABLE 1 : OAXACA-BLINDER DECOMPOSITION OF TURKEY’S PERFORMANCE GAINS, 2003-2009......................6 TABLE 2 : OAXACA-BLINDER DECOMPOSITION OF THE DIFFERENCE IN PISA 2009 READING SCORES BETWEEN TURKEY AND BENCHMARKING COUNTRIES................................................................7 TABLE 3 : SHARE OF THE VARIATION IN READING PERFORMANCE EXPLAINED BY AGE, GENDER, AND SOCIOECONOMIC INDEX OF THE STUDENT’S FAMILY......................................................9 FIGURES: FIGURE 1 : PISA 2009 SCORES IN TURKEY AND OTHER COUNTRIES IN THE REGION................................................4 FIGURE 2 : AVERAGE PISA SCORES AND PER CAPITA GDP OF ALL PISA PARTICIPANTS, 2009.................................4 FIGURE 3 : AVERAGE PISA SCORES AND PER CAPITA GDP OF ALL PISA PARTICIPANTS AFTER CONTROLLING FOR LEVEL OF SOCIOECONOMIC INDEX, 2009....................................................................5 FIGURE 4 : TURKEY’S PISA SCORES BY DISCIPLINE, 2003-2009.............................................................................................5 4 Promoting Excellence in Turkey’s Schools FIGURE 5 : STANDARD DEVIATION OF TURKEY’S SCORES BY DISCIPLINE, 2003-2009................................................5 FIGURE 6 : PROGRESSION IN PISA SCORES BY PERCENTILE OF ACHIEVERS, 2003-2009.............................................6 FIGURE 7 : PROGRESSION IN PISA SCORES BY PROFICIENCY LEVEL, 2003-2009............................................................6 FIGURE 8 : COMPARISON OF TURKEY’S AVERAGE SCORES IN PISA 2009 WITH BENCHMARKING COUNTRIES..................................................................................................................................7 FIGURE 9 : DISTRIBUTION OF 15-YEAR-OLDS BY PROFICIENCY LEVEL IN READING, PISA 2009............................8 FIGURE 10 : PISA PERFORMANCE 2003-2009 BY QUINTILES OF SOCIOECONOMIC INDEX.........................................8 FIGURE 11 : KNOWLEDGE OF EFFICIENT LEARNING STRATEGIES AND AVERAGE PISA SCORE........................... 11 FIGURE 12 : THE PROJECTED EFFECT OF “GOOD� LEARNING STRATEGIES ON TURKEY’S PISA RESULTS......... 11 FIGURE 13 : SHARE OF VARIATION IN READING SCORES EXPLAINED BY INDIVIDUAL FACTORS/IN PISA 2009 FOR PARTICIPATING COUNTRIES............................................................................ 13 FIGURE 14 : SOCIAL STRATIFICATION IN THE PISA 2009 BY COUNTRIES....................................................................... 14 FIGURE 15 : AVERAGE PERFORMANCE IN READING AMONG STUDENTS BY SCHOOL............................................. 14 FIGURE 16 : AVERAGE PERFORMANCE IN READING AND AVERAGE NUMBER OF REPETITIONS IN PRIMARY SCHOOL AMONG STUDENTS IN EACH PROGRAM................................................................ 14 FIGURE 17 : DISTRIBUTION OF PISA 2009 STUDENTS BY QUINTILE OF SOCIOECONOMIC BACKGROUND ANF TYPE OF PROGRAMMES................................................................................................... 15 FIGURE 18 : SHARE OF GIRLS ENROLLED BY STREAM, PISA 2003-2009............................................................................. 16 FIGURE 19 : A PROFILE OF TEACHERS IN TURKEY.................................................................................................................. 19 FIGURE 20 : EFFECT OF TEACHING QUALITY ON STUDENT PERFORMANCE.............................................................. 20 FIGURE 21 : TEACHING CAREER PATHS IN SINGAPORE....................................................................................................... 22 BOXES: BOX 1 : WHY PISA MATTERS.....................................................................................................................................................3 BOX 2 : ABOLISHING SELECTION AND INTRODUCING COMPREHENSIVE SECONDARY SCHOOLS: THE SUCCESS OF POLAND’S REFORMS.......................................................................................... 16 BOX 3 : SINGAPORE-THE PURSUIT OF EXCELLENCE IN TEACHING........................................................................ 21 BOX 4 : THE STALLINGS METHOD OF CLASSROOM OBSERVATION......................................................................... 22 BOX 5 : SCHOOL CHOICE IN SECONDARY SCHOOLING............................................................................................... 24 Promoting Excellence in Turkey’s Schools 5 Acknowledgements T his report was prepared by Cristian Aedo, Naveed Hassan Naqvi, and Paul Cahu. From within the World Bank, helpful guidance, comments, and assistance were provided by Martin Raiser (Country Director, Turkey), Alberto Rodriguez (Education Sector Manager, Europe and Central Asia), Cristobal Ridao-Cano (Country Sector Coordinator, Turkey), Florian Fichtl (Lead Operations Officer, Turkey), and Ina-Marlene Ruthenberg (Country Program Coordinator, Turkey). We thank the report’s peer reviewers Juan Diego Alonso (Senior Economist, Latin America and the Caribbean), Harry Patrinos (Sector Manager, Human Development Network), and Ernesto Cuadra (Lead Education Economist, Middle East and North Africa) for their useful and insightful comments and guidance. Finally the team would like to acknowledge the very valuable insights and contributions of the Ministry of National Education. 6 Promoting Excellence in Turkey’s Schools Promoting Excellence in Turkey’s Schools v 7 Executive Summary 1. The education system in Turkey has made in Turkey and increased effectiveness shown remarkable improvement since 2003 in the delivery of education. The country’s in terms of better student performance rapid economic growth has improved the and reduced inequality with a concurrent socioeconomic conditions of the population and sustained increase in enrollments. The and led to the gains of around 20 points in all country’s scores in the three PISA disciplines PISA disciplines. However, the socioeconomic (reading, math, and science) increased sharply status of each student’s family mattered less by 20 points or more between 2003 and 2009. for his or her PISA results in 2009 than in This represents the equivalent of having 2003. This is because the delivery of education around half a year of additional schooling in services has become more effective in the each of the tested subjects. At the same time, period between these PISA tests, improving inequalities in student performance have the schooling provided to almost all Turkish decreased as the gains in PISA scores have students and thereby reducing inequalities. come overwhelmingly from low and medium achievers. For example, the performance of 3. The Government’s educational policies the bottom 1 percent of achievers has gone up have contributed to the improvements by 30 points in reading, by 33 points in math, in education outcomes – including and by 25 points in science; the performance of curriculum reform, phased modernization high-achievers has, on average, also improved of teaching and learning materials and although by less than that of lower achievers. practices, stronger focus on measuring During the same period, the enrollment rate of learning outcomes through large scale 15-year-olds has grown by a strong 7.8 percent national and international assessments, per year according to PISA, which makes these and enhanced monitoring and evaluation improvements even more remarkable. systems. Notwithstanding these successes, some challenges remain. The performance 2. This report finds that most of the of Turkey’s average 15-year-old is still one full progress in the education system since 2003 year (or 40 PISA points) behind the OECD has been the result of a combination of average. Around 25 percent of the Turkish the overall socioeconomic progress being 15-year-olds do not read well enough to be able vi 8 Promoting Excellence in Turkey’s Schools to analyze and understand what they read and are therefore considered by the OECD to be Recommendations: “functionally illiterate�; however, this rate has � Offer targeted support to expand been declining at a fast rate since 2003 when the enrolment in kindergarten to the lagging equivalent proportion was 36 percent. Despite provinces. some progress in narrowing the achievement � Ensure equity in access to ECE services by gap in between students from the richest and targeting the poorest households nationwide. poorest income quintile groups, it was still � Improve quality by rigorously monitoring, about 100 points (or more than two years of evaluating, and enforcing standards for pre- education) as of 2009. school education. � Ensure adequate funding for ECE and 4. This report continues the work involve private and not-for-profit actors in the undertaken in the previous World Bank ECE system. (2011) report which examined a cross- � Reduce effective entrance age to Grade 1. section of countries that participated in PISA 2009, and pointed out that the average 15- year old in Turkey is 1 school year behind 7. Improving teaching methods by: the average OECD counterpart in reading, math and science skills. The current analysis a. Promoting the use of effective learning strategies: An analysis of the PISA results relies on the subsequent availability of detailed indicates that learning in Turkey’s schools data for PISA 2009, and exploits similar data could be significantly improved if all from previous rounds of PISA in 2003 and students were taught how to use effective 2006. This time trend analysis offers a key learning strategies. As in other PISA insight- it shows that while Turkey lags its countries, students who report having OECD counterparts, it has nonetheless made been taught “good� learning strategies remarkable improvement since 2003, especially (those used to understand, remember, and for students from poorer households. summarize information) tend to produce better test results. In Turkey, estimates 5. The report finds that the main areas for have shown that an increase of about one further improvements are: standard deviation of the OECD indexes related to the use of the effective learning strategies would increase average student 6. Encouraging an earlier start in school: performance respectively by 24 points for the understanding and remembering According to PISA, across all OECD countries techniques and by 23 points for the one year of pre-school was correlated with 34 summarizing ones. Two-thirds of the additional points in reading, and around two- benefits of the good learning strategies thirds of this impact was due to peer effects. On actually operate through peer effects. the basis of these estimates, Turkey’s average In total, an increase of just one standard score in reading would increase by 24 points deviation in the number of students that if 100 percent of the country’s students could are taught these techniques would increase attend a good quality pre-school for one year. the average reading score in Turkey by 68 Also, for the 15-year-olds who were tested in points (or the equivalent of almost one and 2009, the average age of starting first grade a half years of schooling). Moreover, if all Turkish students had at least the average was 6.8 years. Estimates show that bringing the amount of knowledge of these techniques, average starting age closer to 6 could improve the share of the “functionally illiterate� reading results by up to 10 points. would drop from 25 percent to 10 percent. Promoting Excellence in Turkey’s Schools vii 9 Sweden and UK) may offer lessons for Turkey. Recommendations: These countries have only one type of school, � Through pre-service and in-service in some cases allowing choice of academic training, train teachers to educate students programs within the school, which means in the use of effective learning strategies. This that the stakes at play when moving to upper can be done separately or in the context of the secondary education are not as high as those in wider reform of teacher career management Turkey, where an examination streams people and incentives. into different types of schools that vary widely � The training should incorporate a module in terms of quality (World Bank, 2013). to address the specific needs for effective learning strategies of disadvantaged students. Recommendations: � Continue expanding access to the best b. Improving classroom practice: Teachers’ educational programs for students from all practices play a significant role in how socio-economic backgrounds. students learn in Turkey. PISA measures � Consider reducing the types of upper teachers’ practices in three dimensions: secondary schools in Turkey, while expanding (i) the disciplinary climate; (ii) the quality the number of academic programs within of teacher-student relationships: and (iii) schools. stimulation of student reading engagement. By improving these teaching practices 9. The findings in this report could be within the classroom by one standard usefully complemented with further research deviation, Turkey’s scores in the three PISA including: disciplines would increase by about 11 points. a. A corroboration of PISA findings using other student assessments, such as TIMSS and national test results. The Recommendation: use of additional student achievement � Carry out a nationally representative scores could allow to test the robustness assessment of teaching practices and classroom of the relative weights accorded to socio- dynamics using the “Stalling method� of economic and school specific effects and classroom observations. thus help policy makers identify measures to maximize learning opportunities 8. Reducing inequalities of opportunities in for socio-economically disadvantaged students. upper secondary education: In Turkey there are several types of upper secondary schools b. An assessment of current teacher and average performance in PISA of these practice and classroom dynamics in school types is quite different. Students in the Turkey’s schools. This would be based best program – science and Anatolian high on an application of the Stallings method schools – had PISA scores that are equivalent of classroom observations to a nationally to about additional three years of schooling representative sample of schools and its than students in the streams with the lowest aim is to provide useful information to score (e.g. vocational schools). The selection understand what makes teachers effective process that streams students into different in the classroom; and types of upper secondary schools according to their abilities is likely to have an effect on c. A diagnostic on Teacher Career their academic performance not only on that Management to assess current teacher education level but also in earlier stages in the management structures at the central, school cycle. It is noteworthy that streaming provincial and school level. This analysis has become gender neutral in Turkey, with the would be based on structured interviews share of girls enrolled increasing significantly with key informants from central, in the best schools. Some of the less stratified provincial and sub provincial levels, and countries of the OECD (Canada, Denmark, the application of questionnaires to a Finland, Iceland, New Zealand, Norway, sample of teachers. viii Promoting Excellence in Turkey’s Schools 10 Promoting Excellence in Turkey’s Schools 1 Chapter I: Introduction 1. The education system in Turkey has Turkey’s average 15-year-old is still one full shown remarkable improvement since 2003 year (or 40 PISA points) behind the OECD in terms of better student performance average. Around 25 percent of Turkish 15- and reduced inequality,1 with a concurrent year-olds do not read well enough to be able and sustained increase in enrollments. The to analyze and understand what they read and country’s PISA scores increased sharply (by are therefore considered by the OECD to be 20 points or more) between 2003 and 2009 in “functionally illiterate�. However this rate had the three PISA disciplines (reading, math, and declined rapidly since 2003 when the equivalent science), which represents about half a year proportion had been 36 percent. Despite some of schooling in each of the tested subjects.2 progress in narrowing the achievement gap At the same time, inequalities in student between students from the richest and poorest achievements have decreased, and the gains in income quintile groups, it was still about 100 PISA scores have come overwhelmingly from points (or more than two years of education) as low and medium achievers.3 Between 2003 of 2009. and 2009 the standard deviation at national level in PISA has decreased by 10 percent in 3. The educational policies and reforms reading, 12 percent in math, and 15 percent that the Government has implemented in science. The performance of the bottom 1 have made important contributions to percent of achievers has gone up by 30 points improving outcomes in recent years. Turkey in reading, by 33 points in math, and by 25 made remarkable progress in increasing points in science. The performance of students access to basic education since the Basic between the 80th and 90th percentile has Education Law (Law No. 4306) was passed in also improved, although by less than that of August 1997, which mandated eight years of lower achievers. During the same period, the compulsory Education. As result of that law, enrollment rate for 15-year-olds has grown by a Turkey launched an unprecedented expansion strong 7.8 percent per year according to PISA, of public primary and secondary schooling. which makes these improvements even more The expansion involved a broad range of remarkable. actions including construction of new schools; renovation or expansion of existing schools; a 2. Notwithstanding these successes, some massive provision of computers, educational challenges remain. The performance of equipment, and educational materials; new 1 See Table 1 in Annex 1. 2 The recently released TIMSS 2011 country results tell a similar story. Girls outperform boys in both mathematics and science; students who have many resources as measured by the TIMSS’s Home Educational Scale (based on the number of books at home and parent education) outperform those with just a few resources by 160 TIMSS points in science and by 185 TIMSS points in mathematics. 3 See Table 1 in Annex 1. 2 Promoting Excellence in Turkey’s Schools recruitment and additional staff training. and secondary education; and over 844,000 Teachers in primary schools increased from computers have been allocated to schools as 384,170 in 2003 to 515,852 in 2012, and student part of the setting up of Information Tech- teacher ratio fell from 27 to 21 students per nology Classrooms in primary and secondary teacher. education. Several web-based services are now available for students, parents, teachers, public 4. The expansion of primary and secondary employees and citizens. Key web based services schooling was accompanied by the are (i) Management Information System: development of new curriculum in primary a school management information system and secondary education, especially since software which includes the whole process 2004. A recent independent assessment of from a student’s first registration to his/her the revised secondary school curriculum graduation; (ii) Parent Notification System: a regarded it as an improvement over the system geared towards parents with children curriculum used in the past.4 Curriculum in schools, both public and private, that gives reform was accompanied by initiatives to access to information about announcements, improve the teaching profession policies. class schedule, absences, grades, exams and Turkey introduced incentives to improve projects; (iii) Information Acquisition System: a teachers’ professional and career development, system that allows asking questions and making and to ensure that teachers are appointed to suggestions and complaints and monitor them distant rural schools. They have also focused on in accordance with the Information Acquisition using technology to keep teachers connected Law; (iv) Distance Education Platform: a model and overcome potential isolation. The country of distance education through which users can has been implementing several activities for get visual and audio computer training online; improving pre-service teacher education, for and (v) Education Information Network: selecting the most highly qualified candidates designed for all education stakeholders, starting with teachers and students, which provides from the pool of teacher candidates and for e-content for different grades. improving their career development and rewards system. For example, new teachers 6. The rest of this report will analyze are being asked to work in schools in lagging the evolution of regional and income provinces allowing them to accumulate inequalities in PISA scores, the overall points which make them eligible later to effectiveness of the delivery of education apply for schools and postings of their choice. in Turkey, and identify areas on which Additionally, teachers are assigned to schools policymakers can focus to further improve rather than to provinces to insulate deployment educational outcomes. The report pays decisions at the provincial level from political particular attention to the role played by pressures. improved teaching and greater school accountability in improving educational 5. Turkey is also modernizing its outcomes. In a recent report (World Bank, teaching and learning equipment, by 2011) the World Bank examined a cross- providing computers and smart boards section of countries that participated in PISA under the Prime Minister’s Fatih Project. 2009, and pointed out that the average 15- Simultaneously web-based services and tools year old in Turkey is one school year behind have been made available to key stakeholders the average OECD counterpart in reading, to improve accountability and transparency math and science skills. The current analysis of the education system. Over 97 percent of relies heavily on the subsequent availability schools in Turkey are now connected to the of detailed data for PISA 2009, and exploits internet, including many of its rural schools, similar data from previous rounds of PISA in increasing technological access to schools 2003 and 2006. This time trend analysis offers communities and improving connectivity and a key insight- it shows that while Turkey lags educational networks. Over 921 million school its OECD counterparts, it has nonetheless books have been distributed during 2003- made remarkable improvement overall, and 2010 free of charge to all students in primary especially in reducing inequalities, since 2003. 4 The assessment was carried out by a consortium of Birmingham University-UK, Gazi University-Ankara, Turkish Cur- riculum and Instruction Association (EPODER)-Ankara and Global Advisory and Training Services (PGlobal)-Ankara. MONE (2012) Promoting Excellence in Turkey’s Schools 3 Chapter II: Turkey’s Performance in PISA 7. The Program for International Student every three years thereafter. The aim of PISA is Assessment (PISA) is an international to improve educational policies and outcomes large scale assessment conducted by the around the world. Turkey has participated in Organization for Economic Cooperation this international large scale assessment since and Development (OECD) of the academic 2003. This chapter analyses performance results performance of 15-year-olds in mathematics, for the country in a time trend perspective as science, and reading. The tests were first well as in a regional perspective. administered in 2000 and have been repeated BOX 1 Why PISA Matters Why does PISA matter for economic growth? Recent international evidence suggests that the quality of education is one of the strongest drivers of economic growth in the long term. A recent paper by Hanushek and Woessmann (2007 and 2010) built international measures of education quality by pulling together the results of international test scores, including TIMMS, PISA, PIRLS, and earlier international tests. Using this measure, they showed that countries that had better quality of education in the years 1960s, such as the Eastern Asian countries, experienced faster economic growth between 1960 and 2000. According to the authors’ computations, on average, a 50 point increase in a country’s PISA score will raise its growth rate by 1 percentage point over the following four or five decades. These effects will appear very gradually as the better trained cohorts enter into the labor force. Therefore, the economic benefits that will arise from a commitment to improving the quality of learning can be considerable. Why does PISA matter for educational policy? The objective of the PISA testing is to determine how well students are prepared to meet some of the challenges of their future lives. In particular, levels of Reading literacy, Mathematical and Scientific literacy are assessed using items that require students to show how effectively they can apply their reading, mathematical and scientific skills and knowledge to solve problems set in real-world contexts. These problems are designed to reflect some of the kinds of thinking, tasks and activities the students might be expected to carry out in their day-to-day lives. A sample question from Math illustrates the applied nature of the PISA tests: “Nick wants to pave the rectangular patio of his new house. The patio has length 5.25 meters and width 3.00 meters. He needs 81 bricks per square meter. Calculate how many bricks Nick needs for the whole patio�. Additional sample questions can be found at Source: http://pisa- sq.acer.edu.au/ 4 Promoting Excellence in Turkey’s Schools FIGURE 1 PISA 2009 Scores in Turkey and Other Countries in the Region 520 500 Bulgaria 480 Serbia Russian Federation 460 Turkey New EU members 440 Croatia 420 OECD Advanced European 400 Hungary Poland 380 reading math science Source: Authors based on PISA datasets Note: Blue line indicates OECD average II. A Turkey’s Performance in PISA in FIGURE 2 Average PISA Scores and Per Capita GDP of All PISA a Regional and Time Perspective Participants, 2009 8. Turkey’s performance in PISA is higher Average PISA scores, 2009 PISA 2009 mean scores, averaged across 3 disciplines 600 than might have been expected given its level of economic development. Its current KOR FIN SGP PISA scores in reading, math, and science EST JPNCAN NLD AUS CHE DEU BEL 500 are all around 450. This means that its current LVA LTU POL HUN PRT SVK SVN FRA ISL GBR DNK CZE ITA SWE ESP AUT IRLUSANOR LUX HRV GRC performance in reading remains not far from RUS TURKEY TUR ISR ARE SRB CHL the average of the new EU members but its THA BGR URY ROU MEX TTO performance is weaker in math and science than 400 JOR MNE BRA COL KAZ TUN IDN ALB AZE that of other countries of the region (see Figure PER PAN 1). However, Turkey’s education performance KGZ is better than average when its level of GDP 300 8 9 10 11 per capita is taken into account (see Figure 2). per capita GDP (2005 PPP), log scale The average PISA score of Turkish students in 2009 was approximately 10 points above Source: Authors based on PISA datasets average given its level of economic development (10 PISA points is approximately 2.5 standard average once GDP per capita and the countries’ deviations and it is statistically significant at average socioeconomic index are taken into 5 percent). This is even more impressive once account. In fact, on those terms, Turkey’s the socioeconomic level as well as the level of performance is the highest among the 65 per capita GDP is taken into account. In Figure participating countries, even higher than Korea 3 Turkey achieves almost 70 PISA points above or Singapore. Promoting Excellence in Turkey’s Schools 5 FIGURE 3 Average PISA Scores and Per Capita GDP of All PISA Participants after Controlling for Level of Socioeconomic Index,5 2009 Average PISA scores, 2009 PISA 2009 mean scores, averaged across 3 disciplines 100 150 KOR SGP TURKEY TUR POL JPN IDN THA MEX HUN PRT FRA ESP FIN 50 CZE ITA BELNLD IRL CHE COL LVA CHL EST HRVSVK AUS USA TUN BRA GRC DEU ISR GBRCAN SWE URYRUS LTU DNK NOR SVN AUT PER JOR ALB BGR ROU TTO ISL ARE LUX 0 SRB PAN KAZ MNE AZE -50 KGZ -100 8 9 10 11 per capita GDP (2005 PPP), log scale Source: Authors based on PISA datasets FIGURE 4 Turkey’s PISA Scores by Discipline, 2003-2009 9. Turkey’s PISA scores increased sharply between 2003 and 2009 in all disciplines (reading, math, science). This progress represents about half a year of schooling in each of the tested subjects (see Figure 4). At the same time, inequalities in results have decreased substantially as measured by the standard deviations in scores (see Figure 5). 10. Gains in PISA scores have come overwhelmingly from low and medium achievers. The performance of the bottom 1 percent of achievers has gone up by 30 points in reading, by 33 points in math, and Source: Authors based on PISA datasets by 25 points in science (see Figure 6). The performance of students between the 80th FIGURE 5 Standard Deviation of Turkey’s Scores by Discipline, and 90th percentile in achievement has also 2003-2009 improved although by less than that of lower achievers. The performance of Turkey’s top PISA achievers declined between 2003 and 2009, but this decrease only affected a limited share of students ‒ only 4 percent in reading and math and 8 percent in science ‒while the average performance of the majority of students remains high. The same pattern can be observed by proficiency level (see Figure 7). Considerable progress was made by the lowest achievers (proficiency level 1) and by the moderate and strong achievers (proficiency levels 3 and 4), with only the top performers showing a slight decline in performance. Source: Authors based on PISA datasets 5 The Programme for International Student Assessment (PISA) index of economic, social and cultural status was created on the basis of the following variables: the International Socio-Economic Index of Occupational Status (ISEI); the highest level of education of the student’s parents, converted into years of schooling; the PISA index of family wealth; the PISA index of home educational resources; and the PISA index of possessions related to “classical� culture in the family home. Source OECD (http://stats.oecd.org/glossary/detail.asp?ID=5401). 6 Promoting Excellence in Turkey’s Schools 11. The improvement in educational FIGURE 6 Progression in PISA Scores by Percentile of Achievers, outcomes in Turkey is due both to socioeconomic progress and system- 2003-2009 wide improvements in effectiveness. A decomposition of the changes in learning outcomes can be used to isolate the relative contribution made by various factors. Here an Oaxaca-Blinder decomposition6 is performed to examine how much of the improvement in education outcomes in Turkey between 2003 and 2009 can be explained by improvements in students’ socio-economic background and changes in the composition of the student sample (explained variation) and how much is due to system-wide effects (unexplained variation) such as improvements in effectiveness.7 As shown in Table 1, better socio- Source: Authors based on PISA datasets economic conditions (related to the country’s rapid economic growth) have produced gains FIGURE 7 Progression in PISA Scores by Proficiency Level, around 20 points in all disciplines. At the 2003-2009 same time, the extension of upper secondary education resulted in a more diverse school population and increased enrollments in lower grades. With this expansion the share of “late�8 students (that is, students attending grades 9 or lower) has indeed increased sharply from just over 8 percent in 2003 to 29 percent in 2009, while the share of students ahead of the appropriate grade enrolled in grade 11 or higher has decreased from around 39 percent to 4 percent in the same period (see Figure 1 in Annex 1). This phenomenon cost Turkey around 10 PISA points. The most important development has been the simultaneous increase in the effectiveness and quality of the education system and the decrease in the Source: Authors based on PISA datasets returns to socioeconomic background. While TABLE 1 Oaxaca-Blinder Decomposition of Turkey’s Performance all students experienced an improvement in their socioeconomic status during this Gains, 2003-2009 period, the benefits derived from this declined, Reading Math Science Total change 2003-2009 18 15 16 because the education system became better at of which explained by changes in: allowing students from less advantaged families ..socioeconomic and wealth factors 16 20 18 to achieve their full potential. The result, as ..grade structure -10 -11 -10 shown in Table 1, is that system-wide effects ..other factors* 4 1 2 over compensate for the decline in returns ..returns to socioeconomic and wealth -36 -67 -23 to socioeconomic background, leading to an ..other returns 4 2 4 overall improvement in educational outcomes ..country effectiveness 41 70 25 and a decline in the variation of educational Source: Authors based on PISA datasets achievements. Note: See Annex 2 for technical details. * Other factors include; Students’ learning strategies and teacher practices. 6 See Annex 2 for technical information on the Oaxaca-Blinder decomposition. 7 Ideally the decomposition should control for the resources spent in education. However, this information is not available at the school level in the PISA dataset. 8 According to PISA a 15 year old is expected to be enrolled in grade 10. Promoting Excellence in Turkey’s Schools 7 12. While Turkey was 40 to 50 PISA points or Poland, which also affects the country’s below Hungary and Poland (see Figure 8), relative performance. In addition, Turkish the Turkish education system appears to be students tend to be less familiar with the most more efficient after all observable factors efficient learning strategies than students are accounted for. In other words, once all in other PISA countries resulting in lower observable factors are taken into account PISA scores for Turkish students. However, Turkey is able to get higher returns in terms teacher practices (discipline, relationship with of PISA results than those countries. Based students, encouragement of reading, and the on the Oaxaca-Blinder decomposition of the structure of lessons) are better in Turkey, which gaps between Turkey and the benchmarking partially compensates for these variables. countries and once socioeconomic factors are After all observable factors have been taken taken into account, Turkey’s system appears into account; the Turkish system appears to more efficient (see Table 2). First, as students be much more efficient. The effectiveness gap in the PISA sample in Turkey come on measures what is left of the achievement gap average from lower grades their PISA scores once all observable factors have been taken are lower. Second, Turkey’s socioeconomic into account and is a measure of the overall conditions lag behind those of Hungary effectiveness of the educational system. This effectiveness is substantially higher in Turkey FIGURE 8 Comparison of Turkey’s Average Scores in PISA 2009 than elsewhere, for example, 24 PISA points with Benchmarking Countries higher than in Hungary, 13 PISA points higher than in Poland, and 81 PISA points higher than in Bulgaria. 13. Despite these remarkable improvements, Turkey’s performance is still one full year behind the OECD average. As mentioned in Chapter I, around 25 percent of Turkish 15-year-olds still do not read well enough to be able to analyze and understand what they read and are therefore considered by the OECD to be “functionally illiterate�9 (see Figure 9), though this is down from 36 percent in 2003. So Turkey has further catching up to do to compete with the most advanced OECD Source: Authors based on PISA datasets countries – an important factor in its future TABLE 2 Oaxaca-Blinder Decomposition of the Difference in PISA competitiveness. At the same time, as shown 2009 Reading Scores between Turkey and Benchmarking by the analysis above, as Turkey’s income and Countries socioeconomic conditions further improve, this is likely to lead to further improvements in vs. Hungary vs. Poland vs. Bulgaria Total gap in reading vs. benchmark -34 -42 21 its PISA scores. of which is captured by differences in: ..socioeconomic and wealth endowments -60 -43 -58 14. Also, inequalities in learning ..grade structure -11 -6 -17 achievement among students from different ..other factors -2 0 5 socioeconomic backgrounds remain large. ..returns of socioeconomic and wealth 11 -6 15 Even though the achievement of Turkey’s ..other returns 3 0 -5 ..country effectiveness 24 13 81 students from all socioeconomic groups has increased and though the achievement Source: Authors based on PISA datasets Note: See Annex 2 for technical details 9 Those students may be able to decipher a text, but they do not read fast and well enough to extract the relevant informa- tion from what they read. 8 Promoting Excellence in Turkey’s Schools gap between the richest and the poorest FIGURE 9 Distribution of 15-year-olds by Proficiency Level in socioeconomic groups has shrunk, the Reading, PISA 2009 achievement gap among these income groups in the 2009 PISA scores was still about 100 % 520 90 % 100 PISA points or more than two years of 500 80 % education (see Figure 10). This is critical as 480 70 % Turkey moves towards a more skill-intensive 60 % 460 specialization in the international division 50 % Levels 5 & 6 Level 4 of labor, which will require an increase in the 40 % 440 Level 3 supply of skilled labor across all socioeconomic 30 % 420 Level 2 groups. This in turn will help Turkey further 20 % 400 Functionally illiterate Average reading score reduce the level of income inequality, which 10 % 0% 380 remains at the higher end of the OECD ia 3 a ion 9 rs tia Eu D Hu n y d n F erbi ar 0 EU 200 a lan C ar be spectrum. 20 pe oa OE ng at lg em Po Cr ro S er ey y Bu Ne rke ed m rk Tu Tu ed sia nc w va ss Ru Ad 15. The following section examines the PISA results in more detail to identify potential Source: Authors based on PISA datasets areas of focus for policy that would allow Turkey to close the remaining achievement gaps to other OECD countries. FIGURE 10 PISA performance 2003-2009 by Quintiles of Socioeconomic Index II.B Identifying the Factors that Drive Turkish Students’ Performance in PISA 16. There are three broad determinants of students’ performance in the PISA tests: (i) the individual student’s characteristics; (ii) the characteristics of the other students in his or her school; and (iii) the school’s characteristics. The first category contains all the factors that cannot be modified by policy in the short to medium run, including the students’ gender, age, and socioeconomic characteristics (such as their parent’s education, occupation, Source: Authors based on PISA datasets income, and wealth). The second category exists because student performance is also affected can be attributed to their teacher’s efforts), by the characteristics of the other students in the autonomy of the school, its accountability, the school. This phenomenon is called the peer and the design of the education system.11 effect and has been proven to be very strong.10 In this section an analysis of variation in The third category includes information about performance within and between schools in those school characteristics that can be affected Turkey is performed in order to determine the by education policy, including the school role of individual and school characteristics and classroom environment, the teacher’s on student’s performance. Our main interest practices and qualifications, the student’s is in variables that may be affected by policy. learning strategies (which at least partially Results here rely on PISA 2009 only and exploit 10 A practical way to control for differences in performance caused by peer effects is to introduce variables capturing average characteristics of the students at the school level. 11 The design of the education system can affect peer group effects to some extent. The Bank has carried out a study of international evidence on social stratification and school choice, and pointed to some lessons for Turkey (WB, 2013). Further work in this area is planned to deepen our understanding of the issues. Promoting Excellence in Turkey’s Schools 9 the cross-sectional dimension of the data. As outcomes both through a direct impact shown in Table 1, variables such as teaching of the quality of teaching and learning practices and learning strategies do not account materials and through influencing the social for much of the improvement in learning stratification of schools. In many countries, outcomes between 2003 and 2009. However, and Turkey is no exception, the education below we show that they matter significantly system consists of a range of schools of different in a cross-sectional comparison of student quality, at the top of which are a network of performance within and between schools. selective schools (either private or public). Local regulations and customs can play a role Importance of Individual Student in minimizing or reinforcing the differences Characteristics and Peer Effects between these schools. These differences are reflected in the peer effect on student 17. A range of models was used to performance on PISA. In most countries measure the effect of the student’s family in Europe, the weight of individual factors characteristics and school-level peer effects (the family’s age, gender, and socioeconomic on his or her performance, and the results index) on a student’s performance is in a band are presented in Table 3. The individual around 20 percent (31 percent for the OECD socioeconomic index accounts for 20 percent as a whole), while Turkey is at the upper end of the variance (model 1 in Table 3), while the with 41 percent for individual and school level average socioeconomic index (a measure of socioeconomic factors. We return in the next peer effects) at the school level accounts for Section to study school characteristics and 36 percent (model 4 in Table 3). Controlling social stratification in greater detail so that we for gender, age, and individual socioeconomic can compare Turkey to other OECD countries index (model 2) explains 26 percent of the to identify the direction of policy change. variance at the individual level but up to 41 percent (model 5) of the variance when the Importance of School Characteristics school average socioeconomic index is also included.12 Overall, this means that the size 19. PISA data contain a wealth of of the individual student effects on PISA information about schools that can be performance is somewhat lower than the peer affected by policy and can be useful to flag effects in Turkey. relevant policy recommendations (see Table 2 in Annex 1). However, the data need to 18. Education policy can affect learning be taken with caution as they do not make it possible to identify any causal relationships TABLE 3 Share of the Variation in Reading Performance Explained between school practices and student by Age, Gender, and Socioeconomic Index of the Student’s performance.13 Another caveat is that for Family PISA the information is provided by school Model (1) (2) (3) (4) principals who may under-represent or over- Individual socioeconomic index Yes Yes Yes represent some behaviors/information due to Gender Yes Yes strategic reasons. Nevertheless, the correlation Age Yes Yes that emerge from the PISA data between school School average socioeconomic index Yes Yes level characteristics and student performance Share of variation explained 0.20 0.26 0.36 0.41 can shed some light on which policies promise Source: Authors’ elaboration the highest potential impact.14 Note: See Annex 3 for a technical discussion 12 It is notable that the share of the variance explained by both individual and school-level factors together (model 4) is less than how much individual (model 1) and the average socioeconomic index at the school level (model 3) explain separately. This occurs because individual and school characteristics are correlated. The correlation between the socioeconomic index at the individual level and averaged at the school level was 0.60 in Turkey in 2009. 13 This issue is not relevant when one looks at the relationships between characteristics such as age, gender, socioeconomic factors, because such variables are given and are a priori exogenous to performance. As a consequence, there is clearly a causal link between age for example and performance. 14 To be as unbiased as possible given available information, each regression included controls for age, gender, and socioeconomic index, mother tongue, school environment, rural/urban differences, and region. 10 Promoting Excellence in Turkey’s Schools 20. Crowded classes may be hampering 22. The availability of educational materials educational quality in Turkey. The 2009 PISA is also not correlated with schools’ PISA results were significantly lower in schools performance.15 This may indicate that that have a large number of students per availability of educational resources is teacher. Moreover, this effect may have large considered not a major bottleneck, however implications on inequities across schools as: the quality of teaching materials is not (i) each additional student per teacher is currently measured in PISA and leaves open associated with a reduction of 0.8 points in the the question of the quality and effective use of school’s reading score and (ii) student-teacher these educational materials. ratios vary and can be very high in some schools. Moreover, the negative impact of 23. The school and classroom environment high student-teacher ratios is 50 percent larger affect student performance. First, the in math and science, and this effect remains disciplinary climate (the frequency and severity significant once differences in school types are of classroom disruptions by students) plays a taken into account. major role in students’ academic performance, both at the individual and the school levels. 21. However, average student-teacher ratios In schools where the disciplinary climate is in Turkey are not unusually high. Turkey’s better, student performance on PISA is also average student-teacher ratio (around 18 higher. It makes sense to consider the impact students per teacher) is about normal for of the disciplinary climate at the school level European countries. Almost one fifth of as discipline and rules enforcement depends Turkish students attend schools with student- not only on each individual teacher but also on teacher ratios of over 25, and in 3 percent of school management. In fact, the disciplinary Turkish schools, the student-teacher ratio climate plays a much stronger role at the exceeds 39 students per teacher. There is even a school level than at the individual level. The school in the Turkey PISA sample that reports total effect of discipline on performance is very large ‒ an improvement of one national having a ratio of 65 students per teacher. This standard deviation in the disciplinary climate most likely reflects the relative shortage of is associated with an average gain of 31 points teachers in remote villages and smaller towns. in reading, 35 in math, and 28 in science. The Turkey already has a point system to incentivize average disciplinary climate in Turkey’s schools teachers to teach in remote areas, and these was slightly above the OECD average in 2009. results suggest closing the gap in teachers available across locations in Turkey could 24. Teachers’ practices also affect student significantly reduce inequalities in educational performance. Although the PISA survey does outcomes. Interestingly, there is no statistical not allow us to measure or monitor teachers’ link in Turkey between student-teacher ratios practices, it contains a few variables that and the shortage of teachers as reported by can be considered as proxies. One indicator principals. In addition, these reported teacher is the promotion by teachers of students’ shortages have not been shown to be correlated reading. In Turkey, this is clearly associated with students’ PISA performance, which with significant gains in reading, with an suggests that principals do not recognize these improvement of one national standard shortages as a major issue. At the national level, deviation being associated with a gain of 9 Turkey should target its efforts on reducing points in students’ reading scores. Given this student-teacher ratios on those 20 percent evidence, it would be desirable to have more of schools where the student-teacher ratio is robust measures of teaching practices and over 25. A nation-wide reduction in average their effect on student achievements, so that student-teacher ratios would be expensive and best practice examples could be disseminated unlikely to have much impact. throughout the country. 15 No actual measure is included in PISA, but an indicator from the subjective answers of principals was built and used in the analysis. Promoting Excellence in Turkey’s Schools 11 their awareness and use of various learning FIGURE 11 Knowledge of Efficient Learning Strategies and strategies, and the OECD recruited a panel of Average PISA Score experts to construct indices of these strategies based on students’ responses about how well they: (i) memorized; (ii) elaborated; (iii) controlled; (iv) structured; (v) understood; and (vi) summarized information.16 In Turkey, an increase about one standard deviation in these OECD indexes would increase TURKEY average performance by 24 points for the understanding and remembering techniques17 and by 23 points for the summarizing ones.18 26. Students perform better not only when they are aware of the best learning strategies but when their peers are aware of them as well. Learning strategies play a bigger role at the school level than at the student level (see Figure 12). Two-thirds of the benefits of the Source: Authors based on PISA datasets learning strategies actually operate through peer effects. When a student learns how to 25. Students in Turkey who report having learn, this awareness will benefit his peers been taught “good� learning strategies tend more than himself. This suggests that there is to achieve better results. This is consistent still considerable potential for encouraging with results found for other OECD countries more Turkish teachers to teach students these (see Figure 11). In PISA, students report methods. In total, an increase of one national FIGURE 12 The Projected Effect of “Good� Learning Strategies on Turkey’s PISA Results A: Increasing PISA Reading Scores B: Reducing the Share of “Functionally Illiterate� Students among 15-year-olds PISA Reading Score Share of “Functionally Illiterate Students Reading Score of Average 15 Year Share of Functionally Illiterate olds (including peer effects) Students among Turkish 15 Year olds 68 Points Difference ~1.5 %68 Points OECD average OECD average Difference years of schooling Without improving With improved Without improving With improved “good� Learning “good� Learning “good� Learning “good� Learning Strategies Strategies Strategies Strategies Source: Authors based on PISA datasets 16 OECD (2010), p. 112. 17 The average PISA score in reading was about 475 points in 2009. 18 As better students are likely to use more than one relevant strategy at the same time, the impact of all of these has been tested together. The combined effect of the two indices of learning strategies is to increase average PISA scores by 28 points. 12 Promoting Excellence in Turkey’s Schools standard deviation in students’ awareness of of the 65 PISA 2009 participating countries, efficient learning strategies would increase the one year of pre-school was correlated with 23 average reading score in Turkey by 68 PISA additional points in the country’s average PISA points (or the equivalent of almost one and score, while more than one year of pre-school a half years of schooling), with a 95 percent appeared to be associated with 44 additional confidence interval between 55 and 81 PISA points. As with the effects of learning strategies, points. Moreover, if all Turkish students had two-thirds of this impact flows through peer at least an average knowledge of these efficient effects and is not captured directly by the strategies, the share of the functionally illiterate student who attended pre-school. Using these would drop from 25 percent to 10 percent. estimates, Turkey’s average score in reading would jump by 32 PISA points if 100 percent 27. In the area of school autonomy, those of its students could benefit from more than schools in Turkey with the most control over one year of pre-school (with a 95 percent the curriculum, its resources, or the selection confidence interval ranging from 21 to 43 PISA of students did not perform significantly points). Scores in math and science would better than others. One cannot draw statistical improve by 23 PISA points with a 95 percent inference from this simple correlation about confidence interval ranging from 13 to 32 PISA the importance of school autonomy.19 But the points. results are not inconsistent with the recent literature on school autonomy. Thus, a recent 29. Encouraging children to start school study20 showed that schools tend to benefit at an earlier age by effectively reducing the from having more autonomy only when a entrance age to grade 1 could also bring strong accountability system is already in place. benefits to Turkey. Starting school at a According to OECD (2011a) and results from later age has a negative effect on academic rigorous impact evaluation studies (Barrera- performance, an effect which has been found Osorio et al, 2009; Clark, 2009; and Machin and in many countries. On average across all of the Vernoit, 2010 and 2011), countries in which PISA 2009 participating countries, a one-year schools have greater autonomy over what is delay in starting school decreased a country’s taught and over how students are assessed tend academic performance by 14 points. The to perform better than those in which schools school entrance age of the student cohort tested are more centrally managed. This is also the by PISA in 2009 was 6.8 years. Since then it has case for countries where schools make their declined to 6.2 years for the 2012 cohort and is students’ achievement data publicly available likely to fall further in 2013 given government and have considerable autonomy over how to policy changes in 2012. A decline from 6.8 allocate their resources. In countries where years to 6 years is associated with an increase in there are no such accountability arrangements, the country’s reading results by 10 PISA points. schools with greater autonomy over their Recent reforms to boost early school entry are resource allocations tend to perform worse. likely to show up in improved PISA scores in the future. Importance of early childhood education II.C Turkey’s Inequalities in a 28. According to cross-country PISA Regional and National Perspective evidence, substantial gains in educational outcomes may result from an expansion of 30. In every educational system, children early childhood education, as foreseen by from disadvantaged socioeconomic the Turkish authorities. On average across all backgrounds tend to perform worse than 19 When comparing countries, the OECD found that educational systems that promote school autonomy tend to perform better academically. The absence of a statistical link between autonomy and results within Turkey does not contradict this general finding. First, the degree of autonomy of schools may be insufficiently diverse to allow statistical inference. Second, it would be necessary to control for many unobservable characteristics of schools, for example, by using a randomized impact evaluation, to do a proper assessment. 20 Hanushek et al (2011). Promoting Excellence in Turkey’s Schools 13 students who have benefitted from better 32. In Turkey, inequalities of opportunity social conditions. Poorer children have due to socioeconomic differences are large less access to educational resources (such compared to other OECD countries. Figure as books) and cultural goods, and, because 13 presents the share of the variance in their parents have little education themselves, reading performance explained by individual their children receive little benefit from that factors (age, gender, and socioeconomic education. Therefore, inequalities in academic index) and school factors (socioeconomic performance induced by individual differences index) for several countries. It appears that are to a certain extent inevitable, are hard individual factors explain a large share of to minimize, and are contingent on initial Turkey’s academic performance ‒ around socioeconomic inequalities.21 26 percent versus 21 percent in the OECD as a whole. This situation can be explained by 31. However, education policy can offset (at the fact that socioeconomic inequalities are least partially) other aspects of the system greater in Turkey than in the average OECD that can induce inequalities. In particular, country. Moreover, once school socioeconomic those educational systems that segregate factors are taken into account, the total students from different socioeconomic levels weight of socioeconomic factors on students’ into different schools or classes tend to have the performance in Turkey appears to be one of largest inequalities of academic performance the highest in all of the PISA participating among their students. Limited coverage of countries ‒ 41 percent of the variance as pre-primary education, a variable entrance age opposed to 21 percent in the average EU for primary schools, the frequent use of grade country. repetitions, the use of entrance exams to select students, and the use of streaming according 33. Turkish schools are much more socially to ability levels are all examples of practices stratified than schools in the average OECD that can result in highly stratified student country, and students tend to attend schools populations across schools and thus widen that are socially homogeneous. In most inequalities in academic performance due to countries, students tend to go to school with dominant peer effects. peers from similar backgrounds, which results in the socioeconomic index at the individual level being correlated with the same index FIGURE 13 Share of Variation in Reading Scores Explained by averaged at the school level. Figure 14 shows Individual Factors / in PISA 2009 for Participating the correlation between the socioeconomic Countries index at the individual and at the school level. A correlation of 0 means that all students Individual factors are randomly assigned to schools, while a Both individual and school factors HUN correlation of 1 means that the students are Share of the variance in reading explained by... BGR segregated according to their socioeconomic TUR level. Figure 14 shows that schools in Turkey are much more stratified than those in the average OECD country. POL 34. In Turkey, there are several types of FIN upper secondary schools and average performance in PISA of these school types is quite different. At least ten different streams or programs can be identified from the PISA 2009 sampling of schools, including vocational, technical, science, and others. As can be seen in Source: Authors based on PISA datasets 21 See OECD (2010). 14 Promoting Excellence in Turkey’s Schools Figure 15, there are considerable differences in FIGURE 14 Social Stratification in the PISA 2009 by Countries academic performance between students in the different streams at the age of 15. Students in 0.8 the best program – science high schools – had 0.7 0.5981 PISA scores that were on average 147 points 0.6 0.5 higher than students in the stream with the 0.4 lowest scores – vocational schools. This gap 0.3 represents more than three years of schooling. 0.2 0.1 35. These differences in the performance 0 NOR FIN MNE LIE DNK CAN NZL SRB MAC HRV SVK TTO IRL ITA MLT LTU PRT GRC QAT RUS CZE MYS ALB BEL USA AZE FRA ARE TUR IND-HP BGR CRI MEX ARG IND-TN PAN VEN-MI THA of the students in these various streams can be attributed to a large extent to the selection process. As shown in Figure 16, the Source: Authors based on PISA datasets best programs are also the ones that welcome students who did not repeat any grades during primary school, while the streams with the FIGURE 15 Average Performance in Reading among Students by lowest performing students tend to contain School students who repeated grades. As the streaming occurs only one year prior to the age at which most students are tested in PISA, it is likely that differences in achievement are more reflective of the way in which students are selected in the streaming process than any innate differences in the quality of the upper secondary programs. 36. Streaming is also to some extent the reason behind school stratification in Turkey. As shown in Figure 17, students from different socioeconomic levels are not uniformly distributed among streams. Half of the students that enter the vocational and technical streams, which are those for which the PISA scores are lowest, are from the lowest (first and second) quintiles. Only 8 percent of the students in Source: Authors based on PISA datasets these streams come from the top quintile. Conversely, students in the most elite streams, the Anatolian general high schools and the FIGURE 16 Average Performance in Reading and Average science high schools, are overwhelmingly Number of Repetitions in Primary School among from the upper socioeconomic levels, with Students in Each Program 65 percent of the students in the science high schools coming from the top (fifth) Average reading performance Average frequency of repetition socioeconomic quintile. Thus, access to the elite streams is highly skewed toward the upper social classes, which contributes to the social stratification of schools. e Fin r.. An rts n Vo cial Ge l l l e l na ra ca na nc m ia he in primary ne A i ol tio tio m ni n 37. In addition, the most elite streams seem ie c e ch at ra ch ea Sc ca ca Te og An an T Te Vo n to have the best education environment, Pr An ian ia n i ol ia ti ol ol ul at ol at at M which reinforces these inequalities of at An An opportunity. The education environment encompasses teacher practices, the disciplinary Source: Authors based on PISA datasets Promoting Excellence in Turkey’s Schools 15 climate, and the transmission to students of Indeed, the limited number of available the best learning strategies. The quality of places in the elite programs fuels competition the teaching environment varies significantly between students. It also creates an artificial from one stream to another. Because of these threshold of performance below which inequalities, otherwise similar students (after students are not admitted into the elite schools. controlling for all other factors) will score 23 This is likely to create three undesirable effects. points more on the PISA reading test if they First, it may discourage children from the attend a science high school than if they attend lower socioeconomic quintiles who start too a vocational school. far from the threshold and are unlikely to be able to compete with children from better- FIGURE 17 Distribution of PISA 2009 Students by Quintile of off families. Second, the competition is likely Socioeconomic Background and Type of Program to spur middle and upper class families into investing extra money in their children’s Quintile 5 education by employing private tutors. Third, it may also prompt parents from the higher Quintile 4 quintiles to segregate their children from Quintile 3 those from less favored households as early as the primary level in an acknowledgement Quintile 2 of the importance of peer effects for students’ Quintile 1 academic performance. As a result, the Science high Anatolian Anatolian Others General high Vocational Average streaming process probably exacerbates school schools high schools voc. & tech. schools & technical schools reading score stratification throughout the education cycle, schools even though it only begins at the end of lower Source: Authors based on PISA datasets secondary school. International evidence confirms that early tracking hampers results 38. The selection process that streams for the less favored students without inducing students into different kinds of upper a noticeable improvement in the performance secondary schools according to their abilities of the students from more prosperous social is likely to have an effect on their academic backgrounds. Therefore, bringing forward performance. The streaming process occurs the age at which students are streamed would in the 9th grade however the prospect of this very likely not only increase inequalities but streaming is likely to increase performance also eventually cause the average student inequalities much earlier in the school cycle. performance to deteriorate. 16 Promoting Excellence in Turkey’s Schools BOX 2 Abolishing selection and introducing comprehensive secondary schools: The success of Poland’s reforms In 1999, Poland reformed its basic education system in order to raise the level of education in society, increase educational opportunities, and improve the quality of education. The new government at that time restructured basic education by converting the old 8-year primary school that was followed by early vocational tracking, into a 6-year primary education followed by three years of lower 3 general secondary education. Only after 9 years of schooling would a decision about what type of upper secondary education – academic or vocational – be undertaken. In other words, the new system postponed the choice of type of curriculum at the secondary level (general or vocational) for one year. This structural change was accompanied by curricular reform. A concept of core curricula was developed which aimed to provide schools with extensive scope of autonomy and responsibility. A system of examination and tests at the end of primary and lower secondary were introduced. Jakubowski et al (2011) used the variation created by the policy change in 1999 to test the impact on test scores over time. Specifically, they estimated a difference in difference model that compared the change in test scores of the likely vocational school students that were able to study in the general, academic track because of the change in school policy. The purpose of the study was to explain the significant improvement in international achievement tests by Poland in recent years. They found, on average, that the reform was associated with significant improvements. In math, Poland improved its score by 0.25 of a standard deviation. In reading the increase is 0.28 of a standard deviation. In science, the scores increased by 0.16 of a standard deviation. They confirmed these results using an evaluation model – propensity score matching and differences in differences to create counterfactual scores for the group of likely vocational students in subsequent years – and the OECD’s Program for International Student Assessment (PISA), an internationally comparable standardized student test conducted every three years totest reading, mathematics and science achievement of 15-year-olds, data for 2000, 2003 and 2006, using 2000 as the baseline since most of the existing students were continuing their lower secondary schooling under the old system. They explored threats to identification using among other things decomposition analysis. They concluded that the reform was associated with an improvement in likely vocational students’ scores of about 100 points, or a whole standard deviation. They explored the implications using as well a 2006 special application of PISA in Poland to 16 and 17 year-olds, and warn of the dangers of early vocationalization. 39. It is noteworthy that streaming has become been growing since 2003 in favor of girls (see more gender-neutral in Turkey. Figure 18 Figures 2 – 4 in Annex 1). shows the share of girls enrolled in the different FIGURE 18 Share of Girls Enrolled by Stream, PISA 2003-2009 streams according to PISA 2003 and 2009, and it can be seen that all types of streams (with the exemption of vocational schools) are moving closer to gender neutrality. In general in Turkey, the gender enrollment gap has been reduced since 2003 but still remains with a secondary net enrolment rate of 69 percent for boys against 66 percent for girls as of 2012. At the same time, there is a reverse achievement gap between girls and boys attending school: (i) girls were ahead in grade attainment in 2003 and this is even more true in 2009; (ii) girls outperform boys in reading and science but they underperform in math; and (iii) the gender performance gap has Source: Authors based on PISA datasets Promoting Excellence in Turkey’s Schools 17 Chapter III: Areas of Policy Focus 40. The preceding analysis suggests that the learning strategies: An analysis of the main areas where further improvements are PISA results indicates that learning in needed are encouraging an earlier start in Turkey’s schools could be significantly school, improving teaching methods, and improved if all students were taught reducing inequalities of opportunities in how to use the most effective learning upper secondary education: strategies. As in other PISA countries, Turkish students who report having a. Encouraging an earlier start in school. been taught the use of “good� learning According to PISA, across OECD countries strategies (those used to understand, one year of pre-school was correlated with remember, and summarize information) 34 additional points in reading, and around tend to produce better test results. In two-thirds of this impact was due to peer Turkey, estimates show that an increase effects. On the basis of these estimates, if 100 of about one standard deviation of the percent of the students could attend a good OECD indexes related to the use of quality pre-school for one year, Turkey’s the effective learning strategies would average score in reading would increase by increase average student performance 24 points;22 scores in math would improve by 24 points for the understanding and by 22 points;23 and scores for science remembering techniques and by 23 would increase by 29 points.24 Also, for the points for the summarizing techniques. 15-year-olds who were tested in Turkey in Two-thirds of the benefits of these 2009, the average age of starting first grade effective learning strategies actually was 6.8 years. Estimates show that bringing operate through peer effects. In total, an the average starting age closer to 6 could increase of just one standard deviation improve reading results by 10 PISA points, in the number of students that are with a 95 percent confidence interval taught these techniques would increase ranging from 8 to 12 PISA points. The Bank the average reading score in Turkey by has already undertaken a detailed analysis 68 points (or the equivalent of almost of the early childhood education sector in one and a half years of schooling) Turkey, and its main recommendations can with a confidence interval at 5 percent be found in the World Bank report on early level between 55 and 81 PISA points. childhood education in Turkey, but they are Moreover, if all Turkish students had at summarized in section III.A below.25 least an average amount of knowledge of those techniques, the share of the b. Improving teaching methods: “functionally illiterate� would drop i. Promoting the use of effective from 25 percent to 10 percent. 22 With a 95 percent confidence interval ranging from 16 to 32 PISA points. 23 With a 95 percent confidence interval ranging from 12 to 32 PISA points 24 With a 95 percent confidence interval ranging from 20 to 38 PISA points 25 World Bank (2012). 18 Promoting Excellence in Turkey’s Schools Improving ii. classroom practice: lagged in particular in the provision of early Teachers’ practices play a significant role childhood education (ECE), and summarizes in how students learn in Turkey. PISA recent government efforts to address this measures teachers’ practices in three challenge. dimensions: (i) the disciplinary climate; (ii) the quality of teacher-student 42. The report outlined four broad categories relationships; and (iii) the stimulation of recommendations for increasing the of students’ reading. By improving these effectiveness of Turkey’s ECE program teaching practices within the classroom which are key to encourage an earlier by one standard deviation, Turkey’s start in schools. The first category focuses scores in the three PISA disciplines on ensuring equity in access to ECE services would increase by about 11 points. by targeting public spending in a progressive manner, in other words, by targeting the c. Reducing inequalities of opportunities disadvantaged groups within the population. in upper secondary education: In Turkey, The second encourages the Government to students in the best program had PISA put systems in place that will enhance the scores that are equivalent to about additional quality and accountability in Turkey’s ECE three years of schooling than students in the subsector. The third advises policymakers streams with the lowest score. The selection how to make ECE management more effective process that streams students into different by increasing the involvement of private types of upper secondary schools according and not-for-profit actors in the sector. The to their abilities is likely to have an effect on last category of recommendations describes their academic performance not only at the the levels of financing that will enable ECE secondary level but also in earlier stages in programs to achieve Turkey’s ambitious goals the school cycle. of expanding ECE coverage. Specifically the recommendations are: III.A Encouraging an Earlier Start in Schools • Target spending in a progressive manner: The ECE expansion strategy will be more 41. A recent Bank’s report on “Expanding equitable and will have a greater impact if and Improving Early Childhood Education funds allocated to this initiative are distributed in Turkey� (World Bank, 2012) points progressively. This can be done in two ways: out that early childhood development (ECD) interventions have been shown to a) Targeting low-enrollment provinces; and yield many direct and indirect benefits to b) Targeting poor households to ensure society. First, investing in children early in equity in access. their developmental cycle (from the time of • Systematically assess and enforce national conception to 6 years of age) yields the highest quality standards: To ensure high quality returns in terms of their economic productivity service provision, the Government could relative to investments later in life. Second, they systematically assess and enforce standards provide a mechanism to enhance equity, as on the curriculum, infrastructure, teacher focused early childhood interventions targeted qualifications, teacher training, teaching at the vulnerable segments of population have materials, and educational supplies. The been known to reduce the intergenerational Government could also develop a system for transmission of poverty. Third, ECD policies assessing such outcomes as school readiness can greatly improve the personal development among ECE students. Some tools for doing this and life skills of young children and increase include: the opportunities available to them later in life by, for example, increasing rates of childhood a) School self-evaluations in which survival and school readiness. Lastly, policies schools would evaluate their own that focus on young children can have positive performance every year against externalities that benefit other members of national standards; and these children’s families, as well as society as b) School external evaluations in which a whole. The report also shows that among schools would be evaluated every ECD interventions, Turkey until recently has three years by peers and expert groups, Promoting Excellence in Turkey’s Schools 19 including assessing such outcomes as Provincial Program. The report suggests that children’s school readiness. the Core National Program should have three components: (i) funding infrastructure and • Involve private and not-for-profit actors teachers in all provinces (55 percent of the in the ECE system: While the bulk of services budget); (ii) providing targeted subsidies for will have to be delivered by the public sector, the poorest 40 percent of households in all the effectiveness of ECE provision can be provinces (25 percent of the budget); and (iii) enhanced by involving the private sector and devoting the rest (5 percent of the budget) non-governmental organizations (NGOs). to innovative schemes for improving quality Non-governmental actors and public-private and increasing accountability. The task of the partnerships can support and complement the Supplementary Provincial Program would Government of Turkey’s efforts to provide then be to ensure equity of access to ECE in ECE services, deliver targeted support, low-enrollment provinces and can consist of produce innovative media programs, and interventions that address province-specific conduct external evaluations. binding constraints.26 • Ensure adequate funding for ECE: To III.B Improving Teaching Methods achieve this ambitious policy agenda, the resources dedicated to ECE will need to be 44. Over the past decade, both developed raised from the current level of 0.03 percent of and developing countries have become GDP to 0.23 percent of GDP in 2014. increasingly concerned with raising the effectiveness of their teachers in classroom 43. Based on international best practices, practice. This interest seems to have been the government might consider taking a motivated by a growing body of research dual-program approach built around the suggesting that teacher effectiveness is the options outlined above to encourage an most important school-based predictor of earlier start. The report recommends two student learning and that several consecutive complementary programs to achieve this years of outstanding teaching can offset the objective- a Core National Program, and learning deficits of disadvantaged students. Supplementary Provincial Program. The report This section explores research on this issue, recommends that the majority (85 percent) of examines the role played by teachers and the resources devoted to ECE in the coming teaching in Turkey’s academic performance years should be allocated to the Core National and presents some recommendations. Program. The remaining resources (15 percent) would be allocated to the Supplementary 45. Turkey’s demographic profile means that unlike in most other OECD countries, FIGURE 19 A Profile of Teachers in Turkey its student population has been increasing A: Teachers in Turkey B: Main Teacher Practices Identified by for some time and will continue to grow in Head Teachers as a Hindrance to Learning the near future. The number of students in Turkey Turkey’s basic education (primary and pre- OECD primary levels) was more than 12 million as of 2011. In the last decade, more than 137,000 new children per year have entered the system, and Turkey’s teaching workforce for basic education expanded by roughly 50 percent from just over 361,000 teachers in 2001 to more than 550,000 teachers in 2011 (see Figure 19). While this has gone hand in hand with Lack of Pedagogical Preparation Arriving Late Absenteeism overall improvements in educational outcomes, Figure 19 also shows that inappropriate teacher Source: World Bank (2011) based on MoNE (2011) and OECD (2009) practices as reported by principals remain a 26 Options could include: (i) implementing information campaigns and outreach to families; (ii) hiring extra staff to plan or deliver expansion targets; (iii) giving performance grants to schools when they reach their targets for increasing enrollments; and (iv) giving performance grants to sub-provinces if they succeed in increasing their pre-school enrollments. 20 Promoting Excellence in Turkey’s Schools more important constraint in Turkey than in FIGURE 20 Effect of Teaching Quality on Student Performance the average OECD country.27 Student 46. International evidence suggests that Performance teaching quality is the main school-based 100th Percentile Student A – with a Low Quality Teaching over 3 Years predictor of student achievement and that Student B – with High Quality Teaching over 3 Years 83rd Percentile several consecutive years of outstanding teaching can offset the learning deficits of disadvantaged students.28 One seminal 54 Percentile Points paper29 looked at the effect of quality of Difference 50th Percentile teaching on student performance in the US Student B state of Tennessee. The authors found that the difference in the mean performance of 29th Percentile an average 2nd grade student (measured by Student A Student B Student A assessment results) of having high quality teaching30 versus low quality teaching over three years is greater than 50 percentage points 0th Percentile (as measured in Grade 5). As Figure 20 shows, Age 8 Age 11 Student Age not only does the average student with high quality teaching outperform the one with low Source: Based on Sanders and Rivers (1996) quality teaching by over 50 percentage points, the performance of the student with low measures to promote excellence in teaching quality teaching actually falls by 21 percentage in Turkey: (i) Focus pre-service and in- points over three years. service training of teachers on the effective use of “good� learning strategies in class and 47. Given this preliminary evidence, it (ii) promote effective teachers’ practices and is clear that good teaching is critical to classroom dynamics. increasing effective learning among Turkey’s students. Teaching practices and classroom - Recommendation 1: Focus pre-service and in-service training of teachers dynamics also play a significant role in how on the effective use of “good� learning students learn in Turkey. An analysis strategies. Based on the discussion in of PISA results confirms this hypothesis this chapter, education policymakers in for Turkey, and the analysis presented in Turkey should review the country’s teacher Chapter 2 indicated how this process of training programs to ensure that teachers improvement might be started. However, are trained to promote the use of effective this is clearly not the whole picture. Turkey’s learning strategies by their students. Since teacher policies and the actual practices being this initiative is fundamental to promoting used in schools are also crucial determinants excellence in teaching, it is also important of effective learning in school. While studies to consider teacher training in the wider around the world have documented the context of teachers’ careers and incentives. importance of teachers in student learning The training should incorporate a module outcomes, much research remains to be done to address the specific needs for effective on the various policies that are most effective in learning strategies of disadvantaged students. attracting, retaining, training, and motivating Singapore is a case study of a country that has teachers. revised its teacher training as part of a wider reform of teachers’ careers and incentive 48. Based on the analysis performed the structure. This case study is presented in Box report recommends the following two 3. 27 Turkish school principals cite the lack of pedagogical preparation of teachers as key constraint to student learning. Specifically, 43 percent of Turkish teachers are in schools where the principal has reported that a lack of pedagogical preparation is a factor hindering instruction a lot or to some extent (see Figure 20 and OECD, 2009). 28 Hanushekand Rivkin, 2006; Nye et al, 2004; Park and Hannum, 2001; Rivkin et al, 2005; Rockoff, 2004; Sanders, 1998; Sanders and Rivers, 1996; and Vignoles et al, 2000. 29 Sanders and Rivers (1996). 30 High quality teaching is defined as being done by a teacher in the top 20 percent of all teachers as measured by ability. Low quality teaching is defined as being done by a teacher in the bottom 20 percent of teachers as measured by ability. Promoting Excellence in Turkey’s Schools 21 BOX 3 Singapore-The Pursuit of Excellence in Teaching Singapore has introduced the “Education Service Professional Development and Career Plan� (Edu-Pac) for teachers to develop their potential and fulfill their aspirations. The Edu-Pac gives teachers a choice between three different career tracks: (i) a Teaching Track, where they specialize in teaching or pedagogy and can advance to be Principal Master Teachers; (ii) a Specialist Track, where they can specialize in areas where deep knowledge and skills are essential for breaking new ground in educational developments and in which they can advance to become a Chief Specialist; or (iii) the Leadership Track, which trains them to take on leadership positions in schools and the Ministry of Education’s headquarters, and where they can advance to become the Director-General of Education (see Figure 21). These career tracks are associated with an Enhanced Performance Management System (EPMS). The EPMS is competency-based and defines the knowledge, skills, and professional characteristics appropriate for each track, thus making clear the expectations and kinds of behavior needed for success in each of the tracks. It is developmental in nature and supports teacher improvement and performance. The EPMS process involves: a. Performance planning: The teacher starts the year with a self-assessment and develops his/her goals for teaching, instructional innovations, improvements at the school, professional development, and personal development and meets with his/her reporting officer who is usually the Head of Department for a discussion about target setting and performance benchmarks. b. Performance coaching: Performance coaching takes place throughout the year and particularly during a formal mid-year review where the reporting officer meets with the teacher to discuss progress and share needs and to coach and provide feedback and support. c. Performance evaluation: At the end of the year, the reporting officer conducts the teacher’s appraisal interview and reviews his/her actual performance against the performance plan. The reporting officer gives the teacher a performance grade, which affects the value of the teacher’s annual performance bonus. It is also during the performance evaluation phase that decisions regarding the teacher’s promotion to the next level are made based on “current estimated potential (CEP).� The reporting officer makes this CEP decision in consultation with senior staff who have worked with the teacher, and it is based on “observations, discussions with the teacher, evidence of portfolio, and knowledge of the teacher’s contribution to the school and community.� The professional development and training of teachers is a key component in the pursuit of excellence in education service delivery in Singapore. First of all, Singapore is very selective about choosing the right candidates for pre-service teacher training, focusing on their academic achievement, their communication skills, and their motivation for teaching. For every 100 applicants to teacher training colleges, only 20 get accepted, but almost 90 percent of those who are accepted graduate and enter the teaching force. Salaries are competitive with other professions. This selectivity has contributed to the reputation of teaching as a prestigious career. In line with several other high performing systems in the world, pre-service teacher training is highly centralized and quality controlled so that all teachers receive pre-service training at the National Institute of Education (NIE) at Nanyang Technological University in Singapore. In-service teacher training and professional development is recognized as critical to ensuring that teachers keep up with rapid changes and continually improve their practice. Towards this end they are entitled to 100 hours of professional development every year. Teachers, in consultation with their performance coaches, can undertake courses at the NIE that may lead to diplomas or degrees participate in networks of teaching practices, access school-based teacher training from experts, or even take online courses. Schools also have dedicated funds that they can use to finance additional teacher professional development in addition to the specific requirements mandated by the central government. Source: Singapore Ministry of Education website accessed on June 4, 2012, Lee and Tan (2010),World Bank (2010),OECD (2011b) Chapter 7, and Sclafani and Lim (2008). 22 Promoting Excellence in Turkey’s Schools – Recommendation 2: Promoting effective FIGURE 21 Teaching Career Paths in Singapore teachers’ practices and classroom dynamics. It is critical to delve deeper into Director - General of Education current teaching practices and classroom dynamics in Turkey’s schools. One way of Director Chief Specialist doing this that has been popular in other Deputy Director Principal Specialist countries is by applying the Stalling method Cluster Superintendent of classroom observations to a nationally Principal Master Teacher Principal Lead Specialist representative sample of teachers to assess Master Teacher Vice Principal Senior Specialist 2 what makes teachers effective (see Box Lead Teacher Head of Department Senior Specialist 1 4). This technique would make it possible Senior Teacher Subject Head / Level Head to collect information on teachers’ use of instructional time, materials, and core pedagogical practices, as well as their ability Classroom Teacher to keep students engaged in learning. This assessment would be a crucial input for Teaching Track Leadership Track Senior Specialist Track policymakers when considering how best to modify teacher training and practice in Turkey. Source: Ministry of Education, Singapore BOX 4 The Stallings Method of Classroom Observation Getting Inside the Black Box Student learning is the prime determinant of individual returns and economy-wide gains from education (Hahushek and Woessman, 2007). As teacher quality is the prime determinant of student learning, understanding what makes teachers effective in the classroom is of key importance. What the education literature indicates is that having materials and the teachers in the classroom are not enough. Education is the transformation of inputs into learning outcomes and this happens in the classroom. Therefore there is the need to have a tool for analyzing “the black box� of the classroom. The Stallings method is one of the observational procedures or techniques which have been widely used to understand what makes teachers effective in the classroom. This is a quantitative method of measuring classroom behaviors from direct observations that specify both the events and behaviors that are to be observed and how they are to be recorded. The Stallings method “Classroom snapshot� instrument measures the following: (a) teacher’s use of instructional time; (b) use of materials in the classroom, including information and communications technologies; (c) the domain of core pedagogical practices; and (d) the ability to keep students engaged. In its implementation 10 observations of each class are taken at regular intervals and coded using a standardized grid in a statistically representative sample of schools. These interactive coding systems allow the observer to record nearly everything that students and teachers do during a given time interval. These interaction systems are very objective and typically do not require the observer to make any high inferences or judgments about the behaviors they observe in the classroom. In other words, these low-inference observational systems provide specific and easy identifiable behaviors that observers can easily code. The methods advantages are (i) reliability: its highly reliability across observers and countries; (ii) benchmarking: its results can be compared to US/OECD good practice indicators; (iii) simplicity: the method requires relatively short training to produce qualified observers; and iv) analytical power: Stallings variables are statistically representative of the education system and highly correlated with learning results. Classroom observation has many valid and important educational purposes. Three important purposes or areas where systematic classroom observation has been widely used are: (1) description of instructional practices; (2) investigation of instructional inequities for different groups of students; and (3) improvement of teachers’ classroom instruction based on feedback from individual classroom or school profiles. Promoting Excellence in Turkey’s Schools 23 III.C Reducing Inequalities of Turkey has now achieved gender neutrality Opportunities in Upper Secondary in access to high quality secondary schools. Education Even with these improvements enrollment in these schools is skewed towards students 49. In Turkey, a high-stakes centralized from better off families. Expanding access entrance exams for upper secondary schools to the best programs for all students leads to segregation of students both by would reduce performance gaps due to Socio Economic Status (SES) and academic socioeconomic differences. This may ability.31 Students in the best program – require a review of the current system of science and Anatolian high schools – had entrance exams, which encourage out of PISA scores that are equivalent to having about class tutoring which only the middle and three additional years of schooling compared upper income families can afford (see students in the streams with the lowest scores World Bank, 2011). Another option is to (i.e. vocational schools). The selection process increase the diversity of academic programs that streams students into different types of within schools. Some of less stratified upper secondary schools according to their countries of the OECD (Canada, Denmark, abilities is likely to have an effect on their Finland, Iceland, New Zealand, Norway, academic performance not only on that cycle Sweden and UK) may offer lessons for but also in earlier stages in the school cycle. Turkey. These countries have only one type of school, in some cases allowing choice - Recommendation: Continue expanding of academic programs within the school, access to the best educational programs which means that the stakes at play when for students from all socio-economic moving to upper secondary education are backgrounds. Turkey has been expanding not as high as those in Turkey, where an the share of students attending the best examination streams people into different schools (Science and Anatolian high types of schools that vary widely in terms of schools). At the same time the share of girls quality (World Bank, 2013). enrolled in those schools has increased and 31 See section II.C for details. 24 Promoting Excellence in Turkey’s Schools BOX 5 School choice in secondary schooling Most countries apply rules that restrict school choice in secondary schooling. The most common criteria include location, whether there are siblings in the same school, academic record, admission test, financial status, and religious affiliation. However, in most cases these rules are combined into a more elaborate mechanism. New Zealand, for instance, has an open enrollment scheme, i.e. children are no longer guaranteed a place in their local school and oversubscribed schools are free to choose the selection criteria. In Finland, students that live in the school neighborhood have preference over others. Nonetheless, schools can apply their own criteria for those outside their catchment area. In Spain, oversubscribed schools follow an index at the individual level that accounts for proximity to residence, attendance of siblings, and family’s financial situation. Finally, in the USA and England local authorities administer the school choice mechanism, which leads to wide variation of rules. Most imposed restrictions end up spurring certain degree of stratification by SES. The soaring prices of houses in districts with good schools are an example. Academic achievement can be highly correlated with SES. Hence, schools that select students based on the former generate high social stratification. The less stratified countries contains several members of the OECD that may offer lessons for Turkey (Canada, Denmark, Finland, Iceland, New Zealand, Norway, Sweden and UK). These countries shared some characteristics. For example, all of them offer only one school type for 15 year-old students, while Turkey offers several types of schools. In Denmark, Finland, Iceland, Norway, Sweden and UK between 7 and 24 percent of the students attend schools where academic performance is considered for admission. Canada and New Zealand have 53 and 43 percent, respectively; while in Turkey that figure is 66 percent. It is interesting to observe, however, that a big share of the students in these countries (even higher than that in Turkey) is in schools that group by ability (within the school). In Finland selection into the academic track considers prior academic performance. However, vocational graduates can still pursue higher education. Sweden grants free school choice to parents, who can even use vouchers in private schools. Admissions are on a first-come-first-served basis. Finally, out of these eight comparators, five can’t apply any selection criteria in lower secondary education, Finland and England consider academic performance, but Scotland doesn’t apply any criteria. The fact that all the comparison countries have only one type of school means the stakes at play when moving to upper secondary education are not as high as those in Turkey, where an examination streams people into different schools that vary widely in terms of quality. Source: World Bank (2013), “School Choice, Inequality and School Transitions- a short note for the Government of Turkey�, World Bank: Washington D.C. (processed) Promoting Excellence in Turkey’s Schools 25 Summary of Recommendations 50. The report finds that the main areas for b. Improving teaching methods by: further improvements are i. Promoting the use of effective a. Encouraging an earlier start in school: learning strategies: An analysis of the According to PISA, across all OECD countries PISA results indicates that learning in one year of pre-school was correlated with 34 Turkey’s schools could be significantly additional points in reading, and around two- improved if all students were taught thirds of this impact was due to peer effects. On how to use effective learning strategies. the basis of these estimates, Turkey’s average As in other PISA countries, students score in reading would increase by 24 points who report having been taught “good� if 100 percent of the country’s students could learning strategies (those used to attend a good quality pre-school for one year. understand, remember, and summarize Also, for the 15-year-olds who were tested in information) tend to produce better test 2009, the average age of starting first grade results. In Turkey, estimates have shown was 6.8 years. Estimates show that bringing the that an increase of about one standard average starting age closer to 6 could improve deviation of the OECD indexes related reading results by up to 10 points. to the use of the effective learning strategies would increase average Recommendations: student performance respectively by � Offer targeted support to expand 24 points for the understanding and enrolment in kindergarten to the lagging remembering techniques and by 23 provinces. points for the summarizing ones. � Ensure equity in access to ECE services by Two-thirds of the benefits of the good targeting the poorest household nationwide. learning strategies actually operate � Improve quality by rigorously monitoring, through peer effects. In total, an evaluating, and enforcing standards for pre- increase of just one standard deviation school education. in the number of students that are � Ensure adequate funding for ECE and taught these techniques would increase involve private and not-for-profit actors in the the average reading score in Turkey by ECE system. 68 points (or the equivalent of almost � Reduce effective entrance age to Grade 1. one and a half years of schooling). 26 Promoting Excellence in Turkey’s Schools Moreover, if all Turkish students had at c. Reducing inequalities of opportunities in least the average amount of knowledge upper secondary education: of these techniques, the share of the “functionally illiterate� would drop In Turkey there are several types of upper from 25 percent to 10 percent. secondary schools and average performance in PISA of these school types is quite different. Students in the best program – science and Recommendations: Anatolian high schools – had PISA scores that � Through pre-service and in-service are equivalent to about additional three years training, train teachers to educate students of schooling than students in the streams with in the use of effective learning strategies. This the lowest scores (e.g. vocational schools). The can be done separately or in the context of the selection process that streams students into wider reform of teacher career management different types of upper secondary schools and incentives. according to their abilities is likely to have an � The training should incorporate a module effect on their academic performance not only to address the specific needs for effective on that cycle but also in earlier stages in the learning strategies of disadvantaged students. school cycle. It is noteworthy that streaming has become gender neutral in Turkey, with the share of girls increasing significantly in Improving ii. classroom practice: the best schools. Finally, some of less stratified Teachers’ practices and classroom countries of the OECD (Canada, Denmark, dynamics play a significant role in Finland, Iceland, New Zealand, Norway, how students learn in Turkey. PISA Sweden and UK) may offer lessons for Turkey. measures teachers’ practices in three These countries have only one type of school, dimensions: (i) the disciplinary climate; in some cases allowing choice of academic (ii) the quality of teacher-student programs within the school, which means relationships: and (iii) stimulation that the stakes at play when moving to upper of student reading engagement. By secondary education are not as high as those in improving these teaching practices Turkey, where an examination streams people within the classroom by one standard into different types of schools that vary widely deviation, Turkey’s scores in the three in terms of quality (World Bank, 2013). PISA disciplines would increase by about 11 points. Recommendation: � Continue expanding access to the best educational programs for students from all Recommendation: socio-economic backgrounds. � Carry out a nationally representative Consider reducing the types of upper assessment of teaching practices and classroom secondary schools in Turkey, while expanding dynamics using the “Stalling method� of the number of academic programs within classroom observations. schools. Promoting Excellence in Turkey’s Schools 27 Proposed Future Research Agenda 51. The findings in this report could be also assess how teacher’s instruction time usefully complemented with further research correlates with student learning. including: c. A diagnostic on Teacher Career a. A corroboration of PISA findings using Management to assess current teacher other student assessments, such as management structures at the central, TIMMS and national test results. The provincial and school level. This analysis use of additional student achievement would be based on structured interviews scores could allow to test the robustness with key informants from central, of the relative weights accorded to socio- provincial and sub provincial levels, economic and school specific effects and and the application of questionnaires thus help policy makers identify measures to a sample of teachers. The analysis to maximize learning opportunities would attempt to address the following for socio-economically disadvantaged questions: How effectively do MoNE, students. Higher Education Council (YOK), and the Student Selection and Placement Center b. An assessment of current teacher (OSYM) interact to prepare and select well- practice and classroom dynamics in qualified teachers? What role does MoNE Turkey’s schools. This would be based play in the recruitment, training, and on an application of the Stallings method management? How are teachers rewarded of classroom observations to a nationally for good performance? The information representative sample of schools and its collected from these sources could be aim is to provide useful information to complemented by de jure information understand what makes teachers effective collected by the application of the System in the classroom. The following questions Approach for Better Education Results could be addressed: How much time (SABER) –a tool developed by the World teachers allocate to instruction within the Bank to benchmark teacher policies classroom? What is their degree of use according to evidence-based global of available education materials? What is standards and best practice. The World their overall domain of core pedagogical Bank could also provide an in-depth look practices? How effective they are to keep and draw lessons from country experiences students engaged? If the sample of schools of potential relevance for Turkey in the to be used coincides with the ones used area of teacher career management (e.g. in national or international assessments Singapore’s Education Service Professional (e.g. PISA or TIMMS), the analysis could Development and Career Plan). 28 Promoting Excellence in Turkey’s Schools References Barrera-Osorio, F., T. Fasih, H. Patrinos, and L. Machin, S. and J. Vernoit, (2010). “Academy Santibáñez (2009). “Decentralized Decision- schools: who benefits?� CentrePiece: The Making in Schools: The Theory and Magazine for Economic Performance 325, Evidence on School-Based Management�; Centre for Economic Performance, LSE. The World Bank: Washington, DC. Machin, S. and J. Vernoit (2011). “Changing Clark, D. (2009). “The Performance and School Autonomy: Academy Schools Competitive Effects of School Autonomy.� and their Introduction to England’s Journal of Political Economy 117(4): 745- Education.� CEE Discussion Papers 0123, 783. Centre for the Economics of Education, Hanushek, E. A. andS.G. Rivkin (2006). LSE. “Teacher Quality� in E. A. Hanushek & F. MoNE (2011). “Milli Eğitim Istatistikleri: Welch (Eds.), Handbook of the Economics Örgün Eğitim 2010-2011 (National of Education (Vol. 2). Amsterdam: Elsevier. Education Statistics: Formal Education Hanushek, E. A., Suzanne Link, and Ludger 2010-2011)�, Ankara: Ministry of National Woessmann (2011). “Does School Education. Autonomy Make Sense Everywhere? Panel MoNE (2012). “A Research into the Assessment Estimates from PISA,� IZA Discussion of Beneficiaries of the Curriculum Reform Papers 6185, Institute for the Study of of the Secondary Education Project�, Labor (IZA) Ankara: Ministry of National Education. Hanushek, E. and L. Wöessmann (2007). “The Nye, B., S. Konstantopoulos, and l.V. Hedges Role of Education Quality in Economic (2004). “How Large Are Teacher Effects?� Growth�, World Bank Policy Research Educational Evaluation and Policy Analysis, Working Paper 4122, Washington, D.C.: 26(3), 237-257. The World Bank. OECD (2009). “Creating Effective Teaching and Hanushek, E. and L. Wöessmann (2010). “The Learning Environments: First Results from Economics of International Differences in TALIS�, Paris: Organisation for Economic Educational Achievement�, NBER Working Co-operation and Development. Paper #15949, Cambridge, MA: National OECD (2010). “PISA 2009 Results: Learning Bureau of Economic Research. to Learn Student: Engagement, Strategies, Harmon, C., H. Oosterbeek, and I. Walker and Practices, Volume III.� Paris, (2003). “Returns to Education: Organisation for Economic Co-operation Microeconomics�, Journal of Economic and Development. Surveys, 17(2):115-155. OECD (2011a). “PISA in focus�, Paris: Lee, Christine Kim-Eng and Mei Ying Tan Organisation for Economic Co-operation (2010), “Rating Teachers and Rewarding and Development. Teacher Performance: The Context of OECD (2011b). “Strong Performers and Singapore�, Paper presented at APEC Successful Reformers in Education: Conference on Replicating Exemplary Lessons from PISA for the United States�, Practices in Mathematics Education, Koh Paris: Organisation for Economic Co- Samui, Thailand, 712 Mar. 2010 operation and Development. Jakubowski M., H. A. Patrinos, E. Porta, and Park, A., and E. Hannum (2001). “Do Teachers J. Wiśniewski (2011). “The Impact of the Affect Learning in Developing Countries?: 1999 Education Reformin Poland�, Policy Evidence from Matched Student-Teacher Research Working Paper 5263, The World Data from China.� Paper presented at the Bank. Conference Rethinking Social Science Promoting Excellence in Turkey’s Schools 29 Research on the Developing World in As A Model for Teacher Development�, the 21st Century. Park City, Utah: Social Washington, D.C.: The Aspen Institute Science Research Council. Education and Society Program. Rivkin, S. G., E.A. Hanushek, and J.F. Kain Vignoles, A., R. Levacic, J. Walker, S. Machin, (2005). “Teachers, Schools, and Student and D. Reynolds (2000). “The Relationship Achievement.� Econometrica, 73(2), 417- between Resource Allocation and Pupil 458. Attainment: A Review.� Report 228. Rockoff, J. (2004). “The Impact of Individual London, UK: Centre for the Economics of Teachers on Student Achievement: Education, London School of Economics Evidence from Panel Data�, American and Political Science. Economic Review 94 (2): 247-52. World Bank (2011). “Improving the Quality and Sanders, W. L. (1998). “Value Added Equity of Education in Turkey, Challenges Assessment.� School Administrator, 11(55), and Options.� Human Development 24-27. Department, Europe and Central Asia Sanders, W. L., and J.C. Rivers (1996). Region, Washington DC. “Cumulative and Residual Effects of World Bank (2012).“Expanding and Improving Teachers on Future Student Academic Early Childhood Education in Turkey�, Achievement.� Research Progress Report. World Bank: Washington D.C. Knoxville, TN: University of Tennessee World Bank (2013), “School Choice, Inequality Value-Added Research and Assessment and School Transitions- a short note for Center. the Government of Turkey�, World Bank: Sclafani, S. and E. Lim (2008). “Rethinking Washington D.C. (processed) Human Capital in Education: Singapore 30 Promoting Excellence in Turkey’s Schools Annex 1 TABLE 1 Evolution of performance and inequalities in Turkey TABLE 2 Analysis of the determinants of school quality Source: WB staff calculations using PISA 2009. Effects are measured by a linear regression at the student level, when age, gender, socioeconomic index of both the pupil and the school (average index) and mother tongue are controlled for. The indexes of “teaching environment� and “awareness of strategies� are computed as a weighted average using the multipliers estimated by the model (2). These indexes are rescaled such as their standard deviation at the National level is unity. Therefore, the estimated multipliers of these indices in the models (3) to (6) are the marginal effects of an increase of one standard deviation. Standard errors are computed after clustering observations at the school level. Statistical significance at respectively the 0.1%, 1% and 5% level is indicated by ***,**, and *. Promoting Excellence in Turkey’s Schools 31 FIGURE 1 Distribution by grade of the 15 year old according to PISA sampling 70 66.62 60 52.12 50 % of students 39.19 40 2003 30 25.17 2009 20 10 0.84 4.39 3.2 3.79 0.25 0.7 3.5 0.22 0 7 8 9 Grade 10 11 12 Source: Authors based on PISA datasets. FIGURE 2 Proportion of Girls and Boys in Turkey PISA sample, 2003 and 2009 Source: Authors based on PISA datasets. FIGURE 3 Grade distribution in PISA sample in Turkey, 2003 and 2009 by gender Source: Authors based on PISA datasets. 32 Promoting Excellence in Turkey’s Schools FIGURE 4 Relative performance by gender and subject, Turkey PISA 2003 and 2009 Source: Authors based on PISA datasets. Promoting Excellence in Turkey’s Schools 33 Annex 2 Oaxaca-Blinder decomposition One uses the underlying linear model to compute the Oaxaca-Blinder decomposition, with Perf the test score for reading, math or science, β the vector of returns that differ for 2003 and 2009, X, the vector of explaining variables and c an intercept that also differ for 2003 and 2009: The following explaining variables are included: - Gender - Grade - Socioeconomic index at the individual level - Average socioeconomic index at the school level - Wealth index at the individual level - Mother tongue - Students’ learning strategies (memorization, elaboration, and control) - Teacher-students relation and disciplinary climate Equation (E1) is estimated separately using 2003 and 2009 variables. One then uses the results to compute the Oaxaca-Blinder decomposition: Changes in factors Changes in returns Change in effectiveness Oaxaca-Blinder decomposition of the performance gap across countries The approach is similar to the decomposition of performance gains. One uses the same underlying model described by equation (E1), except one compares Turkey in 2009 with a benchmarking country in 2009 rather than Turkey in 2009 versus Turkey in 2003. This gives the following model, with i denoting a country. The corresponding Oaxaca-Blinder decomposition of the gap between Turkey and a benchmark country (BNC) Gap due to factors Gaps linked to returns Effectiveness gap 34 Promoting Excellence in Turkey’s Schools Annex 3 Linear models can be used to measure the weight of students’ characteristics on performance. Let us denote a the age, g the gender, m the mother tongue and h the socioeconomic index at the individual level and let us denote by the average socioeconomic index at the school level. Let us denote by c a constant term and by ε a random term normally distributed. Linear model 1 measures the effects of individual factors on reading performance: Model 1: The derived measure of the variance in the reading performance explained by age, gender and the individual socioeconomic index is then: Linear model 2 measures the effects of individual factors at the school level on reading performance: Model 2: The derived measure of the variance in the reading performance explained by individual factors at the school level is then: Model 3: The derived measure of the variance in the reading performance explained by individual factors at the school level is then: Model 4: The derived measure of the variance in the reading performance explained by individual factors at the school level is then: