101562 Poland: Skilling up the next generation An analysis of Poland’s performance in the Program for International Student Assessment Poland: Skilling up the next generation An analysis of Poland’s performance in the Program for International Student Assessment Contents Figure 11. PISA scores in mathema cs improved between 2000 and 2012 across every ESCS percen le........... 27 Acknowledgments .................................................................................................................................................. 6 Figure 12. Improvements in performance and ESCS in Polish schools between 2000 and 2012 ......................... 28 Abbrevia ons and Acronyms.................................................................................................................................. 6 Figure 13: Between-school variance in mathema cs performance is limited in Poland ...................................... 29 Execu ve Summary ................................................................................................................................................ 7 Figure 14: Stra fica on in the educa on system according to PISA 2012 scores ................................................ 30 1. Why Skills Ma er for Poland ............................................................................................................................ 11 Figure 15. Index of equality of opportuni es: Poland and other countries, 2012 ............................................... 31 The importance of cogni ve skills ........................................................................................................................ 13 Figure 16. PISA 2012 score gaps by loca on and gender, Poland and comparator countries .............................. 32 Poland’s educa on system ................................................................................................................................... 16 2. Cogni ve Skills of Polish 15-year-old Students ................................................................................................. 19 Tables Snapshot of Poland’s performance in PISA........................................................................................................... 20 Table 1: Three waves of reform in Poland’s general educa on system ................................................................ 23 Performance and equity ....................................................................................................................................... 26 Table A1: Percent of popula on aged 15+ by highest level of schooling a ained and average years of schooling in Poland, 1990-2010 ....................................................................................................................... 40 3. Policy Implica ons: Remaining Challenges in the Polish Educa on System ..................................................... 33 Addressing performance gaps in upper-secondary educa on ............................................................................. 34 Improving problem-solving skills .......................................................................................................................... 34 Promo ng equity.................................................................................................................................................. 35 References ............................................................................................................................................................ 37 Annex ................................................................................................................................................................... 37 Boxes Box 1: Reforms to the Polish general educa on system since the 1990s ............................................................. 18 Box 2: Digging deeper: Performance in mathema cs .......................................................................................... 26 Box 3: PISA’s Index of Economic, Social, and Cultural Status ................................................................................ 27 Figures Figure 1. Poland’s income convergence, 1995 - 2013: A sizeable catch-up but s ll a long way to go .................. 12 Figure 2. Three dimensions of skills ..................................................................................................................... 14 Figure 3. A growing intensity of use of non-rou ne cogni ve and interpersonal skills in Poland ........................ 15 Figure 4. The Polish educa on system before and a er the 1999 reform ........................................................... 17 Figure 5. Poland’s PISA 2012 scores were above OECD averages and most neighboring countries ..................... 20 Figure 6. Poland’s PISA performance by discipline, 2000-12 ................................................................................ 21 Figure 7. Distribu on of students by proficiency level in math: (a) Poland’s progress in 2000-2012; (b) Poland and comparators in 2012.......................................................................................................................... 22 Figure 8. PISA scores in mathema cs by public expenditures per student, Poland and other PISA countries .......................................................................................................................... 23 Figure 9. Problem-solving scores and comparison with mathema cs scores (PISA 2012) ................................... 24 Figure 10. Poland’s PISA 2012 performance on mathema cs subscales compared to the average mathema cs performance...................................................................................................................... 25 Acknowledgments This report was prepared by a World Bank team consis ng of Chris an Bodewig, Lucas Gortazar, Ka a Herrera Sosa, Daniel Kutner, Jeremie Amoroso and Mar n Moreno under the overall guidance of Mamta Murthi, Country Director, Central Europe and the Bal cs, Marina Wes, Country Manager for Poland and Cris an Aedo, Educa on Global Prac ce Manager. The report benefited from comments from peer reviewer Harry Patrinos and from Zbigniew Sawiński, Michał Sitek and Jerzy Wiśniewski. Marc De Francis edited the report. Abbrevia ons and Acronyms ESCS Economic, Social, and Cultural Status Executive ECA ECE EU GDP OECD Europe and Central Asia Early childhood educa on European Union Gross domes c product Organisa on for Economic Co-opera on and Development Summary OLS Ordinary least squares PIAAC Program for the Interna onal Assessment of Adult Competencies PIRLS Progress in Interna onal Reading Literacy Study PISA Programme for Interna onal Student Assessment RIF Re-centered influence func ons TIMSS Trends in Interna onal Mathema cs and Science Study UN United Na ons UNESCO United Na ons Educa onal, Scien fic and Cultural Organiza on VET Voca onal educa on and training page 8 page 9 Executive Summary 15-year-olds prior to the 1999 reform persist today in upper secondary educa on, where the performance of Facing the prospects of rapid demographic aging and decline over the coming decades, Poland needs a highly students in voca onal upper-secondary schools trails that of their peers in general educa on. Moreover, Poland’s skilled workforce to help generate the produc vity growth that it needs to fuel con nued convergence of its living performance in problem solving in the 2012 PISA was well below the OECD average. Lastly, equity remains an issue standards with those of its West European neighbors. in need of further a en on: The difference in mathema cs performance between the top and bo om quin les are the equivalent of nearly three years of schooling – a wider gap than the OECD average and much wider Skilling up the workforce starts with equipping youth with the right cogni ve and socio-emo onal founda on than in other top-performing countries such as Japan and Korea. While Poland’s school system today is not very skills. Interna onal research has iden fied three dimensions of skills that ma er for good employment outcomes stra fied according to the socioeconomic background of its students, socioeconomic background s ll ma ers for and economic growth: cogni ve skills, such as literacy, numeracy, and crea ve and cri cal thinking or problem- performance. solving; socio-emo onal skills and behavioral traits, such as conscien ousness, grit, and openness to experience; and job- or occupa on-specific technical skills, such as the ability to work as an engineer. Cogni ve and socio- emo onal skill forma on starts early in a person’s life. Good cogni ve and socio-emo onal skills provide a necessary founda on for the subsequent acquisi on of technical skills. Put differently, poor literacy and numeracy skills severely undermine a person’s ability to benefit from further training and lifelong learning. An impressive transforma on in cogni ve skills has occurred among youth in Poland. This report focuses on cogni ve skills. It examines results for Poland from the Program for Interna onal Student Assessment (PISA), which assesses the mathema cs, reading, and science competencies of 15-year-olds. The findings point to an impressive transforma on in the cogni ve skill set of the youth in Poland between 2000 and 2012. Poland raised its scores in mathema cs, reading, and science to the equivalent of what students would have learned in one addi onal year of schooling. Its PISA scores are now above the OECD average and at the same levels as in countries such as Finland and Germany. Aggregate gains in quality have gone hand in hand with improvements in educa onal equity. Poland has seen an increase in the propor on of its students at the top performance level, and the share of its students who are poor performers has declined. Students from poor and well-off socioeconomic backgrounds alike saw performance improvements. All this has been achieved with stable levels of educa on spending, at about 5 percent of GDP and below the OECD average. The overall effects of reform on Poland’s PISA scores has been posi ve, although isola ng the precise impact of each reform element is difficult. Given the many educa on (and non-educa on) reforms in Poland since the early 1990s, it is difficult to isolate the effects of each element of reform on Poland’s PISA results. The overall effects are clearly posi ve. In addi on to changes in the accountability arrangements with a strengthening of the role of local governments and the introduc on of standardized examina ons, the Polish authori es introduced a change to the structure of the educa on system in 1999, which delayed selec on between general and voca onal tracks and effec vely added one year of exposure to general curriculum content. Taking effect a er the first PISA test in 2000, when Poland performed rela vely poorly, this change has been rigorously evaluated and shown to have had a significant posi ve effect on performance. Despite the many successes, some challenges remain, par cularly regarding problem-solving skills and equity in achievement. There is evidence from PISA assessments replicated for older students in upper-secondary educa on in 2006, 2009 and 2012 that performance gaps previously found between voca onal and general schools for page 11 Chapter 1 Why Skills Matter for Poland page 12 page 13 1. Why Skills Matter for Poland But what about skills? This report places a spotlight on the next genera on and examines whether Poland’s How can Poland achieve convergence in living standards with its Western European neighbors when its youth are leaving the compulsory educa on system with the right set of skills needed for further educa on and popula on is aging and shrinking? Poland’s income growth over the last two decades has been remarkable, but training and for produc ve employment. It finds that Poland has made major strides in raising the skills of the next the country s ll has a long way to go to catch up with the living standards of its EU15 neighbors. In 1995, Poland’s genera on and to prepare them for the demands of a growing and changing economy. GDP per capita stood at about 37 percent of the EU15 average, and by 2013 it had risen to 66 percent (Figure 1). Looking ahead, Poland’s long-term economic growth prospects are put at risk by demographic change: According The importance of cognitive skills to United Na ons projec ons, the working-age (ages 15 to 64) share of the popula on is expected to fall from 69.5 Interna onal evidence shows how much the skills of a country’s workforce ma er for economic growth percent in 2015 to 57 percent by 2050. In contrast, the share of the popula on 65 and above will increase from and shared prosperity. Interna onal evidence suggests that quality of educa on is one of the most important 15.3 percent in 2015 to 29 percent by 2050.1 With fewer workers and more old-age dependents, labor produc vity determinants of long-term economic growth.2 Recent research (Hanushek and Woessmann, 2007 and 2012), improvements will be key to sustained economic growth. drawing on student assessment surveys from 1960 onward, es mates that an improvement of 50 points in scores on the Organisa on for Economic Co-opera on and Development’s (OECD’s) Program for Interna onal Student Figure 1. Poland’s income convergence, 1995 - 2013: A sizeable catch-up but s ll a long way to go Assessment (PISA) would imply an increase of 1 percentage point in the annual growth rate of GDP per capita.3 Both the share of students achieving basic literacy and the share of top-performing students ma er for growth (Hanushek and Woessmann, 2007; OECD, 2010). A recent OECD (2015) report presents economic returns to universal basic skills, defined as all students achieving level 1 skills (420 points) in PISA. While low-income countries with lagging educa on systems stand to gain the most, advanced middle- and high-income countries can expect a significant boost to long-run economic growth simply by making their educa on systems deliver be er for the weakest students: The report finds that on average, high-income countries could gain a 3.5 percent higher discounted average GDP over the next 80 years if they were to ensure that all students achieved basic skills (as defined above). As will be presented in this report, a declining yet s ll significant share of Polish 15-year-olds currently perform poorly in PISA. Ensuring universal basic skills in Poland could add 2 percent discounted future GDP. Ensuring basic cogni ve skills for all also helps to make growth inclusive. Beyond aggregate economic growth, educa on improves the living standards of individuals. With more educa on people are able to acquire more and higher-order skills, making them more produc ve and employable and extending their labor market par cipa on over their life me. That in turn leads to higher earnings and be er quality of life.4 Educa on is an engine of social mobility: Human capital is a key asset in income genera on and hence cri cal to reducing poverty and increasing Source: World Bank Staff es mates using Eurostat data. shared prosperity (Bussolo and Lopez-Calva, 2014). Making the best use of human capital is at the heart of policies that can sustain increases in living standards. “Skills” can be differen ated along separate though mutually reinforcing dimensions: cogni ve, socio-emo onal, Mi ga ng the risk to economic growth from popula on aging and decline involves expanding the number of workers and technical skills. Figure 2 presents the differen a on across the different dimensions. CogniƟve skills include and enhancing their produc vity. Expanding their numbers means increasing the employment rate at all ages, literacy and numeracy, such as that measured in PISA, but also include competencies like cri cal thinking and especially among young and older workers, and encouraging immigra on. Enhancing produc vity means raising problem-solving. Socio-emoƟonal skills capture one’s ability to interact with others as well as traits such as the skills of the current and future workforce, in addi on to other measures such as reforms in product and capital determina on and focus on ge ng a job done. Technical skills capture one’s ability to perform technical tasks markets. According to Eurostat labor force survey data, Poland faces high youth unemployment which, at 24.2 percent in 2014, was above the EU average. In addi on to the lost income, poor labor market outcomes at the beginning of 2 a person’s professional life may have long-las ng, nega ve impact on his or her long-term labor market outcomes, 3 See Sala-i–Mar n, Doppelhofer, and Miller (2004). See Hanushek and Woessmann (2007) and Hanushek (2010). Using these tests as measures of cogni ve skills of the popula on, they show that countries limi ng the possibili es of young people (Schmillen and Umkehrer, 2013; Kahn, 2010; Gregg and Tominey, 2005). that had be er quality of educa on in the 1960s experienced faster economic growth during the years 1960-2000, controlling for other factors. 4 See Hanushek (2013). 1 United Na ons Popula on Projec ons, 2012 Revision. page 14 page 15 in any occupa on, for example in work as a plumber or engineer. Measuring the level of educa onal a ainment Figure 3. A growing intensity of use of non-rou ne cogni ve and interpersonal skills in Poland does not automa cally mean measuring actual skills. While many countries in Central and Eastern Europe have seen increases in educa onal a ainment (years of educa on, level of educa on completed) since the start of the economic transi on, they have not necessarily seen improvements in their performance in interna onal student assessments that measure cogni ve skills, such as PISA (Sondergaard and Murthi, 2011). Figure 2. Three dimensions of skills Source: Aedo et al., 2013. This report focuses on cogni ve skills and examines evidence from the PISA assessment of mathema cs, reading, and science competencies of Polish 15-year-olds. Introduced by the OECD in 2000, PISA is a worldwide study of 15-year-old school students’ performance in three different disciplines: mathema cs, science, and reading. PISA focuses on the competence of students and their ability to tackle real-life problems in those three disciplines and emphasizes skills that are cri cal for individuals’ personal and professional development. A sample ques on from mathema cs illustrates the applied nature of the PISA tests: “Nick wants to pave the rectangular pa o of his new house. The pa o has length 5.25 meters and width 3.00 meters. He needs 81 bricks per square meter. Calculate how many bricks Nick needs for the whole pa o.”6 In assessing the performance of Polish 15-year- Source: Bodewig and Badiani-Magnusson (2014) olds, PISA mainly captures those students who are in the third year of lower secondary educa on (gymnazjum). The skills content in the labor market in many economies is changing, and Poland is no excep on. In a widely Given that skills forma on is cumula ve, scores reflect not just the competencies acquired in those schools but cited study, Autor, Levy and Murnane (2003) show that the demand for non-rou ne cogni ve (analy cal and competencies acquired even earlier in the educa on system. PISA’s scoring system is standardized so that the interpersonal) skills has been on the rise in the United States since 1960. A similar change can be observed in mean score for each discipline among OECD countries in year 2000 (reading), 2003 (mathema cs) and2006 Poland. Overall, the skills required for different tasks (or occupa ons) are star ng to include a higher propor on (science) is 500 points, with a standard devia on of 100 points.7 According to OECD, a gain of 40 points in PISA is of non-rou ne cogni ve analy cal and non-rou ne interpersonal components than before.5 Tasks today usually equivalent to what students learn in one year of schooling.8 Poland has been par cipa ng in all PISA rounds since require higher-level skills and are associated with higher value-added. The analysis does not say whether more of 2000. these skills are being demanded in Poland but not possessed by available workers or if these skills are in excess supply. However, it does seem to suggest that jobs in the country now favor graduates with higher levels of cogni ve, problem solving, and socio-emo onal skills (e.g. interpersonal), as compared to the situa on a few years 6 Addi onal sample ques ons can be found at Source: h p://pisa-sq.acer.edu.au/ ago when the economy was not yet liberalized (Aedo et al., 2013). 7 PISA results have been comparable for reading since 2000, for mathema cs since 2003 and for science since 2006. 8 PISA 2009 Technical Report (OECD 2012). 5While the skills percentages are lower for the Poland data compared to those observed in the original paper of Autor, Levy, and Murnane (for instance, the original ar cle shows about 12 percentage points for non-rou ne cogni ve skills analy cal for the USA, compared with an increase of less than 2 percentage points in Poland), the original paper covered 40 years of data (between 1960 and 2000). The data used to construct the Poland analysis covers just eight years, between 2002 and 2010. page 16 page 17 Cogni ve skills built in childhood and youth are a necessary founda on for successful acquisi on of technical Figure 4. The Polish educa on system before and a er the 1999 reform and job-specific skills later in life. The founda ons of cogni ve (and behavioral) skills are formed early and are Before the reform of 1999 After the reform of 1999 the pla orm upon which later skills are built. The most sensi ve periods for building a skill vary across the three age grade age grade dimensions of skills, and skill forma on benefits from previous investments and is cumula ve. Technical and 6 Zero class (primary school or 0 6 Zero class (primary school or kindergarten) 0 kindergarten) job-specific skills – o en acquired last, through technical and voca onal educa on and training (TVET), higher 7 I 7 I educa on, or on-the-job learning – benefit from strong cogni ve and behavioral skills acquired earlier in the 8 II 8 II educa on system. In other words, the cogni ve skills acquired in childhood and youth, such as those measured by 9 III 9 III PISA, will help workers to con nuously update their technical skills during their working lives. This is of par cular Comprehensive primary school 10 IV 10 IV importance in aging economies such as Poland´s where workers need to adapt to technological progress during Comprehensive primary school their longer working lives. 11 V 11 V 12 VI 12 VI The report is part of a series of World Bank reports that examine PISA data in depth to analyze educa on 13 VII Final test systems and provide policy makers with op ons for evidence-based reforms. Due to its focus on policy, the series 14 VIII 13 I aims to address key challenges in several countries, with a focus on improving educa on quality and equity. This Comprehensive lower secondary school (gimnazjum ) Entrance exam 14 II report provides a snapshot of the performance of Polish 15-year-old students in PISA compared with other peer ISCED 2A 15 I 15 III countries and over me. The report presents: (i) Poland’s performance in PISA since 2000 un l 2012, (ii) the overall General Basic upper Upper evolu on of the system and of socioeconomic condi ons of students and schools and its rela onship with PISA 16 secondary vocational II Final exam secondary school scores; (iii) the remaining challenges for Poland – improving problem solving skills of its students and addressing 17 vocational III 16 General Profiled I school Basic upper upper gen. Upper (liceum ) school vocational remaining equity issues. 18 (technikum ) IV 17 secondary secondary secondary II schools school school vocational 19 Matura V 18 ISCED 3C III ISCED 3A ISCED 3B schools ISCED 3A Poland’s education system Matura 19 Matura Matura IV The Polish educa on systems has high student-popula on coverage and follows a 6-3-3 educa on structure. Source: Jakubowski et al. (2010). Matura Poland’s general educa on system has undergone a significant transforma on since the early 1990s, including a change to its structure in 1999-2000 with the introduc on of a comprehensive lower-secondary school, the Na onal assessments take place at the end of each stage of the system. There is an assessment at the end of Gimnazjum (see Box 1 and Figure 4). Primary educa on lasts six years, followed by three years of lower-secondary primary schooling, when students are 12 years old, for system evalua on purposes but not for placement.9 Un l school and three years of upper-secondary school (four years in the case of voca onal secondary schools, see recently, there was an exam at the end of lower secondary, when students are 15 years old, a er which students Figure 4). Since 1999 pre-primary educa on has been op onal at ages three to five, and gross enrolment at ages a end a three-year upper secondary general school (lyceum), a technical or professional profile, or a two-year three, four, and five has increased significantly, rising from 28, 38, and 48 percent (respec vely) in 2005 to 51, 65, upper-secondary voca onal school, but it was phased out. Moreover, there is a high- stakes (Matura) exam a er and 94 percent (respec vely) in 2012 (OECD, 2014). In 2012 Poland’s enrolment of five-year-olds was on par with upper-secondary school, when students are 18 years old. It is not compulsory but students need to pass it, in the OECD average, while enrolment of three- and four-year-olds was well below average. One year (zero grade) order to qualify for entrance to higher educa on. Lastly, there is an op onal voca onal exam for those students preparatory class for six-year-olds is compulsory before primary educa on. The net enrollment rate for primary a ending a basic or technical voca onal school. and secondary schools are 97 and 98 percent respec vely. As of 2015, preschool at age five will be compulsory and there will be a guarantee of a preschool place for all children ages three and four. Primary school will start from age six (Jakubowski, 2015). 9In principle, the place of residence, and not the outcomes of the assessment, determines the lower secondary school that a student will a end: Schools are supposed to select the students from the catchment area first, but they can also admit other students on the basis of the results of the exam. page 18 page 19 Chapter 2 Cognitive Skills of Polish 15-year-old Students page 20 page 21 2. Cognitive Skills of Polish 15-year-old Students Figure 6. Poland’s PISA performance by discipline, 2000-12 Snapshot of Poland’s performance in PISA Poland’s 15-year-olds performed strongly in mathema cs, reading, and science in 2012. Poland’s scores were above the OECD country average in all subjects: mathema cs (518 points), reading (518 points), and science (526 points) (Figure 5). Poland scored well above the OECD average in mathema cs and at par with countries such as Germany and Finland. These rapid improvements in PISA scores make Poland an interes ng case to examine. Figure 5. Poland’s PISA 2012 scores were above OECD averages and most neighboring countries Source: World Bank staff es mates using PISA data. Note: PISA results have been comparable for reading since 2000, for mathema cs since 2003 and for science since 2006. Performance in PISA mathema cs, science, and reading has increased across the en re performance distribu on, accoun ng for more than one year of schooling in all three subjects (see Figure 7, Panel A, for mathema cs; the shapes are similar for reading and science). Low achievers have benefited more than high achievers. PISA categorizes scores in six levels of proficiency; students who score below level 2 in the reading and mathema cs tests are considered func onally illiterate and innumerate, respec vely. This means that they are not able to understand and solve simple problems, severely limi ng their development and subsequent cogni ve and technical skill acquisi on process. In Poland, the share of students below basic PISA proficiency levels has declined significantly and to levels far below those in neighboring countries (Figure 7, Panels B and C). Conversely, there has been a boost in the share of students performing at the top of the proficiency distribu on. Source: World Bank staff es mates using PISA data The significant gains in Poland’s PISA scores have extended across three dis nct phases. First, Poland saw significant improvements between 2000 and 2003. Its scores presented a broadly stable picture between 2003 and 2009, followed by another significant jump between 2009 and 2012 (Figure 6). The aggregate increases for reading, mathema cs and science across the en re period are around 40 points, the equivalent of one year of instruc on. What have been the drivers of the sudden improvements in performance between 2000 and 2003 and between 2009 and 2012? While evidence shows a causal link between the first jump and the 1999 reform, with its extension of general educa on by one year due to the introduc on of the comprehensive lower secondary schools (Jakubowski et al., 2010), no causal evidence has been found to explain the 2009-2012 improvements. However, one credible hypothesis is that the changes in curriculum and student assessment led to improvements in cogni ve skills, with students tested in 2012 having completed three years of lower secondary educa on under the new curriculum (see also Marciniak, 2015, and Jakubowski, 2015). page 22 page 23 Figure 7. Distribu on of students by proficiency level in math: (a) Poland’s progress in 2000-2012; (b) Poland and embarked on a sustained reform process since the early 1990s. comparators in 2012 Table 1 Poland’s public spending on educa on 2004 2005 2006 2007 2008 2009 2010 2011 Government expenditure on educa on as % of GDP 5.4 5.5 5.3 4.9 5.1 5.1 5.2 4.9 as % of total government 12.7 12.6 12 11.6 11.8 11.4 11.4 11.4 expenditure Government expenditure per student (in PPP$) Primary educa on 3268.3 4072.5 4986.9 5803.7 Secondary educa on 3055.7 3678 4597.4 5263.2 Ter ary educa on 2787 2954.1 2775.1 3603.6 4543.9 Source: UNESCO UIS (2015) Figure 8. PISA scores in mathema cs by public expenditures per student, Poland and other PISA countries Source: World Bank staff es mates using PISA data Poland’s above average PISA performance has come even though below-average public spending. Poland’s public spending on educa on in 2011 was 4.7 percent of GDP (5.5 percent total spending, with 0.8 percent private Source: World Bank staff es ma ons using PISA 2012 data and UNESCO 2012 data. Note: The curve represents a logarithmic approxima on spending), compared to an OECD public spending average of 5.3 percent of GDP (and 6.1 percent total). The of the sca er plots. per-student expenditure has increased over the last years, in part due to the declining student popula on (Table However, Polish 15 year-olds performed less well in the problem solving assessment in PISA 2012. In 2012, the 2). However, PISA performance in 2012 was significantly above what would be expected given the level of public OECD introduced a new element of assessment: a problem-solving category that measures students’ capacity to expenditure per student in 2012 (see Figure 8). Poland’s aggregate PISA performance is similar to that of countries respond to non-rou ne analy cal problems in a digital environment. This capacity is deemed essen al for students that invest more than twice as much per pupil, such as Finland, Netherlands, and Belgium. Poland has guaranteed to achieve their poten al as construc ve and reflec ve ci zens by making use of the new technological tools.1 a level of public spending on educa on over the years to ensure access to a minimum standard of quality and then 1This assessment responds to new demands for skills that people need in order to solve more intensive abstract tasks in their workplace. As computers and computerized machines are being introduced in greater numbers, workers are required to deal with unexpected analy cal tasks that require both high-order skills and computer mastery in order to make the best use of the machines. See Autor, Levy, and Murnane (2003) for a detailed descrip on. page 24 page 25 While Poland’s 15 year-olds score above the OECD averages in reading, mathema cs and science, their results in par cipate in society and for the economy to prosper. This survey is useful to gauge how effec vely educa on the problem solving test were below OECD averages. Figure 9 presents the evidence on problem-solving (Panel A). systems are preparing students with the skills they will need in life and work. Among par cipa ng countries, It also shows the gap compared to the mathema cs assessment and the influence of computer skills on rela ve Poland scores the second lowest, below economies like Estonia and the Czech Republic and far below Finland and performance in problem solving: a significant part of the varia on for Poland in this test of 45 points compared Korea (OECD, 2013c). to the paper-based mathema cs test can be a ributed to a lack of computer skills, reflec ng a gap in both digital Weaknesses in problem solving among Polish students can also be observed when analyzing the subscales of the literacy and problem-solving skills (Panel B). Lastly, a look at the mathema cs “subscales” in PISA 2012, which mathema cs assessment. On the content subscale, students in Poland performed the best in problems related to measure in more depth mathema cal content and mathema cal processes, illustrates problem solving-related space and shape; they also performed above the average in algebra and geometry. Polish students did not do as dimensions of mathema cal competencies in which Polish students have deficits (Box 2). well in problems related to change and rela onships (Figure 10). The analysis of the processes subscales suggest Figure 9. Problem-solving scores and comparison with mathema cs scores (PISA 2012) that there is room for improvement in the process of interpre ng and evalua ng mathema cal problems, which is key for managing real-life situa ons that require mathema cal skills. Figure 10. Poland’s PISA 2012 performance on mathema cs subscales compared to the average mathema cs performance Source: World Bank staff es ma ons using PISA 2012 data. Source: World Bank staff es ma ons using PISA 2012 data. Poland’s results on the PISA problem-solving test are consistent with those from the Survey of Adult Skills for the younger age cohorts. The Program for the Interna onal Assessment of Adult Competencies (PIAAC) conducts the Survey of Adult Skills and measures key cogni ve and workplace skills needed for individuals to successfully page 26 page 27 Figure 11. PISA scores in mathema cs improved between 2000 and 2012 across every ESCS percen le Source: World Bank staff es ma ons using PISA 2012 data. Note: A polynomial of order 6 was used to smooth the do ed line The educa on reform in 1999 introduced a change in the structure of the system with an effec ve delay Performance and equity of selec on between general and voca onal tracks by one year. This has paid off in improved aggregate Poland’s overall gains in PISA since 2000 have been equally shared and are underpinned by strong gains performance. The reform introduced three years of comprehensive, mandatory lower-secondary educa on, among students from disadvantaged backgrounds. PISA results allow one to assess equity in educa on systems known as Gimnazjum. In an analysis of the reform, Jakubowski et al. (2010) found that delaying the year when and interac ons between student performance and socioeconomic status, because the assessment collects a student must choose between the general and voca onal tracks, which also allowed greater exposure to general informa on on both performance and student background informa on using the OECD’s Index of Economic, Social, curriculum content, had a posi ve and significant impact on student achievement on the order of one standard and Cultural Status (ESCS Index; see Box 3). Figure 11 shows Poland’s performance in 2000 and 2012 by percen le devia on. of socioeconomic status (as measured by the ESCS index) with a clear overall message: The higher the ESCS, the be er the performance. However, the picture clearly depicts a significant and balanced improvement across the Poland’s general educa on system prior to the 1999 reform was highly stra fied by socioeconomic background, en re distribu on. The flat lines (“steps”) show the average for each quin le of socioeconomic status. For all school type, and performance. Panel A in Figure 12 depicts PISA performance by school and the school-specific quin les, the average improved by 42 to 49 points – the equivalent of more than one year of schooling. average ESCS in 2000, just prior to the implementa on of the 1999 reform. Three key messages emerge. First, in 2000 PISA performance varied significantly by the student’s socioeconomic background. Second, performance varied by type of school. Third, 15 year-olds from socioeconomically disadvantaged backgrounds were page 28 page 29 dispropor onately in basic voca onal schools and voca onal secondary schools, where aggregate performance In contrast, by 2012 PISA performance in the comprehensive lower-secondary schools had significantly was significantly behind that of general secondary schools. Panel A summarizes the above evidence in a single increased. Figure 12 (Panel B) depicts PISA performance by school and the school-specific average ESCS in 2012. sca erplot, showing how students from different socioeconomic strata are distributed into different educa onal First, because of the introduc on of a comprehensive lower-secondary school in grades 7-9, the chart only tracks and achieve widely differing levels of cogni ve skills as measured by PISA. captures one type of school instead of three. Second, compared to 2000, performance improved significantly across the board, especially in schools scoring low on the ESCS index. Third, more schools had an average ESCS Figure 12. Improvements in performance and ESCS in Polish schools between 2000 and 2012 index above the OECD average. In sum, in 2012 the Polish educa on system was performing be er overall and students were be er off socioeconomically. Most performance varia on in the PISA test at age 15 in Poland today is within and not between schools. The varia on in performance between schools is a measure of how big the “school effects”1 are, and this is closely related to how students are allocated or selected into schools. Poland’s varia on in performance between schools has substan ally decreased since 2000 (OECD, 2013a). Between-school differences in mathema cs performance in 2012 accounted for as li le as 20 percent of the varia on in student performance in Poland. This is significantly below the OECD average (see Figure 13) and that of neighboring countries such as Hungary, Slovak Republic, Germany, and Bulgaria – countries with educa on systems that select students into general and voca onal tracks at an earlier stage than Poland does and whose 15 year-olds are in different types of secondary schools compared to Poland’s comprehensive lower secondary Gimnazjum – but above that in many countries with similar unitary structures like Poland, e.g. most Nordic countries. Figure 13: Between-school variance in mathema cs performance is limited in Poland Source: OECD (2014b). Today’s low stra fica on of performance among lower-secondary schools in Poland goes hand in hand with Poland’s moderate socioeconomic stra fica on. The index of school social stra fica on is defined as the correla on between the PISA student’s socioeconomic status and the school’s average socioeconomic status. Source: World Bank Staff es mates using PISA 2000 and 2012 data. Notes: OECD average of ESCS index is 0. The ESCS school average is calculated by compu ng the weighted average of student’s ESCS at each school. In a world with li le social stra fica on (thus an index near zero), students from different backgrounds would study together, making schools as diverse as society as a whole. Figure 14 depicts countries par cipa ng in PISA according to their index of school social stra fica on and mean mathema cs scores. Poland is in the top le 1According to the OECD, school effects are the effect on academic performance of a ending one school or another, usually schools that differ in resources or policies and ins tu onal characteris cs. See OECD (2013b) for a detailed descrip on. page 30 page 31 quadrant of countries, with its PISA mathema cs scores being above and its school social stra fica on index being the evidence from those two measures. Slightly more than 15 percent of the variance in performance in Poland below the OECD averages. The figure also shows that the highest performing countries in PISA tend to have less can be explained by socioeconomic status, slightly above the OECD average (Figure 15, Panel A), reflec ng a socially stra fied school systems. Poland is one of them. While Poland’s index shi ed significantly from 0.48 in reasonable equality of educa onal opportuni es. Learning in Poland depends less on socioeconomic status than in 2000 to 0.58 in 2009, it is now back at 0.52 in 2012, just below the OECD average. other countries, giving students from disadvantaged backgrounds a be er chance at learning and, with that, be er future opportuni es. However, there are countries, such as Estonia, that have be er performance in PISA and, at Figure 14: Stra fica on in the educa on system according to PISA 2012 scores the same me, much be er equality of opportuni es, sugges ng that there is s ll room for improvement. The difference between the top and bo om ESCS quin les in PISA reading and mathema cs scores remains large. In 2012, the difference between these quin les was 108 points – the equivalent of almost three years of schooling and larger than the OECD average of 98 points (Figure 15, Panel B). This gap is even more striking when compared with that of other high-achieving educa on systems such as those in Finland, Japan, and Korea, whose gaps are only 73, 82, and 87 points, respec vely. Figure 15. Index of equality of opportuni es: Poland and other countries, 2012 Source: World Bank staff es mates using PISA 2012 data. Note: PISA mathema cs scores on ver cal axis. Index of School Social Stra fica on on horizontal axis. The index ranges from 0 to 1. A higher index indicates a higher correla on between students’ and schools’ socioeconomic status. OECD mathema cs score average 500 points. OECD average Index of School Social Stra fica on 0.525. However, there is evidence that performance stra fica on emerges in upper-secondary educa on that is similar to the pre-1999 situa on. In addi on to 15-year-olds in grade 9, the PISA test has also been administered to grade 10 students in upper-secondary educa on in 2006 and 2009. Analysis using the data for the grade 10 test shows significant performance varia on between voca onal and general upper-secondary schools and a high share of students in basic voca onal secondary schools performing below level 2 in the PISA test (Chłoń-Domińczak and Federowicz (2014). The evidence of a re-emergence of such performance gaps in upper-secondary educa on was one reason the authori es decided to introduce an iden cal curriculum in the first year of upper-secondary educa on for both general and voca onal schools. Moreover, while Poland’s school system today is less stra fied by performance and socioeconomic background, socioeconomic background s ll ma ers in predic ng PISA performance. There are two measures that are used in this report to examine equity in educa on: (i) the strength of the rela onship between student performance and socioeconomic status, and (ii) the PISA score gap between top and bo om ESCS quin les.1 Figure 15 summarizes Source: World Bank staff es ma ons using PISA 2012 data. Note: The index is the percent of the variance in reading scores explained by the 1The strength is defined as the amount of varia on in outcomes in the outcomes (PISA Scores) explained by the ESCS Index, which means the predic ve power of ESCS on learning outcomes (the most common measure is R-squared of linear regressions, see Ferreira and Gignoux (2011), who define this measure circumstances. If a significant share of the results is explained by these characteris cs, then the equality of opportuni es is low). Larger differences between as “equality of opportunity”: a measure of educa onal opportunity using the share of variance in test scores that is explained by individual predetermined top and bo om SES students in PISA scores indicate a greater impact of ESCS on student performance, that is, more inequality (a more general measurement, which is used here, is the performance gap between top and bo om socioeconomic quin le groups). For a broader discussion, see Willms (2006). page 32 page 33 Performance gaps also remain between students in urban and rural areas and between girls and boys. Even though performance variance between schools is lower than in other countries within schools, school loca on s ll ma ers: The PISA score gaps between students in urban and rural areas account for the equivalent of almost one year of schooling. Meanwhile, girls outperform boys in reading, but not in mathema cs. However, evidence from the Progress in Interna onal Reading Literacy Study (PIRLS) in 2006 and 2011 evidence suggests a narrowing of the gap in grade 4. Figure 16. PISA 2012 score gaps by loca on and gender, Poland and comparator countries Chapter 3 Policy Implications: Remaining Challenges in the Polish Education System Source: World Bank staff es ma ons using PISA 2012 data. page 34 page 35 3. Policy Implications: Remaining Challenges in the Polish Education System schools and has involved the teaching of cri cal thinking and problem-solving skills (Tan and Gopinathan, 2000). Despite Poland’s success in improving the cogni ve skills of 15-year-olds as measured by PISA since 2000, In 2009 Singapore developed a 21st century competencies framework, which now guides the development of important challenges remain. There are three issues that will need a en on to ensure that the Polish educa on a new na onal curriculum (OECD, 2014c). Korea recently adopted a new curriculum that places more emphasis system con nues to provide the adequate skills for future jobs and remains equitable: (i) the re-emergence of on cri cal thinking skills and crea vity than the previous curriculum. Both Korea and Singapore have also made performance differen als between students in voca onal and general educa on in upper secondary educa on, changes to assessment methods and approaches and use teacher professional development as tools to promote (ii) differences in performance across students from different socioeconomic quin les, and (iii) rela vely poor good teaching prac ces. In Korea, the university entrance exams use essays that test wri ng and logical thinking, problem-solving skills. while university admission criteria in Singapore were expanded beyond the secondary graduate cer ficate and an entrance examina on to include project work in schools and extracurricular ac vi es (Bodewig and Badiani- Magnusson, 2014). The Province of Alberta in Canada adopted a new framework for student learning that Addressing performance gaps in upper-secondary education iden fies cri cal thinking and problem-solving as key competencies across the curriculum. Interes ngly, the Students in voca onal schools in upper-secondary perform poorly in reading and mathema cs. There is evidence Alberta framework was developed in a highly consulta ve process with contribu ons from students, parents, and from PISA assessments replicated for older students in upper-secondary educa on in 2006, 2009 and 2012 employers (OECD, 2014c). (“na onal op on”) that performance gaps between voca onal and general schools for 15-year-olds found prior to the 1999 reform persist today in upper-secondary educa on (Chłoń-Domińczak and Federowicz, 2014). Despite Poland can learn from and adapt emerging interna onal experience and experiment with new approaches. successes at the lower-secondary level, the performance of students in voca onal upper secondary schools trails Poland introduced a new curriculum in 2008 to help foster advanced cogni ve skills, and it appears to have that of their peers in general educa on. contributed to improvements in PISA performance in mathema cs, reading, and science since 2009. Poland’s rela vely poor performance in the PISA 2012 problem-solving test suggests the need for con nued innova on and Developing a policy response to this phenomenon of performance differen als by school type at the upper- experimenta on over the coming years in classroom and teaching prac ces to take advantage of the possibili es secondary level would benefit from more analysis and research. This should include more in-depth research to that the new curriculum provides. It is also advisable to explore in depth how socio-emo onal skills – cri cal understand the nature of the transi on from lower to upper-secondary educa on and the drivers of selec on of contributors to problem-solving – can be reflected in curriculum and classroom and teaching prac ces. school types. Depending on the findings of such research, policy op ons might include further expanding exposure to general curriculum content, teaching prac ces, and assessment in upper-secondary voca onal schools and embedding cogni ve skills in voca onal subjects, and introducing flexibility for students to switch streams while Promoting equity s ll allowing them to con nue on to higher educa on. Poland’s educa on equity challenge is different from that of several of its neighbors. Its school system at the lower-secondary level is less stra fied by socioeconomic background, and rela vely li le of the varia on in overall performance can be explained by the varia on in performance between schools. This suggests that Poland has a Improving problem-solving skills more equitable school system than many of its neighbors. However, that is not to say that equity is not an issue Poland’s 15-year-olds perform rela vely poorly in the PISA 2012 problem-solving assessment. The increased in Poland. As shown, variance in mathema cs and reading performance between students from the top and demand and recognized importance of crea ve problem-solving skills in the labor market has triggered a bo om ESCS quin les is large (almost the equivalent of three years of schooling). Students from the bo om two discussion worldwide on what types of interven ons could help foster these skills. At the center of the discussion ESCS quin les are significantly more likely to score at the bo om performance level in mathema cs and problem- is how the curriculum, the assessment system, and classroom and teaching prac ces can help create learning solving. The associa on between socioeconomic disadvantage and poor performance points toward a policy opportuni es to help students solve real-life problems. The curriculum should clearly define the acquisi on of response that combines expanding access to quality preschool educa on and strengthening social assistance to cogni ve skills, such as problem-solving and crea ve and cri cal thinking, as an objec ve. Teaching prac ces need improve the socioeconomic condi ons of children and youth. to be geared toward achieving this objec ve (with implica ons for teacher professional development) and these skills need to be captured in student assessment. Early childhood educa on has been shown worldwide to be a key tool to improve opportuni es for disadvantaged children in their educa on and throughout life. Evidence from PISA is consistent with this: Poland’s Several countries have started to revise their curricula, teaching prac ces, and assessment systems to 15-year-olds who had a ended pre-primary educa on scored 62 points higher in mathema cs in 2012 than those emphasize the importance of problem-solving, and several have started experimenta on. Singapore and Korea, who did not (the equivalent of a year and a half of schooling), and they scored 32 points higher a er accoun ng the two top performers in the PISA 2012 problem-solving assessment, provide interes ng examples. Singapore for socioeconomic status. This suggests that Poland’s con nued emphasis on expanding access to preschool – adopted the “Thinking Schools, Learning Na on” ini a ve in 1997, which created the ground for a revision of making preschool at age five compulsory and guaranteeing a preschool place for children ages three and four – is a the curriculum and assessment system. It aims to promote ac ve learning and crea ve and cri cal thinking in promising vehicle for ensuring school readiness for disadvantaged children. page 36 page 37 Poland’s social assistance system for low-income families is under review. Poland’s social assistance system comprises a considerable number of permanent and temporary cash and in-kind programs, mostly means-tested and with differing objec ves that include poverty reduc on, support to families, and support to people with disabili es. There are also significant, well-targeted child tax credits to low-income families. Social assistance overall, as well as the biggest benefit program, the family allowance, are well targeted, but they suffer from low coverage of the poorest households and low adequacy in their benefit levels. As a result, the poverty reduc on impact is limited. This suggests the need to examine the overall effec veness of the social assistance system, especially from the perspec ve of poor families with children. Poland’s Ministry of Finance and Ministry of Labor and Social Policy are in fact engaged in a spending review exercise focusing on strengthening financial assistance to low income families. References Annex page 38 page 39 References Poland.” Policy Research Working Paper No. 5263. Washington DC: World Bank Aedo, C., J. Hentschel, J. Luque, and M. Moreno. 2013. From Occupa ons to Embedded Skills: A Cross-Country Kahn, L. B. 2010. “The Long-Term Labor Market Consequences of Gradua ng from College in a Bad Economy.” Comparison. Washington, DC: World Bank. Labour Economics 17: 303 - 16. Autor, D., F. Levy, and R. Murnane. 2003. The Skill Content of Recent Technological Change: An Empirical Marciniak, Z. 2015. “Reviewing Polish Educa on Reform in the Late 1990s – Possible Lessons to Be Learned, Poland Explora on. Quarterly Journal of Economics, 118(4), November 2003, 1279-1334. as a Global Development Partner Series. Washington DC: World Bank. Amermueller, A. 2004. “PISA: What Makes the Difference? Explaining the Gap in PISA Test Scores Between Finland OECD. 2010. “The High Cost of low Educa onal Performance: The Long-Run Impact of Improving PISA Outcomes.” and Germany.” Discussion Paper No. 04-004, ZEW Center for European Economic Research. OECD Publishing. Retrieved May 12, 2015 from h p://www.oecd.org/pisa/44417824.pdf Barro, R., and J. Lee. 2013. “A New Data Set of Educa onal A ainment in the World, 1950-2010.” Journal of OECD. 2012. “PISA 2009 Technical Report.” OECD Publishing. Retrieved April 10, 2014 from h p://www.oecd.org/ Development Economics 104: 184-98. pisa/pisaproducts/50036771.pdf Bodewig, C., and R. Badiani Magnusson. 2014. “Skilling Up Vietnam: Preparing the Workforce for a Modern Market OECD. 2013a. “PISA in Focus: Does It Ma er Which School a Student A ends?” PISA in Focus 27. OECD Publishing. Economy.” DirecƟons in Development. Washington, DC: World Bank. Retrieved May 13, 2015 from h p://www.oecd.org/pisa/pisaproducts/pisainfocus/pisa%20in%20focus%20n27%20 Bussolo, M., and L. F. Lopez-Calva. 2014. “Shared Prosperity: Paving the Way in Europe and Central Asia.” (eng)--FINAL_version2.pdf Washington, DC: World Bank. OECD. 2013b. What Makes Schools Successful? Resources, Policies and PracƟces, Volume 4. OECD Publishing. Chłoń-Domińczak, A., and M. Federowicz. 2014. “How Did Educa onal Reforms in Poland Lead to Significant Retrieved May 31, 2015 from h p://www.oecd.org/pisa/keyfindings/Vol4Ch1.pdf Progress in PISA?” Presenta on for the IBE Educa onal Research Ins tute. Accessed on June 1, 2015 at OECD (2013c), OECD Skills Outlook 2013: First Results from the Survey of Adult Skills, OECD Publishing h p://www.kwalifikacje.edu.pl/download/konferencje/pla2015/Educa onal_reforms_in_Poland_prezentacja.pdf OECD. 2014a. PISA 2012 Results: What Students Know and Can Do, Volume 1. OECD Publishing. Retrieved April 10, Ferreira, H. G., and J. Gignoux. 2011. “The Measurement of Educa onal Inequality: Achievement and Opportunity.” 2014 from h p://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf IZA Discussion Paper No. 6161. OECD. 2014b. PISA 2012 Results: Excellence through Equity, Volume 2. OECD Publishing. Gregg, P. and E. Tominey. 2005. “The Wage Scar From Male Youth Unemployment.” Labour Economics 12: 487 - 509. OECD. 2014c. PISA 2012 Results: CreaƟve Problem Solving, Volume 5. OECD Publishing. Hanushek, E. 2013. “Economic Growth In Developing Countries: The Role of Human Capital.” Economics of OECD. 2015. Universal Basic Skills: What Countries Stand to Gain. OECD Publishing. EducaƟon Review 37: 204 - 12. Sala-i-Mar n, X., G. Doppelhofer, and R. I. Miller. 2004. “Determinants of Long-Term Growth: A Bayesian Averaging Hanushek, E. 2009. “School Policy: Implica ons of Recent Research For Human Capital Investments in South Asia of Classical Es mates (BACE) Approach.” American Economic Review 94 (4): 813 - 35. and Other Developing Countries.” EducaƟon Economics 17 (3): 291 - 313. Schmillen, A., and M. Umkehrer. 2013. “The Scars of Youth: Effects of Early-Career Unemployment on Future Hanushek, E. 2010. “The High Cost of Low Educa onal Performance: The Long-Run Economic Impact of Improving Unemployment Experience.” IAB Discussion Paper No. 6/2013. PISA Outcomes.” OECD Publica ons. Sondergaard, L. & M. Murthi. 2011. Skills Not Just Diplomas. World Bank, Washington DC. Hanushek, E., and L. Woessmann. 2006. “Does Educa onal Tracking Affect Performance and Inequality? Differences-in-Differences Evidence Across Countries.” The Economic Journal 116 (510): C63 - C76. Tan, J., and S. Gopinathan. 2000, “Educa on Reform in Singapore: Towards Greater Crea vity and Innova on?” NaƟonal InsƟtute for Research Development Review 7 (3): 5-10. Hanushek, E., and L. Woessmann. 2007. “The Role of Educa on Quality in Economic Growth.” World Bank Policy Research Working Paper 4122. World Bank, Washington, DC. Willms, JD. 2006. Learning Divides: Ten Policy QuesƟons about the Performance and Equity of Schools and Hanushek, E., and L. Woessmann L. 2012. “Do Be er Schools Lead to More Growth? Cogni ve Skills, Economic Schooling Systems, UNESCO Ins tute for Sta s cs, Montreal. Outcomes, and Causa on.” Journal of Economic Growth 17: 267 - 321. Jakubowski, M. 2015. “Opening Up Opportuni es: Educa on Reforms in Poland.” IBS (Ins tute Badan Strukturalnych) Policy paper 01/2015 Jakubowski, M., H. A. Patrinos, E. E. Porta, and J. Wiśniewski. 2010. “The Impact of the 1999 Educa on Reform in page 40 page 41 Annex Figure A1. Evolu on of ESCS 2000-2012 by percen les Source: World Bank staff es ma ons using PISA 2012 data. Table A1. Percent of popula on aged 15+ by highest level of schooling a ained and average years of schooling in Poland, 1990-2010 Highest level a ained Average Years of Schooling Year Primary Secondary Ter ary Total Primary Secondary Ter ary Total Completed Total Completed Total Completed 1990 47.5 41.7 44.9 40.1 5.9 3.4 9.06 6.94 1.94 0.19 1995 34.2 32.1 54.1 47.7 9.6 5.5 9.86 7.14 2.42 0.30 2000 27.3 26.0 59.6 52.8 10.8 6.2 10.26 7.24 2.68 0.34 2005 21.6 20.8 60.7 54.7 16.7 9.8 10.94 7.43 2.98 0.53 2010 17.1 16.1 63.7 58.4 19.0 11.4 11.32 7.39 3.32 0.61 Source: Barro-Lee Dataset (2012)