DIREC T ION S IN DE VELOPMEN T Human Development 64852 OVERVIEW Skills, Not Just Diplomas Managing Education for Results in Eastern Europe and Central Asia Lars Sondergaard and Mamta Murthi with Dina Abu-Ghaida, Christian Bodewig, and Jan Rutkowski OVERVIEW Skills, Not Just Diplomas Managing Education for Results in Eastern Europe and Central Asia Lars Sondergaard and Mamta Murthi with Dina Abu-Ghaida, Christian Bodewig, and Jan Rutkowski This booklet contains an Overview of the forthcoming World Bank book, Skills, Not Just Diplomas: Managing Education for Results in Eastern Europe and Central Asia, along with a list of contents of that book. To order copies of the full-length book, published by the World Bank, please use the form at the back of this booklet. © 2012 The International Bank for Reconstruction and Development / The World Bank 1818 H Street NW Washington DC 20433 Telephone: 202-473-1000 Internet: www.worldbank.org All rights reserved This volume is a product of the staff of the International Bank for Reconstruction and Development / The World Bank. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgement on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development / The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; telephone: 978-750-8400; fax: 978-750-4470; Internet: www .copyright.com. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2422; e-mail: pubrights@worldbank.org. ¸ Cover photo: The graduating class of 2007, Alexandru Ioan Cuza University, Iasi, Romania, march to their commencement ceremony. © Mediafax Foto/Liviu Chirica. Contents Foreword to Skills, Not Just Diplomas v Acknowledgments to Skills, Not Just Diplomas ix Abbreviations xi Contents of Skills, Not Just Diplomas xiii Overview 1 The Skills Challenge 2 Why Are Skills an Emerging Problem if Education Systems Are Delivering? 4 Priority Areas for Action 7 Managing Education Systems for Results 9 Build the Foundations of Adult Learning Systems 14 Conclusion 14 Notes 15 Figures O.1 Analysis of Reading Competency of 15-Year-Old Students on the PISA 2009 3 O.2 Distribution of Firms in ECA Region that Consider Worker Skills a “Major� or “Very Severe� Constraint, 2008 4 iii iv Contents O.3 Primary School Student-Teacher Ratios in ECA Compared to Other Regions of the World, 1990–2008 8 O.4 Status of Measuring and Using Data on Student Learning Outcomes in the ECA Region, 2009 10 Foreword to Skills, Not Just Diplomas It is no secret that good education lies at the heart of economic growth and development. At the same time, improving the quality and relevance of education is enormously difficult not least because there is no one single policy measure that will do so effectively. This book contributes to our understanding of how to improve education by examining the recent experience of the countries of Eastern Europe and the former Soviet Union. Fabled for uniform access and high quality of education 20 years ago, the countries in this region have struggled to maintain their reputation. Three factors have contributed to the slide in quality and relevance. First, one of the legacies of central planning is that the countries pay too much attention to measuring inputs into learning—such as the number of schools and the number of teachers—and not enough to outcomes. Indeed, they have been late in developing systems to assess how much students are learning and whether learning is leading to employment. In this sense, their education systems operate in the dark, which makes policy making extremely difficult. Second, the system of management, also a vestige of the past, limits the ability of schools to improve the learning environment for students, as well of municipali- ties that may want a different mix of programs to meet local labor v vi Foreword to Skills, Not Just Diplomas market needs. As with assessment, the countries have been slow to embrace the governance and accountability reforms that are now part of the landscape of education systems the world over. These limitations to autonomy and accountability for outcomes have reduced the energy and the incentives for improvements from within the system, contrib- uting to a shortage of skills. Indeed, firm complaints about the shortage of relevant skills for expansion and growth have risen to a crescendo in most countries. Finally, the systems increasingly allocate resources where they are not needed. For example, the sharp decline in student numbers in the past 20 years has not resulted in a commensurate decline in the number of classrooms and teachers. As a result, resources are increasingly tied up in buildings and teachers where they may be better spent elsewhere. Most of these limitations are found not just in schools but extend to higher education and to training. The result of these limitations is that the quality and relevance of education in the countries in the region is increasingly questionable. A large proportion of students finish lower secondary school with a mini- mal command of literacy and numeracy. Their failure to learn even the basics is not picked up early enough by assessment systems, nor are schools and municipalities given sufficient incentives to ensure that all students learn the basics. The systems are also struggling with imparting higher order skills beyond the basics, for which well motivated and high quality teachers are critically needed but which few school systems are able to attract, given the low pay and historical overstaffing. And without the vision or resources to make vocational and technical education an attractive option for students—indeed, this remains one of the most unreformed subsectors of the education system—quality has eroded and students have shied away from this form of education, possibly contrib- uting to the widespread shortage of vocational and technical skills now found in these countries. This book makes the case that improving the quality and relevance of education requires a fundamental change of approach to education in the countries of the region. To start with, education systems need to “turn the lights on� and take seriously the measurement of what students actually learn as opposed to measurement of the inputs into the education process on the implicit assumption that learning follows. This assessment needs to inform both teaching and policy making. Policy makers also need to move away from controlling inputs and processes and instead increase the emphasis on incentives to improve student learning, whether in school or in higher education. And, finally, for these reforms to be financially fea- Foreword to Skills, Not Just Diplomas vii sible, current spending on education needs to be used much more effec- tively. In particular, countries in the region cannot afford to maintain one of the lowest class sizes in the world while heating and lighting half- empty buildings when resources are needed elsewhere. It is our sincere hope that this book will stimulate debate about how to improve education and training both for the countries under discussion and in the world at large, and thereby encourage action to help realize prosperity for all. Philippe Le Houerou Vice President Europe and Central Asia Region World Bank Washington, D.C. Acknowledgments to Skills, Not Just Diplomas This study was managed by Lars Sondergaard, who authored chapters 3 and 4, and Mamta Murthi, who authored the Overview and chapter 7. Jan Rutkowski authored chapter 1; Dina Abu-Ghaida authored chapter 2 with inputs from and Christian Bodewig and Lars Sondergaard; Lars Sondergaard authored chapter 4 with inputs from Alex Usher; and Christian Bodewig authored chapter 6 with inputs from Sarojini Hirshleifer. Dina Abu-Ghaida co-managed the study in its early stages with Lars Sondergaard. The book grew out of a regional research program launched and sup- ported by Europe and Central Asia’s Chief Economist’s office, and during the writing of the book, two chief economists helped shape it: Pradeep Mitra and Indermit Gill. The authors are grateful to both for their guid- ance and support in writing this book. The book is underpinned by significant contributions, including back- ground papers, by Michael Mertaugh, Alex Usher, Iveta Silova, Andras Benedek, Algerlynn Gill, Rostislav Kapelyushnikov, Hakan Ercan, and ´ one jointly by Jerzy Wisniewski, Maciej Jakubowski, Harry Anthony Patrinos, and Emilio Ernesto Porta. We also thank Juan Manuel Moreno and Nina Arnhold for reading and providing comments on various drafts ix x Acknowledgments to Skills, Not Just Diplomas and helping to shape the book’s main messages, and Algerlynn Gill for her excellent research assistance and inputs into several of the book’s chapters. In addition, Nadezhda Lepeshko and Carmen Laurente helped prepare graphs and figures for the document. The book also benefitted from the excellent editorial work of Peggy McInerny and Patricia Carley. Larry Forgy provided editorial assistance on chapter 1. We also benefitted from valuable comments from our peer reviewers: Richard Murnane, Amit Dar, Bernard Hugonnier, Halsey Rogers, and Manfred Wallenborn. And we benefitted from inputs, comments and suggestions from numerous colleagues, including Mohamed Ihsan Ajwad, Gordon Betcherman, Mary Canning, Isak Froumin, Sachiko Kataoka, Arvo Kuddo, Toby Linden, Lily Mulatu, Bojana Naceva, Reehana Rifat Raza, Alberto Rodriguez, Marcelo Selowski, Jan Sadlak, Jamil Salmi, Luis Crouch, and countless others. Any and all errors that remain in this volume are the sole responsibility of the authors. Abbreviations BEEPS Business Environment and Enterprise Performance Survey CIS Commonwealth of Independent States ECA Europe and Central Asia EU European Union EU10+1 Bulgaria, the Czech Republic, Estonia, Hungary, Latvia, Lithuania, Poland, Romania, the Slovak Republic, and Slovenia, plus Croatia GDP gross domestic product OECD Organisation for Economic Co-operation and Development PIAAC Program for International Assessment of Adult Competencies (OECD) PISA Programme for International Student Assessment (OECD) PPP purchasing power parity Country abbreviations used in figures and tables: ALB Albania BGR Bulgaria ARM Armenia BIH Bosnia and Herzegovina AUT Austria BLR Belarus AZE Azerbaijan CYP Cyprus BEL Belgium CZE Czech Republic xi xii Abbreviations DEU Germany MDA Moldova DNK Denmark MKD Macedonia, former ESP Spain Yugoslav Republic of EST Estonia MLT Malta FIN Finland NLD the Netherlands FRA France NOR Norway GBR United Kingdom POL Poland GEO Georgia PRT Portugal GRC Greece ROU Romania HRV Croatia RUS Russian Federation HUN Hungary SRB Serbia IRL Ireland SVK Slovakia ISL Iceland SVN Slovenia ITA Italy SWE Sweden KAZ Kazakhstan TJK Tajikistan KGZ Kyrgyz Republic TKM Turkmenistan LTU Lithuania TUR Turkey LUX Luxembourg UKR Ukraine LVA Latvia UZB Uzbekistan Note: These three-letter country codes are part of the International Organization for Standardization (ISO) 3166 standard to represent countries. Contents of Skills, Not Just Diplomas Foreword Acknowledgments Abbreviations Overview Chapter 1 The Demand for Skills in ECA Chapter 2 Education and the Supply of Skills to the ECA Market Chapter 3 Resolving the Skills Shortage in the ECA Region: A Policy Framework Chapter 4 Managing for Results at the Pre-University Level of Education Chapter 5 Managing for Results in the Tertiary Education Sector Chapter 6 Advancing Adult Learning in ECA xiii xiv Contents of Skills, Not Just Diplomas Chapter 7 Extended Summary: The Path for Education Reforms in the ECA Region References Overview The countries of Europe and Central Asia (ECA)1 are currently emerging from the deepest recession suffered by any developing region. Post-crisis conditions are very different from those of preceding years. Financial resources are more limited and more expensive, and export growth is restrained by potentially slower growth in destination countries. Restoring and sustaining growth in this context require reforms that boost com- petitiveness and increase labor productivity. Such reforms are all the more important given the shrinking of the working-age population in many countries of the region. Earlier boom years in the ECA region exposed significant bottlenecks to growth, particularly with respect to the skills of the labor force (Mitra et al. 2010). Paradoxically, for a region with relatively high and expanding educational attainment (as measured by the number of years of com- pleted schooling) and relatively high-quality education in the early years of schooling, a shortage of worker skills has emerged as one of the most important constraints to firm expansion. This book seeks to answer the following questions: Why do ECA firms increasingly complain that they cannot find graduates with the right skills? What can ECA countries do to close the skills gap? 1 2 Skills, Not Just Diplomas When trying to answer these questions, this book faces a fundamental problem: data exist on the number of students who graduate (i.e., how many diplomas are issued) in ECA countries, but internationally compa- rable data on whether graduates of upper secondary and tertiary institu- tions (from which the bulk of ECA graduates now enter the labor market) have the right skills and competencies for the job market do not exist. This book uses a range of different data sources to argue that the skills problem in the ECA region relates more to the quality and relevance of the education provided in ECA countries than to problems of access. A central argument of the book is that ministries of education are con- strained in a number of ways from effectively managing their education and training sectors. The three most important and interrelated impedi- ments to improving quality and relevance are the lack of systematic data on key skills-related performance issues (i.e., how much students are learning and whether they are finding jobs after they graduate), the legacy of central planning, and inefficient use of resources. Lack of data on student learning and employment outcomes makes it difficult for education ministries to address the legacy of central planning, which emphasizes centralized management based on inputs. Ministries of education in the region continue to micromanage the sector using detailed norms and regulations. This input-oriented style of management leads to the inefficient use of resources and results in a rigid education sector—not the type of flexible sector needed by ECA to create modern, skilled workforces. This book highlights how these constraints manifest themselves and then presents ways of overcoming them, relying on the experience of ECA countries that have successfully addressed them, together with international experience. Recommendations are presented in separate chapters for pre-university, tertiary, and adult education. The Skills Challenge The ECA region had a well-regarded education system before the transi- tion. While the intervening years have taken some of the shine off this reputation, these countries continue to have strong achievements. Notably, enrollments are high at all levels of education. The communist legacy is particularly visible in low-income countries in the region, which have the highest secondary enrollments in the world for their income level. Tertiary enrollments, which have grown rapidly in the past two decades, are also high relative to income levels, with the exception of a few low-income Commonwealth of Independent States (CIS) countries, such as Azerbaijan and Uzbekistan. Overview 3 The ECA region is reasonably successful not only in terms of enroll- ment rates, but also in terms of providing quality education in the early grades. In particular, in international tests that measure student compe- tencies in primary and lower secondary education, students in many (but not all) ECA countries outperform their peers in countries with the same income level (see figure O.1). For instance, students in Estonia, where gross domestic product (GDP) per capita was $21,6442 in 2008, outper- formed their peers in Denmark, Sweden, Iceland, and Austria—where per capita income is almost twice as high—on the mathematics part of the 2009 Programme for International Student Assessment (PISA). The pic- ture for educational quality within the ECA region is varied, however, as more than several countries, such as Bulgaria, Romania, Montenegro, Azerbaijan, and the Kyrgyz Republic, have students (in the early grades) that underperform relative to their income level. In spite of these positive achievements of ECA education systems, the European Bank for Reconstruction and Development (EBRD)–World Bank Business Environment and Enterprise Performance Surveys (BEEPS) show that ECA firms’ perception of skills constraints changed dramati- cally around 2005.3 By 2008, skilled labor shortages had become the second most commonly reported constraint to growth in the BEEPS sur- vey across all countries in the region, second only to tax rates (see figure O.2). On average, 30 percent of firms considered education and skills to Figure O.1 Analysis of Reading Competency of 15-Year-Old Students on the PISA 2009 550 POL EST 2009 mean performance 500 HUN LVA SVN TUR HRV CZE LTU SVK 450 RUS SRB BGR ROM MNE 400 KAZ ALB AZE 350 KGZ 300 0 20,000 40,000 60,000 80,000 100,000 2008 GDP per capita, PPP (current international $) Source: PISA 2009 Database and World Bank staff calculations based on World Development Indicators Database. Note: PISA = Programme for International Student Assessment. The figure shows a regression line representing countries’ predicted PISA reading scores based solely on GDP per capita, compared to the Organisation for Eco- nomic Co-operation and Development mean reading score (horizontal line) and GDP per capita in 2008. See “Acronyms and Abbreviations� for a key to country abbreviations. 4 Skills, Not Just Diplomas Figure O.2 Distribution of Firms in ECA Region that Consider Worker Skills a “Major� or “Very Severe� Constraint, 2008 9 mean = 30.2 8 TJK 7 UZB number of countries 6 AZE CZE 5 ARM 4 BIH GEO EST KOS KGZ HRV UKR 3 ALB LVA MDA 2 MKD MNE SRB BGR POL LTU KAZ 1 HUN SVN TUR SVK ROM RUS BLR 0 % % % % % % % % % 10 20 30 40 50 60 70 80 90 x< x< x< x< x< x< x< x< x< ≤ ≤ ≤ ≤ ≤ ≤ ≤ ≤ % % % % % % % % 10 20 30 40 50 60 70 80 EU10+1 and Turkey Western Balkans LI CIS MI CIS Source: Authors’ calculations based on BEEPS 2008. Note: EU10+1 = European Union, LI = low-income, MI = middle-income, x = % of firms (in respective countries) that consider education as an obstacle. Figure shows data obtained from the fourth round of the BEEPS carried out in 2008–09, which covered approximately 11,800 enterprises in 29 countries. See “Acronyms and Abbrevia- tions� for a key to country abbreviations. be a major or severe constraint in 2008. The highest proportion of firms reporting constraints were found among the middle-income CIS coun- tries, where upwards of 40 percent of firms were dissatisfied with the availability of skilled workers. A smaller proportion of firms in the Western Balkans reported similar levels of dissatisfaction, with significant variabil- ity in the dissatisfaction levels of firms in the EU10+1 (European Union 10+1) countries and Turkey (BEEPS dataset 2008). While the recession may have provided something of a respite from these labor shortages, as the economies in the region recover, labor demand will tighten once again and skilled labor shortages will likely be worse than before. Why Are Skills an Emerging Problem if Education Systems Are Delivering? Despite generally high average enrollment and attainment rates, as well as respectable quality education (where measured, at the early grade level) Overview 5 for their income level, ECA countries’ growth is constrained by skills shortages. This is a puzzle that cannot be fully resolved, mainly because crucial information is missing. Put simply, there is no internationally com- parable information on the quality of upper secondary or tertiary educa- tion or the relevance of education at these levels—information that would indicate whether graduates have the right skills for the modern job market There are international assessments that provide information on stu- dent competencies up to the age of 15 (usually the end of lower second- ary education), but this is not an age when most people in the region are entering the job market. Beyond these early-stage assessments, no com- prehensive, fully reliable information exists on student competencies. What is more, the information on student competencies that is available relates only to fundamental cognitive skills, not behavioral skills— involving such issues as work ethic and teamwork—which are empha- sized by employers. For policy makers to better understand the causes of the emerging skills bottleneck—and how to address it—this informa- tional gap needs to be closed. Given the lack of data on the skills of new labor market entrants, the book’s conclusions are necessarily based only on the data that are obtainable. Two problems related to quality seem particularly acute in the ECA region: too many students are failing, and education systems have dif- ficulties imparting problem-solving skills. In terms of the many stu- dents that fail, PISA 2009 data show that all ECA countries (except a handful of new EU member states) have very large proportions of 15-year-olds who have such poor numeracy and literacy skills that their success in the modern workplace is highly doubtful. As work becomes more demanding even for this group, their poor skills will become more apparent. Although ECA countries perform strongly on international assessments that measure students at the primary level (grade 4), their performance is weaker on assessments of students around the end of lower secondary education (grade 8). These results suggest that their education systems are adept at imparting basic skills, but have problems imparting higher order skills such as problem solving—the very skills that firms increasingly seek. More troubling, the quality of education in the ECA region also does not show signs of consistent improvement. In fact, up to 2006 it appeared to be getting worse in many countries. Rather than narrowing the gap between their scores and Organisation for Economic Co-operation and Development (OECD) mean scores, many ECA countries that participate 6 Skills, Not Just Diplomas in international learning assessments at this level—which are likely the better-managed countries—either seemed stuck or had regressed in terms of their scores. It is probable, moreover, that the situation is even worse in ECA countries that do not participate in efforts to measure and compare the academic performance of their students. Encouragingly, the latest round of PISA (2009) may have marked a turning point for 11 ECA coun- tries that showed improvements compared to their 2006 performance (including significant increases in Bulgaria, Romania, Serbia, the Kyrgyz Republic, and Turkey). However, seven ECA countries continued the downward slide observed previously with a deterioration compared to their 2006 performance. The quality of upper secondary and tertiary education in the region is also unlikely to have improved, although lack of data makes it hard to confirm this contention. Since educational quality is not showing consis- tent improvement at the lower secondary level, this weakness is probably mirrored at the upper secondary level (albeit with a lag). Enrollments at the tertiary level used to be tightly controlled and reserved for the few, best performers. In the past two decades, however, enrollments have doubled, tripled, or quadrupled, but without the benefit of the quality assurance mechanisms needed for the many new programs, institutions, and types of students, and without the kind of information required for parents and students to make informed choices. As a result, it is unclear if the expansion of the system has been accompanied by growing quality. Graduates of upper secondary and tertiary education may also be graduating with the wrong set of skills. During the early years of the tran- sition, the vocational school system—which once produced more than half of all secondary graduates in most ECA countries—collapsed very quickly. Voting with their feet, students have left vocational schools in favor of general secondary education and the prospect of pursuing a ter- tiary degree. It may be that the pendulum has swung too far in one direc- tion, however, as employers assert that it is now hard to find graduates with technical skills. Yet, until the region’s vocational schools are reformed to ensure that they can produce graduates with appropriate skills, it may be too soon to be encouraging students to return to this sector. The contribution of education to the skills gap is not only a story of the uncertain quality and relevance of formal education. It also involves too little progress in providing workers with options for further training. Growing evidence shows that continuous adult education and training fosters employment and greater productivity. The development of this sector should accordingly be central to the region’s economic growth Overview 7 agenda, particularly in more advanced ECA countries and countries that are facing a significant demographic decline—a decline that suggests that labor force participation and individual productivity will have to increase in order to generate higher rates of economic growth. Yet many of these countries have only started to prepare for an expansion of the adult train- ing sector, needed not only to compete in the global economy, but also to address the lag effects of the transition to market economies. In light of growing employer demands for greater skills, adult education and training in ECA countries can no longer be ignored. Priority Areas for Action As noted at the outset, three major impediments are inhibiting the cre- ation of flexible and responsive education systems in the ECA region. First, these systems have been operating “in the dark� because they design policies and take management decisions without systematically collecting and analyzing data on the learning and employment outcomes of stu- dents and graduates. Second, the legacy of central planning has kept the management of school systems highly centralized, with central policy makers intensely involved in operational details. While virtually all OECD countries have embraced performance-oriented management in educa- tion since the 1980s (see OECD 2004)—albeit at different speeds—most ECA countries continue to use management practices that focus on com- pliance with detailed regulations and financing schemes based on inputs, not outputs. This means that most local education authorities and school principals lack the autonomy and authority to make crucial management decisions for their own institutions, including how much and what type of vocational content students can choose from and how many teachers to employ. As a result, the system is inflexible and does not respond either to labor market needs or changes in student numbers. Third, education systems in the ECA region use financial resources highly inefficiently. Nowhere is this more apparent than in the pre- university sector, where few countries have adjusted teacher staffing levels in response to falling student numbers over the past 20 years. Consequently, student-teacher ratios have fallen sharply (and per student costs have risen)—more so than in any other region in the world (see figure O.3). This implies that scarce resources are tied up in paying meager salaries to too many staff members and heating half-empty buildings. To be fair, these impediments affect ECA countries differently and vary accordingly to the level of education. However, no ECA country 8 Skills, Not Just Diplomas Figure O.3 Primary School Student-Teacher Ratios in ECA Compared to Other Regions of the World, 1990–2008 29 27 student-teacher ratio (primary) 25 23 21 19 17 15 90 91 92 93 94 95 96 97 98 99 00 01 02 03 04 05 06 07 08 19 19 19 19 19 19 19 19 19 19 20 20 20 20 20 20 20 20 20 East Asia and Pacific Europe and Central Asia Latin America and the Caribbean Middle East and North Africa Sources: EdStats database, plus authors’ linear interpolations for missing years. Note: Actual data are marked with dots. One of the problems with international data on student-teacher ratios (in both the World Bank’s EdStats database and the other international education databases) is that it is unclear whether or not data for a particular country are reported on a full-time equivalent basis. has fully escaped the legacy of central planning.4 That system focused on controlling inputs—that is, it checked whether local actors were in compliance with detailed norms for all inputs. Education ministries remained uninformed, however, about whether students acquired skills and competencies. In terms of how these impediments affect the different levels of educa- tion, they are most clearly apparent at the pre-university level, which absorbs two-thirds of total education funding in the ECA region. The same three impediments affect tertiary education in a slightly different way, as this sector has already undergone significant reform over the past 20 years, during which time it has doubled or tripled in size in many countries (see chapter 5). Because most ECA countries do not yet have adult education sectors in the modern sense, these impediments cannot be fairly characterized as problems of this sector. However, the policy recommendations offered in this book do apply to adult education and offer ideas and principles for how this subsector could be developed and managed in the region (see chapter 6). Overview 9 Managing Education Systems for Results To address the three impediments outlined above, countries in the ECA region need to manage their education systems for results. First and fore- most, they need to collect more information on “performance,� that is, on what students are learning, what jobs they are finding, and what salaries graduates are earning. In addition, ECA countries could benefit from fol- lowing the lead of OECD countries and several developing countries and replace input-oriented management of the education sector with perfor- mance-oriented management. This change implies devolving more autonomy to frontline providers and putting in place accountability mechanisms—for instance, in the form of performance contracts and performance-based budgeting—that emphasize performance and not compliance with norms. With a greater availability of performance data and with frontline providers empowered with greater decision-making power, the preconditions will be in place for education systems in the region to become both more responsive to labor market needs and more efficient. Focus Attention on Learning and Employment Outcomes Countries in the ECA region would benefit from greater measurement of student learning and employment outcomes, information that they can then use to design better policies. There is significant consensus on the key skills and competencies that education is expected to deliver, as well as standardized tests to measure them.5 Many countries in the region already participate in international assessments, but have moved more slowly than other regions in generating useful information from national assessments. Although all school systems in the region have some kind of national assessment system, many are in the early stages of development (see figure O.4) and there is very little evidence to date that ECA countries are using these assessments to inform policy. Rather, national assessments continue to be seen as a certification mechanism, one that confirms a student has gained a mastery of a predetermined curriculum, thus enabling a diploma to be issued or the student to prog- ress to upper secondary or tertiary education. Student assessments are rarely seen as an opportunity to identify the strengths and weaknesses of an education system. When students graduate, ministries of education in the ECA region do not systematically collect, analyze, and disseminate information on their employment outcomes. Such information is, however, critical for policy 10 Skills, Not Just Diplomas Figure O.4 Status of Measuring and Using Data on Student Learning Outcomes in the ECA Region, 2009 several years of experience in several years of measuring, experience with analyzing, and own assessments making use of and regular learning results to early piloting of participation in improve education own assessment international policy instruments, assessments some participation in Bulgaria not started, very international early stages, no, or assessments Hungary rare participation Latvia in international Lithuania assessments Albania Romania Croatia Armenia Serbia Czech Republic Azerbaijan Slovenia Estonia Belarus Georgia Bosnia and Herzegovina Macedonia, FYR Kazakhstan Moldova Kosovo Montenegro Kyrgyz Republic Poland Tajikistan Russian Federation Turkey Slovak Republic Turkmenistan Ukraine Uzbekistan Sources: Authors’ assessments based on data from UNICEF (2007, table 2.1); the extent of countries’ participation in PISA, Progress in International Literacy Study (PIRLS) and Trends in International Mathematics and Science Study (TIMSS) since 1995; and inputs from World Bank country experts. The UNICEF data is drawn from “Table 2.1: Status of Reforms of Assessment and Examination Systems, 2006,� regarding “Introduction of other school exams or assessments (e.g. basic school)� and “Introduction of sample-based national assessment.� In this table, UNICEF scores the progress of countries on a range of 0 to 4, with 0 representing “not planned or started� and 4 repre- senting “operational.� In addition, the figure uses World Bank staff compilations on the number of international assessments in which each country has participated. makers and higher education institutions because it helps them detect which programs and fields of study are in high demand among employers. Moreover, this data can help students make better choices about which university and field of study to pursue. Again, some member countries of the OECD (including Hungary, Italy, and the Netherlands), together with Romania, provide examples in this area for others to emulate. Overview 11 Introduce Autonomy and Accountability Based on Results Overcoming the legacy of central planning involves moving away from detailed norms and instead holding actors accountable for performance (or results). At the central level, this implies that policy makers in the ECA region would relinquish certain duties and assume others so that education systems may innovate and improve student learning outcomes. The opportunity before central governments in these countries is to move away from managing schools and classrooms and focus instead on setting goals, policies, and standards; defining responsibilities; mobilizing financial resources; ensuring political consensus; targeting poor and excluded students; and monitoring and evaluating service delivery and system quality. Extending autonomy throughout education systems means placing authority and responsibility in the hands of the people most able to innovate and improve the quality of education: local managers and education authorities (see Osborne and Gaebler 1992). It also means holding these actors accountable for improving the learning out- comes of the bulk of their students. Aligning the incentives of these stakeholders with the student learning outcomes desired by policy makers requires education ministries to set overall performance goals; articulate who is responsible and accountable to whom and for what; and ensure that these responsibilities are agreed, accepted, and understood. At the pre-university level, policy makers can expand the autonomy of lower-level actors in the school system by granting them greater decision- making power over school operations and budgets and relaxing norms on class sizes. Greater autonomy is particularly needed in vocational educa- tion and training, where programs and institutes need the ability to expand or contract course offerings in response to student and employer demand, not rigid governmental norms. Simultaneously, policy makers will need to implement a range of accountability mechanisms to hold schools accountable for results. Potential mechanisms include the creation of school councils that involve parents and local communities in school decision-making pro- cesses; requiring schools to prepare school development plans that out- line each school’s strengths and weaknesses, together with an action plan for making improvements; preparing “school scorecards� that include basic indicators on a school’s performance; as well sanctioning low-per- forming schools and rewarding high-performing schools. Here, however, it is important to define a high-performing school as one that delivers 12 Skills, Not Just Diplomas improved learning outcomes to all types of students, including those from low-income and minority households. Significant improvements in the direction of greater autonomy and accountability will, however, be difficult to achieve in the school system unless policy makers in the region address the demoralization of the teaching force, an effort that will involve resolution of serious overstaff- ing and improving the pay, professional development, and work condi- tions of teachers. At the tertiary level, greater autonomy and effective accountability are equally critical. The first part of this equation has been carried out in a number of countries in the region—mainly the new EU member states and, to a lesser degree, the countries of South Eastern Europe as well as the Russian Federation and Ukraine—which have already granted tertiary providers considerable autonomy. These countries, while continuing to make needed improvements in the devolution of managerial authority, must now face the primary (but not exclusive) challenge of improving academic and fiscal integrity in the sector by introducing stronger accountability mechanisms. Other countries in the region, especially Belarus and most of the low-income CIS states, have yet to change how their tertiary education systems are managed. These countries face the simultaneous challenges of enhancing university autonomy while devel- oping accountability mechanisms and introducing performance-based financing. Regarding the need to monitor and evaluate system delivery and quality, all countries in the region lag behind; in other words, all need to make greater efforts to “turn on the lights� by vastly improving their data collection on student learning and educational outcomes. Even though fundamental integrity problems remain widespread in the university sector, it is important that ECA policy makers refrain from attempting to resolve these problems before they devolve authority to universities and introduce flexible financing. The group of ECA countries that have not yet initiated tertiary reforms need to introduce autonomy and accountability as mutually reinforcing policy instruments. Policy makers in these countries need not simply transfer autonomy to static, traditional universities, but can instead mandate that increased autonomy be accompanied by more “businesslike� and accountable leadership and management. In view of the relatively weak institutional capacity in the region— including somewhat newly established or nonexistent quality assurance agencies—policy makers may need to employ many different, comple- mentary tools to strengthen accountability for results. These tools include Overview 13 rankings, tracer studies, and creation of an enabling environment to facilitate the growth of the private sector. Counting on a quality assurance agency to quickly establish the capacity needed to externally monitor a rapidly growing sector is risky—this kind of agency needs to be part of the solution, but not the only response. Improve the Efficiency of Resource Use through Performance-based Financing Managing school systems for performance means moving away from inflexible line-item budgeting towards greater use of delegated budgets, with incentives for maintaining enrollment and attendance (i.e., per stu- dent financing—a path on which the majority of ECA countries have already embarked). More flexible, smarter financing in the form of block grants (i.e., contract- or performance-based) can also provide funding to institutions in return for meeting agreed learning outcomes. Not only does this type of financing give local education managers much-needed flexibility that input-based budgets do not permit, it keeps them focused on student results. In addition to performance-based financing, improving financial effi- ciency at the pre-university level will require creating incentives for larger class sizes, encouraging school optimization, and addressing the issue of overstaffing. At the tertiary level, several ECA countries (for example, Poland and Romania) have already adopted per student financing, shift- ing their focus away from inputs and toward results. The remainder of countries in the region will need to introduce performance-based financ- ing to improve the efficiency of spending in this sector. If ECA countries want to simultaneously raise participation rates and increase the quality of tertiary education, more private resources will also be needed—the cost pressures facing higher education in the region are simply too great. Mobilizing private resources will help strengthen results-oriented outcomes in the tertiary sector in two fun- damental ways: (1) by increasing competition, as more nimble and innovative private providers help bring innovations to public providers, and (2) by increasing students’ connection to the education process. When students and parents pay tuition fees—whether to a public or a private provider—they are generally more demanding about the quality and relevance of the education provided. There is no single ideal level of funding for higher education, and no single ideal mix of public and private funding sources. Different countries will make different kinds of trade-offs. 14 Skills, Not Just Diplomas Build the Foundations of Adult Learning Systems Unlike pre-university and university education, adult education systems are largely nonexistent in most ECA countries. Building such systems will require shifting away from government-defined programs towards a well- regulated market of private and public providers that deliver training services to both working and unemployed adults. Governments in the region will, however, continue to play a role in education and training for the unemployed, often by contracting private providers to deliver needed services. Policies that address market failures in this sector are especially important to lay the groundwork for effective adult education systems. In general, successful systems require a high degree of coordination and part- nership between government agencies and the private sector, as well as giving the demand side of training—that is, businesses and individuals—a strong voice in determining training policy. Once a solid adult learning sector is established, governments can then “steer� it by monitoring data on program quality, promoting autonomy in return for accountability, and improving the efficiency of government financing in the sector. Expanding adult learning is a priority for advanced economies in the region that are facing a demographic decline. Their challenge is to ensure that existing coordination mechanisms function well and that regulation and financing are used to facilitate the emergence of a private sector-oriented adult education and training market. The principal prior- ity of less advanced economies in the region experiencing a demographic decline (i.e., many of those in South Eastern Europe and the middle- income CIS countries) is to introduce a policy framework for adult learning and create the tools needed to implement this strategy (e.g., coordination mechanisms, plus initial steps toward regulation). For the less advanced economies in the region that are not facing a demographic decline (i.e., those of low-income CIS countries and Albania), it may be more productive to limit efforts to establish policy frameworks and coordination mechanisms for this sector. For many countries, participat- ing in the OECD’s Programme for International Assessment of Adult Competencies (PIAAC) would be an important first step in understand- ing the current skills and competencies of their work forces. Conclusion It will take time to steer education systems in the ECA region away from a focus on inputs toward a focus on delivering a quality education to the Overview 15 majority of students. Waiting to begin this process will only cause coun- tries to fall farther behind in delivering the skills their economies need to compete in a global economy. Overall, much greater measurement of student learning outcomes is needed at all levels of education if policy makers are to understand the strengths and weaknesses of their respective systems and design effective policies to improve them. If the role of cen- tral governments is redefined to focus on strategic policy, system goals, and regulation, central policy makers will be freed to use incentives to manage lower-level actors to meet goals and then measure the results. In schools, this process will involve greater use of per capita financing, more autonomy, and greater accountability for learning outcomes. At the level of higher education, strengthening national quality assurance mech- anisms is needed to improve the accountability of education providers, as is greater dissemination of information on learning and employment outcomes (e.g., results from tracer studies, or university rankings) to inform student choice. In addition, existing financing incentives will need strengthening through the introduction of more performance-based bud- geting and, where warranted, greater institutional autonomy. Although adult learning systems are critically important for the region, their priority in individual ECA countries will depend on economic and demographic conditions and other, perhaps more pressing education challenges. All countries in the region can, however, begin to develop a coherent strategy for adult education and training. Notes 1. The countries included in the Europe and Central Asia (ECA) region include Albania, Armenia, Azerbaijan, Belarus, Bosnia and Herzegovina, Bulgaria, Croatia, Czech Republic, Estonia, Georgia, Hungary, Kazakhstan, Kosovo, Kyrgyz Republic, Latvia, Lithuania, the former Yugoslav Republic of Macedonia, Moldova, Montenegro, Poland, Romania, the Russian Federation, Serbia, the Slovak Republic, Slovenia, Tajikistan, Turkey, Turkmenistan, Ukraine, and Uzbekistan. 2. Adjusted for Purchasing Power Parity, or PPP. 3. The EBRD–World Bank Business Environment and Enterprise Performance Surveys (BEEPS) provide successive snapshots of the self-reported constraints to expansion by firms in the ECA region. The surveys were conducted in virtually all ECA countries in 1999, 2002, 2005, and 2008. Survey samples are constructed by random sampling from the national registry of firms (or equivalent) and cover both industry and service sectors. 16 Skills, Not Just Diplomas 4. It should be noted that Turkey does not have a socialist past, but does operate “in the dark� to a large extent and has not made the performance-oriented management reforms of other OECD countries. The expenditure efficiency issues are, however, different in nature. See World Bank 2005a. 5. While the use of learning assessments is not uncontroversial, testing method- ologies have improved in recent years, making these tests better and more useful for assessing student learning than was previously the case. Restoring and sustaining growth in Eastern Europe and Central Asia requires reforms to boost competitiveness and increase labor productivity. Among the required changes are reforms to education. In surveys conducted immediately before the economic crisis, companies in the region reported shortage of skills as one of the most significant bottle- necks in their operations, suggesting that education systems in Eastern Europe and Central Asia— with a reputation for high enrollment rates and well-trained teachers—still need to improve their performance. In fact, international test results show that many students—outside of a handful of coun- tries in the region—are failing to acquire more than the most basic literacy and numeracy skills. Anecdotal evidence also indicates that the rapid expansion in higher education has led to a decline in the quality and relevance of education provided. At the same time, there are few opportunities for adults to retrain, upgrade, or acquire new skills—the life-long learning needed for employability. As Skills, Not Just Diplomas suggests, the shortage of skills is a wake-up call to reform education and training systems to provide higher quality education with the flexibility for students and training institutions to better respond to market signals. Such deep reform will have to center on the following: • Focusing more on measuring whether students learn and graduates find jobs, and using this information to actively improve teaching and learning. • Using incentives across the education system, including granting greater autonomy to institutions on curriculum, teaching methods, resource use and institutional mission, and increasing accountability for learning. • Addressing the huge inefficiencies in the sector by downsizing the school network in response to dwindling student numbers, and using freed-up resources to restore the attractiveness of the teaching profession, expand adult education and training, and measure learning outcomes. Only by embracing and seriously pursuing such difficult and long-term reforms will the countries in Eastern Europe and Central Asia substantially improve their education outcomes and support the competitiveness of their economic systems. SKU 32439