Moving toward Effectiveness, Equity and Efficiency (RESUME3) Contents Acknowledgements ....................................................................................................................................... 3 1. How does Ukraine’s education system perform? ............................................................................. 5 2. What does Ukraine need to do to strengthen its reform agenda? ................................................ 11 A. Extend the NUS vision for competency-based and student-centered learning across the sector, particularly in higher education .............................................................................................................. 13 B. Strengthen institutional capacity and governance structures .......................................................... 14 C. Target resource flows to promote performance and sustainability ................................................. 16 D. Align individual incentives and capabilities with learning goals .................................................... 19 E. Provide effective feedback and information on systemic results .................................................... 22 2 Acknowledgements This report was written by a team led by James Gresham and Diego Ambasz, and included Suhas Parandekar, Juan Manuel Moreno, Simon Thacker, Lucia Casap, Subhashini Rajasekaran, Iryna Kohut, Inna Sovsun, Olga Kupets, and Gülcan Yayla. The CEDOS think tank led by Yegor Stadny provided data and background analysis. Additional data and analysis were also provided by Yevhen Nikolaiev and Lyudmyla Sokuryanska. Support was provided by Larysa Khaletska and Patrick Biribonwa. The analysis was carried out under the overall supervision of Satu Kähkönen, Country Director, Ukraine and Harry A. Patrinos, Practice Manager, Education. Valuable comments at various stages of the project were received from Feng Zhao, Halsey Rogers, Elizabeth Ninan, Shwetlena Sabarwal, Jason Weaver, Soren Nellemann, Lars Sondergaard, Anthony Gaeta, Klavdiya Maksymenko, and Anastasia Golovach. The team benefited greatly from discussions at various formal and informal meetings with representatives of various government agencies, local authorities, and the donor community. In particular, valuable comments were received from experts of the Ministry of Education and Science, the Institute for Education Analytics, the Ukrainian Center for Education Quality Assessment, the Institute for Educational Content Modernization, and the National Agency for Quality Assurance in Higher Education. 3 Today, Ukraine is at a crossroads: despite impressive transformation in some sectors of the economy, the foundations of the emerging new economy are still fragile. Openness to the outside world, adoption of new technologies, and a vibrant entrepreneurial spirit is driving this transformation. However, growth has been volatile and unsustainable with a reliance on commodity-based exports, short-term foreign savings, and foreign remittances. The Euromaidan revolution created further acute political and economic challenges which led to a broad agenda of structural reforms. Going forward, the old growth model that relied on legacy industries will not deliver Ukraine’s aspirations.1 Education and human capital development are critical for driving high, sustainable and inclusive growth in Ukraine, yet human capital remains a small share of national wealth. Historically, Ukraine has benefited from a strong education system that has propelled the country’s economic and social development. Since independence, Ukraine has been able to sustain many of its comparative advantages in educational excellence, contributing to high levels of educational attainment and human capital development. However, skills demanded by the expanding sectors are different than those supplied by the education system, and change has been slow. According to recent wealth estimates of 141 countries, human capital comprises only 34 percent of total national wealth in Ukraine, compared to 51 percent for lower- middle-income countries and 62 percent for the Europe and Central Asia (ECA) region.2 Despite high levels of education, human capital has been a relatively weak factor of production in driving economic growth. Per capita income and labor productivity also remain among the lowest in the region, and the population continues to decline. Between 1999 and 2017, the period over which Ukraine’s private sector emerged, the average rate of growth of per capita income was 3.3 percent per year, compared to the average of Commonwealth of Independent States (CIS) countries at 5.6 percent or non-CIS countries at 3.6 percent. Ukraine’s economic transformation remains incomplete, and although the economy has grown, per capita income growth has been volatile.3 At the same time, Ukraine’s population continues to decline due to declining birth rates and emigration. The population of Ukraine has shrunk by around 15.0 percent since 1999, and the youth population has declined by nearly 25.0 percent. There is strong reason to believe that the education system needs to change or risk falling behind, and Ukraine recognizes this need. While there are competing visions of what knowledge and skills will be needed in the future, education and the systems that educate the next generation must constantly evolve and adapt to a fast-changing world. The increasing role of technology in economic activities and everyday life has already led to significant changes in the demand for skills, with a greater need for advanced skills in all types of work.4 However, upgrading cognitive skills alone is not enough: ‘soft’ skills are increasingly important given that interpersonal relations between humans cannot (yet) be replaced by the intervention of technology. Adaptability requires a strong and balanced toolkit of skills, which requires a rethinking of the traditional dividing lines between academic and technical disciplines. Although Ukraine has taken bold steps toward reforming its system, the reform process has been uneven and additional reforms are needed to ensure success and continuity of the process. In Ukrainian discourse, ‘reform’ is often believed to consist of legislative changes only.5 In this sense, reform is well underway with the passage of recent laws, particularly in the general secondary and higher education sectors. However, legislative changes must be implemented materially in practice to create the change in results that is desired. This requires resources, technical know-how, public support, and political leadership. Furthermore, additional reforms are critical to address imbalances that remain. This report seeks to answer two key policy questions: 1. How does Ukraine’s education system perform in terms of effectiveness, equity, and efficiency? 2. What does Ukraine need to do to address constraints to progress under the ongoing reform agenda and position education as a driver of growth? 4 1. How does Ukraine’s education system perform? Ukraine is committed to developing a modern education for the 21st century, and it has taken a number of key steps to make this happen. The reforms introduced following the Euromaidan Revolution have generated great optimism by decentralizing and democratizing the education system, while laying the foundations for greater alignment and integration with European norms and standards in education. Despite Ukraine’s impressive foundations in and history of education, the system appears misaligned with the changing needs of the economy and the population. This section looks at system performance along three dimensions: effectiveness, including quality and relevance; equity and inclusion; and efficiency of resource use. Ukraine has a highly educated population, but the education system needs to focus more on quality over quantity and on meeting the evolving skills needs of the labor market Educational attainment in Ukraine has increased considerably over the last 30 years, particularly at the tertiary level. Between 1980 and 2010, the average years of schooling for the population over age 15 has increased nearly three-fold (figure O.1).6 Educational attainment for the average Ukrainian now exceeds that of other high-income countries, such as the United Kingdom and Germany. Ukraine has also become one of the top countries in terms of higher education coverage of the population, a trend that has accelerated rapidly after independence. This is partially explained by Ukrainian legislation, by which colleges and technical schools became part of the higher education system until recently. Still, as of the 2017/18 academic year, the higher education coverage rate was 82 percent. The share of the population age 25+ with at least some tertiary education has reached about 40 percent, exceeding that of the OECD average and many other countries. Figure O.1 Educational attainment of population in Ukraine and selected countries, 1980–2010 Population Age 25+ with Tertiary Education 45 40 Ukraine 35 Percent of population age 25+ 30 OECD Sweden 25 Germany Poland 20 Hungary Chile 15 Portugal Argentina 10 5 0 1980 1985 1990 1995 2000 2005 2010 Source: World Bank EdStats. However, learning outcomes among secondary school students and literacy proficiency levels among tertiary educated adults lag other countries. While the Human Capital Index predicts that a child born today in Ukraine will receive 13 years of schooling by the time she reaches age 18, this figure drops to 10.2 years after adjusting for the quality of learning. This means that for the average student, 2.8 years of schooling time is ineffective or wasted in the sense that it does not result in learning. This learning gap is substantially higher in Ukraine than in high-performing education systems such as Canada, Finland, or the 5 Republic of Korea. Furthermore, literacy proficiency scores collected through the World Bank’s Skills Towards Employment and Productivity (STEP) survey indicate that higher educational attainment does not guarantee even basic cognitive skills or ensure such skills relative to other countries. Although university graduates have higher reading proficiency scores on average, the top-performing individuals with only a general secondary education scored higher than over 50 percent of university graduates, suggesting wide variation in learning outcomes (figure O.2).7 Figure O.2 Learning outcomes and literacy proficiency in Ukraine and selected countries Learning gap based on HCI Literacy proficiency of adult population 15 (2011-2013) 320 Literacy scale (range 0-500) 14 310 300 13 290 Years of School 280 12 270 Japan Korea, Rep. Finland 260 Canada Germany Sweden Austria 11 United Kingdom 250 Estonia Poland Latvia United States 240 10 USA Armenia Korea, Rep. Canada Estonia Poland Ukraine Japan Finland UK Sweden Austria Germany Ukraine 9 8 Total adult population Expected Years of School Learning-Adjusted Years of School Adult population with tertiary education (ISCED-1997 5A/6) Source: World Bank HLO Database and STEP Survey. The rapid expansion of the higher education system has produced an increasing number of graduates, many of whom have not been absorbed into the labor market except in jobs that do not require a university-level education. Thus, relative to demand, there is a larger supply of tertiary graduates along with a diminished relevance of credentials. This has contributed to education-job mismatch, particularly for young university graduates. Consequently, the share of tertiary educated workers among the unemployed has increased from 32 percent in 2004 to 47 percent in 2013.8 Moreover, as of 2013, 40 percent of young university graduates were working in lower-level jobs that did not require university-level education, in comparison to 29 percent for prime-age and older workers. In terms of employee satisfaction, a serious skills mismatch is identified as 40 percent of employers reported significant skills gaps which harm business objectives. Teaching and learning in universities remain focused on knowledge acquisition, often with outdated content that is not adjusted to the needs of modern students or new trends, research, or technology. Although there is little systematic information on quality of learning outcomes in higher education, there is a common view that curriculum and pedagogical teaching methods need to be modernized to reflect the evolving needs of society and the economy. For example, applied business, financial and management skills are in short supply in Ukraine, and only a few universities actively promote entrepreneurship training and programs.9 Globally, higher education is trending toward multidisciplinary study programs that emphasize experiential learning and project-based approaches to solve complex problems. Ukraine would benefit from more support for thematic teaching, entrepreneurship, and student-centered approaches to learning, as well as involving employers in study program revisions to improve employability. Corruption, including a mass disregard for academic integrity and a high tolerance for academic violations, also poses an ongoing challenge to education quality and the signaling power of 6 credentials. Although corrupt access to the higher education system has diminished since the introduction of the External Independent Test (EIT) in 2008, other issues persist, such as academic dishonesty and systematic violations of academic integrity principles, including cheating during exams and plagiarism. These instances are commonplace and are not perceived as abuses. Different estimates indicate that at least 25–30 percent of students have directly engaged in academic misconduct or bribery, with a much larger share exposed to and familiar with such practices. Given that higher education is central to public and private sector development and is a primary driver of social mobility, this challenge remains a major concern for the higher education system.10 Despite evidence of diminishing quality and skills mismatches, economic returns remain relatively high at the tertiary education level, though they vary by field of study. Available estimates from the period 2006–2016 indicate that the private returns to an additional year of schooling in Ukraine are approximately 6 percent, compared with the average in Europe and Central Asia of 7.3 percent and the world average of 8.8 percent.11 However, workers with tertiary education have much higher returns than graduates of secondary vocational schools as well as lower-educated workers with general secondary education or less, though the returns vary by field (Figure O.3). High returns despite diminishing quality and skills mismatches are driven by a confluence of factors, including an over-emphasis on educational credentials in the labor market, high selectivity into tertiary education by program and institution, and the combined effect of older and younger cohorts of tertiary educated workers comingled together in the labor market. The variation in returns by field of study indicates the importance of strategic development of higher education and the need to provide information to students on their prospects when making choices about their educational pathways. Figure O.3 Returns to education by level and field of study Relative to upper secondary education and less on yearly earnings of employees at their main job ( %) Tertiary, Agriculture Tertiary, Military and defense Tertiary, Arts Tertiary, Education Tertiary, Humanities Tertiary, Natural sciences Tertiary, Technical sciences Tertiary, Medicine Tertiary, Law Tertiary, Economics Vocational 0 10 20 30 40 50 60 70 2006 2016 Source: Authors’ calculations based on HLCS-2006 and 2016 (individual-level data for the fourth quarter). Poor governance and limited labor market relevance have contributed to a decline in public trust and a focus on credentials rather than skills, which threaten Ukraine’s human capital edge. Prior to the Euromaidan Revolution of 2014, nearly 75 percent of Ukrainians believed that corruption was pervasive or widespread in higher education, and about 50 percent reported the same for secondary education. 12 According to the 2016 Life in Transition Survey, Ukraine is an outlier among other countries, with relatively low levels of satisfaction with public education, along with high levels of experience with corruption.13 At the same time, the share of tertiary educated workers among the unemployed increased from 32 percent in 2004 to 47 percent in 2013.14 Low accountability and skills mismatches contribute to a growing level of 7 credentialism and to the public sense that the education system is no longer meeting the needs of students, families and employers. Inequities in Ukraine’s education system start early and limit the potential of learners and the system as a whole High-performing education systems prioritize equity and inclusion of all learners, maintaining the vision that all students are capable of high achievement with the right level of support. Ensuring that every child benefits from high-quality instruction is not only an important end in itself. The evidence from international assessments suggests that strong performance for the system as a whole is dependent on the need to deliver for every child.15 Unequal access to preschool means that inequalities in the opportunity to learn and benefit from schooling start early, particularly in rural areas and for poor families. Ukraine has high rates of enrollment in pre-primary education by international standards, but access remains unequal. The net enrollment rate for children ages 3-5 in urban areas is 85 percent on average, compared to 58 percent in rural areas. The rural enrollment rate drops even further in some oblasts, such as Kharkiv (54 percent), Lviv (44 percent), and Ivano-Frankivsk (39 percent).16 This is concerning, given the results of the recent UCEQA monitoring study of primary school graduates, conducted in 2018, which clearly shows that Grade 4 students who attended preschool scored significantly better in mathematics than those who did not.17 Early inequities persist over time, which prevents many students from acquiring the foundational skills needed to succeed in higher education or the labor market. Ukraine performs well on available international assessments compared to other countries at similar income levels. However, this masks a high degree of inequality. The latest available international assessment, TIMSS 2011, indicates that 28 percent of Ukrainian students reached only the low benchmark for mathematics performance, and another 20 percent of students failed to reach the low benchmark. This means that nearly 50 percent of students in total are at the lower end of the mathematics achievement distribution. School segregation and poor-quality learning environments, particularly in rural areas, contribute to the challenge. Evidence from external learning assessments in Grade 4 and Grade 11 indicate that inequality is driven by (a) clustering of poor students in poor schools; (b) inadequate learning environments in small-sized and rural schools; and (c) selectivity that creates between-school inequality such as the difference in performance between selective, “elite” schools (gymnasiums, lyceums, specialized schools) and regular non-selective schools. Rural schools have substantially less access to learning materials and information and communication technologies (ICTs) and are more likely to have shortages of subject teachers, according to DISO data from 2018. The hub school program is not yet having its intended effect on learning outcomes. Although the hub school program is intended to give students in rural areas access to better quality learning environments, the effects cannot yet be observed in terms of student learning outcomes. In fact, average performance in hub schools is worse than in other schools. This likely reflects the mixed fidelity of implementation of the program, along with the fact that the program is still new so the learning impact has not materialized yet. Higher education represents the top objective and most common path for most young Ukrainians, but unequal access to high-quality curricular options and academic/career guidance in secondary schools means many are not adequately prepared to enter higher education. About 60 percent of grade 9 students choose to continue their education in general secondary schools. Those who enter elite urban schools with specialized curriculum are more likely to secure top scores on the EIT. However, these schools use competitive admissions procedures determined at the school level, meaning that high performance on 8 the EIT is partly the result of stringent admissions criteria at entry. 55 percent of these students choose to take the mathematics EIT, considered to be one of the most difficult EIT subjects, and it is required for admission to many in-demand fields in higher education. Students in urban regular (nonselective) schools are less likely to choose the mathematics EIT (45 percent), compared to only 40 percent in rural regular schools. Rural students are less likely than urban students to exceed the EIT cutoff thresholds and are considerably less likely to apply to and ultimately enroll in higher education. Whereas nearly 70 percent of urban students passed, applied, and ultimately enrolled, this figure drops to 40 percent for rural students.18 Furthermore, rural students are less likely to achieve the high levels of EIT performance required to access state-funded places in HEIs: only 17 percent of state-funded places for bachelor’s programs in 2018 went to students from rural areas (figure O.4). Figure O.4 Higher education admissions by student origin and EIT performance (2018) 70 175 Average EIT score (Ukrainian Admitted students (thousands) 170 60 165 50 160 language) 40 155 150 30 145 20 140 135 10 130 0 125 Urban Rural Urban Rural State-funded places Privately funded contract places Number Average EIT score Source: Authors’ analysis of EDEBO database. Ukraine spends a high share of public resources on education, but with a large network of institutions and a declining student-age population, those resources are not used efficiently Ukraine spends more of its GDP on education than most EU and OECD countries. After a steep devaluation of the hryvnia in 2013, public education spending declined by 35 percent in real terms over two years. Between 2013 and 2017, budget financing shrank from 7.2 to 6.0 percent of GDP. The decline brought Ukraine closer to international benchmarks in terms of the share of national wealth devoted to education, but spending remains high: with public spending on education at 6.0 percent of GDP and with private spending adding another percentage point of GDP, Ukraine’s education spending is among the highest in the world (figure O.5).19 This is driven in part by the law which requires the state to allocate at least 7 percent of GDP for education. However, Ukraine faces serious macroeconomic vulnerabilities and fiscal pressures, including significant debt repayments.20 This means that spending more to implement the education reform agenda and make needed investments in the sector is not an option: making better use of existing resources is an imperative. 9 Figure O.5 Public and Private Spending on Education, Percent of GDP (%) 8 7 6 5 4 Percent of GDP 3 2 1 0 Canada Denmark Israel Australia Finland Germany Argentina Mexico Estonia Czech Republic Korea Colombia OECD average EU22 average Ireland Poland Norway Belgium France Turkey Italy Slovak Republic United States Netherlands Sweden Japan Hungary Luxembourg Ukraine Indonesia United Kingdom Iceland Latvia Portugal Chile Austria New Zealand Slovenia Spain Lithuania Public expenditure on education institutions Private expenditure on education institutions Source: World Bank (2018c). Note: Ukraine data from 2017; OECD member and partner country data from 2014. Spending on education covers preprimary through tertiary, including expenditures not allocated by level. Data covers the majority of OECD member and partner countries as well as EU countries. To meet a growing demand, the tertiary education system expanded significantly in the 2000s, but the subsequent decline in the student population has left the sector oversized and inefficient. Higher education coverage of the population increased from around 45 percent in 1992 to over 80 percent today, according to data from SSSU. This significant increase has been driven in part by a large expansion of the sector. However, the student population has been shrinking in past years: over the same period, the youth population aged 0-17 declined by around 40 percent. While the private higher education sector shrank by almost 4 times between 2007 and 2017, the public sector has shrunk by only 1.6 times. Now, with a total population of around 42.4 million, Ukraine’s network of 327 universities, academies and institutes—of which 231 are public—is quite large. This amounts to 7.7 HEIs per 1 million population, or 5.4 public HEIs per 1 million. Even after considering population size, Ukraine appears to have a relatively large public higher education sector compared to other countries in Europe. Furthermore, many HEIs are relatively small, especially considering specialized institutions (for example, academies and institutes) and branches of universities, which can hinder the quality and conditions of learning. Although the number of HEIs has decreased over time, along with the number of students, the number of university faculty members has decreased more slowly, leading to low student-faculty ratios relative to neighboring countries (figure O.6). Figure O.6 Inefficiencies in higher education Density of HEIs relative to population (2017) and student-faculty ratio in tertiary education (2012–2017) 12.0 Romania 24 HEIs per 1 million population 10.0 22 Student-faculty ratio 8.0 (headcount basis) 6.0 Poland 20 Finland 4.0 18 UK 2.0 Moldova Hungary 16 - Sweden 14 Belarus Ukraine 12 All HEIs Public HEIs 10 2012 2013 2014 2015 2016 2017 Source: Authors’ analysis of SSSU and UIS data. 10 The same demographic and fiscal pressures are affecting general secondary schools, though the situation is more complicated due to the role of local governments in managing their school networks. The need to adapt a general secondary network to the demographic reality has long been recognized in Ukraine. While some areas have seen modest increases in birth rates, and urban student populations have increased as a result of rural-to-urban migration, the student-age population continues to decline in most of the country. This has led to low student-teacher ratios, particularly in grades 5-11 when students switch to subject teaching. The average student-teacher ratio in lower and upper secondary education in Ukraine is 10.9, compared to the OECD average of 13.1.21 Small schools and small classes constitute a fiscal strain on the budget as well as a deficient learning environment that detracts from students’ opportunities to learn and work together with a diverse peer group. However, local governments have responsibility for their school networks, and years of confused responsibilities and poor sectoral and budget management have perpetuated inefficiencies in the school network.22 2. What does Ukraine need to do to strengthen its reform agenda? Ukraine is a middle-income country with significant potential for growth. Ukraine has historically benefited from its human capital, particularly its robust education system and highly skilled labor force. However, Ukraine has not translated this human capital into productivity and national wealth. The rapid expansion of the education system has led to high levels of educational attainment, but quality and relevance of learning have deteriorated while lack of innovation, isolation, and corruption have led to dissatisfaction, skills mismatches, and credentialism. Despite having a highly educated labor force, human capital represents only 34 percent of total wealth in Ukraine (compared to the ECA average of 62 percent), and labor productivity is only 22 percent of that in the European Union (Figure O.7). This suggests that education is not contributing its full potential to the wider economy. Figure O.7 Despite high levels of education, Ukraine is not tapping its full human capital potential 120% 100% 80% 60% 40% 20% 0% Labor productivity as share of European Union average (2017) Human capital wealth as share of total per capita national wealth (2014) Source: World Bank World Development Indicators and Wealth Accounting databases; World Bank (2018a). Demographic decline and rapid technological change mean that education must constantly evolve and adapt. The increasing role of technology in economic activities and everyday life has already led to significant changes in the demand for skills, with a greater need for advanced skills in all types of work. 23 However, upgrading cognitive skills alone is not sufficient: ‘soft’ skills are increasingly important given that interpersonal relations between humans cannot (yet) be replaced by the intervention of technology. 11 Adaptability requires a strong and balanced toolkit of skills, which means that the dividing lines between academic and technical disciplines will need to change over time. This leads to several conclusions: Ukraine cannot afford to take its human capital endowment for granted, and the cost of underperformance is rising. The workers of tomorrow—including the teachers, nurses and doctors that develop and safeguard human capital—are the product of today’s education system. The education reforms introduced since 2014 represent a major departure from the past and are in line with many good practices in high-performing education systems in Europe and elsewhere. They have generated great optimism by decentralizing and democratizing the education system. However, the large-scale transformations remain nascent. Reforms in such an environment inevitably produce imbalances between the objectives of reform—the movement towards a new education paradigm—and the inertia of history. Therefore, how can Ukraine strengthen its reform agenda moving forward? This review has identified five priority areas for reform going forward in order to address systemic imbalances which undermine Ukraine’s education reform agenda. These imbalances, which are interconnected, pertain to vision, governance, financing, incentives, and information. To rebalance the system, this review proposes reforms in these five key areas in order to address the identified imbalances, strengthen the reform agenda, and steer the system towards longer term competitiveness, innovation, productivity growth, and sustainability. The sections below describe the five imbalances, as well as what Ukraine needs to do to address them. Key priority actions for the short-term and medium-term are also summarized in Table O.1 at the end of the Overview. Figure O.8 Priority areas to ensure that reforms promote effectiveness, equity, and efficiency Source: Authors’ elaboration. 12 A. Extend the NUS vision for competency-based and student-centered learning across the sector, particularly in higher education The vision for reform in general secondary education—articulated in the New Ukrainian School—is a positive step in the right direction. This vision is rooted in a clear argument for change: today’s Ukrainian school should better equip pupils with the skills needed to learn throughout life, think critically, set and achieve goals, work in teams, and communicate in a multicultural environment. The attempts to modernize Ukraine’s general secondary school system to address these fundamental challenges are ambitious and badly needed. The Law on Education and the New Ukrainian School will prepare students for the 21st century through a combination of several elements: new educational content, more motivated teachers, greater decentralization and autonomy, child-centered approaches to teaching, a new schooling structure with 12 years of secondary education, fair allocation of public funds, and contemporary educational environments.24 Going forward, it will be important for Ukraine to accelerate the expected reform of upper secondary education to introduce the new three-year curriculum structure and streamline educational pathways from secondary into tertiary education. The current structure of upper secondary education in Ukraine is fragmented, with upper secondary general education offered in general secondary schools as well as vocational schools, colleges, and technical colleges. The movement toward a student-centered and competency-based approach to learning in line with the NUS vision will be challenging at the upper secondary level without advancing reform to concentrate resources —schools, teachers, funding, and instructional time—on what matters most. This reform will increase the duration of upper secondary education from 2 to 3 years, creating more time for students to gain exposure to the curriculum at a critical transition point in their educational trajectories, while also affording more choice, similar to the upper secondary school reforms that took place in Finland in the 1980s and Poland in the 1990s.25 The reform would also concentrate resources in a smaller network of specialized institutions, allowing schools to offer a higher quality of education with more elective subjects for students while also using fiscal resources and educational facilities more efficiently. The Law on Education requires grade 12 to be introduced by 2027 but accelerating the reform would ensure the transition to three-year upper secondary education sooner. This reform also presents an opportunity to accelerate optimization of the school network in upper secondary education, as underutilized schools and colleges could be merged or closed by local authorities. In higher education, the challenge is greater and increasingly urgent. There is no clear vision that links higher education to the positive developments for reform in secondary education or to the skills requirements of the labor market. The Law on Higher Education was the first large systematic reform measure adopted by Ukraine’s parliament in the immediate aftermath of the Euromaidan Revolution. It represented a compromise on the part of many different interest groups, coming on top of years of absence of a clear development strategy for higher education: a policy of nonpolicy.26 However, it did not address this challenge. To this day, there is no clear or coherent vision for the development of the higher education sector or individual universities. This is a fundamental problem: higher education in Ukraine cannot serve the needs of the people and the economy without clear objectives and a strategy for how to achieve them. Although the 2014 Law on Higher Education made a major step toward dismantling the centralized structures of the past, it provided more autonomy without the attending accountability mechanisms or financial flows. Without a strategy for higher education development, individual HEIs use their autonomy to achieve individual goals rather than working to achieve a broader goal for the system and nation. Furthermore, many important decisions governing the sector need to be taken by the Council of Ministers, including any that would lead to changes in funding and because many government agencies oversee subordinate HEIs. This further complicates the lack of a strategic vision. 13 The vision for higher education in Ukraine needs to prioritize modern approaches to curricula, pedagogical teaching methods, and learning support systems in line with the NUS and labor market needs, while also transforming the system to promote diversity and sustainability. Successful modern mass higher education systems are characterized by a high level of institutional diversity in which individual institutions have different missions and profiles.27 Currently, the higher education system is expansive, with a large number of specialized HEIs that are relatively small in size and scope. This has caused the system to become oversized and inefficient, especially in relation to the shrinking student population. Despite spending a relatively high share of public resources, funds are spread thinly across many institutions and staff, contributing to the incoherence: although the system spends a lot, rectors and faculty believe that lack of funding is the key problem, limiting their ability to invest in modern curricula or equipment.28 This also contributes to institutional stratification. A strategic vision for higher education should reflect modern approaches to curriculum and pedagogical teaching methods, stronger linkages with employers and the labor market, and greater institutional diversity with larger and more comprehensive HEIs. It is also important to maintain a focus on Ukraine’s vision for equitable and inclusive education. High-performing education systems prioritize equity and inclusion of all learners, maintaining the vision that all students are capable of high achievement with the right level of support. Global evidence from international assessments suggests that strong performance for the system as a whole is dependent on the need to deliver for every child. Top-performing systems show a low correlation between learning outcomes and the home background of the individual student, meaning that these systems have produced mechanisms and approaches to ensure that schools can compensate for the disadvantages that result from the student’s home environment.29 Maintaining a focus on equity and inclusion should be a key aspect of Ukraine’s vision for education going forward. In particular, priority should be given to (a) expanding access to quality pre-primary education with a focus on vulnerable groups, and (b) strengthening the capacity of Inclusive Education Resource Centers to support the transition to inclusive education within a decentralized context. B. Strengthen institutional capacity and governance structures On one hand, reforms have greatly expanded the autonomy of HEIs, local governments, schools, and teachers. This reflects a major departure from the centralized direction of the system in the past. However, the capacity of decentralized institutions and governance structures which are designed to ensure quality and promote accountability remain weak. The newly established State Service for Education Quality (SSEQ) is charged with developing a quality assurance system for secondary education, including audits of education institutions, supervision and monitoring compliance with requirements of the Law on Education, and monitoring education quality. Unlike the previous form of inspection which focused on detecting violations through a rigid structure, the new SSEQ is expected to support education institutions and local authorities to improve outcomes. This shift moves Ukraine closer to the norm in other European countries with external school evaluations, and this is a positive step given the research showing that school evaluations and support for school improvement can have positive effects at the school and system levels.30 However, this institution and its functions are new, and capacity at the central and regional levels remain weak. The capacity of local authorities to manage their school networks, particularly in rural areas, remains weak as well. However, there are some good practice examples of effective and collaborative approaches in newly amalgamated hromadas which could be further developed to strengthen capacity. Many HEIs are similarly ill-equipped to use their greater degree of autonomy to improve the quality of teaching and learning. Managerial capacity within HEIs for internal quality assurance and institutional self-improvement is limited in many cases, and the limited degree of financial autonomy afforded to HEIs constrains capacity even further. Ukraine is moving towards a system of accreditation and quality assurance more in line with the Standards and Guidelines for Quality Assurance in the European Higher Education 14 Area, and the newly established National Agency for Quality Assurance in Higher Education (NAQAHE) is leading the effort for institutional and program accreditation, along with university ranking mechanisms, regulation of doctoral degrees, and support for internal quality assurance systems. However, accreditation procedures are only now being developed, and there is a substantial shortage of trained experts.31 Evidence on effective education service delivery shows that autonomy needs to be paired with accountability and capacity to operate in a decentralized approach. As such, Ukraine should prioritize three areas of support: (a) capacity for decentralized management and education service delivery; (b) internal governance capacity within HEIs; and (c) mechanisms for accreditation and quality assurance. (a) Strengthen capacity for decentralized management and delivery of education at regional, local, and school levels A key area for accelerating education reforms in secondary education is the development of managerial capacity for decentralized service delivery. Reforms have provided local authorities and schools with more autonomy in how they use their budgets and organize curricula, but they need to have proper levels of capacity in order to manage this autonomy. This is a long-term objective, but the Government can support this through the State Service of Education Quality (SSEQ) and through the dissemination of information and management tools needed to build managerial capacity. For example, in Brazil, the structured planning and management decision-making process known as the “management circuit” introduced through the Jovem de Futuro program prompted positive changes on various managerial practices as well as learning outcomes.32 Improving communication, outreach and stakeholder engagement is also key to building this capacity and generating support at the local level. Ukraine can also build on good practices from across the country. For example, the Swedish project supporting decentralization in Ukraine has developed and implemented an interactive tool to help communities prepare for the school network optimization process, as well as a database33 of good practices from newly amalgamated communities. Both these initiatives could be developed further and institutionalized in the system. (b) Strengthen internal governance capacity in HEIs As in secondary schools, HEIs also face challenges in managing the increased levels of autonomy provided to them. Many HEIs have demonstrated limited capacity for democratic decision making and strategic planning and management aimed at improving quality of teaching and learning.34 The decision from 2014 to allow HEIs to elect rectors without external interference did protect them from external political influence, though it also led to a consolidation of the status quo. Proper training and support systems should be developed to support internal transformation and build the managerial capacities needed in HEIs. One option is to strengthen the capacity and authority of the university governing boards, which is a common feature in many European higher education systems. 35 These supervisory boards should include representatives from society and the economy, such as internationally acclaimed academics, business, civil society, and government representatives. In 20 of 28 European higher education systems investigated by the European University Association (EUA) in 2010, institutions were required to include external stakeholders in the internal governance of their institution at the central level. These boards should have the reputation and authority to carry out strategic management of the HEI, coordinate efforts to achieve designated goals, and provide independent assessment of the progress of the HEI. (c) Strengthen systems for accreditation and quality assurance There is an urgent need to strengthen internal and external quality assurance (QA) functions in the higher education system, including through improved capacity of the newly established National Agency for 15 Quality Assurance in Higher Education (NAQAHE), as well as in individual HEIs. In particular, there is a need to accelerate QA reforms in line with the Standards and Guidelines for Quality Assurance in European Higher Education Area (ESG). In terms of accreditation, Ukraine could also explore the potential for partial or conditional accreditation. Currently, there are two potential outcomes of accreditation: full accreditation or failure to become accredited. Without accreditation, students will not receive a recognized diploma, which creates strong pressure on universities and the accreditation committee to accredit the program. Given the high-stakes nature of the decision, this leads to rampant corruption and pressures on all levels of the decision-making process. Partial or conditional accreditation, in which the HEI is given a period of time to address relevant issues, is a possible alternative which also supports improvement-oriented feedback and a more phased approach. Additionally, statistical information on accreditation should be collected and published, so the general public and prospective students can see the percentage of programs that were not fully accredited. C. Target resource flows to promote performance and sustainability Given macroeconomic vulnerabilities and fiscal pressures, and the fact that Ukraine already spends a high share of its GDP on education, there is a need to ensure that resource flows are made strategically to create the right incentives within the education system to achieve sustainable results. However, there is currently a disconnect—particularly in higher education—between resource flows which prioritize the status quo and the need to achieve strategic objectives in the sector. Ukraine’s input-based model of public funding for higher education has created perverse incentives in recent years for HEIs to lower admissions and quality standards, while maximizing the number of fee-paying students to compensate for declining public funds. Public funding is allocated through a system of quotas determined for each study field and level of education by the Ministry of Economic Development and Trade, similar to the system inherited from the USSR. Given the decline in student numbers over time, HEIs have become more dependent on public funds, creating at times perverse incentives for HEIs to lower entrance requirements to admit more and more students, including those with lower levels of preparation for higher education (on both state-funded and privately funded places). Today, 55–60 percent of current higher education students are paying fees (figure O.9). This is particularly the case in lower-cost fields such as social sciences and humanities. About half of those fee-paying students are enrolled in distance and evening programs, in which the quality and value of the education may be weaker. This dual-track funding system disadvantages poorer students and rural students who are less likely to achieve the high EIT scores needed to be admitted to state-funded places. Figure O.9 Enrolled students, by program, type of funding and form of study (2018) Specialist Master Bachelor Junior Specialist 0 100 200 300 400 500 600 700 800 Number of enrolled students (thousands) State-funded, intramural State-funded, extramural and evening Self-funded, intramural Self-funded, extramural and evening Source: Authors’ analysis of EDEBO database. 16 Despite several recent positive changes to improve transparency and competitiveness of funding, the public funding model for higher education remains unsustainable. Several mechanisms have been put in place in recent years to improve transparency in distribution of state-funded places and standardize EIT admissions requirements across HEIs offering the same field of study. However, these changes do not address fundamental concerns with the public funding model for higher education in Ukraine. First, despite the greater degree of autonomy afforded to HEIs, financial autonomy still remains quite limited. The amount of budget funding for higher education and the number of state-funded places are determined independently in practice, leading to persistent funding disparities even for the same study field. For example, with medicine, one state-funded place in 2016 afforded the average university under MOES 12 percent more funding than to a university under the Ministry of Health, despite having the same curricula. Without information on quality and relevance of education, higher education funding remains focused on quantity of admitted students rather than learning or graduates’ employment outcomes. Furthermore, the new mechanism for allocating budget seats at the bachelor’s level means that public funding depends on the number of applications. This creates a challenge for HEIs, which cannot adequately plan for the number of faculty they will need. This differs from good practice in higher education financing in other countries such as the Netherlands and Latvia, where differences in actual cost delivery are prioritized as a principle for ensuring financial sustainability of a given university. In secondary education, recent reforms to the financing formula provide strong economic incentives to improve efficiency, along with quality and equity, but the formula should be carefully monitored and adjusted. School funding according to a transparent formula that includes at least a component based on a per-student amount is considered good practice in the field, even though systems employ a wide range of different criteria in their funding formulas.36 Ukraine has had a per-student formula since 2014, when it was introduced along with a large budget decentralization reform, but it largely represented a de facto continuation of the previous financing system. However, in 2018, the formula included a crucial change by providing a hard budget constraint for local governments in which average class sizes were smaller than the norm in the formula. This has created a wedge between the actual and desired school network, in which local governments will be in a state of surplus or deficit vis-à-vis the subvention formula. At the same time, local governments have flexibility to reallocate resources across budget years and redeploy savings, for example, to purchase learning materials or provide preschool services. The possibility for an annual review represents a good opportunity to develop indicators on network efficiency and systematically revise the formula as needed. Three areas should be prioritized to improve targeting of resources: (a) reform of the public funding model in higher education, (b) financial incentives for efficiency, and (c) alignment of the secondary school financing formula and hub school program with the need to consolidate the school network. (a) Reform the public funding model for higher education to promote competition, performance and excellence and to consolidate resources Ukraine has an urgent need to move away from its input-based method of public funding for higher education, which has created strong incentives to lower quality standards while also contributing to funding disparities. At the same time, there is evidence that the network of HEIs is oversized relative to a shrinking student-age population. With the input-based method of funding, which does not rely on actual cost of delivery, many HEIs are in the position of having insufficient resources to deliver quality education that is relevant to the needs of the labor market. Furthermore, the current system does not incentivize excellence at the level of programs, faculties or institutions. Therefore, there is a need for systemic reform to the funding model to ensure transparency and efficiency, while also introducing a more differentiated approach based on indicators of quality. Special financial incentives may also help to encourage the development of centers of excellence within universities and stimulate top-performing programs. Systemic reform would 17 help to prioritize critical objectives of enhancing quality and improving links with the labor market, while also supporting consolidation in the sector and optimization of HEI finances. The concept of strategic financing in higher education, including performance-based funding mechanisms, have been a topic of discussion in Ukraine for several years now. A funding formula has been developed, along with draft legislative acts, which would allocate funds based on the adjusted number of students. However, the formula and its legal bases have not yet been formally approved or implemented. Moving forward, it will be important to improve the information base on which to institute a new funding model. This would include more information on the actual estimated cost of service delivery in different fields of study and formats, as well as more robust quality assurance information to mitigate the risk that formula- based funding exacerbates institutional stratification rather than rewards performance. 37 (b) Introduce additional incentives to reward or penalize HEIs on efficiency of resource use Ukraine could introduce additional mechanisms to incentivize the consolidation or merger of higher education programs and/or institutions. For example, the sector could establish an incentive program to create economies of scale and scope through voluntary strategic cooperation or mergers. A mix of top-down and bottom-up approaches may be suitable here, whereby the state provides incentives for consolidation, but the suggestions of where and what to consolidate are made by institutions, considering regional aspects and equity of access. For example, competitive funding could be provided as a top-down incentive to HEIs that have voluntary plans to merge, to build joint units or to collaborate to increase sector efficiency. A bottom-up development of models for collaboration and consolidation by HEIs would engender ownership on the part of HEIs and less political opposition.38 For example, Denmark presents a good practice example of comprehensive consolidation in higher education in which the government does not regulate which institutions should merge but supports the autonomy/ownership of HEIs and provides financial incentives to stimulate institutions’ participation in the process.39 At the same time, Ukraine could also consider additional financial penalties for HEIs that fail to improve efficiency of resource use and reduce waste, for example through the introduction of performance agreements. Unlike performance-based funding, performance agreements look at future performance, awarding institutions on the basis of expected performance rather than actual performance.40 Such agreements have been introduced in several European countries, including Croatia, Estonia, Finland, Germany, Latvia, and the Netherlands. They are individual agreements between an HEI and the funding authority, and the agreement usually includes a financial penalty or sanction if objectives are not achieved.41 (c) Monitor and adjust school financing formula, along with the hub school program, to incentivize optimization of school network and pedagogical workforce The education subvention formula for secondary education should be carefully monitored going forward and adjusted as needed to continue encouraging local governments to optimize their school network and consolidate resources. One option going forward is to gradually raise the desired class size goal specified in the formula, which is currently 13 students per class for rural areas. This would strengthen the economic incentive on the part of local governments to consolidate classes and schools. An additional consideration in the future is to create school size goals in the financing formula, which currently are not included. This adjustment of the formula needs to go hand-in-hand with a strategy for rationalizing the pedagogical workforce while making the teaching profession more attractive. The consolidation of classes and schools, leading to fewer but larger institutions with more efficient use of resources directly requires the rationalization of teachers and non-teaching staff in schools. This is particularly important to consider if teachers’ salaries continue to increase, either due to subsequent increases in the base pay or if more and 18 more teachers become certified and earn associated pay increases. Going forward, Ukraine needs to consolidate resources at the local level and raise student-teacher ratios while also rationalizing the workforce, perhaps through creating incentives for retired teachers to leave the workforce while introducing additional measures to improve the attractiveness of the profession. At the same time, the hub school program should be evaluated for implementation fidelity and revised accordingly. As mentioned, students in hub schools perform marginally worse on the EIT than other students, even though hub schools are supposed to provide more effective and efficient learning environments than comparator schools. Although this could be explained by the fact that the program is relatively new, the practical implementation of the program varies considerably, meaning that ‘hub school’ is just a title rather than a substantive indication of school conditions. The hub school criteria have changed over time as well, meaning that schools may meet different conditions for achieving hub school status. The hub school program, together with the education subvention formula, have great potential to improve efficiency of resource use along with equal access to quality learning environments in rural areas, but only if hub schools meet minimum quality conditions and represent a substantive change over their alternative. D. Align individual incentives and capabilities with learning goals The teaching load (Stavka) system for educators devalues the requirements of professional teachers and provides the wrong incentives for upgrading teachers’ capacity and the status of the profession. The teaching load system for organizing teaching and compensating teachers fragments their work into ‘piece-meal’ tasks, separating teaching hours from other important but non-teaching tasks. Because of this system, only 53-56 percent of take-home salary is from the base salary, while the remainder is tied to various top-ups. This system creates incentives that are misaligned with the expectations of teachers under the NUS, which requires a massive paradigm shift in terms of how teachers deliver instruction and interact with students. Voluntary teacher certification can be part of a strategy to upgrade teachers’ skills in line with the NUS vision, but it needs to be monitored carefully along with broader investments in teacher professional development opportunities. Substantial improvements are needed to upgrade the quality of training provided by in-service teacher training institutes while aligning both in-service and initial teacher education with the competency-based approaches to learning envisioned in the NUS. Furthermore, the professional status of teaching in Ukraine is relatively unappealing, further limiting incentives to enter the teaching profession. For example, the salary progression of a teacher’s career in Ukraine is relatively small, with those at the top of the scale making only 30 percent more than a new teacher; in OECD and EU countries, teachers at the top of the scale make 70–80 percent more than new teachers (figure O.10).42 Although teachers’ salaries were recently increased in an effort to improve the social status of the profession, they are lower than the salaries of other tertiary educated workers in Ukraine. Furthermore, the large share of top-ups reduces the transparency of the overall remuneration package for teachers. These conditions deter many bright students from considering teaching as a profession, unlike in top-performing education systems which consistently attract high-performing students into teacher preparation programs and the teaching profession.43 Students entering teacher preparation programs in Ukrainian pedagogical universities tend to have relatively lower scores on the EIT than those entering many other fields including sciences, health and welfare, and social sciences, and evidence suggests only a share of those entering teacher preparation programs will go on to become teachers. 19 Figure O.10 Teachers' salary progression: Ukraine vs. comparator countries 2.5 Ratio between starting and maximum base salary 2 1.5 1 0.5 0 Source: World Bank (2018c); OECD (2018a). There is a need to improve incentives for managing the overall teacher workforce. The teacher workforce in Ukraine is large and aging, with 25 percent over the age of 55 and 15 percent working while in retirement. While the teacher workforce has declined over time, it remains misaligned with the declining student-age population, leading to low student-teacher ratios and an inefficient use of funds. The hard budget constraint introduced in the education subvention formula will help local authorities to manage the size of the teacher workforce, but in general Ukraine needs to pursue a comprehensive approach that improves fiscal sustainability and raises the professional status and capabilities of the teaching profession. Both students and educators, particularly in higher education, have incentives to engage in corruption and academic misconduct which detracts from quality and diminishes the value of educational credentials. Estimates indicate that at least 25-30 percent of students have directly engaged in academic misconduct or bribery, with a larger share exposed to and familiar with such practices.44 Students face incentives to engage in academic misconduct, namely to compensate for an intense testing schedule (often in mandatory courses unrelated to a student’s specialization), living conditions in dormitories, obligations to work at least part-time while studying, and inadequate preparation for study at the higher education level.45 At the same time, declining public funding for universities has stretched the availability of resources, and salaries for faculty members are low relative to what they could get working in other fields, which further contributes to the corruption and integrity risks.46 There is also evidence in Ukraine that experience with bribery in HEIs is correlated with less valued diplomas on the part of employers, lower perceptions of institutional prestige, and lower perceptions that university study will help achieve a corresponding salary.47 This is particularly concerning given the global evidence showing that corruption in higher education normalizes corrupt practices among young people and increases social inequality.48 Moving forward, Ukraine needs to tackle priority areas to improve incentives: reform the career path for educators in schools and higher education institutions, harmonize the EIT to the NUS and vision for higher education, and introduce stronger tools to incentivize academic integrity and anti-corruption. (a) Reform teacher career path, including the teaching load (stavka) system for organizing and compensating teachers’ work and opportunities for professional development High-performing education systems around the world make teaching an attractive profession by improving its status, compensation policies and career progression structure, while also making good use of teachers’ time with students.49 However, the stavka system for organizing and compensating teachers’ work is poorly aligned with these objectives, as well as the new expectations of teachers under the New Ukrainian School. By fragmenting teachers’ work into teaching and non-teaching tasks, the result is that nearly half of 20 teachers’ take-home pay comes from various top-ups. This creates incentives for abuse and nontransparent allocation of teaching hours, while also devaluing the work of professional teachers. This stavka system is in contrast to the weekly workload system used in many OECD countries, where the income of all employees, including that of teachers, is based on 36–40 hours of work per week, of which 22–29 hours are allocated for teaching.50 Given that the New Ukrainian School reform expects teachers to tackle increasingly complex tasks associated with the new curriculum, this workload system of organizing and compensating teachers’ work may ultimately create disincentives for teachers and undermine reform. Teacher certification can play a significant role in supporting the NUS reform, but it must be part of a larger coordinated set of reforms aimed at making the teaching profession more attractive while also rationalizing the teacher workforce. High-performing education systems around the world attend to multiple teacher policy goals in a coherent manner, in order to ensure that policies like teacher certification are aligned with other aspects of teacher training, recruitment, and management. To function most successfully, teacher certification may need to be part of a comprehensive set of policies to reform the teacher career path and workforce that will include: (i) transparent remuneration packages for teachers; (ii) reform of workload system of organizing teachers’ work; (iii) dynamic opportunities for professional development, including coaching and mentoring opportunities for teachers to practice and demonstrate new teaching methods; (iv) rationalization of the teacher workforce through voluntary and involuntary attrition; and (v) optimization of schools and classes which drives the demand for teachers. (b) Reform career path for academic teaching staff in higher education institutions, including the teaching load system As with secondary schools, the teaching load (stavka) system is used to organize and compensate the work of teaching faculty and lecturers in HEIs. This system fragments their work and creates incentives for faculty members to collect as many teaching hours as possible, which may ultimately hinder their effectiveness as educators. The career path for academic teaching staff in HEIs needs to be reviewed as revised, putting remuneration within a wider framework of incentives. For example, more holistic approaches combine monetary and non-monetary rewards, including compensation, benefits, and opportunities for personal development.51 A more strategic approach is needed to improve the attractiveness of the academic profession and strengthen human resource management in HEIs. (c) Harmonize University Admissions Exam (EIT) to NUS and vision for higher education Given that the EIT plays such a crucial role in establishing quality of and regulating access to higher education, it is important that Ukraine continue to invest in and modernize the EIT to ensure it remains a state-of-the-art tool that is fit for purpose. On one hand, the EIT is used as a summative evaluation of secondary education (the state attestation exam) for all graduates to ensure that they pass a minimum competency threshold. Going forward, this will mean that the EIT content and test items will need to be adapted to reflect the new competency-based approach to learning that is envisioned in the New Ukrainian School curriculum. On the other hand, the EIT maintains its original purpose of providing transparency in regulating access to higher education for a subset of secondary school graduates who intend to continue their education in universities. The recognition that many students entering higher education today are not prepared for advanced studies has led some universities and programs to set minimum entrance thresholds on the EIT subtests for applicants. Given this trend, there is a clear argument to be made for reviewing the content of the EIT subtests, as well as their psychometric design, to ensure alignment with global best practices. Finally, the regular administration of EIT presents an opportunity to collect more information on students’ backgrounds, educational objectives, and pathways, so the incorporation of more survey questions could help to shed light on this and also track changes over time. 21 In addition, the introduction of more external exams for admissions to master’s degree programs could help to improve transparency while also controlling access to this level of education. The recent introduction of external exams have helped to control access for high-demand programs such as medicine and law, but this could be expanded to other regulated professions as well. (d) Strengthen incentives for academic integrity and anti-corruption and implement tools for oversight Strengthening the capacity of the MoES and the newly established National Agency for Quality Assurance in Higher Education (NAQAHE) to address issues of academic integrity is essential. However, Ukraine can seek to address this challenge on multiple fronts simultaneously. One important institutional reform that can help to strengthen incentives is to operationalize the Office of Education Ombudsman, as anticipated by the Law on Education.52 This office should create an institutional grievance redress mechanism for addressing instances of corruption or integrity violations, both in higher education and other elements of the education sector. The ombudsman position has recently been filled and the office is now in the process of being staffed. This could also be supplemented by support to establish ombudsman offices in HEIs, as well as a review and introduction of stronger legislation to penalize violations of academic integrity, including a retrospective check for plagiarism for persons who apply for high-level positions in HEIs. Additionally, Ukraine should seek to implement tools for combating plagiarism and academic misconduct in higher education, such as the National Repository of Academic Texts. This repository was designed to serve as a universal database of all academic work published in Ukraine. Having such a database would greatly facilitate the detection of plagiarism in students’ papers, theses, and dissertations. However, there has been little progress in this area since the resolution was passed by the Cabinet of Ministers in 2016. Ukraine could also disseminate information on academic integrity in universities based on student surveys. Such tools could identify high instances of integrity violations, while also encouraging and disseminating good practices among other HEIs. For example, in Romania, a coalition of education stakeholders developed detailed questionnaires which assessed university governance in several areas, including transparency and responsiveness, academic integrity, enforcement of rules, governance quality, and financial management. The exercise resulted in an immediate improvement in university transparency in procurement and recruitment, along with some significant improvements in awareness about integrity violations in universities at a high level.53 There is also a need to work directly with HEI students, faculty and administrators to expand awareness of academic integrity principles and raise awareness at higher levels. This is aligned with research on corruption which advises against fighting corruption in general, and instead focusing on specific malpractices.54 Ukraine can build on existing data sources and successful projects and programs, such as Profrights.org, a database containing information on violations of the rights of teachers and students in HEIs, as well as the Strengthening Academic Integrity in Ukraine Project (SAIUP). E. Provide effective feedback and information on systemic results The final imbalance relates to the availability of effective feedback and information to students, teachers, employers, and the system. As mentioned above, autonomy in education only works if also paired with accountability. However, through the provision of effective feedback and information, assessment is also required to hold actors accountable for improving service delivery and achieving results. 22 Assessment capacity to monitor learning should be improved, along with the EIT to better regulate access to higher education. Ukraine has initiated a sample-based external assessment of learning outcomes in grade 4, and this represents a positive development in the sector, since external assessment information on student learning is crucial to hold schools and the education system accountable. This brings Ukraine in line with most OECD countries, which have some form of a summative assessment at the primary level. However, Ukraine would also benefit from an external assessment in grade 9, before students transition into upper secondary school. The university admissions exam (EIT) has brought more transparency and trust to the admissions process, and it is now being used in place of the school leaving exam. However, it is not clear that the EIT in its current form is well designed to fulfill both functions of reform: (a) to assess that secondary school graduates have attained a minimum acceptable level of knowledge, and (b) to regulate the quality of higher education through a high-stakes examination. Furthermore, the EIT does not yet reflect or measure the competency-based approach to learning envisioned in the NUS. Limited access to academic and career guidance counseling in secondary schools complicates the choice of study field and institution. Ukraine does not have any national or large-scale programs on guidance counseling in secondary schools. While there are various nongovernmental organizations that aim to support students and their families in this area, the scale is limited. Youth who did not have access to academic and career guidance are more likely to randomly select study fields and occupations, according to the ILO School-to-Work Transition Survey. Currently, there is also very limited public information allowing young people to compare programs, fields of study, university rankings, or information on graduates’ employment. Although the government has identified professional counseling as a priority in its 2019 Priority Action Plan, the focus remains limited to vocational education, despite the fact that most students pursue general secondary education instead. Without information on the quality and relevance of individual degree programs or the skills of individual graduates, employers and education institutions become locked in a “credentialist equilibrium.” The supply and demand for educational credentials remains high in Ukraine. There is a strong preference for tertiary education as the means to enter and succeed in the competitive labor market, but there is also evidence that employers may over-emphasize educational credentials. 40 percent of young university graduates were working in jobs which did not require university-level education, as of 2013, compared to 29 percent for prime-age and older workers.55 A comparison of the structure of job vacancies and the required educational profiles seems to support this hypothesis.56 While employers highly demand57 cognitive, socioemotional and technical skills among new hires, more so than any education level, employers still preference ever higher credentials for at least two reasons: (i) they find that credentials do not accurately signal workers’ underlying skills, and (ii) employers do not know how to value or compare some credentials given changes in the structure of the education system over time.58 For example, there is evidence that employers have a fairly low demand for young bachelor’s degree holders because this is still perceived by many local employers as “incomplete” tertiary education compared to the Master or Specialist degrees.59 Without improving the flow of information between education institutions and employers on the supply and demand for skills, the credentialist equilibrium and associated education-labor market mismatches will persist. Moving forward, Ukraine needs to prioritize 3 areas: strengthening student assessment systems in basic education, developing a program for counseling and guidance services, and improving data collection systems and performance monitoring in higher education. (a) Strengthen systems for student assessment in basic education and disseminate results High-performing education systems in the EU and OECD have summative assessments of student learning. Twenty-nine education systems had such assessments at the primary education level, and 27 had them at the lower secondary education level.60 Ukraine has rolled out a grade 4 monitoring survey which is an 23 important step, but other reforms are needed to improve the student assessment system as well to track and improve learning outcomes in general secondary education. In particular, there is a need to introduce an external summative sample-based assessment at grade 9 to monitor the development of key NUS competencies and student readiness to progress into specialized upper secondary education. The system would also benefit from a more systematic approach to measuring quality in preschool education and student readiness for primary school. This could involve the integration of standardized assessments 61 of preschool quality into the quality assurance process managed by the SSEQ. UCEQA has developed a Strategy for Learning Assessments in General Secondary Education until 2030, which represents a strong step towards expanding and improving the student assessment system in Ukraine in line with the learning objectives of the NUS. Moving forward, this Strategy should be further developed and implemented. (b) Develop national program for counseling and guidance services as key pillar of upper secondary reform Educational and career guidance counseling plays an important role in motivating students and keeping them engaged in education by providing information on study options and work prospects and identifying careers that may interest them. Guidance staff also support young people in developing the skills they need to make smart decisions and take responsibility for personal growth and professional development. In many European countries, academic and career guidance is explicitly stated as a measure to facilitate the transition through secondary education and combat early school leaving. For example, some systems like Finland, Sweden, Norway, Spain and Italy specify that providing guidance, supporting students in their decision making, and preparing them to cope with real-world challenges are among the main tasks of all school staff, while in the United Kingdom, schools have a statutory duty to provide access to career advice.62 In Ukraine, students have little information on the labor market relevance of different higher education pathways, including labor market outcomes for graduates of vocational and higher education and different fields of study across higher education institutions. As a result, students end up making momentous educational and occupational choices based on anecdotal information from their peers and families, or based on random selection. A national program for counselling and career/educational guidance services at the upper secondary education level would be instrumental in helping to address this gap in the system. It could include more integrated site visits between schools and local employers, outreach efforts between schools and HEIs, and provision of current and relevant data on labor market outcomes. This should also be paired with transparent and accessible information about HEIs and possible outcomes based on graduate tracer studies, surveys of current students, and other similar sources. (c) Improve systems for data collection and monitoring performance of HEIs and higher education system, including through HEMIS, student surveys, and graduate tracer studies Current mechanisms for collecting data on HEIs and the wider HEI system as a whole should be strengthened in order to better inform policies. Although the EDEBO database contains a considerable amount of information, it is poorly suited for policy analysis purposes. Furthermore, there is a lack of basic data on a number of aspects, such as the number of personnel in HEIs and tuition fees across programs and HEIs. This could be linked to the EIT database and improved into a more effective higher education management information system (HEMIS). The establishment of a student experience and engagement survey could also improve quality assurance processes, accreditation, and monitoring. According to the European University Association (EUA), student experience and engagement surveys are the most common way for institutions to introduce quality assurance processes.63 There are several examples of such surveys that could inform Ukraine’s reform, such as the National Student Survey (NSS) in the United Kingdom, which is used for external quality assurance 24 and is obligatory for publicly funded universities in the UK. Additionally, surveys such as the North American National Survey of Student Engagement (NSSE) and the Student Experience in the Research University (SERU) survey are voluntary and used for institutional self-improvement and internal quality assurance efforts. Allowing Ukrainian universities to ask university-specific questions on such a survey may be an option to increase the response rate and buy-in. Other metrics of university performance and graduates’ employability would be helpful to monitor quality. This could include the introduction of university rankings, graduate tracer studies, and other such mechanisms. One possible model is Poland’s Graduate Tracking System, which relies on data submitted by HEIs as required by the law, and is managed by the same agency that manages Poland’s HEMIS.64 *** Ukraine has embarked on an ambitious reform agenda with great potential to transform and reposition the education system as a driver for economic growth and social prosperity. However, without attending to the imbalances identified in this report, reforms may simply “tinker” with the status quo, rather than achieving the broad-based changes that Ukrainians expect and deserve. The experience of other high-performing education reformers has shown that reform can succeed if it is backed by political will, broadly supported through engagement with stakeholders, fiscally sustainable, and coherent within the education system. Addressing the imbalances and tensions that remain in the sector will help to secure these conditions, putting Ukraine’s education reform agenda on a path to success for the benefit of the next generation and the prosperity of the country. 25 Table O.1: Priorities for Action under Ukraine’s Education Reform Agenda Priority Areas Short-Term Medium-Term • Develop coherent and evidence- • Accelerate reform of upper secondary based sector strategy for higher education to introduce 3-year education, prioritizing labor curriculum structure and streamlined Vision market relevance, institutional educational pathways diversity, and sustainability • Maintain and expand support for inclusive education • Complete accreditation procedures • Build managerial and technical Institutional for higher education capacity for schools, local authorities, and HEIs on internal quality Governance assurance, planning, and resource Structures management • Reform public funding model for • Implement additional incentives to higher education to promote reward or penalize HEIs on efficiency strategic objectives and of resource use Resource consolidate resources Flows • Monitor and evaluate secondary school financing formula • Monitor and evaluate teacher • Restructure career path and Stavka certification program workload system for teachers and • Introduce more tools for detection faculty Incentives and oversight of academic • Modernize EIT to align with NUS and misconduct higher education strategic vision • Develop criteria for university • Develop counseling and guidance rankings services for secondary school students • Develop and implement graduate • Expand EMIS to other sub-sectors, tracer survey introduce capabilities for tracking Information individual students and staff, and link systems to broader investments in e- government and digital teaching and learning materials 26 Notes 1 World Bank (2019a). 2 World Bank (2018a). 3 World Bank (2019a). 4 World Bank (2019c). 5 KAS (2017). 6 Jorda and Alonso (2017). 7 Del Carpio and others (2017). 8 Kupets (2016). 9 World Bank (2017a). 10 Denisova-Shmidt and Prytula (2017); OECD (2017a); Osipian (2017, 2009). 11 Psacharopoulos and Patrinos (2018). 12 OECD (2017a). 13 EBRD (2016). 14 Kupets (2016). 15 OECD (2018b); Barber and Mourshed (2007). 16 Based on authors’ analysis of SSSU data from 2017. 17 UCEQA (2018) 18 Based on data for 2018 university admissions campaign. 19 World Bank (2018c). 20 World Bank (2019d). 21 World Bank (2018c). 22 Herczynski (2017). 23 World Bank (2019c). 24 MOES (2017b). 25 OECD (2011). 26 KAS (2017). 27 World Bank (2016). 28 KAS (2017). 29 OECD (2018b); Barber and Mourshed (2007). 30 For example, see OECD (2013). Synergies for Better Learning: An International Perspective on Evaluation and Assessment. OECD Reviews of Evaluation and Assessment in Education. 31 Sovsun (2019). 32 Paes de Barros, et al. (2019). 33 http://wiki.sklinternational.org.ua/ 34 KAS (2017). 35 Arnhold, Kivisto, Puttmann, Vossensteyn, and Ziegele (2018). 36 OECD (2017c); World Bank (2012). 37 World Bank (2016). 38 Arnhold and others (2018). 39 Pruvot, Estermann, and Mason (2015). 40 De Boer and Jongbloed (2015). 41 Curaj, Deca, and Pricopie, eds. (2018). 42 OECD (2018a). 43 World Bank (2019b). 44 DIF (2015); Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 45 Denisova-Schmidt, Prytula, and Rmuyantseva (2018). 46 Osipian (2007); Klein (2012). 47 Authors’ analysis of Developers of Ukraine survey of 2,938 current students and recent graduates of information technology (IT) programs, conducted in May 2018. 48 Altbach (2013); De Waal (2016); Hallak and Poisson (2007). 49 World Bank (2019b); OECD (2018b). 27 50 Steiner-Khamsi (2016). 51 Arnhold, Pekkola, Puttmann, and Sursock (2018). 52 MOES announcement on education ombudsman office: https://mon.gov.ua/ua/news/z-1-sichnya-2019-roku-v- ukrayini-zyavitsya-osvitnij-ombudsmen-sho-zahishatime-prava-uchniv-studentiv-osvityan-i-naukovciv-uryad- prijnyav-vidpovidnu-postanovu 53 Transparency International (2013). 54 Shekshnia and Denisova-Schmidt (2017); Denisova-Schmidt (2018). 55 Kupets (2016). 56 Del Carpio and others (2017). 57 Muller and Safir (2019). 58 Kupets (2016). 59 Nikolaiev (2017). 60 OECD (2017). 61 Such as those developed under the Measuring Early Learning Quality and Outcomes (MELQO) initiative. 62 European Commission (2014). 63 Loukolla and Zhang (2010). 64 For reference, click here for the link to Poland’s graduate tracking system: http://ela.nauka.gov.pl/en/ 28